Page 105 of 139 FirstFirst ... 5559596979899100101102103104105106107108109110111112113114115 ... LastLast
Results 1,041 to 1,050 of 1384
  1. #1041
    Retired 10,000+ Posts
    Need some advice on learning networking

    slimslob's Avatar
    Join Date
    May 2013
    Location
    Bakersfield, CA
    Posts
    34,132
    Rep Power
    989

    Re: Need some advice on learning networking

    I h ave found that the problem with Facebook today is that someone was making a change to a router table and made a mistake. Started sending everyone in the wrong direction. Facebook tech staff tried to correct it and ended up making it worse. At least that is the story that is going around Computer Sciences academia.

  2. #1042
    Service Manager 2,500+ Posts rthonpm's Avatar
    Join Date
    Aug 2007
    Location
    Pennsyltucky
    Posts
    2,790
    Rep Power
    108

    Re: Need some advice on learning networking

    Facebook ended up taking all of its associated DNS entries off the core routers of the Internet. Then when they tried to roll back the change, the team couldn't do it because DNS names wouldn't resolve. Complicating matters further, onsite staff didn't have access to the servers to directly make the change at the console.

    Taking down a production network is easier than you think!

    Sent from my BlackBerry using Tapatalk

  3. #1043
    Service Manager 10,000+ Posts
    Need some advice on learning networking

    BillyCarpenter's Avatar
    Join Date
    Aug 2020
    Location
    Long Beach, Mississippi
    Posts
    13,375
    Rep Power
    448

    Re: Need some advice on learning networking

    Quote Originally Posted by slimslob View Post
    I h ave found that the problem with Facebook today is that someone was making a change to a router table and made a mistake. Started sending everyone in the wrong direction. Facebook tech staff tried to correct it and ended up making it worse. At least that is the story that is going around Computer Sciences academia.

    I'm sure that they were using some kind of routing protocol. That's gonna automatically pick the best (fastest) route. However, these routes can be manipulated by manually changing the "cost" of a particular route. Cost is sometimes changed for load balacing reasons. In other words, we don't want all of our traffic going down one path but rather it's better to have 2 paths balance the load.


    And, yes, it'd super easy to take down a production network. Even though I'm learning this stuff, it's more for my own personal enjoyment. I woldn't want to do it for a job. Too much pressure.
    Growth is found only in adversity.

  4. #1044
    Service Manager 10,000+ Posts
    Need some advice on learning networking

    BillyCarpenter's Avatar
    Join Date
    Aug 2020
    Location
    Long Beach, Mississippi
    Posts
    13,375
    Rep Power
    448

    Re: Need some advice on learning networking

    PS - Whomever took down that network will be in the unemployment line. Can you image being THAT guy?
    Growth is found only in adversity.

  5. #1045
    Service Manager 2,500+ Posts rthonpm's Avatar
    Join Date
    Aug 2007
    Location
    Pennsyltucky
    Posts
    2,790
    Rep Power
    108

    Re: Need some advice on learning networking

    Quote Originally Posted by BillyCarpenter View Post
    PS - Whomever took down that network will be in the unemployment line. Can you image being THAT guy?
    I reached out to a long time network engineer that I've known for years to ask his opinion on this. He admitted that this type of misconfiguration happens more often than most people think. The only difference in this case is that it was such a large company, and the split of duties didn't allow someone onsite to deal with the issue.

    Sent from my BlackBerry using Tapatalk

  6. #1046
    Service Manager 10,000+ Posts
    Need some advice on learning networking

    BillyCarpenter's Avatar
    Join Date
    Aug 2020
    Location
    Long Beach, Mississippi
    Posts
    13,375
    Rep Power
    448

    Re: Need some advice on learning networking

    Quote Originally Posted by rthonpm View Post
    I reached out to a long time network engineer that I've known for years to ask his opinion on this. He admitted that this type of misconfiguration happens more often than most people think. The only difference in this case is that it was such a large company, and the split of duties didn't allow someone onsite to deal with the issue.

    Sent from my BlackBerry using Tapatalk

    I've watched a ton of videos on this type of scenario where a switch or router is misconfigured. Most of the time the router/switch that's being configured is at a remote location and it's easy accidentally configure the wrong piece of equipment.

    For the record, I also read that Facebook are building their own routers. At least some of 'em.
    Growth is found only in adversity.

  7. #1047
    Retired 10,000+ Posts
    Need some advice on learning networking

    slimslob's Avatar
    Join Date
    May 2013
    Location
    Bakersfield, CA
    Posts
    34,132
    Rep Power
    989

    Re: Need some advice on learning networking

    Quote Originally Posted by rthonpm View Post
    I reached out to a long time network engineer that I've known for years to ask his opinion on this. He admitted that this type of misconfiguration happens more often than most people think. The only difference in this case is that it was such a large company, and the split of duties didn't allow someone onsite to deal with the issue.

    Sent from my BlackBerry using Tapatalk
    To further complicate matters and delay getting them back up, the offsite personnel who had router access rights needed to go onsite to do so. When they arrived at location,their security ID badges were accepted by the scanners but access also required possession of a physical key for the mechanical lock(s). They had to contact the onsite people to get someone down to the door to let them in.

  8. #1048
    Service Manager 10,000+ Posts
    Need some advice on learning networking

    BillyCarpenter's Avatar
    Join Date
    Aug 2020
    Location
    Long Beach, Mississippi
    Posts
    13,375
    Rep Power
    448

    Re: Need some advice on learning networking

    Spanning Tree is whack butt I've about got it down.


    Spanning Tree is to used to prevent broadcast loops that can bring down the network. You don't have to worry about it if you've only got one connection from one switch to another.


    Anyway, whomever came up with Spanning Tree is some kind of mad genius.

    First a root bridge is elected. Than a root port. Then the designated ports are elected. It's quite the process and I'm glad I learned it. I still need to practice or I'll quickly forget.
    Growth is found only in adversity.

  9. #1049
    Retired 10,000+ Posts
    Need some advice on learning networking

    slimslob's Avatar
    Join Date
    May 2013
    Location
    Bakersfield, CA
    Posts
    34,132
    Rep Power
    989

    Re: Need some advice on learning networking

    A little more on the Facebook outage. It seems that Facebook shot itself in the foot. Or to be more exact, the BGP, Border Gateway Protocol. It left them without a DNS to stand on. "According to Cloudflare—which has published an excellent explanation of what it saw—Facebook’s trouble started when its Autonomous System issued a BGP update withdrawing routes to its own DNS servers."

    Facebook shoots own foot, hits Instagram and WhatsApp too - Malwarebytes Labs | Malwarebytes Labs

  10. #1050
    Service Manager 10,000+ Posts
    Need some advice on learning networking

    BillyCarpenter's Avatar
    Join Date
    Aug 2020
    Location
    Long Beach, Mississippi
    Posts
    13,375
    Rep Power
    448

    Re: Need some advice on learning networking

    Quote Originally Posted by slimslob View Post
    A little more on the Facebook outage. It seems that Facebook shot itself in the foot. Or to be more exact, the BGP, Border Gateway Protocol. It left them without a DNS to stand on. "According to Cloudflare—which has published an excellent explanation of what it saw—Facebook’s trouble started when its Autonomous System issued a BGP update withdrawing routes to its own DNS servers."

    Facebook shoots own foot, hits Instagram and WhatsApp too - Malwarebytes Labs | Malwarebytes Labs

    I haven't covered BGP and I'm not sure if it's covered in the CCNA course. I briefly looked it up just now and the protocol is the only one that will route over the internet for huge networks. Without knowing how it works, it's impossible (for me) to understand what went wrong.
    Growth is found only in adversity.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Get the Android App
click or scan for the Copytechnet Mobile App

-= -= -= -= -=


IDrive Remote Backup

Lunarpages Internet Solutions

Advertise on Copytechnet

Your Link Here