Snynet Solution Logo
MON - SUN: 10 AM - 6 PM
+60 11 5624 8319

Blog

One of the largest internet outages ever recorded occurred this weekend

Image Description

Following a misconfiguration in one of its data centers, the US internet service provider (ISP) CenturyLink suffered a major technical outage that spread across the internet taking down many popular sites and services on Sunday.

The error at the company's data center spread outward from its network and also ended up impacting other ISPs, which led to connectivity problems for many other companies including Amazon, Twitter, NameCheap, OpenDNS, Reddit, Discord, Hulu, Steam and others.

Cloudflare was also severely impacted by CenturyLink's outage and in a blog post, CEO and co-founder of the web infrastructure and website security company Matthew Prince explained how the incident affected the internet as a whole, saying:

“Because this outage appeared to take all of the CenturyLink/Level(3) network offline, individuals who are CenturyLink customers would not have been able to reach Cloudflare or any other internet provider until the issue was resolved. Globally, we saw a 3.5% drop in global traffic during the outage, nearly all of which was due to a nearly complete outage of CenturyLink’s ISP service across the United States.”

Incorrect Flowspec rule

Based on information from a CenturyLink status page, it appears the issue originated in the ISP's CA3 data center in Mississauga, located in Canada's Ontario province.

As its own services were affected by the outage, Cloudflare paid close attention and believes that an incorrect Flowspec rule that came at the end of a long list of BGP updates may have caused it. 

If this was the case, every router in CenturyLink/Level(3)'s network would have received the Flowspec rule and started blocking BGP, which would lead them to stop receiving the rule.

The devices would then start back up, work their way through all the BGP rules until they got to the incorrect Flowspec rule and BGP would once again be dropped, creating an endless loop.

BGP routes are a type of message that internet companies relay between each other to inform each internet provider which group of IP addresses is available on their network. However, CenturyLink's incorrect Flowspec rule also brought down some routers outside of its network which began to announce incorrect BGP routes to other Tier 1 internet services. This brought down other networks, causing the major internet outage experienced over the weekend.

Thankfully, CenturyLink was able to fix the issue by telling all other Tier 1 internet providers to ignore any traffic coming from its network. This type of action is usually a last resort as it results in all of the company's customers losing internet connectivity.

Via ZDNet

Date

31 Aug 2020

Sources


Share


Other Blog

  • AMD Radeon RX 6700 XT launched to steal Nvidia RTX 3070's 1440p crown

    The AMD Radeon RX 6700 XT finally follows the Radeon RX 5700 XT, bringing Big Navi to 1440p gamers.

    Read More
  • After EIC, Web.com and Miss Group continue round of consolidations in web hosting market

    More acquisitions emerge as companies look to consolidate geographically and strategically

    Read More
  • Apple M1 MacBooks will soon be able to run Android apps, as well as iOS

    Apple’s M1 MacBooks can run iOS apps designed for iPhones and iPad – and soon they will be able to run Android apps as well.

    Read More
  • India tops the list of countries that are for cashless economy - 79% Indians in favour

    A survey by MoneyTransfers.com spoke to residents from 21 different countries whether a move towards a cashless society would be “positive” or “negative”. The answers gave a clear idea of the countries most approving of the change.

    Read More

Find Out More About Us

Want to hire best people for your project? Look no further you came to the right place!

Contact Us