September… when school is back in session, the leaves begin to change, and stable Internet connectivity apparently remains elusive in many countries. Although the Internet disruptions observed around the world this during last month were not due to the change of seasons or the start of school, their underlying causes were very familiar, including power outages, national exams, severe weather, network issues, and DDoS attacks.

Power Outages

Internet connectivity in Venezuela was disrupted several times over the course of the month due to multiple widespread power outages. On September 12, NetBlocks reported that Caracas was experiencing its second power outage in two days, corroborated by a Tweet from @CORPOELECinfo, an account associated with Venezuela’s national electric corporation.

The impact of these two outages can be seen in the figures below for the Capital District and Miranda – Caracas straddles both states. A minor disruption can be seen mid-day on September 11, with a more significant disruption occurring mid-day on September 12.

• 

• 

Just over a week later, another significant power outage disrupted Internet connectivity in Venezuela for several hours, as shown in the figures below. Starting around 1500 UTC on September 20, the disruption lasted until approximately 0100 UTC on September 21.

• 

• 

The disruption was also very visible at a local level as the CAIDA IODA graphs below for the Venezuelan states of Lara, Tachira, and Zulia show, with marked drops in the Active Probing and Darknet metrics.

• 

• 

• 

Brief power outages early in the day (GMT) on September 24 and mid-afternoon (GMT) on September 27 again caused Internet disruptions visible at a country level, as seen in the figures below.

• 

• 

National Exams

After seeing several Internet disruptions during the last week of August intended to prevent cheating on national exams, two more disruptions occurred in Iraq during the first few days of September. As the figures below show, partial disruptions were observed on September 1, 2, and 3, with diminishing severity across the three days.

Hurricanes

Several significant Atlantic Ocean hurricanes impacted Internet connectivity during September, starting with Hurricane Dorian. On September 1 & 2, Dorian made multiple landfalls in the Bahamas, causing significant damage. As the Oracle Internet Intelligence and CAIDA IODA figures below show, Internet connectivity in the Bahamas began to degrade shortly after midnight (GMT) on September 2, and then degraded significantly around 0730 GMT. The impact can also be observed at a city and network level, as seen in the additional CAIDA IODA figures below. However, in the main city of Freeport, the Internet disruption started earlier, with a significant drop in the Active Probing metric observed around 0200 GMT. While a slight decline was observed in the Active Probing metric for AS15146 (Cable Bahamas) around 0200 GMT, a more significant decline occurred around 0730 GMT. As the Google Transparency Report graph for the Bahamas below shows, connectivity took several weeks to return to “normal” levels.

• 

• 

• 

• 

A week later, after moving up the eastern seaboard of the United States, Hurricane Dorian landed in the Canadian province of Nova Scotia. There, the strong winds impacted Internet connectivity for subscribers of local network provider Eastlink in cities including Halifax, Dartmouth, and Truro, according to Fing Internet Alerts. The figure below from Fing shows that the disruption began around 1320 GMT, which corresponds to the time that the Active Probing and Darknet metrics began to decline in the CAIDA IODA figure below.

On September 19, Hurricane Humberto battered Bermuda, reportedly knocking out power for 80% of the island’s electricity customers. This widespread impact is best reflected in the lower peak in the DNS Query Rate metric seen in the Oracle figure below, the significant drop in the Darknet metric in the CAIDA IODA figure, and the visibly lower peak and deeper trough in the Google figure. Oracle’s Traceroute Completion Ratio metric and IODA’s Active Probing metric also saw declines, as would be expected, but the drops were less pronounced. This may be because a large portion of the targets selected for these active measurements are infrastructure components kept online with backup generators or other alternative power sources.

• 

• 

• 

Network Issues

On the morning of September 1 (GMT), country-level disruptions are visible in the Oracle Internet Intelligence Country Statistics graph for Haiti, shown in the figure below. Three separate disruptions occurred – the first lasted approximately four hours, starting at 0300 GMT, followed by two separate hour-long disruptions, starting at approximately 0740 GMT and 0900 GMT.

The observed disruption was apparently due to a fiber cut experienced by local provider Digicel Haiti, as it explained in the Tweet below. (English translation, via Google Translate: “We want to apologize to all customers who encountered a difficulty yesterday or morning while using the network! Unfortunately one of our underwater fiber optic cables was cut off! Our technicians work tirelessly to repair it quickly, now all our services are working.”) The Oracle Internet Intelligence Traffic Shifts figure shown below illustrates the impact of the fiber cut in Digicel Haiti – a near complete failure of traceroutes through upstream providers to reach the network, with a significant spike in latency for the few that did get through.

On September 16, Internet users in St. Thomas on the U.S. Virgin Islands experienced an Internet disruption due to a fiber cut. In a post to its Facebook page, local service provider Viya noted that a public works contractor had severed a fiber optic cable, impacting connectivity. The CAIDA IODA figure below shows a visible drop in the Active Probing metric for St. Thomas starting just after 1200 GMT. (However, the apparent corresponding increase in the Darknet metric is unusual – it is not clear why it occurred.)

Singaporean provider StarHub experienced a multi-hour disruption on September 17 that impacted both fixed and mobile Internet users. To their credit, StarHub was quick to respond, and ultimately posted an update on its Web site, providing more information on the incident:

“An internet routing issue has been identified as the root cause of the incident, and StarHub has ruled out a cyber attack. StarHub takes this incident seriously and is implementing measures to prevent a recurrence.”

The CAIDA IODA figures below, ordered by the number of IPv4 prefixes originated, show the impact of the routing issue on five StarHub autonomous systems. The most significant impact appeared to be with AS55430, associated with StarHub’s Next Generation Nationwide Broadband Network. Connectivity for AS10091, associated with StarHub’s discontinued cable Internet service, also saw a major impact.

• 

• 

• 

• 

• 

Fing Internet Alert Tweeted about an Internet disruption affecting New Zealand provider Trustpower on September 29. The CAIDA IODA figure below for the network shows that the disruption was brief but significant, with clear declines occurring in both the Active Probing and Darknet metrics. Trustpower posted several apology Tweets like the one shown below, explaining that it had experienced a brief outage in its fiber services.

Somewhat unsurprisingly, chronically problematic submarine cable SEA-ME-WE-3 suffered another failure on September 20, according to a published report. While the cable connects Australia to Asia and Europe, the Perth-to-Singapore segment suffered issues this time, with the problem apparently occurring between repeaters 345 and 346. The segment has experienced documented problems in the past, but traffic has been able to fail over to the Australia-Singapore Cable (ASC) since September 2018.

DDoS Attack

Cool Ideas, one of South Africa’s largest Internet Service Providers, was targeted by a DDoS attack on September 21 & 22. The attack reportedly used DNS Amplification to overwhelm Cool Ideas’ connections with traffic. Cool Ideas posted status updates to its Web site while the attack was underway, alerting users to potential connectivity loss and degraded performance.

Based on the CAIDA IODA figure below, it appears that the attack may have come in several waves, with multiple declines seen in the Active Probing metric during the two day period. The Oracle Internet Intelligence figure below shows how the attack impacted latency for traceroutes to targets within the Cool Ideas network, with increases observed during the waves of the attack. The figure also shows how Cool Ideas’ upstream connectivity shifted as a result of the attack, with traffic moving from AS6939 (Hurricane Electric) to AS3741 (Internet Solutions). A Cool Ideas representative noted that they had integrated attack mitigation mechanisms with their Internet Solutions connectivity, which is likely why they shifted to having more traffic flow through that upstream provider (so it could be scrubbed).

This upstream connectivity shift is illustrated in more detail in the figure below from IIJ Lab’s Internet Health Report. The graph shows the AS dependencies for Cool Ideas’ ASN during the duration of the attack. Before the attack began, Cool Idea had a significant dependency on AS6939, but this dropped over the course of the attack, with other network providers, including AS3741 and AS174 (Cogent), seeing increased AS hegemony.

Correction

In the August 2019 Internet Disruption Report, we classified the Internet shutdown in the Jammu and Kashmir region of India as a sub-national shutdown that was not visible on publicly available monitoring tools. A CAIDA IODA team member alerted us that the disruption was in fact visible in their tool. The figure below shows a decline in the Active Probing metric on August 4, when the shutdown began, along with a corresponding drop in the Darknet metric. We thank the CAIDA team for the update, and apologize for the error.

Conclusion

September also saw a number of Internet disruptions where the cause was unclear – research did not uncover any associated local event or information published by a service provider on their Web site or social media presence. That latter point continues to be a source of frustration, with multiple service providers again failing to respond to requests made via social media for additional information about Internet disruptions observed to have impacted their networks.

Having said that, the underlying reasons for government-directed Internet shutdowns ultimately become well-known, and in early September, the New York Times published an article that examined both the human and economic costs of such shutdowns, looking at several that have recently occurred in African nations.

“The shutdowns do more than stunt the democratic process. They can batter whole economies and individual businesses, as well as drastically disrupt the daily life of ordinary citizens, turning the search for mobile service into a game of cat and mouse with the police and driving people across borders just to send emails for work.”

And although this has been in planning for quite some time, Russia will apparently soon begin testing its ability to effectively isolate the Russian Internet (RuNet) from the global Internet. Driven by legislation signed earlier this year, according to a published report, the legislation mandates that Russian Internet service providers have the technical means to disconnect from the rest of the world and reroute Internet traffic through exchange points managed by Roskomnadzor, Russia’s telecommunications and media regulator. If they are successful in implementing such a system, tests and/or activation will likely result in Internet disruptions being visible in popular monitoring tools not only for Russia, but also neighboring countries that route traffic through Russia.