San Francisco-based Fastly blames global internet outage on software bug
Fastly, the San Francisco company hit by a major outage that caused many of the world’s top websites to go offline briefly this week, blamed the problem on a software bug that was triggered when a customer changed a setting.
The problem at Fastly meant internet users couldn’t connect to a host of popular websites early Tuesday including the New York Times, CNN, the Guardian, Twitch, Reddit and the British government’s homepage.
“We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change,” Nick Rockwell, Fastly’s senior vice president of engineering and infrastructure, said in a blog post late Tuesday.
He said the outage was “broad and severe,” but the company quickly identified, isolated and disabled the problem and, after 49 minutes, most of its network was up and running again. The bug had been included in a software update that was rolled out in May. Rockwell said the company was trying to figure out why it wasn’t detected during testing.
The U.S. government recovered millions of dollars in Bitcoin paid by Colonial Pipeline to ransomware hackers who locked up its computer system last month.
“Even though there were specific conditions that triggered this outage, we should have anticipated it,” Rockwell said.
Fastly provides what’s called a content delivery network — an arrangement that allows customer websites to store data such as images and videos on various mirror servers across 26 countries. Keeping the data closer to users means the content shows up faster when users visit those websites.
But the incident highlighted how much of the global internet is dependent on a handful of behind-the-scenes companies like Fastly that provide vital infrastructure, and it amplified concerns about how vulnerable they are to more serious disruption.
Start your day right
Sign up for Essential California for news, features and recommendations from the L.A. Times and beyond in your inbox six days a week.
You may occasionally receive promotional content from the Los Angeles Times.
