One of the biggest-ever IT outages has disrupted companies across the world, from airlines to financial services and media groups, after what should have been a routine software update cascaded into crisis.
Thousands of flights were cancelled on Friday, workers in cities from Tokyo to London were unable to log on to their computers, hospital operations were postponed and some television channels went off air.
The outage has been blamed on a security update from US group CrowdStrike, which caused a problem with Microsoft’s Windows. PCs and servers are affected, suggesting that millions of computers may need to be fixed for the issue to be resolved.
More than 12 hours after the problems began, some services, including airlines and media groups, had begun to come back online. But the unprecedented scale of the fault means it could take days for every Windows user to recover.
“I don’t think it’s too early to call it: this will be the largest IT outage in history,” said Troy Hunt, a prominent security consultant, in a social media post. “This is basically what we were all worried about with Y2K, except it’s actually happened this time.”
Tesla’s chief executive Elon Musk wrote on social media: “Biggest IT fail ever,” followed by “Microsoft” next to an angry emoji face.
In an online advisory, CrowdStrike told customers they would need to restart every affected computer, delete the botched update file, then reboot again. Allie Mellen, analyst at IT research group Forrester, said: “Fixing this issue requires manual, hands-on keyboard work — in some cases, for hundreds or thousands of affected machines.”
In China, some workers welcomed an early start to the weekend after their employers told them to go home. “Thank you Microsoft for an early vacation” trended on microblogging site Weibo, with users posting pictures of blue error screens.
Australian businesses were the first to warn of problems, with the operations of retailers including Woolworths and 7-Eleven hit. Sydney airport said “a global technical outage” had affected its operations.
In Europe, airlines and airports warned of disruption on what was set to be the busiest day for departures from the UK since October 2019.
The US Federal Aviation Administration said Delta, United and American airlines grounded flights due to take off. The three largest US carriers later resumed flying, but almost 8,000 flights in the US were delayed or cancelled, according to aviation data provider FlightAware.
“This is a very, very uncomfortable illustration of the fragility of the world’s core internet infrastructure,” said Ciaran Martin, professor at Oxford university’s Blavatnik School of Government and former head of the National Cyber Security Centre.
In an online post earlier on Friday, Microsoft said it was aware of “an issue” affecting Windows devices running the CrowdStrike Falcon agent, which may “get stuck in a restarting state”. That appears to have contributed to the problems that hit some of its cloud computing customers.
CrowdStrike is one of the world’s largest providers of “endpoint” security software, used by companies to monitor for security problems across a huge range of devices, from desktop PCs to checkout payment terminals.
Shares in CrowdStrike were down 9 per cent and Microsoft 0.5 per cent in early afternoon trading in New York.
CrowdStrike’s chief executive George Kurtz said on Friday morning that the company was “working with customers impacted by a defect found in a single content update for Windows hosts”.
“This is not a security incident or cyber attack,” Kurtz said. “The issue has been identified, isolated and a fix has been deployed.”
Microsoft said: “Earlier today, a CrowdStrike update was responsible for bringing down a number of IT systems globally. We are actively supporting customers to assist in their recovery.”
However, the incident is far from resolved, with multiple sectors reporting continued problems on Friday:
Financial services
Global banks, brokers and financial technology companies including JPMorgan Chase, UBS and Bloomberg were hit, leaving some traders unable to access their systems in order to process trades.
JPMorgan’s trading execution systems were affected, while UBS and fintech company ION Group also faced problems on Friday morning, according to people familiar with the matter.
Some Bloomberg Terminal users also faced problems and interruptions before the UK market opened on Friday. Broker CMC Markets said it was experiencing issues “currently impacting access to our trading platforms”.
“It’s a mess,” said a senior trader at an asset manager. “This is the biggest upset in years.”
The London Stock Exchange Group said trading was running normally on its venue but its news service was unavailable in the morning before it was restored later.
Transport
Airline passengers faced the prospect of extended disruption as carriers began the painstaking process of restoring flight schedules following the outage.
Flight cancellations built up throughout the day, with some of the worst problems in the US. United said the problem affected multiple systems, including those for calculating aircraft weight, checking in customers and call centre phones.
More than 5,100 flights had been cancelled by Friday afternoon in New York, more than 4 per cent of all scheduled flights, according to data provider Cirium.
The disruption came at one of the busiest travel times of the year, leaving airlines with little leeway to recover operations quickly, and with planes and crew stuck out of position.
Airports from the US to the UK, India and Australia reported problems on Friday, with Dutch carrier KLM temporarily suspending operations after finding it “impossible to handle flights”.
Others including British Airways and Ryanair continued operating but warned customers of delays.
“Not all airports and airlines are CrowdStrike customers so that is why not everyone is affected equally,” said Christiaan Hen, chief executive of aviation technology company Assaia.
“However, [given] the highly connected nature of the industry, even if not affected directly, most airports and airlines are affected in some way.”
Media, consumer and sport
Sky News was forced to suspend programming on Friday morning but has since resumed broadcasting. David Rhodes, executive chair of Sky News, said on X that the UK broadcaster had “not been able to broadcast live TV this morning”.
Manchester United was one of many football clubs in England and Scotland to be forced to delay the release of tickets to its matches, blaming a “global Microsoft Servers outage which is affecting many systems, including ours”.
The organisers of the Paris Olympic Games have said the outage has hit their IT activities, just a week before the city is set to welcome millions of visitors.
Health
The majority of GP practices in the UK have been affected by an issue with EMIS, an appointment and patient record system, forcing practices to use non-digital methods of communicating with patients and providing services. UK pharmacies’ ability to dispense vital medicines has been affected.
In Germany, Schleswig-Holstein university hospital, one of Europe’s largest, has been forced to cancel all elective operations and close its walk-in clinics.
The biggest hospital network in the US state of Massachusetts, Mass General Brigham, cancelled all non-urgent surgical procedures and medical appointments.
Reporting by Daniel Thomas, Philip Georgiadis, Stephanie Stacey, Akila Quinio, Robert Wright, Sarah Neville and Nikou Asgari in London, Cheng Leng and William Langley in Hong Kong, Oliver Barnes in New York, Claire Bushey in Chicago
*This story has been amended to correct the quote from a Forrester analyst to clarify that hundreds or thousands of machines would need updates, not hundreds of thousands
