Air traffic control meltdown made worse by password problems of engineer working from home

Resolving an air traffic control (ATC) meltdown in August 2023 was made more difficult because of delays in verifying the password of an engineer allowed to work remotely, an inquiry has found.

More than 700,000 passengers suffered disruption when flights were grounded at UK airports on August 28 last year after ATC provider National Air Traffic Services (Nats) suffered a technical glitch while processing a flight plan.

An inquiry set up by regulator the Civil Aviation Authority (CAA) found that Nats rostered a Level 2 engineer to be on call rather than on site that day, despite it being one of the busiest of the year in terms of flight passenger numbers.

A more junior Level 1 engineer, who was on site at Nats’ headquarters in Swanwick, Hampshire, began checks as soon as automatic flight planning systems failed.

The Level 2 engineer was contacted 34 minutes later but their password login details “could not be readily verified due to the architecture of the system”, the report stated.

After exhausting remote intervention options, it was agreed they would attend the control centre but it took a further one-and-a-half hours for them to arrive, which was three hours and 15 minutes after the incident began.

Nats should consider rostering a Level 2 engineer on site during busy periods such as the summer, the inquiry found.

It acknowledged this would be a “significant” expense, but insisted it should be viewed in “the context” of the overall cost to the industry and passengers from the August 28 2023 failure, which it estimated at reaching up to £100 million.

The inquiry was led by Jeff Halliwell, who has served as a chief executive and non-executive director in roles across the private and public sector.

He said: “Our report sets out a number of recommendations aimed at improving Nats’ operations and, even more importantly, ways in which the aviation sector as a whole should work together more closely to ensure that, if something like this does ever happen again, passengers are better looked after.”

The report recommended that Nats should give earlier notice to airlines and airports of possible disruption.

Some aviation organisations said it took too long for them to be informed about the August 2023 failure, with several first hearing of it from media coverage.

The report said: “Most of the airlines and airport representatives agreed very strongly that earlier warning of a potential problem would have made a considerable difference to their ability to make precautionary preparations, which in turn would have reduced the negative impact on passengers.”

There were also concerns over the “quality and style” of Nats’ communications, and “considerable frustration about the inability to ask questions or to find out detailed information”, it added.

EasyJet chief executive Johan Lundgren said: “The report makes clear once again that airlines and passengers were severely let down by Nats due to its failure of resilience and lack of planning.

“Airlines were then left picking up the pieces and costs, which ran into millions.

“Lessons must now be learnt and the recommendations urgently actioned by Nats, as an ATC failure of this scale, a crucial part of national infrastructure, can never be allowed to happen again.”

The inquiry found that an automatic flight planning system and its back-up shut down within 20 seconds after a plan for a flight from Los Angeles to Paris (Orly) was received.

This was because of a “unique set of circumstances not previously encountered”, including a pair of duplicate three-letter waypoints, which are used to identify locations.

The system had previously processed more than 15 million flight plans without this scenario being seen.

The failure led to flight plans being manually processed, decreasing the rate from up to 800 per hour to 60 per hour.

A Nats spokesman said: “We would like to apologise again for the inconvenience passengers suffered because of this very unusual technical incident.

“Over the 15 months since this incident, we have worked hard to address the lessons from it, and to ensure it cannot ever happen again.

“Our own internal investigation made 48 recommendations, most of which we have already implemented; these include improving our engagement with our airline and airport customers, our wider contingency and crisis response, and our engineering support processes.

“We fixed the specific issue that caused the problem last year as our first priority and it cannot reoccur.

“We will study the independent review report very carefully for any recommendations we have not already addressed and will support their industry-wide recommendations.”

The inquiry noted that a number of affected passengers waited “many weeks, and in some cases months” for airlines to refund their out-of-pocket expenses.

It recommended that the CAA is given the power to “take consumer enforcement action” without going through the courts, which could include the ability to fine airlines.

Transport Secretary Louise Haigh said: “The Nats IT failure last year was an unprecedented event that we all hope never happens again, so I welcome the final report and its recommendations to strengthen the sector and restore passenger confidence.

“I’ve said before that I will be the passenger-in-chief and my priority is to ensure all passengers feel confident when they fly.

“That’s why my department will look to introduce reforms, when we can, to provide air travellers with the highest level of protection possible.”

Under the Conservative government, in June last year the Department for Transport set out plans to give the CAA “stronger enforcement powers”, but no legislation on the issue was introduced to Parliament.