AI system designed to improve aircraft turnaround should include SAFETY applications

By developer • September 17, 2024

JDA Aviation Technology Solutions

Below is an excerpted, informative article about the use of AI at commercial airports and the inventor of this APP, as well as author of the article focuses primarily on airport managements’ use of this information to “improve aircraft turnaround performance.” Its application to AVIATION SAFETY is only glancingly mentioned. He reports that his invention uses cameras at the gates that capture images of the ground crew, wing walkers, ground directors, baggage handlers and fuelers. The AI has LEARNED HOW TO IDENTIFY SPECIFIC people, aircraft and ground equipment. Monitoring of the cameras has been shown to be able to improve the gate utilization on a real time basis; why not for SAFETY???

What is the size of risks in these areas?

Looked at superficially, an airport appears to be a single integrated public utility; it’s not. The Owner/Operator, typically a municipal organization, is responsible for the terminal, runways, ramps, taxiways, roads within the ALP, hangars, outlying buildings (hangars, repair facilities, kitchens, etc.), parking, ATC tower/navaids/radars/offices, etc. From that position as a “landlord,” the airport grants leases, permitting access to areas within the boundaries, to airlines, terminal vendors, airlines, and a long list of service providers. This web of entities authorized to do business there is frequently disorganized.

Based on data developed by the Flight Safety Foundation, that preeminent aviation safety organization estimates that 27,000 ramp accidents and incidents — one per 1,000 departures — occur worldwide every year. About 243,000 people are injured each year in these accidents and incidents; the injury rate is 9 per 1,000 departures. Ramp accidents cost major airlines worldwide at least US$10 billion a year, the data indicates. These accidents affect airport operations, result in personnel injuries, and damage aircraft, facilities and ground-support equipment.

Recent serious accidents have made it clear that more attention on these airport areas of risk; here are a few press publications on these events:

As the inventor, MR. SCHÄFER, recognizes his system holds potential use in SAFETY. The recent record reviewed above should expedite enhancement of this AI system. A joint test in 2023 involving AAAE Airport Consortium on Consumer Trust (ACT), ADB Safegate, Synaptic Aviation, DFW and American Airlines appears to have widened the scope of its coverage, i.e. FOD, and may provide a quick option.

EXCLUSIVE FEATURE: How are airports revolutionizing the turnaround process with AI and computer vision?

By Anthony James

August 16, 2024

Previously blind to possible delays during the turnaround process, more and more airports are turning to AI and computer vision to see ‘inside the box’.

Everywhere you looked at this year’s Passenger Terminal Expo in Frankfurt in April, it seemed there was a new solution designed to help airports improve aircraft turnaround performance.

“Until very recently, the turnaround has been a ‘black box’,” says Royal Schiphol Group’s Jeffrey Schäfer[1], process owner, aircraft turnaround. Traditionally, Schäfer says, airports allocated a gate to an airline for a flight, but what happened beyond that point was largely out of their control.

“Most of the processes are not carried out by the airport; instead, they are conducted by THE HANDLING AGENT OR THE AIRLINE. As a result, we didn’t have any insight into what was happening. For decades, we only knew we had a problem when we noticed the flight should have departed – there was no early warning or ability to predict off-block times.”

Handling agents kept their own records, applying time stamps to mark the completion of each stage (Figure 1). However, they were not always the most robust in their application. “Getting high-quality data is challenging,” notes Schäfer. “But we need to know because there are other flights – your delay might not be a problem for you, but it is for the next aircraft waiting to use your gate.”

…

FIGURE 1: 10 Key turnaround steps

The airport could switch the incoming flight to a new gate, but this is never popular – with passengers or the airport: “A gate change incorporates so much movement, for staff on the ground, vehicles, baggage, cargo – everything must be moved, often at the last minute. This is why so many airports are now looking for better information – we want to have a predictable delay. Flights will always be delayed, but we want to know it as soon as possible, so we can act without hampering other processes and protect the continuous flow of flights, avoiding serious disruption.”

…

“The next flight has to wait until the stand is available, which creates extra emissions – so it’s not just about waiting times.”
Jeffrey Schäfer, Royal Schiphol Group

Deep dive

To remedy the situation, Schiphol is now using a dedicated turnaround monitoring system, developed by Aviation Solutions, called Deep Turnaround. It uses historic, real-time and predictive insights to radically improve the aircraft turnaround process for all stakeholders.

Using AI image-based processing, an algorithm detects and reports more than 70 unique turnaround events in 30 turnaround processes. It can detect delays as early as 40 minutes before the targeted off-block time, helping Schäfer and his team make more informed decisions.

Two cameras mounted above every stand track each turnaround event, sending snapshots to the cloud every five seconds so an end-to-end AI model can accurately register exactly what’s going on at the apron, in real time.

“The algorithm can recognize each individual vehicle or process that’s going on – not just when the fuel truck arrives and is in position, but also when the actual hose is connected from the truck to the aircraft,” explains Schäfer

…

“Process managers can now communicate with ground handlers – when they receive an alert, they can liaise with the relevant stakeholders,” continues Andersson. “They are already responding to alerts from the system when events such as catering, baggage loading and off-loading are not occurring as planned; and are getting in touch with the ground handlers to rectify the situation in a timely manner, so that the passengers can still reach their destinations on time.”

“Process managers can now communicate with ground handlers.”
Thomas Hoff Andersson, Berlin Brandenburg Airport

The insights gained allow process managers to have fact-based conversations in the operations center and act in real time to step in to try to remedy any potential problems.

“We have created a resilience team whose task is to ensure turnaround events are occurring as per the predictions and in all situations,” explains Andersson. “For example, if we receive alerts about an event such as baggage unloading not occurring according to schedule, the process manager can request they step in to assist the ground handlers to ensure the passenger is not delayed and the aircraft can take off for its next destination on time. This is also helping us ensure all stakeholders adhere to the agreed service delivery plans. The beauty now is that we can combine the effectiveness of our human talent, digital and physical assets to ensure we are giving the passenger the best-possible experience.”

Fault finder

Schiphol’s Schäfer says these new AI tools are not there to apportion blame, but to assign help: “We did not implement this to prove the handlers are making mistakes, but to find where the exact problem lies,” he says.

…

Dashboard showing key performance data from Deep Turnaround, in use at Schiphol.

…

“This is just the beginning – we are only using a limited amount of the potential we can get from the technology.”
Thomas Hoff Andersson, Berlin Brandenburg Airport

“This is just the beginning – we are only using a limited amount of the potential we can get from the technology. We are already thinking ABOUT HOW IT CAN HELP US IN OTHER AREAS – for example, keeping passengers updated on when their aircraft will be ready to board, based on real-time data rather than the current estimations we all use. I feel there will also be further benefits to come. For example, enabling us to identify SAFETY MATTERS, such as if passengers are on the apron or in the wrong place.”

…

“All these details can be – and are – captured by our data,” he says. “This enables reporting to become easier and brings more accurate information at the same time. This way of working and better understanding the origin of delays will put us all on the right track from the early start as we are now focused on improvement strategies instead of finding root causes.”

However, the airport is not the only actor in the ecosystem that has a role in the end-to-end journey that an aircraft, passenger, bag or piece of cargo makes through the airport. Also, most delays and complex processes occur at airports – not in the air. As a result, Schäfer argues that airports have the most knowledge, but only if they collect the right information in the right processes.

Credit: Royal Schiphol Group

“The airport should not only act as a HUB for flights but also FOR data and insights: if airports can standardize insights and make them available for all, everyone can benefit and improve their own operation,” he says. “Then I’d like to coin the performance strategy, which in our opinion the airport is in charge of – the four CARE pillars: capacity, assets, reliability and execution (Figure 2). At Schiphol, all performance management activities can be put under the umbrella of the CARE pillars and are the core of our own activity, aside from assisting others (small and big) in improving their performance through collaboration and information sharing.”

All airlines operating at Schiphol are constantly analyzed for their on-time performance, with the airport using the generic ADAM algorithm results to see the breakdown of causes of flight delays. Schäfer says it’s fascinating to see the differences between them (see graph above).

“This helps us tremendously in finding the right things to talk about with the specific airlines, as we understand the nature of their challenges,” he concludes. “If I were to approach Airline 7 to address reactionary delays or talk to Airline 10 to address schedule delays, they would call me a fool!”

…