AI-ENABLED SIMULATOR THAT BUILDS PILOT SKILLS WITHOUT JUDGMENT
Simnest and AviatePro Services, see article below, are joining forces, combining their expertise in flight simulation with experience in detecting human factors[1] to enhance training (initial and recurrent) for pilots. Their product, SIMVIDENCE (described in detail in the attached), catches moments in the SIM by recording “synchronised simulator video, multichannel audio, speech, and session events with observable human-performance signals.” Those now preserved events are used to work with the pilots to develop personal skills to respond to taxing events-
- “…the instructor and pilot can revisit what happened and discuss the pilot’s own interpretation.”
- “The expected recovery time will depend on the pilot’s stage of training. Following performance across successive sessions could therefore allow instructors to see whether recovery becomes faster as experience develops
EXTERNAL COMMENTS
Third‑party commentary on Simvidence is that it is consistently viewed as one of the most mature, instructor‑centric attempts to bring AI into professional simulator debriefing without crossing into automated pilot evaluation. From a learning perspective, SIMVIDENCE by recording the “tells” provides a basis for instructors not to score pilots or automate assessments, but position the pilot/student to see an objective, real evidence of the key moment(s) and consider what skills should be relied upon in future similar events.
OTHER THOUGHTS
Simvidence is unusually well‑suited to helping pilots rebuild and maintain stick‑and‑rudder skills because it gives instructors high‑fidelity, multimodal evidence of how pilots actually fly the aircraft, moment by moment, without trying to automate judgment or replace hands‑on training. A potential class curriculum might include something like the following:
Using this AI powered system allows the trainer to set up a scenario in which the pilot is practicing scanning skills during a long flight. After an extended time of the simulation, at a difficult point, the Auto Pilot fails. The synchronized camera and biometric sensors will capture the student’s reaction with a number of objective data points that show him of her the reaction to the crisis, display how the return to stick-and-rudder worked and give the individual information (NOT CRITICISM) on which to reconstitute REACTING TO SUCH A CRISIS.
CoPilot adds these additional insights into this pedagogical tool:
- The “resume flying” moment — what actually happens
When automation disengages or becomes unreliable, pilots often show recognizable behavioral signals:
-
-
- Hesitation before taking the controls
- Over‑controlling or freezing the controls
- Breathing changes, vocal tension, clipped speech
- CRM breakdowns (“You have control — wait — I have it — no—”)
- Delayed recognition of pitch/energy drift
- Cognitive tunneling (fixating on one instrument)
-
These are normal human reactions under stress — but they are rarely captured cleanly in traditional simulator debriefs.
- Why Simvidence is uniquely suited to capturing these “tells”
Simvidence’s multimodal evidence system makes these reactions visible and reviewable:
-
-
- Synchronized audio — tone of voice, breathing rate, hesitation
- Video cues — posture changes, facial tension, scanning breakdown
- Control‑input traces — over‑correction, delayed response, abrupt inputs
- Automation‑mode transitions — when the pilot hesitates to disconnect or re‑engage
- Crew coordination signals — communication patterns under stress
- How this helps pilots deal with distress, anxiety, and panic
-
- SIMVIDENCE DOESN’T JUDGE PILOTS — it reveals patterns instructors can coach:
-
-
- Recognizing early signs of cognitive overload
- Practicing calm, deliberate control takeover
- Improving breathing and workload management
- Strengthening CRM during high‑stress transitions
- Building confidence in manual flight through repetition
- Reducing fear of “automation loss” events
-
Pilots often don’t realize how much stress shows up in their voice, hands, or scan until they see it. Simvidence makes that visible in a supportive, instructor‑led environment.
SIMVIDENCE’s value is applicable for a wide range of skills and behaviors that are necessary to cockpit proficiency. Any student, hearing from an instructor about “failures,” will likely respond defensively. The benefits of an objective source of “teachable moments” is that there is no third party judgment. The pilot learns from the tape.
AI Makes the Invisible More Visible in Simulator Training
28 September 2026
Pilot instructors are accustomed to spotting the moment a trainee becomes overloaded, startled or begins to struggle. But what if technology could help them identify those moments more precisely, and then take the instructor and pilot straight back to the evidence?
That is the thinking behind SIMVIDENCE Powered by Simnest, an AI-assisted simulator training-review platform being developed through a collaboration between AviatePro Services, a developer of AI-supported human factors and debriefing technology for aviation training, and Simnest Group, which combines flight simulator manufacturer Simnest Aviation and training organisation Simnest Pilot Academy.
The platform combines synchronised simulator video, multichannel audio, speech, and session events with observable human-performance signals. Rather than attempting to assess the pilot, it identifies moments that may warrant closer examination during debriefing.
For AviatePro founder and CEO KRISZTIAN MAKAI, that distinction is fundamental.
“We don’t want to provide mind reading,” he told CAT at APATS 2026. “We’re just reading the signs.”
From observation to evidence
Those signs can come from multiple sources. The system analyses facial movements, voice tone, head orientation, gaze, and the speed of head movements, looking for changes in behaviour and deviations from an individual pilot’s baseline.
A
viatePro has spent more than 18 months developing what Makai describes as an “interpretation layer” that turns raw AI signals into understandable behavioural patterns.
Crucially, the technology does not conclude that a pilot is stressed, overloaded or lacking resilience. Instead, it can identify a change in behaviour and direct the instructor to the relevant moment. “The system highlights a change from the pilot’s own baseline,” Makai explained.
Different signals can also reinforce one another. A startle response, for example, might be visible through facial signals, voice tone, or changes in scanning behaviour. A pilot might develop tunnel vision or, conversely, begin rapidly scanning instruments and controls.
But Makai is particularly interested in what happens next. “HOW THEY RECOVER FROM THE STARTLE – THAT MATTERS,” he said.
The expected recovery time will depend on the pilot’s stage of training. Following performance across successive sessions could therefore allow instructors to see whether recovery becomes faster as experience develops.
Giving instructors somewhere to look
Simvidence combines synchronised recordings with behavioural analysis to help instructors identify and review relevant moments in a simulator session. If it detects an event, warning or significant behavioural deviation, instructors can return directly to that point afterwards rather than searching through an entire recording.
Instructors can also flag moments themselves using a small button device, perhaps identifying an example of strong teamwork or leadership, or something they want to revisit during the debrief.
The result is a collection of reviewable moments rather than an automated verdict. “It’s like a starting point for a facilitated debriefing discussion,” Makai said. “Here’s the tool. It highlighted these five moments. Let’s discuss it, guys.”
Instead of simply telling a trainee they appeared overloaded, for example, the instructor and pilot can revisit what happened and discuss the pilot’s own interpretation. The system also provides role-separated transcripts alongside the recordings.
That evidence may be particularly valuable as younger pilots enter training, Makai believes. He said cadets increasingly expect detailed feedback and the ability to review their own performance rather than simply being told where they need to improve.
Keeping AI away from the grade
As AI becomes more capable, an obvious question is whether technology like this could eventually move from supporting an instructor to assessing the trainee.
Makai is emphatic that this is not what Simvidence is designed to do. He emphasises that evidence captured during a simulator session forms only part of the wider context that informs an instructor’s assessment. “The instructor starts building the bigger picture during the briefing, before the simulator session even begins, and the debriefing can add further context to what was observed,” he said. “Even a multimodal fusion AI system does not have access to everything that informs that professional judgement. Simvidence provides insights and traceable evidence from the session, but the instructor brings those findings together with observations and discussions beyond the system’s view. Our role is to enrich that picture, not to turn individual moments into competency grades.” The distinction is particularly important in Competency-Based Training and Assessment (CBTA), where observations during the simulator form only part of the training and assessment process.
Makai therefore rejects the idea that Simvidence should automatically determine competency grades. “AI should support the instructor’s judgement, not replace it or determine competency grades,” he said. “We decided to show highlights: please review this moment.”
The technology could nevertheless support instructor standardisation. Because moments are timestamped and shareable, instructors and standardisation teams can examine the same evidence and discuss why they interpreted it differently.
But Makai argues that the industry itself needs much greater agreement about how human performance should be interpreted before asking AI to make those decisions.
One million hours
That need for evidence sits behind an ambitious AviatePro objective: collecting one million hours of simulator training data.
Makai calls it the “Wunderlich Project,” inspired by 19th-century physician Carl Wunderlich’s large-scale work measuring human body temperature. The principle is that reliable knowledge requires large amounts of evidence.
“Our long-term ambition is to collect one million hours of simulator training data to support the validation and refinement of the system’s findings.”
The collaboration combines AviatePro’s AI-supported human-performance technology with Simnest’s simulator expertise and real-world training environment. Simvidence is being
developed and validated using Simnest Aviation simulators at Simnest Pilot Academy, where approximately 800 cadets train each year. “The academy’s instructors are highly experienced pilots, whose operational experience and instructional expertise strengthen both the quality of training and the rigour of the software validation process,” said Makai. “Simnest’s forward-thinking approach as a simulator manufacturer is equally valuable, both in bringing Simvidence into the training environment and in exploring how simulator telemetry can enrich our multimodal behavioural insights. That combination of instructional expertise and engineering collaboration is helping shape the next stage of Simvidence’s development.”
“The real value of Simvidence is in giving instructors and cadets more evidence to work with during debriefing. By developing it together with AviatePro in a real training environment, we can create a tool that supports more evidence-based feedback while keeping the instructor at the centre of the process,” said Simnest Group CEO Gyula Kühtreiber.
According to Simnest, use is voluntary, but instructors are already reviewing the system’s insights and using selected material during debriefing, while cadets are returning to review their own performances.
As part of the collaboration, the partners are also exploring integration of simulator telemetry, aircraft-state, and event data to provide additional technical context.
Deployment is intended to remain non-intrusive. Makai said the system can be installed using a computer, four small cameras and an instructor interface, with further installations at Simnest planned as the partners increase the amount of training data available.
For Makai, however, accumulating more data does not change the underlying principle: AI should help instructors identify and examine relevant patterns, while leaving their significance for training and assessment to the instructor.
“AI can help us find and examine the evidence. The instructor remains responsible for interpreting it in context and assessing competence.”
For Simnest and AviatePro, that principle sits at the heart of Simvidence Powered by Simnest: using AI not to replace instructor judgement, but to make more of the evidence behind that judgement visible.
[1] The system analyses facial movements, voice tone, head orientation, gaze, and the speed of head movements, looking for changes in behaviour and deviations from an individual pilot’s baseline
