Image: MetaVR VRSG real-time scene visualizing the gaze of the wearer of a Varjo XR-3 mixed-reality headset through the collection of eye-tracking data. The scene features an FA-18C entity with Finnish markings in flight over 3D terrain of Albacete Air Base, Spain.

MetaVR and Varjo Introduce Eye Tracking Capability for Pilot Performance Review Training

Sudbury, MA, – MetaVR and Varjo have combined the capabilities of Virtual Reality Scene Generator (VRSG) and the Varjo XR-3 mixed-reality headset to provide a real-time, pilot eye gaze tracking facility to enhance After Action Review (AAR) for simulator-based pilot training.

The Varjo XR-3 mixed-reality headset includes a built-in, real-time 200 Hz pupil-tracking capability that can precisely capture even the smallest eye movement of the wearer when training in virtual reality and mixed-reality environments. MetaVR’s VRSG version 6.6* includes the means to harness this eye-tracking data. At the end of the training mission, VRSG exports this data via DIS as a PDU log and visualizes the pilot’s head position, orientation, and gaze vector (with the gaze of each eye depicted as a color-coded cone) over the events of the training session for AAR.

During training mission debrief, the trainer and the pilot can see where the pilot’s head position and eye movements were directed throughout the simulated air engagement, providing the ability to objectively assess the engagement so as to fully understand what lead to the mission outcome. From a tactical perspective, the eye-tracking playback can help identify missed moments of attention to instrumentation or the direction of an important activity, help identify areas for improvement and track pilot progress throughout the training cycle. It also has application for beyond-visual range systems management review.

In addition to recording the eye tracking visualization for after-action playback, the VRSG session with the eye tracking visualization can also be monitored by instructors in real time during the simulated engagement, to further enhance understanding of the nature of the training engagement.

Among the options for configuring VRSG with a Varjo XR system is foveated rendering. Foveated rendering increases the performance of rendering the entire scene in the headset. This option enables the GPU to render the scene at full fidelity in the area that the headset wearer is looking, while reducing fidelity in the peripheral areas of the scene.

Urho Konttori, Co-Founder and Chief Technology Officer, Varjo, said: “What’s best, pilot training can be enriched without adding to the trainee’s cognitive burden. This can help achieve better learning outcomes and improved live missions down the line. The realistic simulation of flight scenarios in virtual and mixed reality is not science fiction. It’s here today already with Varjo XR-3, already producing tangible value for pilot training.”

“Essentially what we are doing here is turning a normally subjective process into a quantitative review, enriching pilot simulator training and improving learning outcomes for trainees,” Bert Haselden, Mixed-Reality Lead, MetaVR, said. “The Varjo XR-3 gives pilots the most natural and fully immersive training possible; when combined with VRSG the result is a capability that is – we believe – unique to the current simulator training space.”

* Eye tracking visualization is a VRSG version 6.6 feature being offered in advance.

About MetaVR

MetaVR, founded in 1997, develops commercial PC-based software for the military simulation and training markets, featuring high-speed 3D visualization content and rapid creation of networked virtual worlds using real-world data. MetaVR’s real-time visual systems provide the fidelity of geospecific simulation with game-quality graphics. Users can build (with real-world photographic imagery, elevation data, and feature data) high-fidelity virtual worlds with our terrain generation tools, and render in real time, at 60Hz frame rates, the resulting virtual world with our real-time 3D visualization application, Virtual Reality Scene Generator. MetaVR systems are used for applications such as UAS/RPA trainers, manned flight simulators, mission planning and rehearsal, joint fires and JTAC simulation training, urban operations training, and emergency response management training. For more information visit www.metavr.com
Media contact: Claire Apthorp +44 (0) 7920403068 / claire@thatwordbird.com

About Varjo

Varjo (pronounced var-yo) is creating the world’s most advanced VR/XR hardware and software for industrial use, merging the real and virtual worlds seamlessly together in human-eye resolution. For more information visit www.varjo.com
Media contact: press@varjo.com