This is not a game

The military looks toward immersive gaming tech to train its troops

U.S. military trainers have leaned on simulators and virtual reality technology for many years. Yet the technology has improved tremendously in the past decade. Processing power has increased, graphics cards now render photorealistic environments, and latency, or lag, has been reduced.

And the development is continuing. Faster frame rates, cloud computing, artificial intelligence advancements and the introduction of 5G wireless technology will all contribute to more realistic training, and make it more affordable.

“Before you needed big, bulky simulators, racks of computers and a large physical space like a gym, warehouse or hangar,” said Harry Buhl, Raytheon Technologies Synthetic Training Environment lead investigator. “Now, it’s a very small footprint locally – it could be something you wear, like goggles driven wirelessly by a laptop, and the processing power could be done in the cloud. You could train pretty much anywhere in the world.”

Raytheon Intelligence & Space, a Raytheon Technologies business, is incorporating these improvements into a Soldier/Squad Virtual Trainer as part of a pursuit for the U.S. Army’s Synthetic Training Environment. The trainer will allow soldiers to sharpen their skills at their home station in a realistic virtual world built on a video game engine. The Army should award an initial contract some time in 2020.

“Much of today’s training is just paper and pencil. Soldiers get PowerPoint slides and a textbook and are then told, ‘Here, go study this; go learn this,’” said Dean Hoover, RI&S Extended Reality chief engineer.

If soldiers instead learn by using 2-D and 3-D representations of the actual system over and over again, it improves emotional engagement, movement coordination, retention and understanding, according to Hoover.

“In the end, it’s going to shorten your training time because you’re looking at a representation of what the real deal looks like, feels like, acts like, reacts like,” Hoover said. “And then, when you get out to the real thing, you’ve already seen that.”

Hoover cited one example where soldiers wearing VR headsets trained for 30 minutes on a virtual weapon system. Then they were allowed to fire the real weapon. Nine of 10 trainees hit the target on the first try, he said.

Training in a virtual environment allows soldiers to get their “reps and sets” in at their home station or while deployed regardless of range availability. Space is limited at the Army’s premier training grounds, known as combat training centers, which are booked many months in advance.

“The ultimate test before being deployed in theater is still going to be ‘live’ training,” Buhl said. “It’s hard to replicate the sweat, the dirt, the bugs, the smell and fatigue in a virtual environment. You also don’t want to put your soldiers unnecessarily at risk with very dangerous training events like in-flight emergencies or live artillery, so that’s why the blending of synthetic and live together, using augmented reality technologies, is the ultimate goal.”

The U.S. Air Force and U.S. Marine Corps currently use a Raytheon Technologies-developed virtual trainer for maintaining the V-22 Osprey tiltrotor aircraft. The trainer works in 2-D on a three-touchscreen computer that mimics the V-22 multi-function display. There is also a virtual reality version with 3D animation that lets maintenance crews work on the inside and outside of the aircraft. The trainer incorporates the real V-22 mission computer software into the gaming engine.

“The trainees can do something as simple as flipping on the avionics power switch to...folding up the wings so it can fit on the deck of a ship,” Hoover said. “That procedure is very complex, and it’s probably something they’ll only get to do once in real-world training. If you do just one of the steps wrong, there’s a chance of breaking something. So, they do it over and over and over again, ad nauseam, until they get it right in the virtual space.”

While Raytheon Technologies is using commercial game engines, gaming graphics cards and VR headsets, Buhl and Hoover said that the military isn’t using the technology to “play games.”

The Army is using virtual reality and augmented reality “to immerse their people into a synthetic environment and provide realistic visualizations that will enable tough realistic training, when and where it’s needed, all to a high degree of cost effectiveness,” Buhl said. “It also trains soldiers with faster and more accurate decision skills, because they replay scenarios and see how different decisions result in different outcomes.”