In Fortnite, you can wage a battle royale against other players around the world.
And you can do so regardless of the videogaming platform you use; PS5, Xbox Series X, Nintendo Switch, Mac, PC or smartphone. It’s called cross-platform play, or crossplay. Fortnite’s shared, virtual world allows this kind of interaction.
The U.S. Army is interested in that approach, exploring complementary concepts and technologies to train its units. The service wants to use commercial virtual and gaming technology adapted for military applications, relying on a common set of data within a modular, open systems architecture. By using off-the-shelf equipment, the “Synthetic Training Environment,” or STE, as the Army calls it, will be “plug-and-play” and quickly available to soldiers.
“With STE, the Army won’t have to deal with the logistics of scheduling and transportation of soldiers to training centers, where they compete for precious training time,” said Harry Buhl, STE lead investigator for Raytheon Intelligence & Space, a Raytheon Technologies business. “They can train at their home stations with minimal overhead, or even while deployed.”
The Army faces a challenge. While legacy training systems enable combined-arms, synthetic training in an operational environment, using its virtual simulators for helicopters and tanks (and simulations for commanders and staffs). These systems lack a common synthetic environment for effective collective training with seamless communication and cooperation across these proprietary systems.
Putting these different legacy systems together for a training exercise requires significant time and overhead. That's where the technology comes in.
STE will train for air, land, sea, space and cyber using virtual and constructive environments that will be initially synthetic-virtual to replace the Army’s aging and costly simulators. The Training Simulation Software and Training Management Tools, or TSS//TMT, foundation will expand to bring in live training, from squads at home stations to brigade combat teams at Combat Training Centers, and constructive training for commanders and their staffs across distributed locations.
“Live training will always be the final test before the Army sends units into combat; it’s graduate-level training,” Buhl said. “But before live training, they’ll do their undergraduate training in synthetic-virtual and constructive worlds. Soldiers can rehearse a mission dozens of times without risking life or limb. They can practice battle drills, emergency situations and mission-critical tasks – what is known as high-consequence training – that can’t be replicated safely in live training, like having an engine fail on a real aircraft.”
This undergraduate training is live and synthetic, and with STE can bring all technology environments together to realistically train and provide mission rehearsal for all echelons—from squads to divisions.
The key to make this happen is a common synthetic battlefield like Fortnite in which all soldiers can fight, and interfaces that are realistic so that soldiers enter this synthetic environment through intuitive means. For the dismounted soldier, the Army is looking for a Soldier and Squad Virtual Trainer that will allow trainees to take actions together or on their own. Soldiers will wear the in-development mixed-reality “Integrated Visual Augmentation System,” which uses Microsoft’s HoloLens goggles.
For combat vehicle crews, the Army is developing reconfigurable vehicle simulators that replicate Abrams, Bradley and Stryker combat vehicles. These simulators will be transportable so that training and mission rehearsal is not limited to institutional locations. With STE, all these interfaces can connect from multiple locations to fight as a cohesive team in the increasingly complex operational environment using the TSS /TMT foundation.
“The virtual or synthetic environments will need to link the live environment for high-consequence training,” Buhl said. “You can’t replace being out in the dirt and mud, sweating, and lacking sleep.”
Similarly, the connection to constructive environment will also follow to bring higher-level commanders and staffs into the same tough realistic and stressful training environment.
RI&S is pursuing both the TSS/TMT, which is foundational to the STE, and the STE Soldier and Squad Virtual Trainer, he said.
“The Army now uses multiple virtual, live, and constructive systems from MILES gear, which is comparable to laser tag, to collective simulators in large buildings at fixed installations,” Buhl said. “These are good capabilities that have served the Army well, but they lack the realism and deployability for the full range of weapons and training situations that the Army uses. With IVAS goggles and reconfigurable simulators, we can bring all brigade soldiers into a realistic training and mission rehearsal experience delivered by RI&S’ TSS/TMT.”
Emerging technologies are an integral element of TSS/TMT. Artificial intelligence and machine learning will allow adaptive training to dynamically adjust the training scenario for maximum value to soldiers as well as commanders who are training their units.
“The technology can tell when you’re bored and ramp up the training, and it can tell if you’re stressing out and decrease the difficulty,” Buhl said. “We want to keep soldiers training in that band of excellence.”