Giving Soldiers a Better View

Augmented reality technology can help make sense of unfamiliar battlegrounds.

By Paul Heltzel

August 6, 2002

A U.S. Marine maneuvering on foot through the streets of Mogadishu, Somalia, spots militia gunmen on a rooftop. With a helmet equipped with a display that superimposes computer graphics over the scene, he can instantly see the street name and building address, check for invisible elements like tunnels and power grids, and, if things get messy, plot the quickest escape routes. 

Had such technology been available in 1993, the military nightmare that inspired last year's film Black Hawk Down, might have been severely mitigated-or even avoided. And while current military gear includes such enhancements such as night-vision goggles and thermal optics, the U.S. Navy is serious about further developing a nascent battlefield information technology called "augmented reality," or AR, committing 2.5 million dollars a year to research.

Augmented reality uses electronics-laden eyeglasses and other head-mounted displays to overlay useful data onto the user's field of vision. While a tracking system locates the soldier's position, a wireless network transmits information about enemy positions, nearby friendly forces, or even hidden infrastructure (which Naval researcher Simon Julier describes as "x-ray vision"). "An AR system is not trying to show a virtual view of a world," says Julier, an onsite contractor from ITT at the Naval Research Laboratory in Washington, D.C. "Rather, it tries to provide information about what the user sees."

University of Southern California Professor Ulrich Neumann directs of the National Science Foundation's research center for multimedia and Internet technologies. "One can imagine little dots representing friend and foe," he says. "If these dots were laid over the scene, it can be a lot more understandable, especially in stressful circumstances, than a map, where you have to translate information into what you see in front of you."

The navy must still overcome several hurdles before augmented reality reaches the battlefield. During the heat of battle, layers of "meta-data" superimposed over one's vision can be distracting-and in war, being distracted is far worse than not having x-ray vision. Says Julier: "If the system shows too much data to the user, it can become overly complicated and uninformative." Julier is working on information filtering software systems, which he says will automatically show only the most critical information.

Story continues below

Tracking is another tricky aspect of augmented reality. The superimposed graphics need to match up precisely to what the user sees, but current technology is too imprecise. Prototypes can point out the building where a sniper is barricaded, but not the window from which he's firing. And finally, augmented reality systems are still too cumbersome.

"The weight issue is a tradeoff between the weight of the computer and the weight of the ammunition," Julier says. Will future Marines trade bullets for heads-up displays? If they can spot a threat, leave maps behind and find a quick exit in unfamiliar terrain, then they just might.