Augmented reality and virtual reality are no longer theoretical concepts, and both have advanced considerably in their design and use in a wide range of industries. While there are many articles, books, research papers, studies and conferences around these two technologies, only a few venues have explored their potential use as threat modeling tools that could, if designed well, enhance security professionals’ understanding of the threat landscape.

A Whole New World

While sophisticated threat modeling tools such as these are still a few years away, let’s take a look at some of the ways augmented and virtual reality might be utilized. Before we do so, however, it’s important to understand key differences between the two emerging technologies.

Virtual reality creates a virtual world for users to interact with. It is more about what users feel or experience in that world than how they connect with it. Augmented reality, on the other hand, blends elements of a virtual world with the real world, so it is more about how the user interacts with those various elements and the components of the real world.

Furthermore, virtual reality is well-suited for the gaming, training and simulation sectors, while augmented reality is better positioned in the commercial, industrial, educational and medical sectors. Of course, there are plenty of use cases for both.

Finally, virtual reality offers only one plausible form factor so far — that of a wearable helmet or goggles. Augmented reality has at least five different form factors in the commercial sector alone and offers one form factor designed specifically for mobile technology.

Virtual Reality for Advanced Threat Modeling

Since virtual reality is designed to provide a wearable experience for end users, it can be used to simulate catastrophic disasters in industrial sectors without putting users in physical danger, destroying company or industrial property, or harming the physical environment. It could make an excellent threat modeling tool for physical threats.

In the industrial sector, for example, virtual reality might be used to simulate a nuclear reactor meltdown, a large fire in a high-rise office building, a coal mine collapse or a major infrastructure failure. Similarly, in the civil sector, it can be applied to replicate a citywide blackout, emergency response in disaster zones, and earthquake and aftershock damage. It can even be used to train astronauts to work on the International Space Station.

There are also many possible use cases for virtual reality in the defense sector. It may be used to simulate terrorist attacks on U.S. or foreign soil, train special forces for overseas missions where the precise physical layouts of buildings are other structures may be unknown, conduct urban warfare exercises and test advanced aircraft.

Unlike augmented reality, virtual reality allows a headset to be integrated with a drone. There are even drone races in which operators are required to fly the devices through structures of which they have no prior knowledge.

Augmented Reality for Threat Modeling

Because augmented reality is primarily designed to enhance how users interact with the world around them, there are dozens, if not hundreds, of potential uses cases for this technology in all facets of threat modeling. Here are just a few:

• Overlaying power lines, water pipes and gas line assets onto a standard map image for field workers in the energy and utilities sector;
• Nuclear reactor safety inspections and walkthroughs;
• Structural inspections of all types in any factory, manufacturing plant, steel mill, oil refinery or oil extraction platform;
• Structural inspections of civil structures;
• Implementation of heads-up displays in all types of automobiles, including self-driving cars, to assist automotive engineers with their work on traffic patterns and algorithms; and
• Visual data modeling for data scientists, statisticians, mathematicians and risk management professionals.

Augmented reality could become advanced enough to allow information security professional to visualize the physical paths an adversary might take through a network, building, city block or industrial facility. It could also provide penetration testers with three-dimensional virtual threat models of applications, software and solution blueprints.

A New Landscape of Threats

While these advanced capabilities of augmented and virtual reality have the potential to revolutionize many industries in positive ways, they also introduce new threats. The implementation of augmented reality in a highly classified environment, industrial control facility, nuclear power station, military base, or any research and development lab that supports defense research could lead to disaster in the wrong hands.

An adversary capable of compromising the application program interface (API) that connects augmented reality software to an automobile’s heads-up display could remotely crash the car. A threat actor could also breach augmented reality glasses to access what amounts to a livestream of personal data without the wearer’s knowledge.

In addition, a malicious insider could wear augmented reality glasses to work to gather intelligence, commit industrial espionage or upload anything the device picks up to a personal cloud. Commercial sector customers might accidentally upload data collected through an augmented reality device to a public cloud. Furthermore, mobile apps designed for integration with augmented reality are susceptible to bad coding practices, insufficient security controls, bad updates, zero-day exploits and malware, just like any other type of software.

Augmented reality is going to be a very unique technology for information security professionals to contend with because of its significant potential to be both helpful and extremely harmful. This technology could solve one problem while simultaneously introducing one or more new ones. Individuals experimenting with the technology, groups developing standards for it, companies creating software and APIs, and organizations deploying augmented reality in one way or another should think very carefully about the direction this technology should move in.