Security researchers have demonstrated how it is possible to use stickers to get computer vision systems in autonomous vehicles to wrongly identify road signs.
Researchers from the University of Washington and other schools recently published a paper that describes a new attack algorithm, known as Robust Physical Perturbation (RP2). The report, “Robust Physical-World Attacks on Machine Learning Models,” detailed how the algorithm makes it possible for errant individuals to alter standard road signs and create havoc for self-driving car systems.
How Does the Attack Work?
The algorithm works in combination with printed images attached to road signs. These images, which could in theory be created by anyone with access to a color printer, confuse the cameras in autonomous vehicles.
The attack relies on undermining the computer vision systems of autonomous vehicles that have been taught to recognize items on or alongside roads using cameras. Computer vision systems in self-driving cars usually rely on an object detector, which identifies pedestrians, signs and vehicles, and a classifier, which works out the nature of the objects and the meaning of the signs.
Systems may be responsive to small alterations to their inputs, known as perturbations, that can cause the vehicles to operate in unexpected ways, reported Car and Driver. Actors would need to access the classifier and then use the RP2 algorithm to create a new, customized image of the existing road sign.
How the Computer Vision Systems Were Tricked
In one of the attacks, the researchers used the RP2 algorithm to create and print a full-size road sign that was placed over an existing warning sign. They created a stop sign that only looked faded to human eyes but was always read as a Speed Limit 45 sign by the computer vision system.
A second technique relied on placing small black-and-white stickers on a stop sign that, once again, led the computer vision system to wrongly identify a Speed Limit 45 sign.
The researchers reported the attacks were effective at a range of distances and angles. In the conclusion to their paper, they stated that they plan to test their algorithm further by altering other conditions that were not included this time around, such as sign occlusion and alterations to other warning signs.
The Implications for Autonomous Vehicle Design
Security fears over autonomous vehicle technology are nothing new. Experts have long directed attention toward the risk of hacks to in-car systems. Earlier this month, in fact, reports centered on a vulnerability in the Controller Area Network (CAN) Bus standard that could impact the security of connected automobiles.
However, this work demonstrated that computer vision systems can also be put at risk. The potential dangers are clear, particularly for vehicles that already use automatic sign recognition. An attacker with access to both the algorithm and the classifier in the in-car system could trick vehicles into responding incorrectly to signs.
While autonomous vehicle development is still at an early stage, self-driving car designers and in-car system manufacturers should take note of the potential dangers. Tarek El-Gaaly, senior research scientist at Voyage, told Car and Driver that such attacks were cause for concern and they could be easier to imitate in the future.
While the risk is limited now, the research highlighted how autonomous vehicle systems could be at risk from malicious actions in the future. Self-driving vehicle manufacturers and computer vision systems designers should take note.