Technologies like artificial intelligence (AI) and neural networks are driven by deep learning — machine learning algorithms that get “smarter” with more data. The deepfake, a severe cybersecurity threat, wouldn’t be possible without deep learning.

Deepfakes aside, we need to be aware that several machine learning models, including state-of-the-art neural networks, are vulnerable to adversarial examples. The threat to the enterprise can be critical.

What is an adversarial example, and why should we care? These machine learning models, as intelligent and advanced as they are, misclassify examples (or inputs) that are only marginally different from what they normally classify correctly. For instance, by modifying an image ever so slightly — even altering just one pixel in some cases — image recognition software can be defeated.

With more companies relying on deep learning to process data than ever before, we need to be more aware of these types of attacks. The underlying strategies behind adversarial attacks are fascinating. Colorful toasters are even involved.

Adversarial Attacks 101

Luba Gloukhova, founding chair of Deep Learning World and editor-in-chief of the Machine Learning Times, finds herself at the hub of the machine learning and deep learning industries. I met Gloukhova, an independent speaker and consultant, at a tech conference in February, where she told me that the more she understands about the capabilities of deep learning, the more potential security risks we are exposing ourselves to become apparent.

“As I saw some of the potential shortcomings of this technology, it got me venturing down this path of adversarial attacks on adversarial examples, and it got me really interested in this industry,” Gloukhova said.

According to Gloukhova, an adversarial attack is one in which inputs to a deep learning neural network ultimately result in unexpected outputs. The example here is the input itself.

“The adversarial examples are generated by a slight modification to the input to a network which is trained to recognize it,” she explained. “The slight modification in one way or another creates completely unexpected results and unexpected outputs.”

This unexpected output exposes vulnerabilities in the architectures of deep neural networks because a bad actor can craft the adversarial examples to disguise malicious content as legitimate to the human eye. One fundamental analogy she often uses is that of a hacker trying to spread malicious code, where the code (input) could fly under the radar of most recognition systems.

“This adversarial attack could contain malicious content that’s hidden because of slight modifications that make it look as if it’s not problematic,” she said.

Toaster? Banana? Other?

Back to that colorful toaster. Back in 2017, at the 31st Conference on Neural Information Processing Systems in Long Beach, a group of data scientists presented evidence of their toaster theory. By placing a sticker of a toaster nearby or on an object (input) for image recognition software, the software (or neural network) would classify it as a toaster. In their experiment, a banana was used as the input.

On its own, it gets classified as a banana with high accuracy. But as soon as we add the toaster sticker, the network thinks it’s a toaster. “It misleads the classification model, and what’s interesting is that the authors of the paper have shown that it is successful in attacking models that have completely different architectures and were trained with completely different differences,” said Gloukhova.

What’s most concerning about these types of adversarial attacks is that the bad actor doesn’t need any information about the model it’s attacking. Fortunately, we’re not highly reliant on this technology at this point.

Threats Well Beyond Image Recognition

Still, Gloukhova warns that the computer security threat is significant, and she doesn’t want to paint a picture that only image recognition systems are vulnerable. In fact, a research paper from Cornell University demonstrates how essentially the same adversarial attacks could be applied to natural language processing systems based on deep neural networks. In the experiment, they were able to take any audio waveform and produce another that is over 99.9 percent similar and transcribe it as any phrase of their choosing.

As our reliance on autonomous vehicles grows, deep neural networks will be a critical component of the transportation ecosystem. This network, not unlike other deep learning networks, is susceptible to adversarial examples. According to Gloukhova, a serious threat has been outlined in research papers that demonstrate how by simply placing some inconspicuous spray paint or a particular sticker on a stop sign, the vehicle may fail to classify it.

“The deep neural network that resides within that vehicle’s recognition system could make it misinterpret the actual stop sign as, say, a yield sign or a speed limit sign,” she warns. “So my goal is to communicate that these adversarial examples do exist. They are a vulnerability of these deep neural networks. We need to be aware of the limits of this technology, especially as our fascination with it and our reliance on it grows.”

We are likely a long way away from this scenario playing out at scale in the real world, but planning to counter these threats now can better position us for safety.

How the Enterprise Can Respond

So what does all this mean for the enterprise? On the surface, the issue can seem too complex to get our minds around, but some of the strategies Gloukhova promotes revolve more around mindset and culture than technology. Of course, getting into the developmental and architectural weeds will still be necessary.

“By leveraging these complex architectures and huge training data sets, we’ve chosen to opt for models and model architectures that make them open to adversarial attacks,” she said. “A big takeaway for this is the overall awareness of this vulnerability.”

She hopes that developers and executives will be more cautious about the kinds of activation functions and neural network architectures they choose, which may come down to using smaller training data sets or adopting less fancy architectures.

Gloukhova recommended developer tactics like non-linear activation functions, adversarial training, JPEG compression, defensive distillation and gradient masking. However, these technical solutions are not so simple, and they require a lot more computing power and some sacrifices.

One of the top priorities for any organization should be to hire the right people for deep learning. “It does take the kind of people that are aware of these threats and are aware of the countermeasures,” Gloukhova said. “I’m excited to spread the word, especially toward the C-Suite. They need to see the value in this kind of knowledge so it can trickle down through the organization to the developers, and maybe it will create some lasting change.”

More from Artificial Intelligence

Data Privacy: How the Growing Field of Regulations Impacts Businesses

The proposed rules over artificial intelligence (AI) in the European Union (EU) are a harbinger of things to come. Data privacy laws are becoming more complex and growing in number and relevance. So, businesses that seek to become — and stay — compliant must find a solution that can do more than just respond to current challenges. Take a look at upcoming trends when it comes to data privacy regulations and how to follow them. Today's AI Solutions On April…

Tackling Today’s Attacks and Preparing for Tomorrow’s Threats: A Leader in 2022 Gartner® Magic Quadrant™ for SIEM

Get the latest on IBM Security QRadar SIEM, recognized as a Leader in the 2022 Gartner Magic Quadrant. As I talk to security leaders across the globe, four main themes teams constantly struggle to keep up with are: The ever-evolving and increasing threat landscape Access to and retaining skilled security analysts Learning and managing increasingly complex IT environments and subsequent security tooling The ability to act on the insights from their security tools including security information and event management software…

4 Ways AI Capabilities Transform Security

Many industries have had to tighten belts in the "new normal". In cybersecurity, artificial intelligence (AI) can help.   Every day of the new normal we learn how the pandemic sped up digital transformation, as reflected in the new opportunities and new risks. For many, organizational complexity and legacy infrastructure and support processes are the leading barriers to the effectiveness of their security.   Adding to the dynamics, short-handed teams are overwhelmed with too much data from disparate sources and…

What’s New in the 2022 Cost of a Data Breach Report

The average cost of a data breach reached an all-time high of $4.35 million this year, according to newly published 2022 Cost of a Data Breach Report, an increase of 2.6% from a year ago and 12.7% since 2020. New research in this year’s report also reveals for the first time that 83% of organizations in the study have experienced more than one data breach and just 17% said this was their first data breach. And at a time when…