When it comes to threat modeling, many businesses plan as if there were only a few possible scenarios in which cybersecurity or privacy-related incidents could occur. We need to plan for more cybersecurity hazards than just basic social engineering, insider threats and product vulnerabilities. Both our businesses and our customers face threats that are messier than what fits into these neat little boxes.

The Complex Emotions of Social Engineering

When most of us think of social engineering, we think of someone being psychologically manipulated into handing over sensitive information to some shadowy criminal figure. This definition implies some things that are not always accurate. The first incorrect assumption is that what everyone considers sensitive is the same from one person to the next. The second is that people are able to guard information against their attackers until they’re tricked into revealing it. 

For many people, the emotional context of social engineering is significantly more complex than we account for in traditional threat modeling. Let’s examine a few different — though unfortunately very common — situations where things get more complicated.

When Everyday Information is Extra Sensitive

Most of us do not consider our legal name to be private information. We tell it to relative strangers, and we sign it on forms or in emails that could be easily intercepted. Seeing it pop up online would not worry us. But lots of people go by chosen names other than their legal ones, and for a variety of different reasons. 

Likewise, most of us aren’t terribly concerned about strangers knowing who we spend time with. We allow ourselves to be tagged in our family members’ social media posts, we allow our friend lists to be publicly displayed, and many of us choose to allow apps to broadcast our location when other users are nearby. For most of us, this information being publicly available really isn’t a problem.

This situation is not so simple for people who need to protect their location and associations, or that of their contacts. This includes mental health professionals, journalists and social workers, whose clients and sources need to be kept private at the risk of this knowledge affecting their life or livelihood. Activists and people seeking to escape domestic violence or stalking need to closely manage who has knowledge of their whereabouts to protect their own lives. 

When the Attacker is Inside the House

As I mentioned in a recent article on stalkerware, we can’t assume that if someone is the victim of unwanted monitoring software that it’s because they failed to follow security “best practices.” Most threat modeling assumes that people are capable of completely protecting data or assets from attackers.

Statistics for child, disability and elder financial abuse, as well as for domestic violence, show that a shocking number of people experience fraud or other financial crimes when someone they know uses their sensitive information fraudulently. The perpetrator is often trusted and may be considered a carer for the victim. Access to the victim’s accounts may be a necessary part of maintaining their housing or health care.

Threat Modeling for Emotional Complexity

As security practitioners, we need to consider a wider variety of possibilities for misuse of data and systems in our care, not just those that affect the majority of people. A shocking number of companies have found themselves in a nasty PR situation because they failed to consider the harm that their products could do to people with exceptional privacy requirements. And this sort of cybersecurity or privacy incident gets much greater traction in the media, due to the emotionally charged nature of that breach of trust.

There are a few questions you can ask to help address these unique situations. 

  • Are there ways to do what you need to do without requiring customers to provide legal name or location information? 

  • Can you allow people to opt-in to providing this information, rather than requiring them to go through the steps to opt-out? 

  • How can you provide features or architect your systems in a way that can help protect people who cannot conform to security “best practices”? 

  • How will you address the concerns of employees or other staff members who have exceptional privacy requirements or who will become a victim of stalking or domestic violence?

Traditional threat modeling scenarios describe the task only in terms of enumeration and systematic analysis. In the end, it’s not just computers we’re protecting, but humans. For many people, threat modeling has a distinctly emotional component, and this is something businesses also need to address.

More from Application Security

Does Follina Mean It’s Time to Abandon Microsoft Office?

As a freelance writer, I spend most of my day working in Microsoft Word. Then, I send drafts to clients and companies across the globe. So, news of the newly discovered Microsoft Office vulnerability made me concerned about the possibility of accidentally spreading malware to my clients. I take extra precautions to ensure that I’m not introducing risk to my clients. Still, using Microsoft Office was something I did many times a day without a second thought. I brought up…

3 Reasons Why Technology Integration Matters

As John Donne once wrote, “No man is an island entire of itself.” With digitalization bridging any distance, the same logic could be applied to tech. Threat actors have vast underground forums for sharing their intelligence, while security professionals remain tight-lipped in a lot of data breach cases. Much like the way a vaccine can help stop the spread of infectious diseases, sharing threat intelligence and defense strategies can help to establish a more secure future for everyone.  So what…

Why Your Success Depends on Your IAM Capability

It’s truly universal: if you require your workforce, customers, patients, citizens, constituents, students, teachers… anyone, to register before digitally accessing information or buying goods or services, you are enabling that interaction with identity and access management (IAM). Many IAM vendors talk about how IAM solutions can be an enabler for productivity, about the return on investment (ROI) that can be achieved after successfully rolling out an identity strategy. They all talk about reduction in friction, improving users' perception of the…

Controlling the Source: Abusing Source Code Management Systems

For full details on this research, see the X-Force Red whitepaper “Controlling the Source: Abusing Source Code Management Systems”. This material is also being presented at Black Hat USA 2022. Source Code Management (SCM) systems play a vital role within organizations and have been an afterthought in terms of defenses compared to other critical enterprise systems such as Active Directory. SCM systems are used in the majority of organizations to manage source code and integrate with other systems within the…