IAM Assessments: Why So Subjective?

What Is an IAM Assessment?

Thousands of identity and access management (IAM) assessments are conducted in the IT industry each year. These can be high-level, comprehensive or focused on one specific IAM function. Most assessment methodologies include gathering information about the current state of security, identifying the desired future state and then comparing both of these against frameworks.

Yet none of the major security frameworks are focused on IAM. The big names in security frameworks — NIST, ISO, GLBA, PCI, ISF Standards of Good Practice, etc. — include excellent IAM criteria. But there are a few problems:

  1. None have comprehensive IAM content.
  2. They are not consistent and occasionally in conflict.
  3. They are vague and difficult to objectively apply.

So when your organization gets an IAM assessment, what is it getting? The answer is: subjective expertise that may apply multiple criteria from multiple frameworks. Responsible IAM practitioners among our ranks use the frameworks that are available, but they are also diligent to cover the topics missing from those frameworks and apply those frameworks in a manner relevant to the organization.

They are creating evaluation criteria and creatively applying standard frameworks. Subjectivity and creativity are essentials parts of an IAM assessment. Is that wrong? I don’t think it is!

The Value of Subjectivity

Look at the cars on the road. There are dozens of major manufacturers, each offering dozens of models. Each model can have three or more option packages and the ability to special order with specific features (e.g., colors, number of speakers, heated steering wheels). There are literally thousands of permutations possible.

This is accepted as normal because we all have different needs and wants. One could argue that I don’t need the 4:1 low range transfer case in my Jeep Wrangler Rubicon, but I want it. Should there only be a few models and options? Certainly not!

Now think of the diversity among organizations: governmental, nonprofit, manufacturing, banking, retail; startup, small, medium, large, multinational; knowledge-focused, production-focused, financially focused. Should there only be one method to assess an organization’s identity and access management? Certainly not!

Back in the 1920s, in the infancy of the auto industry, there were only a few automobile options. That was accepted because our needs and wants were more limited and the industry was in its infancy. Are we still in the infancy of information security? Certainly not!

We have evolved so that every organization has different needs, wants and subjective business values. Subjectivity is an essential part of the IT industry.

Getting Repeatability Through Experience

The ability to repeatedly and consistently assess organizations is the foundation of the public account and audit industry. After more than 150 years, those standards have been well codified, and yet there is still subjectivity. That is not a bad thing; it is an example of how public accounting firms have made their work repeatable.

They achieve this by creating work plans, staffing models and quality assurance practices that assure investors the organization is properly adhering to accounting rules. As a result, it’s possible for investors to objectively compare companies’ financial performance. Hence, repeatability is the goal, and subjectivity is not tied to repeatability.

One could argue, with thousands of IAM assessments conducted every year, that there should be some basis of repeatability: a framework. I believe there is repeatability, it’s just not codified into one framework — yet.

Repeatability does not absolutely require a framework. Frameworks do not absolutely ensure repeatability. Instead, we have to rely on the subjective expertise of the IAM practitioners who are conducting the assessment.

The same evolution that affected the public accounting industry is happening to security and IAM. The frameworks are maturing, more practitioners are gaining experience and more organizations have repeatable methodologies. These three things — frameworks, experience and methodologies — create repeatability.

Strategy Is a Differentiator and It’s Subjective

IAM assessments are helpful to understand the maturity of an environment and opportunities for improvement. It’s often easy to find solutions once the problems are identified. However, when the problems constitute a long list with interdependencies and high costs to remedy, it’s important to have a formalized strategy and road map. In these complex situations, it takes experience and subjective creativity to define a path forward. There simply is no framework for a strategy!

Strategies to improve IAM solutions start with correlating the source and remediation activities. One change — and this is particularly true of strategic changes — can affect multiple findings.

For example, an IAM assessment may include all the following findings:

  • There is poor decision-making ability among the team.
  • The Web access management technology is not supported, leading to a long recovery time objective (RTO).
  • Stakeholder engagement is delayed, or no ongoing engagement exists.
  • There are 16 people required to complete the integration process for each new application.

On first review, it seems logical to address each of these via separate changes. However, these can all be addressed by changing the organizational structure and IAM responsibility matrix. Correlating these and tying them to an organizational change takes creativity.

Once there is a well-correlated list of issues and resolutions, they need to be prioritized. I’ve written about prioritization, and those security portfolio lessons are good ones to apply. In a strategy, the availability of funding and the system dependencies are the most important factors. While risk, duration, resource availability, likelihood and all the other potential factors are important, a strategic initiative should look beyond the tactical and near-term needs toward making significant impacts to the organization. Long-term funding and project dependencies are more important for strategic initiatives than short-term tactical changes.

When building a strategy and road map of activities, it’s likely there will be tactical projects (e.g., implementing intrusion detection for an application platform) and strategic projects (e.g., creating defined and consumable IAM service offerings with service-level agreements). In the latter example, the order of activities and optimal use of funding is essential because it affects processes, communications, organizational behavior, multiple technologies and many other elements.

In light of all of this, every strategy and road map will be subjective because every organization is unique. Experience and creativity are essential to this subjective process being effective.

Leveraging Subjectivity

As we’ve seen, subjectivity is unavoidable and necessary. Further, subjectivity requires experience, the ability to leverage frameworks, effective methodologies and creativity.

Subjectivity is not bad but it requires care to get the best results. When evaluating how to assess your IAM ecosystem and who will do it, you need to evaluate these factors:

  • Team member experience;
  • Team structure;
  • Methodology;
  • Quality assurance approach;
  • Supporting knowledge bases; and
  • Which frameworks will be used.

Once you establish these things, you can learn to embrace the multitude and somewhat fractured state of industry frameworks and trust the subjectivity without compromising repeatability.

Share this Article:
Brett Valentine

Associate Partner, IBM

Brett Valentine is an Associate Partner with IBM Security. He has more than 17 years of industry expertise. He has a Bachelor of Science degree in Computer Science, and a Master of Business Administration. Brett lives in the metropolitan Detroit area.