Day-to-day security functions are typically maintained by a specialized team in a security operations center (SOC). Although the core goal of the SOC is to provide a safe environment for the business, its responsibilities often expand far beyond detecting, analyzing and coordinating the responses to security incidents.

This poses a number of challenges when the time comes to measure the success of your SOC. In addition to their normal duties, SOC analysts must prevent incidents, assist in the development of security policies and governance, support compliance obligations, and consume security-related data sources that report on new threats. By building out consistent measurements to review your SOC’s essential functions, you can evaluate its performance and growth more accurately.

Pen Testing Is Great, but Not Enough

One way to measure the success of your SOC is to analyze its performance in response to a penetration test of the most important company assets. When done properly, this exercise will give you essential insights into how efficiently your SOC can handle security incidents and shed light on which areas need improvement.

However, a proper pen test requires a lot of resources, during the exercise itself and then afterward for reporting and follow-up actions. For most organizations, it will be difficult to organize them continuously, so you have to identify other resources to measure the success of your SOC.

People and Roles

You need to have people with deep technical and business knowledge on your SOC’s board. They don’t all have to be permanent members of the SOC; they can also function as a virtual team. Keeping a pool of experts that can serve as an extension to the SOC during specific incidents can greatly increase your success.

Although counting the certifications held by your staff may not provide a valuable metric, you can still track how your staff improves over time. Developing staff proficiency in a specific domain is one measurement that you can use to demonstrate the quality, maturity and knowledge level of your SOC. These improved skill sets can be built upon with internal or external training.

Technical knowledge should cover the information technology systems that are in use within your organization. Ideally, your staffing will consist of security experts with broad knowledge in incident response, threat intelligence and offensive security capabilities. Staffing is typically done in tiers that correspond to levels of expertise.

Technology

An incident management platform that contributes to your staff’s collaboration isn’t the only technology that will influence the success of your SOC. You obviously need technology to process, triage, analyze, correlate and distribute consumed event data. Most often you’ll use a security platform that includes a security information and event management (SIEM) system. But first you’ll have to define the log data sources that feed into your SIEM. The most common data will be the logs from security devices covering endpoints, servers, networks and authentication services.

Note that success of technology will not be measured by the implementation itself, but how it adds value to the organization. If you start consuming a data source, then you need to make sure that it contains sufficient information to support incident detection, analysis and response.

Policies and Processes

Another element that can influence the success of your SOC is a clear description of policies, responsibilities and escalation, and incident response processes.

These policies and processes set the bar for the entire organization to understand what is appropriate behavior and define who does what, when and why. These policies engage employees and enable consistent actions and repeatable outcomes. The policies will also establish authority to act in case of an incident and will outline consequences for failing to abide by the rules.

Sticking to these controls offers the additional advantage that you can support (and measure) implementation using benchmarks defined for each metric. Your SOC could also draft policies based on the security events measured from the organization’s data, and then suggest that the governance team transform these drafts into ironclad policies. Counting the number of successfully transformed suggestions can also help demonstrate the success of your SOC.

Planning for the Long Term

If your SOC does not have short- and long-term visions that align with the business’ objectives, priorities and risk posture, it will be difficult to sustain a long-running and successful project. You might be able to start the SOC, but keeping the team and technology running over time requires budgeting and resources to reach beyond the kickoff phase.

Identify and classify the company’s critical assets. Your SOC should know what assets belong to the organization, their importance and the business impact of their potential compromise. After all, you can’t defend what you don’t know.

You should also be sure to calibrate expectations. Sometimes someone outside the SOC finds a threat or intrusion sooner than the SOC staff. Which procedure should they then follow? Foresee steps to engage in an ad hoc collaboration with the reporter and the typical answers the SOC would first want to address before reporting an incident further within the organization.

Reporting and Metrics to Measure the Success of Your SOC

To make your SOC a success story, it’s crucial to strike the right balance between sufficient reporting and not overwhelming the organization with too much information. If you measure everything and report every single metric under the sun, you’ll make it very hard for your audience to distinguish what’s essential. That’s not to say you shouldn’t try to measure as much as possible — in fact, some of these measurements can be used internally by the team to improve processes.

One popular metric that can be misleading is time to respond, also called time to resolve. When taken out of context, this data falls short of its potential to help measure the success of your SOC. Assess other metrics similarly for multidimensionality: Does it contribute to a complete, usefully informed picture?

Compiling Your Success Report

So what should you include in your reporting? Obviously, it’s useful to report on the number of incidents. You would ideally classify the number of incidents according to an internally defined taxonomy and include a brief status and summary field. You should also identify a list of key stakeholders in your SOC report, and whether there was any change in this lineup.

Awareness campaigns always provide useful metrics. Quantify them per department or by hierarchy in the organization and include the evolution metrics. Report how many exercises were done during the past year, how they align with the expected outcome and what changes can be observed.

Quantify how many experts outside of the SOC were required to assist with an incident. This can include people within the organization as well as external contributors. Also document how many times an escalation was required, and for which type of incidents. Report the initial criticality when the event was first detected and in which category it was classified.

Track the number of incidents or investigations that were started based on intelligence received from outside the SOC and their results. You can also report the number of times the SOC provided intelligence to a team outside the organization. This can include collaboration with, for example, the national computer security incident response team (CSIRT).

Finally, list the most common incident origin (“attacker”), target (“victim”) and attack vector used by threat actors. Be sure to also include the incident remediation techniques that were used, from both process and technology points of view.

Additional Reports

Besides the above metrics, there is some specific reporting that you can include to give the SOC more exposure. For example:

• Create a regular newsletter and have the SOC routinely report enterprise security news, either on a weekly or monthly basis.
• Report on internal investigations related to high-exposure security news. What has your team done and what was the outcome (even if you have to report that nothing was found)?
• The current results of vulnerability scanning and asset discovery within your organization.
• Information on the evolution of your organization’s digital footprint.

Although you should definitely start the base of your reports with the numbers from your incident management solution, your reports should never solely rely on a “dump” from this platform. Appoint someone to review the data and provide additional context where needed. Similarly to the metrics, the data in the report should be focused on providing relevant, high-quality information. Once you’ve successfully composed the story of your SOC’s success, contextualizing the data for the whole organization will help to improve your overall security culture and practices.