Detecting cyber threats is usually the first critical step in the mitigation of cyber attacks. Common means to achieve this goal are rules or analytics that track network and system behaviors and raise alerts when potentially malicious attacks are identified. Once a potential threat is detected, the staff of the security operations center (SOC) investigates it and, if it is found to be a real risk, responds, contains and mitigates it. Threat detection is a growing challenge, and one that demands constant attention since attackers continuously gain expertise and sophistication and threats are ever-changing. The detection rules/analytics of today must therefore dynamically adapt to the threat landscape over time.

New attacks arise daily and most of their damage is achieved in the first few hours before organizations know they have been breached. Additionally, attackers are using sophisticated techniques to avoid existing detection mechanisms by altering their behaviors. With this challenging reality, pulling together the wisdom of the security community and sharing open-source tools, techniques and best practices is a powerful approach for protecting enterprises against cyber threats. This blog describes how IBM is contributing to this community effort.

We begin by describing the use of an event log rule format that’s an open-source standard and is applicable to any type of log file.

Sigma for Log Events Opens the Doorway

Sigma is a generic and open signature format for log events. It enables the development of shareable detection methods across various SIEMs. As stated on the project’s website: “Sigma is for log files what Snort is for network traffic and YARA is for files.” Moreover, the Sigma repository comes with about 500 built-in rules for several target log sources, including Windows, Linux, web, network, cloud and more. The project has a vibrant community that provides frequent updates. Sigma works with a conversion tool, called sigmac, which converts detection rules from the open Sigma rule format into queries for various target security systems, such as QRadar, Splunk and Elastic.

Sigma contains hundreds of rules for a variety of detection environments, which are mostly Windows. Sigma allows the public to share threat detection rules without being vendor specific, thus providing the community with an excellent collaboration environment.

For example, a new rule was recently added by the community to detect log patterns that might indicate an attack on a Microsoft Exchange server using the ProxyLogon vulnerability:

Figure 1: New Sigma rule for the detection of the ProxyLogon vulnerability

Using Sigma can be a cost-effective way for any organization with any size security budget to become more secure by relying on timely rule updates that can cover emerging threats.

Next, we look at STIX (Structured Threat Information Expression), a pattern that catalogs threat intelligence and forensic findings. Best of all, Sigma rules can be converted to STIX.

STIX for Threat Intelligence Exchange

An open standard called STIX is a language and serialization format used to exchange cyber threat intelligence artifacts. STIX is widely adopted in the industry landscape of threat intelligence. It defines a query language called STIX patterning, which allows users to search threat indicators in STIX data.

The latest piece of the STIX puzzle is the STIX-shifter open-source project. STIX-shifter is a Python library and a command-line tool that allows software to connect to products that house data repositories in different data models. It does this by using STIX patterning to return the results as STIX observations.

The STIX ecosystem gives organizations the ability to query and consume their security data in a standardized way that is better understood by both humans and machines.

The ability to convert Sigma rules to STIX helps security teams use tools that interact well together. The value of being able to convert Sigma rules into STIX patterns is evident. It opens up the world of open-source Sigma rules to the STIX ecosystem, thus allowing organizations to query many security tools for threat indicators and receive the results in a standardized way. It also opens up the use of Sigma rules to a wider world of security products that are not supported by the Sigmac converter, but for which a STIX-shifter module does exist.

Enhancing Security Platforms With STIX

IBM Security is part of the open-source communities of security tools, such as Sigma and STIX. As such, we contributed to Sigma to align it with IBM’s new security platform IBM Cloud Pak for Security, which uses STIX to connect to different security products so they can query and consume data in one place. IBM Cloud Pak for Security is a containerized software platform built on top of Red Hat OpenShift, making it deployable anywhere on-premise or on any cloud.

IBM Cloud Pak for Security is an open security platform that connects to existing data sources to generate deeper insights. Whether the data resides on IBM or third-party tools, on-premise or multicloud environments, the platform helps find and respond to threats and risks — all while leaving the data where it is. This makes it easier to uncover hidden threats, make more informed risk-based decisions and respond to incidents faster.

IBM’s Contribution: Where Sigma Rules Meet STIX Patterns

As part of the community that uses these tools, we set out on a journey to create and contribute the necessary code to convert Sigma rules into STIX patterns. Our goal is to make them available to the community or be leveraged within IBM’s Cloud Pak for Security.

A STIX Backend

Sigma introduced the term backend, which refers to the target system handler, similar to the Splunk or QRadar query language. We created a new backend for the STIX query language. In addition to the backend translation logic (Python code), we also contributed mappings to translate Sigma fields to STIX fields.

Let’s examine what a Sigma rule looks like with the following example. The condition field constructs the final query for the rule. That query can be built from a number of subqueries. In this example, it is made of only one subquery called ‘selection’. The subquery defines two fields to be filtered by a condition: CommandLine and Image. Certain conditions can be applied to these fields, such as equals, starts with, ends with, etc. In this case, the query is looking for the command-line field ending with either .dll or .csproj and the Image field ending with \dotnet.exe.

The first step in converting a Sigma rule to a target query language is mapping the fields of the target system to the relevant fields in the Sigma ontology.

For example, the field in Sigma called filename is mapped to the STIX field file:name.

A more complicated example is the mapping to the Sigma field filehash, since in STIX there is a differentiation between the hash types in the field name, while in Sigma there is none. Hence, the filehash field in Sigma gets mapped to all possible file hash fields in STIX: file:hashes.SHA-256, file:hashes.MD5, file:hashes.SHA-1.

Note that while this translation logic is correct, meaning it will not yield a false positive, it is less efficient since there are multiple comparisons made instead of one.

The next step is to define how the conditions specified in Sigma are converted to a conditional query for the target query language. For example, this is how one would translate the conditional keywords in Sigma to a STIX pattern condition:

Figure 2: Translation of the conditional keywords in Sigma to a STIX pattern condition

After implementing these mappings, the sigmac converter can be executed to convert Sigma rules from their open format to the specific target format. As part of our work, we contributed the mappings needed for converting Sigma to STIX patterns; they can be found here. The sigmac compiler can now be used to translate Sigma rules to STIX.

For example:

Apply the sigmac tool with the STIX backend:

sigma_path/tools/sigmac -t <target_language> -c <mapping_file> <rule_path>

sigma_path/tools/sigmac -t stix -c stix2.0 rules/windows/process_creation/process_creation_dotnet.yml

The STIX pattern results can be used to perform queries using the STIX-shifter or Cloud Pak for Security. Let’s examine what the generated STIX pattern from the previous Sigma rule  example looks like:

[((process:binary_ref.name LIKE ‘%\dotnet.exe’) AND (process:command_line LIKE ‘%.dll’ OR process:command_line LIKE ‘%.csproj’))]

This is what the query looks like in IBM Data Explorer on Cloud Pak for Security:

Figure 3: Screen capture of query in the IBM Data Explorer on Cloud Pak for Security

This query can now be submitted via Data Explorer to any security product that is configured on Cloud Pak for Security, whether it is an endpoint detection and response, a SIEM or an identity and access management tool. The results will be translated into STIX and presented to the user in a normalized fashion. When new rules are contributed to the open-source Sigma project, you can repeat the sigmac conversion and get the most up-to-date set of rules as STIX patterns.

Figure 4: Sigma to STIX conversion flow

To summarize, converting Sigma rules to STIX patterns allows us to use the evolving set of rules in the open-source Sigma project. These rules can be used to hunt for threats in a variety of security products that are supported by the STIX-shifter project or have a native STIX application programming interface. The results can be consumed in a normalized fashion without dependence on the security product that provided the results. This capability can be leveraged on IBM Cloud Pak for Security in a manual threat hunt or proactively during an automated threat investigation using the Threat Investigator app.

Open Standards, Open Source, Open Security

As businesses move further into cloud maturity, applications and data are spread across multiple private and public clouds and on-premise resources. In a recent SANS Institute report, sponsored by IBM Security, more than half of security teams surveyed said they struggle to integrate data with disparate security and analytic tools and combine that data across their cloud environments to spot advanced threats. Attempts to protect this fragmented IT environment often require security teams to undertake complex integrations and continuously switch between different screens and point products.

IBM is committed to making security open, simple and easy to deploy across any hybrid or multicloud environment. As part of that commitment, Cloud Pak for Security features innovations to connect with any security tool, cloud or on-premise system, without moving data from its original source. The platform includes open-source technology for hunting threats, automation capabilities to help speed response to cyberattacks and the ability to run in any environment.

Cloud Pak for Security is the first platform to leverage new open-source technology pioneered by IBM, which can search and translate security data from a variety of sources, bringing together critical security insights from across an organization’s multicloud IT environment. The platform is extensible, so that additional tools and applications can be added over time.

Learn more about using open security tools here.