February 21, 2020 By Joerg Stephan 3 min read

Everything you do in threat intelligence is about indicators or patterns. In a binary world, patterns are actually just how different indicators work together in the chain of a malicious event.

Working with threat intelligence for years now, I’ve often asked myself several fundamental cyberthreat intelligence questions:

  • What exactly is this attack and how can I identify it?
  • Is this attack description related to the technology I use?
  • Is the company I want to protect a possible target?
  • What would this attack look like in my environment?

While thinking about the right answers to these questions and creating your own attack model and hypothesis can be a lot of fun, extracting indicators from intelligence and enriching them is usually not. To speed up this task, I always prefer to write small scripts or code snippets to do these things for me, so I can focus on the fun part.

Extracting Different Types of Indicators From Cyberthreat Intelligence

All code examples below can be found on my public GitHub repository. In this blog, we will mainly look at code snippets used in indifetch.py, which is exactly what it sounds like: fetching indicators from a text or string. As a disclaimer, you should always review the regex used and not trust the code blindly. I always advise some cross-checking and keeping an eye out for a better regex or faster way to do a task — the following is just one way to do this.


Hashes are fairly easy. They normally come in two flavors, md5 and SHA. A function that covers md5 could look like this:

<p class=""><em>def getMD5(text):
thisset = set()
md5_r = re.compile(r"([a-fA-F\d]{32})")
for item in md5_r.findall(text):
return thisset</em></p>

As you can see, we use the simple regex [a-fA-F\d]{32} to fetch the indicator out of a given text. The regex matches any character “a” to “f” and any number in a string of 32-character length. We use a Python set because items in a set are unique, eliminating duplicates right from the start.

Changing this function to cover SHA256 is an easy next step. Besides it being a totally different algorithm, the representation is the same but twice the length of the characters ([a-fA-F\d]{64}), using 64 characters instead of 32.

IP Addresses

Next up are IPs. IPv4, in particular, follows a pretty simple pattern: four numbers, none of which are higher than 255, separated by a dot.

<p class="left"><em>def getIP(text):
IPlist = set()
ip = re.compile(r"\b(?:(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\b")
for item in ip.findall(text):
return IPlist</em></p>

We use the same style of function, as I always try to reuse code if I can. One potential problem in this situation is that, sometimes, version numbers use the same style, which can confuse the code.

URLs and Domains

The last common indicators we will cover in a function are URLs and domains. Many reports will have a large set of URLs since most malware has to connect to a command-and-control (C&C) server or exfiltrate data. In addition, with proxies and firewalls, they are often one of the easier indicators to catch.

<em>def getURL(text):
URLlist = set()
urls = re.compile(r'(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+')
for item in urls.findall(text):
return URLlist</em>

Since you are familiar with the style by now, the regex above worked well for me, but I have also used simpler representations in the past, such as:

<p class="left"><em>http[a-zA-Z0-9./:]*</em></p>

Speed Up Your Threat Analysis

Python is a great tool for scripting these little code snippets and speeding up threat analysis. Some scripts can even extract indicators and make an API call to resources like the X-Force Exchange to get the current scoring of the indicator, further speeding up the process.

One regex you may want to look into is for Common Vulnerabilities and Exposures (CVE) numbers. I tend to use something simple, such as:

<p class="left"><em>CVE[^\w]*\d{4}[^\w]+\d{4,}</em></p>

Note: Remember to always use set() in Python instead of lists to remove duplicates right from the start. This comes in handy especially when you automate API calls as part of the script.

There are many cyberthreat intelligence tools and platforms that can do this dismantling of information for you, but it can be extremely useful to understand the magic behind the process before relying on a tool.

More from Intelligence & Analytics

X-Force Threat Intelligence Index 2024 reveals stolen credentials as top risk, with AI attacks on the horizon

4 min read - Every year, IBM X-Force analysts assess the data collected across all our security disciplines to create the IBM X-Force Threat Intelligence Index, our annual report that plots changes in the cyber threat landscape to reveal trends and help clients proactively put security measures in place. Among the many noteworthy findings in the 2024 edition of the X-Force report, three major trends stand out that we’re advising security professionals and CISOs to observe: A sharp increase in abuse of valid accounts…

Web injections are back on the rise: 40+ banks affected by new malware campaign

8 min read - Web injections, a favored technique employed by various banking trojans, have been a persistent threat in the realm of cyberattacks. These malicious injections enable cyber criminals to manipulate data exchanges between users and web browsers, potentially compromising sensitive information. In March 2023, security researchers at IBM Security Trusteer uncovered a new malware campaign using JavaScript web injections. This new campaign is widespread and particularly evasive, with historical indicators of compromise (IOCs) suggesting a possible connection to DanaBot — although we…

Accelerating security outcomes with a cloud-native SIEM

5 min read - As organizations modernize their IT infrastructure and increase adoption of cloud services, security teams face new challenges in terms of staffing, budgets and technologies. To keep pace, security programs must evolve to secure modern IT environments against fast-evolving threats with constrained resources. This will require rethinking traditional security strategies and focusing investments on capabilities like cloud security, AI-powered defense and skills development. The path forward calls on security teams to be agile, innovative and strategic amidst the changes in technology…

Topic updates

Get email updates and stay ahead of the latest threats to the security landscape, thought leadership and research.
Subscribe today