Everything you do in threat intelligence is about indicators or patterns. In a binary world, patterns are actually just how different indicators work together in the chain of a malicious event.

Working with threat intelligence for years now, I’ve often asked myself several fundamental cyberthreat intelligence questions:

  • What exactly is this attack and how can I identify it?
  • Is this attack description related to the technology I use?
  • Is the company I want to protect a possible target?
  • What would this attack look like in my environment?

While thinking about the right answers to these questions and creating your own attack model and hypothesis can be a lot of fun, extracting indicators from intelligence and enriching them is usually not. To speed up this task, I always prefer to write small scripts or code snippets to do these things for me, so I can focus on the fun part.

Extracting Different Types of Indicators From Cyberthreat Intelligence

All code examples below can be found on my public GitHub repository. In this blog, we will mainly look at code snippets used in indifetch.py, which is exactly what it sounds like: fetching indicators from a text or string. As a disclaimer, you should always review the regex used and not trust the code blindly. I always advise some cross-checking and keeping an eye out for a better regex or faster way to do a task — the following is just one way to do this.

Hashes

Hashes are fairly easy. They normally come in two flavors, md5 and SHA. A function that covers md5 could look like this:

def getMD5(text):
thisset = set()
md5_r = re.compile(r"([a-fA-F\d]{32})")
for item in md5_r.findall(text):
thisset.add(item)
return thisset

As you can see, we use the simple regex [a-fA-F\d]{32} to fetch the indicator out of a given text. The regex matches any character “a” to “f” and any number in a string of 32-character length. We use a Python set because items in a set are unique, eliminating duplicates right from the start.

Changing this function to cover SHA256 is an easy next step. Besides it being a totally different algorithm, the representation is the same but twice the length of the characters ([a-fA-F\d]{64}), using 64 characters instead of 32.

IP Addresses

Next up are IPs. IPv4, in particular, follows a pretty simple pattern: four numbers, none of which are higher than 255, separated by a dot.

def getIP(text):
IPlist = set()
ip = re.compile(r"\b(?:(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\b")
for item in ip.findall(text):
IPlist.add(item)
return IPlist

We use the same style of function, as I always try to reuse code if I can. One potential problem in this situation is that, sometimes, version numbers use the same style, which can confuse the code.

URLs and Domains

The last common indicators we will cover in a function are URLs and domains. Many reports will have a large set of URLs since most malware has to connect to a command-and-control (C&C) server or exfiltrate data. In addition, with proxies and firewalls, they are often one of the easier indicators to catch.

def getURL(text):
URLlist = set()
urls = re.compile(r'(?:[a-zA-Z]|[0-9]|[[email protected]&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+')
for item in urls.findall(text):
URLlist.add(item)
return URLlist

Since you are familiar with the style by now, the regex above worked well for me, but I have also used simpler representations in the past, such as:

http[a-zA-Z0-9./:]*

Speed Up Your Threat Analysis

Python is a great tool for scripting these little code snippets and speeding up threat analysis. Some scripts can even extract indicators and make an API call to resources like the X-Force Exchange to get the current scoring of the indicator, further speeding up the process.

One regex you may want to look into is for Common Vulnerabilities and Exposures (CVE) numbers. I tend to use something simple, such as:

CVE[^\w]*\d{4}[^\w]+\d{4,}

Note: Remember to always use set() in Python instead of lists to remove duplicates right from the start. This comes in handy especially when you automate API calls as part of the script.

There are many cyberthreat intelligence tools and platforms that can do this dismantling of information for you, but it can be extremely useful to understand the magic behind the process before relying on a tool.

More from Intelligence & Analytics

2022 Industry Threat Recap: Finance and Insurance

The finance and insurance sector proved a top target for cybersecurity threats in 2022. The IBM Security X-Force Threat Intelligence Index 2023 found this sector ranked as the second most attacked, with 18.9% of X-Force incident response cases. If, as Shakespeare tells us, past is prologue, this sector will likely remain a target in 2023. Finance and insurance ranked as the most attacked sector from 2016 to 2020, with the manufacturing sector the most attacked in 2021 and 2022. What…

And Stay Out! Blocking Backdoor Break-Ins

Backdoor access was the most common threat vector in 2022. According to the 2023 IBM Security X-Force Threat Intelligence Index, 21% of incidents saw the use of backdoors, outpacing perennial compromise favorite ransomware, which came in at just 17%. The good news? In 67% of backdoor attacks, defenders were able to disrupt attacker efforts and lock digital doorways before ransomware payloads were deployed. The not-so-great news? With backdoor access now available at a bargain price on the dark web, businesses…

Cyber Storm Predicted at the 2023 World Economic Forum

According to the Global Cybersecurity Outlook 2023, 93% of cybersecurity leaders and 86% of business leaders think a far-reaching, catastrophic cyber event is at least somewhat likely in the next two years. Additionally, 43% of organizational leaders think it is likely that a cyberattack will affect their organization severely in the next two years. With cybersecurity concerns on everyone’s mind, the topic received top billing at the recent World Economic Forum’s Annual Meeting 2023 in Davos, Switzerland. At the meeting, Matthew…

2022 Industry Threat Recap: Manufacturing

It seems like yesterday that industries were fumbling to understand the threats posed by post-pandemic economic and technological changes. While every disruption provides opportunities for positive change, it's hard to ignore the impact that global supply chains, rising labor costs, digital currency and environmental regulations have had on commerce worldwide. Many sectors are starting to see the light at the end of the tunnel. But 2022 has shown us that manufacturing still faces some dark clouds ahead when combatting persistent…