SIEM Event Normalization Makes Raw Data Relevant to Both Humans and Machines

A security information and event management (SIEM) system is an indispensable tool for any security operations center (SOC). It collects events from devices in your network infrastructure such as servers, cloud devices, firewalls and Wi-Fi access points to give operations professionals fine-grained visibility into activity on the network and help them spot anomalies that may signal a cyberattack.

In its raw form, this log data is almost impossible for a human to process, so advanced SIEM solutions conduct a process called event normalization to deliver a homogeneous view. Event normalization consists of breaking each field of a raw event into variables and combining them into views that are relevant to security administrators. This is a crucial step in the process of finding meaning in often isolated and heterogeneous events.

Visualize Your Network Activity

There are thousands of vendors and models of devices and software that an organization may want to monitor. It’s impossible for a SIEM to read raw events from all of them, let alone keep up with versions and new releases. Using correlation rules and tools such as a DSM editor, security administrators can translate raw data into a single, normalized stream, making it possible for the SIEM to present data from nearly any device or log source in a meaningful form. Event normalization enables administrators to detect anomalies even when data is streaming in from multiple locations.

For example, a brute-force attack consists of a series of authentication attempts against a system, either from a single IP or multiple addresses. Sorting through authentication logs one by one is a tedious task, but a SIEM solution can solve the problem using correlation rules. This enables administrators to see anomalies such as login attempts from suspicious locations, network scans and simultaneous authentication attempts by the same user from different locations. A SIEM can also monitor network traffic for unusual activity, such as large file downloads.

Behold the Power of Event Normalization

To give you a sense of the power of normalization, here’s an example of a raw log from a firewall:

<;;5>logver=54 dtime=1536072238 devid=FG74E83E17000037 devname=firewall-fort vd=External date=2018-09-04 time=14:43:58 slot=4 logid=0000000013 type=traffic subtype=forward level=notice srcip=10.10.10.200 srcport=44000 srcintf=”DMZ” dstip=172.217.15.206 dstport=443 dstintf=”External” poluuid=55555555-5b5b-5a5a-5c5c-5a5b5c5d5f55 sessionid=555555555 proto=6 action=close policyid=55 policytype=policy dstcountry=”United States” srccountry=”United States” trandisp=snat transip=Pub-IP-Address transport=44000 service=”tcp_1-65535″ duration=11 sentbyte=1699 rcvdbyte=6002 sentpkt=16 rcvdpkt=13 appcat=”unscanned”

Buried in this nearly unreadable stream is important information, including:

  • Hostname;
  • Date and time;
  • Source IP of the traffic;
  • Destination IP;
  • Source port;
  • Destination port;
  • Action taken by the firewall;
  • Source country;
  • Destination country;
  • Application discovered; and
  • Translated IP addresses.

Using correlation rules, we can extract these important details automatically into a report or chart that helps us visualize activity from many sources. The process of creating events consists of finding patterns in raw data, mapping it to known expressions, and assigning unique categories and identifiers. If the SIEM encounters an unknown log source or data type, we can use the editor to define an event and assign variables such as name, severity and facility.

Get the Most Out of Your SIEM Deployment

Good normalization practices are essential to maximizing the value of your SIEM. Tools such as DSM editors make it fast and easy for security administrators to define, test, organize and reuse events, thereby ensuring the maximum visibility into everything that takes place on the enterprise’s computing fabric. It turns steams of machine data into something humans can use.

Contributor'photo

Moises Monge

MSS SIEM Admin

Moises is a network security professional and an IBM MSS SIEM Administrator.