Any woodworker will tell you that finding the right piece of wood, chopping it into perfectly sized chunks and then crafting those chunks into useful items is a complicated process. If a woodworker is making a set of chairs, for example, custom work will be needed for every piece because every tree is different. Incredible skill is needed to ensure consistency and quality craftsmanship.

Information security monitoring works the same way, more or less. Becoming a JavaScript Object Notation (JSON) formatter — and packing up your data in a new way — can reduce or eliminate the need for custom work. Being a JSON formatter can also save your security team precious time.

Logging Inefficiencies

Getting a non-standard log source into your security monitoring tool typically follows a set process. First, you find your non-standard log source and figure out how it logs and how to send those logs across your network. Next, you get those logs sending to your security monitoring tool. (Chances are they’re going via Syslog, so we’ll stick with this assumption for simplicity.) Last, you figure out how to parse out arbitrarily worded and assembled log fields using regular expressions (regex). (Depending on log complexity, this can take a while.)

There’s a massive headache here because of the inefficiency the discrepancies between different custom log sources introduce. The log’s format is essentially arbitrary and non-standardized, created at the mercurial whims of individual developers. Due to this, each time you write a custom parser using regex, it’s going to be a different and potentially substantial piece of work that will require dedicated effort from security integration experts.

JSON Fundamentals

Fortunately, there’s another way. JSON is self-describing, easy to understand and language independent.

  • JSON: This is a lightweight data interchange format that uses human-readable text to transmit data objects consisting of attribute-value pairs. Attribute-value pairs look something like this: {Date: 10/10/2018, DayofTheWeek: Monday, RequiredFood: Lasagna}. In other words? JSON is a simple, consistent syntax for exchanging and storing data.
  • JavaScript Object: This is an abstract data type composed of a collection of pairs so that each possible key appears no more than once in the collection.
  • Extensible Markup Language (XML): This is an HTML-like language that defines a set of rules for encoding documents in a format that is both human- and machine- readable. It’s a comparator to JSON in some ways.

Since the JSON format is text-only, it can easily be sent to and from a server and used as a data format by any programming language. JavaScript can convert JSON format into native JavaScript objects, which makes it easy to use JSON data like any other JavaScript object. JSON is a low-overhead alternative to XML — as JSON does not use end tags (like with HTML, these are the (end/) bits that close off elements) — which makes it shorter and quicker to read and write.

The case for JSON seems pretty compelling. However, its greatest asset (i.e., its flexible nature) also makes it situationally less suitable than XML for transferring data between separate systems or storing data that will be read by third parties. This is because it lacks some key features — in particular, schema support — which can be used to verify each piece of item content in a document. This can cause issues when moving data between systems.

The Need for Simpler Logging

The growing number of devices and application types being implemented in enterprise networks are making it increasingly difficult for log parsers to keep up with the different log formats and data values that can arise — especially from niche or proprietary custom software. Because of this, manual configuration is often needed to enable the parsing and normalizing (consistent value extraction) of logs with uncommon attributes.

When custom configuration is needed, the log’s syntax can have a significant effect on the effort required to work with them. This can include:

  • Attempting to understand the meaning of unclear values;
  • Writing countless, complex regex strings;
  • Finding a consistent anchor point — so regex remains valid with different log message types;
  • Managing the extra burden on system resources if regex is not perfectly written.

Putting JSON to Work

JSON dramatically reduces the effort needed to design and parse logs, both concerning configuration and computational resources. The key value pair (KVP) format of JSON makes it easier for humans and machines to understand what each value in the logs actually means. The JSON field names can also act as a stable anchor for writing extraction expressions. This feature reduces the likelihood of expensive or unreliable strings.

IBM QRadar is a security information and event management (SIEM) system that has a built-in collection of log parsers. In IBM QRadar, for example, it’s simple to extract log event properties created with a JSON formatter using JSON expressions.

To capture the value, one simply types: /”<field-name>”. The IBM QRadar GUI will highlight what your expression is capturing in the workspace on the right. In this case, we are capturing the sender address, so the JSON expression will be: /”SenderAddress”.

We have only scratched the surface of potential use cases of JSON and the power of becoming a JSON formatter. There are multiple ways to tackle logging outside the standard, arbitrary, one-size-fits-all format regex extraction paradigm. IBM QRadar makes it easy to take this way of working with custom log sources to the next level. Because of this, it’s worth considering this approach the next time you plan an integration.

More from Intelligence & Analytics

2022 Industry Threat Recap: Manufacturing

It seems like yesterday that industries were fumbling to understand the threats posed by post-pandemic economic and technological changes. While every disruption provides opportunities for positive change, it's hard to ignore the impact that global supply chains, rising labor costs, digital currency and environmental regulations have had on commerce worldwide. Many sectors are starting to see the light at the end of the tunnel. But 2022 has shown us that manufacturing still faces some dark clouds ahead when combatting persistent…

Cybersecurity in the Next-Generation Space Age, Pt. 3: Securing the New Space

View Part 1, Introduction to New Space, and Part 2, Cybersecurity Threats in New Space, in this series. As we see in the previous article of this series discussing the cybersecurity threats in the New Space, space technology is advancing at an unprecedented rate — with new technologies being launched into orbit at an increasingly rapid pace. The need to ensure the security and safety of these technologies has never been more pressing. So, let’s discover a range of measures…

Backdoor Deployment and Ransomware: Top Threats Identified in X-Force Threat Intelligence Index 2023

Deployment of backdoors was the number one action on objective taken by threat actors last year, according to the 2023 IBM Security X-Force Threat Intelligence Index — a comprehensive analysis of our research data collected throughout the year. Backdoor access is now among the hottest commodities on the dark web and can sell for thousands of dollars, compared to credit card data — which can go for as low as $10. On the dark web — a veritable eBay for…

The 13 Costliest Cyberattacks of 2022: Looking Back

2022 has shaped up to be a pricey year for victims of cyberattacks. Cyberattacks continue to target critical infrastructures such as health systems, small government agencies and educational institutions. Ransomware remains a popular attack method for large and small targets alike. While organizations may choose not to disclose the costs associated with a cyberattack, the loss of consumer trust will always be a risk after any significant attack. Let’s look at the 13 costliest cyberattacks of the past year and…