Security analysts are often bogged down by the rate of false positives generated by security information and event management (SIEM) systems. Supervised machine learning, when applied to historical data to predict alert classification, has the potential to transform the security monitoring industry.

SIEM systems are used to identify potential attacks against a company’s network. These systems usually employ a combination of statistical and pattern-based anomaly detection. They sift through millions of event logs from a variety of security and networking devices and report malicious behavior. SIEM solutions such as QRadar enable security engineers to configure rules that describe patterns of traffic that may indicate an attack in progress. When these rules match, the SIEM produces an artifact called “alert or offense” for further review.

The Negative Effect of False Positives

An analyst typically takes 13–18 minutes to classify a system-generated alert as a false positive or a possible attack. Alerts that classify as escalations are reviewed by second- and third-level analysts and confirmed positives are handled by the incident response (IR) team.

Approximately 70 percent of alerts are ultimately labeled false positives, meaning top-level analysts often spend valuable time researching alerts that have little or no impact. The rapid increase in log sources takes a heavy toll on analysts. Companies are often forced to employ armies of security analysts, resulting in bloated security budgets with little return on security investment.

Slash IT Spending With Supervised Machine Learning

One way to curtail security spending is to employ machine learning techniques. Data scientists use these techniques to predict continuous variables or classify categorical variables by training machines to understand patterns in historical data.

Data scientists use classification algorithms called learners to train machines to understand complex relationships between the features and response variables. The learners include algorithms such as random forests, decision trees, gradient boosting methods, support vector machines, neural networks and more.

Once the training model is built, it is tested against the validation data to predict outcome and evaluate performance.

The selected model is then deployed to generate a prediction response supported by probability confidence levels. This process, in which the machine learns from the training data and evaluates the performance on the validation data, is called supervised machine learning.

Marching Toward the Cognitive Security Era

Supervised machine learning algorithms, when applied to historical alert data, can significantly improve classification accuracy and decrease research time for analysts. It can supplement analysts with additional data and insights to make better judgment calls. Though prediction models based on historical data can improve analyst productivity, they’ll never replace security analysts altogether.

Going forward, machine learning techniques will continue to push to the forefront of analytics and rule-based systems. With these specialized skills and building models to supplement human effort, companies are more likely to invest in people.

IBM Watson for Security is taking this challenge seriously and investing in predictive and cognitive technologies to build scalable systems for managed security services (MSS). IBM strongly believes these methods will significantly tighten classification accuracy, decrease false positive rates, boost analyst productivity and improve customer satisfaction.

Read the white paper: Cybersecurity in the cognitive era

More from Intelligence & Analytics

Email campaigns leverage updated DBatLoader to deliver RATs, stealers

11 min read - IBM X-Force has identified new capabilities in DBatLoader malware samples delivered in recent email campaigns, signaling a heightened risk of infection from commodity malware families associated with DBatLoader activity. X-Force has observed nearly two dozen email campaigns since late June leveraging the updated DBatLoader loader to deliver payloads such as Remcos, Warzone, Formbook, and AgentTesla. DBatLoader malware has been used since 2020 by cybercriminals to install commodity malware remote access Trojans (RATs) and infostealers, primarily via malicious spam (malspam). DBatLoader…

New Hive0117 phishing campaign imitates conscription summons to deliver DarkWatchman malware

8 min read - IBM X-Force uncovered a new phishing campaign likely conducted by Hive0117 delivering the fileless malware DarkWatchman, directed at individuals associated with major energy, finance, transport, and software security industries based in Russia, Kazakhstan, Latvia, and Estonia. DarkWatchman malware is capable of keylogging, collecting system information, and deploying secondary payloads. Imitating official correspondence from the Russian government in phishing emails aligns with previous Hive0117 campaigns delivering DarkWatchman malware, and shows a possible significant effort to induce a sense of urgency as…

X-Force releases detection & response framework for managed file transfer software

5 min read - How AI can help defenders scale detection guidance for enterprise software tools If we look back at mass exploitation events that shook the security industry like Log4j, Atlassian, and Microsoft Exchange when these solutions were actively being exploited by attackers, the exploits may have been associated with a different CVE, but the detection and response guidance being released by the various security vendors had many similarities (e.g., Log4shell vs. Log4j2 vs. MOVEit vs. Spring4Shell vs. Microsoft Exchange vs. ProxyShell vs.…

Unmasking hypnotized AI: The hidden risks of large language models

11 min read - The emergence of Large Language Models (LLMs) is redefining how cybersecurity teams and cybercriminals operate. As security teams leverage the capabilities of generative AI to bring more simplicity and speed into their operations, it's important we recognize that cybercriminals are seeking the same benefits. LLMs are a new type of attack surface poised to make certain types of attacks easier, more cost-effective, and even more persistent. In a bid to explore security risks posed by these innovations, we attempted to…