June 17, 2015 By Jaikumar Vijayan 2 min read

Log management for security-related purposes has become one of the biggest use cases for big data solutions, a survey by the SANS Institute shows.

Big Data Solutions

SANS earlier this year polled some 206 IT managers from medium and large organizations on their experience implementing and using big data technologies for addressing business issues. More than 40 percent of the respondents were from organizations with more than 10,000 employees, and about 52 percent had jobs that related directly to security. The results were compiled in the survey “Enabling Big Data by Removing Security and Compliance Barriers.”

The responses showed that log management applications are a big driver of big data solutions. More than 50 percent of the organizations that have deployed big data technologies are using them to manage log data gathered from the myriad devices and applications running on their networks. Storage and archiving were two other leading use cases for big data platforms, the survey indicated.

“Respondents with big data implementations are currently focused on use cases that are infrastructure-oriented,” SANS said in the report, which was sponsored by Cloudera. Examples include log management, data archival and operational data stores, the report noted.

Machine-generated logs can include data from Web applications, the servers that support these applications, mobile and network devices and other sources.

Invaluable Data

Security analysts consider such data invaluable in helping organizations monitor, troubleshoot and debug issues in network components, systems and applications. Many organizations use log data in forensic analysis applications to identify the root causes for problems on their networks. Banks and financial services companies rely heavily on log data for antifraud purposes and for ensuring that activities on their networks meet regulatory compliance requirements.

Large organizations can generate huge volumes of raw log data on a daily basis. In order to analyze the data and derive value from it, they must be able to ingest, organize, store and retrieve the information in as efficient a manner as possible. Big data solutions can help address some of these issues by offering organizations the scalability and analytics tools they need.

Spiking Interest

The responses showed that interest in big data solutions is spiking across industry sectors, with 27 percent claiming they already have a big data application in production, 28 percent saying they have one in development and another 28 percent planning to deploy a big data application in the next two years. Nearly three-quarters of organizations store personally identifiable information (PII) and sensitive business content in big data environments. As a result, they’re putting greater focus on access controls, encryption and similar security controls for protecting the data, the survey showed.

While log management for security purposes appears to be the most popular use case for the technology, different industries have different use cases for big data solutions, the SANS survey showed. For government organizations, some typical uses cases included fraud detection, compliance and regulatory analysis and climate analysis and weather prediction. Meanwhile, compliance and regulatory analysis was a major use case for financial services firms, followed by risk analysis and management, fraud detection and security analytics and CRM and customer loyalty programs. Within the IT industry, the biggest use case was security analytics and log management.

Big data solutions are just one of the methods organizations can employ to balance the delicate scales of security and compliance. Using analytics to simplify log management — among its many other applications — streamlines security protocols, organizes information and makes it easier for businesses to manage their risk and compliance.

More from

Regulatory harmonization in OT-critical infrastructure faces hurdles

3 min read - In an effort to enhance cyber resilience across critical infrastructure, the Office of the National Cyber Director (ONCD) has recently released a summary of feedback from its 2023 Cybersecurity Regulatory Harmonization Request for Information (RFI). The responses reveal major concerns from critical infrastructure industries related to operational technology (OT), such as energy, transport and manufacturing. Their worries include the current fragmented regulatory landscape and difficulty adapting to new cyber regulations. The frustration appears to be unanimous. Meanwhile, the magnitude of…

Generative AI security requires a solid framework

4 min read - How many companies intentionally refuse to use AI to get their work done faster and more efficiently? Probably none: the advantages of AI are too great to deny.The benefits AI models offer to organizations are undeniable, especially for optimizing critical operations and outputs. However, generative AI also comes with risk. According to the IBM Institute for Business Value, 96% of executives say adopting generative AI makes a security breach likely in their organization within the next three years.CISA Director Jen…

Q&A with Valentina Palmiotti, aka chompie

4 min read - The Pwn2Own computer hacking contest has been around since 2007, and during that time, there has never been a female to score a full win — until now.This milestone was reached at Pwn2Own 2024 in Vancouver, where two women, Valentina Palmiotti and Emma Kirkpatrick, each secured full wins by exploiting kernel vulnerabilities in Microsoft Windows 11. Prior to this year, only Amy Burnett and Alisa Esage had competed in the contest's 17-year history, with Esage achieving a partial win in…

Topic updates

Get email updates and stay ahead of the latest threats to the security landscape, thought leadership and research.
Subscribe today