Full Log Aggregation, Storage and Search

Tenable has released version 3.2 of the Log Correlation Engine (LCE) which includes the ability to store, compress and search any log that is sent to it. This functionality is available to all current LCE customers as a point release upgrade. It also builds upon the existing log normalization, correlation, user tracking and anomaly detection that were already available in prior versions.

Click on the below image for a demonstration of the LCE performing full log searches from within the Security Center:

High Speed Ad Hoc Searches

The LCE can be used to perform a search for any type of ASCII log. Searches can be made with Boolean logic and limited to specific date ranges. There are an infinite number of searches that can be performed, but some of the ones we’ve seen our customers use in early product testing include:

• Searching DNS query records for destination sites that indicate malware or virus infections.
• Seeing which network users visit sites such as Twitter, CNN or YouTube gathered from web proxy logs.
• Tracking down known Ethernet (MAC) addresses in switch, DHCP and other types of logs.
• Generically looking for errors, system crash and reboot messages.
• Finding certain types of Windows event records that indicate system issues.

All search results are saved in a compressed format along with a checksum so that they can be used as forensic evidence. Previous searches can also be re-launched against the latest logs.

Distributed Architecture

The Security Center can manage multiple LCE instances. From a user’s point of view, searches occur across all LCEs that they have access to. If they wish to narrow their search down to just one instance, they can choose to do that.

Each LCE has very high performance for gathering, compressing and searching logs. Tenable has achieved 20:1 compression ratios with some of our evaluation customers. When multiple LCEs are used together, the distributed query is also much faster than performing a similar query against just one LCE. For example, querying three LCE instances with similar logging loads was more than two times faster than one LCE.

Each LCE can use a local disk store or a mounted file system from a remote NAS or SAN. The Security Center can show the disk space usage of each LCE and also predict and alert when it will run out of disk space. Since the LCE does not make use of any third party databases, expanding your logging infrastructure is as easy as procuring a new LCE license and setting up a new server for your logs to be sent to.

Fulfilling Compliance Requirements

Full log aggregation, storage and search are all requirements of many compliance regulations such as PCI and FISMA. Organizations that are already using the Security Center and Nessus to fulfill their vulnerability and configuration auditing requirements for these regulations, can now take advantage of LCE’s ability to manage and search logs.

Many organizations are also subject to mandatory breach disclosure laws. Having direct access to raw logs, correlated events, configurations and vulnerabilities can help incident responders make immediate and correct decisions during a breach. This can result in not only limiting an ongoing breach, but also minimizing the chance of over-reporting or under-reporting the extent of a system compromise.

Obtaining the Log Correlation Engine

Tenable has worked with many customers that have been able to deploy a Security Center, Nessus and Log Correlation Engine solution as a replacement to multiple existing products. The combination of features offered by this solution has often allowed customers the opportunity to replace an array of vulnerability scanning, log analysis, configuration auditing, patch auditing and correlation tools. Tenable customers have also saved time and energy training their staff in one product solution and minimized the number of servers and appliances required to operate their auditing and monitoring infrastructure.

Any Security Center customer can upgrade their solution to full log analysis and correlation with a Log Correlation Engine. Each LCE is available in two options. A smaller version can analyze up to 15 million normalized events and a larger version can track close to 1 billion normalized events. Both versions can make full use of the local disk drive or network storage for full log searching and aggregation. To learn more about Tenable’s log analysis and storage products, please contact us at [email protected] and also consider watching one of our log analysis demonstration videos.