Security Metrics - Is This Network Getting Better?

Metrics that show risk are an excellent way to communicate security information to different people and groups within an organization. However, trend lines can hide a lot of details and nuances. This blog entry discusses an example network where a month’s worth of scan data is used to trend overall vulnerabilities, those that have been around longer than thirty days and correlating systems needing a reboot with residual security issues.

Trending Old Vulnerabilities

Consider the following graph that was rendered on the SecurityCenter dashboard. The top graph trends vulnerabilities detected with Nessus that were of “High” and “Medium” severity levels across a small network of a few hundred hosts. The middle graph plots only vulnerabilities that have been around for thirty days or more. Finally, the last trend line shows the total number of active hosts over a specified time period.

We can tell a lot from these types of graphs:

• Things are generally getting better. There are fewer high severity vulnerabilities from the previous month to the current month.
• There is an increasing trend of older vulnerabilities that is of great concern. However (not shown on these graphs), we did an IP summary and found the bulk of these issues came from two systems that were not covered by the patch management system.
• There were no significant changes in the number of hosts on the network.

Any time someone presents you with a trend line graph, you should ask how the data was obtained. There are many ways that bias can creep into vulnerability graphs. I recently recorded a vulnerability metrics webinar and discussed some issues such as:

• Instead of scanning with an updated set of vulnerability checks, an organization simply repeats the same audit over and over from day one. The trend line will surely go down if systems are patched, but will completely miss graphing “new” vulnerabilities during that time.
• Presenting only certain types of vulnerabilities can also be misleading. I’ve seen customers create graphs of missing Microsoft KBs and completely miss auditing third party applications such as web browsers, network management agents and more.

Tracking When Reboots Are Required

Have you ever had a conversation with your IT group and they swear that they have patched systems, yet Nessus still reports that systems are vulnerable? Very often, this is because machines require a reboot before the patch can fully be applied and Nessus does not consider the system patched if the patch is not in effect.

We have all waited for our Windows laptops to have patches installed on them during shutdown, but in enterprise networks, software automation may be used to silently install patches during the day. This process often leaves the Windows system in a state where several patches are installed and a reboot is all that is needed to make them effective. Unfortunately, for whatever reason, some users never turn their systems off. This often causes IT to “force” systems to reboot.

To see evidence of this, consider the following graphs that were also created with SecurityCenter:

The top line shows a trend of Nessus plugin #35453 that tracks when a computer requires a reboot to finish a software installation. On 7/18, there were six computers that were in this state. The second graph is of the same network and it shows the number of high severity vulnerabilities.

There is a correlation with detected high severity vulnerabilities and systems that need to be rebooted. Systems were being patched but not being rebooted, so they were still vulnerable. There is also a correlation between a drop in systems needing to be rebooted versus measured patches that indicates patches have been applied.

For More Information

Tenable has blogged several times about Security Metrics and how you can measure the effectiveness of your patch management system, calculate risk and much more. We also offer a variety of webinars including a recently recorded one about Vulnerability Metrics in which I discuss vulnerability lifecycles, risk scoring and trending. Tenable’s SecurityCenter product can produce these types of metrics and reports based on continuous scanning and real-time passive network monitoring. If you would like to learn more about SecurityCenter, please consider some of the live videos or contact our sales team.