Event Analysis Training -– An aggressive active worm analysis that isn’t Conficker

Recently, I saw a spike in “compliance” violations in the logs of one of the large research deployments of Tenable’s network and log monitoring products. At first glance, a web server appeared to be sharing content that matched our passive adult media data rules. On further analysis, we discovered that this was actually a worm infection. This blog demonstrates many of the techniques to use network analysis to identify the worm’s activity.

Initial Detection on port 25109

The Passive Vulnerability Scanner (PVS) focuses on finding information about hosts on the network by identifying their vulnerabilities. As it sees new traffic, its database of known systems and their vulnerabilities is updated and sent to the Security Center.

Some of the PVS’s rules are set in “real-time” mode and will generate a syslog entry when the PVS encounters events such as new vulnerabilities or in this case, compliance violations. In this particular case, the logs were sent to the Log Correlation Engine (LCE) and from February 16th through the 19th, had this type of consistent activity:

This particular research site has many university students and occasionally a student server will get compromised and be used to host movies, music and other types of media that has been infected with viruses.

Looking at the actual passively sniffed vulnerability (shown below) it could be seen that the web server was hosted on port 25109.

Hosting web servers on ports other than 80 or 443 is a common way to share content that might not normally be blocked by a firewall or monitored by security tools.

Activity Profile

Now that we know a host is sharing some sort of pornographic content on port 25109, the next step is to profile the activity. This particular environment has the Tenable Network Monitor which is an LCE agent that sniffs packets on a span port and logs network session information.  Snort is also running with the Emerging Threats signature database.

For the last few days, the overall activity profile for the host in question and port 25109 looked like this (click for a high-res image):

The blue area is the past weekend activity for Saturday and Sunday. I’ve indicated where some backdoor events (these are normalized logs from Snort) as well as the PVS’s indication of a new open port occurrence.

We could conclude that sometime on Monday, a compromise took place and then port 25109 was opened as a new service. Once the service is open, many different hosts on the Internet started to connect to it and, at the time of the screen shot, the activity still seemed to be ongoing.

All of the above traffic was related to port 25109. There could be a lot more activity occurring on other ports. Below is the screen shot of the same host’s time period but without the port 25109 filter (click for a full screen shot):

There was a little bit of activity before the initial compromise, but most of the continuous activity and constant communications occurred from Monday onward. There were a few more “never before seen” events on ports other than 25109 as well. These events indicated that the IP address in question had never been attacked like this in the past.

Traffic Analysis

How can we pull apart all of these logs and get a sense of what is occurring? Let’s start by comparing inbound versus outbound traffic.

The Security Center has a tool that can graph all matching events over time based on their direction. In the case of our worm outbreak, the following graph was shown:

There is slightly more inbound traffic than outbound traffic. The traffic loads are also linked. If you look at the spikes and valleys, they correspond almost identically. This means that whenever this host increased its inbound traffic, it also increased its outbound traffic.

This could mean that the compromised server was part of a network of nodes that communicated with each other. This is unexpected since typically, when you have a worm or server controlled by a remote “command and control” node, there is much more scanning and probing traffic than actual command traffic. Based on other worms I had looked at under the Security Center and the Log Correlation Engine, I expected to see a huge amount of outbound connections and only a few inbound connections.

What ports are being used here? The Security Center also has a tool to quickly sort this information and perform a port summary. Below is an example screen shot:

That was my second surprise in this analysis. I was expecting to see a lot of outbound port 445 sessions. Port 445 is a port used by the Conficker worm to infect Windows servers that are still vulnerable to MS08-67.

However, in this case we saw traffic from port 444. Port 444 is commonly known as the Simple Network Paging Protocol (SNPP) service. Sniffing port 444 traffic showed it to be nothing like the paging protocol. Instead, when we graphed out the port 444 activity over the same time period for this infected host, we saw the following:

Reading this from left to right, we can see that the PVS identified new Internet activity for this host on port 444. This was the first time the IP address in question had browsed on port 444. After that, there was a wide variety of almost continuous network sessions of various lengths and bandwidth. The largest percentage of logged connections was “TNM-TCP_Session_Timedout” events. These events indicate TCP sessions that started but didn’t complete. These logs show that our host was involved with almost 60,000 attempted connections over a two day period that did not complete.

Port 25 (SMTP) was also on the list of ports (shown below). This had a similar profile of activity starting shortly after being compromised on Monday. Sniffing this traffic showed that the host was repeatedly sending rejected email and was likely being used to send SPAM.

Although this is conjecture, you can see an initial spike of TCP sessions that taper off within a day. It’s very possible that the content of the SPAM message was fingerprinted by anti-spam vendors and the email messaging attempts were immediately denied.

I thought it would be interesting to compare the amount of traffic and destination for both port 25 and port 25109. The Security Center has a tool that will use the source and destination Class A IP address of each matching event and generate a graph. This is an easy way to see where and how much traffic is occurring. Below are two screen shots for port 25 and port 25091:

Port 25	Port 25109

The X and Y axes are not shown (which would give away our general location). What can we conclude from these graphs? The large circles indicate that there have been more events on port 25109 than on port 25. In addition, port 25 traffic seems to be limited to just a few class A address spaces and not as many as the ones used by port 25109. The large grey circle in the port 25 graph could indicate a Class A address space that was targeted or was particularly successful at sending spam email.

The Compromise

What do the compromise IDS events look like? Consider these sanitized logs:

<158>snort[11035]: [1:2001099:6] ET EXPLOIT Attempt to execute VBScript code [Classification: Misc Attack] [Priority: 2]: {TCP} xxx.yyy.135.66:80 -> aaa.bbb.72.123:1822

<158>snort[11035]: [1:2008803:1] ET CURRENT_EVENTS Possible Downaup/Conficker-A Infection Checking Geographical Location [Classification: A Network Trojan was detected] [Priority: 1]: {TCP} aaa.bbb.72.123:4477 -> xxx.yyy.69.70:80

<158>snort[11035]: [1:2008335:3] ET TROJAN Beizhu/Womble/Vipdataend Controller Keepalive [Classification: A Network Trojan was detected] [Priority: 1]: {TCP} xxx.yyy.121.146:15648 -> aaa.bbb.72.123:25109

<158>snort[11035[1:2001689:6] ET WORM Potential MySQL bot scanning for SQL server [Classification: A Network Trojan was detected] [Priority: 1]: {TCP} aaa.bbb.72.123:1151 -> xxx.yyy.59.186:3306

What we have here is a set of exploit steps. VB Script is downloaded, a potential geo-location attempt is launched by a worm and then we have some potential Trojan and scanning activity. The first two alerts occurred within a minute or so of each other. The other scanning alerts started about an hour later.

End Game

Although fun to watch, the worm had to go. After making the appropriate contacts, the host was turned off as shown below (click for hi-res):

It is easy to see the complete drop off in activity across all log sources. The two spikes of lone “SnortET-Trojan_Activity” events were UDP based worm probes from external hosts.

For More Information

If this type of analysis is of interest to you, you may want to review all of the blog entries in our “Event Analysis Training” category. There are many example articles including working with blacklisted IP addresses and an example Windows NT 4 system being infected. There are also many example demo videos of using Tenable solutions to look at anomalies, network traffic and intrusion detection logs. If you are interested in learning more about the Security Center or Log Correlation Engine, please contact us at [email protected].