The Nessus Port Scanning Engine: An Inside Look

Port Scanning Never Dies

While information security threats constantly evolve from client-side attacks to web application vulnerabilities, there is one activity that is always effective: port scanning. Determining if a port is open or closed is a critical step in the discovery process associated with successfully attacking systems. For example, if port 80 or 443 is not open, it is likely there will not be a public web site associated with that system. Of course, this leads into service identification, which detects web servers listening on non-standard ports. However, you must be able to test if a port is open in the first place before you can determine which service may be running. Therefore, port scanning maintains its position as a necessary practice, even when referencing client-side attacks that can turn the remote client systems into port scanners using JavaScript.

Given the importance of port scanning, I want to cover some of the features and functions of the various port scanners included in the Nessus vulnerability scanner. The Nessus port scanner system has three network-based port scanners:

• TCP Scanner - The TCP scanner sends sequence of packets to initiate a full TCP connect to the target hosts, completing the TCP three-way handshake each time. The TCP port scanner uses a balance of speed and accuracy while using logic to tune itself as the scan progresses. The TCP scanner does not operate on Windows and Mac OS due to operating system limitations, so Nessus initiates the SYN scanner on these systems instead. However, when Nessus is installed on Linux it will implement a full-connect scanner in user space (i.e., without requiring root-level privileges). Early versions of the scanner consisted of a couple of pages of C source code. Over time it has grown in features and complexity to handle many different situations and types of networks. The TCP scanner will dynamically estimate the RTT (Round Trip Time) and make multiple passes on unresponsive ports to determine if there was a problem during the initial attempt. The TCP scanner will also read banners for some services and place this information, along with the open ports, in the Nessus knowledge base where the service identification routine and plugins can find the list of open ports for each host.

  TCP Scanner Settings:
  

  
  • Automatic (normal)(default) - This option can help identify if a firewall (or BSD based IP stack, which limits RST packet rates) is located between the scanner and the target. It uses the scanner’s logic to automatically differentiate between a firewalled host and a host on a slow or unstable link that is dropping packets. For example, if 90% of the ports on a host are closed, but a RST packet was received on 10% of the open ports, it’s likely not a firewall. However, if most probes never receive a response, then it likely is a firewall.

  • Disabled (softer) - This option can improve both accuracy and performance when scanning systems on a slower link, as it removes the logic and timing within the firewall detection. As this scan considers that all ports should answer and that silence is a symptom of congestion, this scan should not be used against firewalled hosts.
  • Do not detect RST rate limitation (soft) - Disables the ability to monitor how often resets are sent (e.g., BSD optimization) and to determine if there is a limitation configured by a downstream network device. This option increases reliability on slow links; however a full scan of an OpenBSD system may take half an hour instead of a couple of minutes.
  • Ignore closed ports (aggressive) - During a port scan, the scanner sends SYN packets to ports which are known to send something back such as a SYN|ACK, a RST or, sometimes, an ICMP unreachable port. The problem is that some packet filters send an ICMP unreachables in return. For the SYN scanner this is OK, but for the TCP connect() scanner we can't distinguish it from a RST. Due to the ICMP rate limitation, this may cause the scanner to believe that "probes" got lost en route, which would artificially slow the scan down.
    This option may lead to “aggressive” timing and is not recommended for use on a production network. However, if a scan runs for over 40 minutes, this option will be turned on by Nessus automatically.

• SYN Scanner - The Nessus SYN scanner generates packets using low level libraries and is fully supported on Linux, Mac OS X and Windows. The SYN scanner behaves a bit differently and simplifies the process by sending packets and waiting for a response, but not initiating the full three-way handshake. It does not open sockets, but generates raw packets using low-level libraries. The SYN scanner has logic to compute the rate of packets to be sent and the amount of time to wait for a response. It will constantly recompute the optimal throughput as it scans periodically, so the calculations are happening several times throughout the scan to maximize the scan’s efficiency. The SYN scanner tends to be slower, but more reliable.

In the Nessus 4.4.1 release, the SYN scanner is improved and a new option has been added to allow the end user to control the rate of packets. An option has been added that allows for local limitation for both scanners or global timing limitation for the SYN scanners. This new option will be available in the nessud.conf configuration file. In previous versions, you could already limit the number of parallel connections (on the first policy configuration screen), that affects the number of parallel connections used by the TCP scanner (per scanner or globally). As the SYN scanner does not really open connections, this option was used to limit the throughput.

SYN Scanner Settings:

The same options that are available for the TCP scanner also apply to the SYN scanner.

• UDP Scanner - There’s not much you can do to make this one go faster (you can cheer it on, but that just helps pass the time) as the UDP scanner will generate UDP packets and send them to the target. Some operating systems respond with ICMP messages, others may crash and some just silently ignore the request. The big problem is that there is no reliable way to know if a port is open because a UDP-based listening service usually won't respond to a probe and also will not respond if the port is closed. Another problem is when ICMP messages are received from a target in response to the UDP probe. Certain operating systems respond slowly and rate limit the response to 1 packet per second for each response. This means a scan of all ports may take as long as 65,535 seconds to complete (approximately 19 hours).

• Netstat Port Scanner - Of course a more reliable way to enumerate open ports on a given host is to login to the system and execute a command that shows all open TCP and UDP ports. This feature is supported across all Windows, Linux, UNIX, and Mac OS platforms that implement the "netstat" command, provided the Nessus scanner has the credentials to login to the target. While this method is typically more reliable, sometimes you may wish to compare the netstat results with what is being reported to be open/closed across the network. You can check the box "Check open TCP ports found by local port enumerators" to enable both the netstat and the network-based port scanners.

Conclusion

Using the port scanners options in Nessus can be of great benefit in tuning your scans. Knowing which situations the SYN or TCP scanner are best utilized in will result in more accurate and faster vulnerability scans. You can refer to the Nessus product documentation for more information about the port scanner settings.