magazine resources subscribe about advertising




 CD Home < Web Techniques < 2000 < August  

Outside Looking In

By John N. Stewart

The largest security team at the largest organization is still many orders of magnitude smaller than the population attempting to break in to the organization's network. Why? The Internet community is enormous and any one member interested in causing harm to your system can easily access the tools he or she needs to break in.

In most cases, would-be intruders use a number of small programs to probe your host computer for weaknesses. When malicious hackers find a hole, they use other tools to exploit the vulnerability and gain access to your network. To protect yourself from these black hats it's important to see your own system from the perspective of a person with the necessary tools. Only then can you effectively fortify your network from probing and potential attacks.

Host Probing

For this article, I put together a demonstration site configured with an out-of-box Solaris installation. I then used a port scanner to probe the system for available services that could potentially be exploited.

When probing a host machine like this, the goal is to find out as much information as possible about the machine. Using that information, black hats can further probe and even attack the remote machine. The most critical pieces of information found during a probe include the host's operating system and the available applications on the computer. With that information, the crackers can discern which vulnerabilities to exploit.

Whenever I came across a potentially dangerous hole on my demonstration system, I patched it and essentially created a countermeasure. With each probing tool and its respective countermeasures, the machine became more secure as it was more difficult for probing software to "see" it. The less information someone can get about your machine, the better.

Port Scanning

To learn which applications are potentially exploitable on a machine without having to log in to the machine, an intruder can use a port scanner. Port scanners step through all the network ports on a remote machine, attempting to establish a connection to each one. (They are similar to the auto-dialer software that was once popular for scanning all the phone numbers in a certain area code in search of connected computers.) The result of scanning is a brute-force report that lists which services are running and accessible to the machine doing the probing.

While there are many port scanning tools, Network SKANner (NSKAN) is a good one and it's free (see " Online"). It's a simple UNIX program that gives feedback on which ports it was able to connect to and, if possible, which protocols are related to those ports. The protocols often indicate that a potentially exploitable application is connected to that port. For example, Web servers (which use the HTTP protocol) are connected to port 80 on most computers. If NSKAN can establish a connection to port 80, you can generally assume that the HTTP protocol is related to that port on that machine.

OS Fingerprinting

The second step is to learn which operating system the remote machine is running, using a technique called OS fingerprinting. Because they are somewhat new, OS fingerprinting techniques are in flux even as you read this. It was once the case that, to determine the OS running on a remote machine, one could use a primitive tool like telnet. Connecting to another computer with this utility would yield basic information regarding the remote system's available services. From that information, you could infer which OS was running on it with some degree of accuracy.

In Example 1, I've connected via telnet to a random computer called somehost. After connecting, somehost sends me text that includes the words SunOS 5.6., which is another name for Solaris 2.6; now I know the machine is running Solaris. This guess may not be entirely accurate, because it's possible that the machine is running a telnet daemon that only makes it look like it's running Solaris 2.6. Because of this, the telnet technique is unreliable at best. Also, the remote machine may not even be running telnet services, so there's no guarantee that you'd even get this far if you're thinking about doing some snooping.

To take the guesswork out of fingerprinting, developers invented a new generation of tools. The new tools use creatively formed IP packets to test for behavioral patterns in a remote machine's IP stack. The software sends packets with particular characteristics that cause only certain operating systems to handle them in predictable ways. Each packet/response pair further limits which operating systems the remote machine could be, until the tool has no further ability to narrow the OS options and it reports back one or more choices.

The best tool for this type of OS fingerprinting is nmap. The May 2000 release of nmap incorporated multiple techniques used in the area of packet-based OS detection and now looks for particular signatures associated with operating systems that can be used to classify the type of remote machine. If the software responds with multiple OS choices, it will likely be the same base OS and different version numbers, as shown in Example 2.


The last step in securing your system from host probing, after port scanning and OS fingerprinting, is to learn which exploits exist for those combinations and which countermeasures can defend against them. Many applications on a given OS have known vulnerabilities.

For example, you can further explore port 25, the standard port for the Simple Mail Transport Protocol (SMTP), using the utility. By doing so, you'll find out whether the system uses the sendmail Mail Delivery Agent (MDA), and whether that agent is misconfigured. You can also determine if the agent can be used as an SMTP relay, and whether it's running sendmail 8.9.3. From there, it's possible to use the agent to send spam or even get access to the system.

Further checks with, from, might uncover that the phf (phone book) utility resides in the cgi-bin/ directory. This is a Web-based application that has well-known vulnerabilities and permits remote users to run commands on the Web server. In this case, it's accessible to anyone over the Web.

Determining which vulnerabilities exist and how to patch them is a matter of doing your research. One good place to start is with the SecurityFocus site. You'll find exploits organized by OS, and other information on common tools and code used by black hats. The exploits are provided there to help secure a site, not to break into it. However, these exploits, like any tool, can also be put to bad use.


To counter potential attacks, you can install IP Filter to mask some of the remote packet OS fingerprinting. The IP Filter utility is a kernel network module that intercepts and analyzes incoming and outgoing packets on a host and determines whether the packets are supposed to be received by or sent from the host. IP Filter also looks for poorly formed packets, illegal packet types, and packets from impossible networks—like the loopback network, for instance. It is, in some ways, a host based firewall package.

I set up IP Filter on my demonstration machine and configured it to IP Then I decided to block all packets originating from the ping utility. The configuration file modification to do this is in Example 3. The example is by no means complicated, but it illustrates an important point. Without support for ping packets, nmap can't effectively probe the system. With this simple step, I've blocked nmap from determining which OS my machine is running.

Another way to safeguard your machine is to turn off the inetd program. Standard UNIX installations often include inetd and cause it to be started automatically when you turn the machine on. The inetd program supports standard telnet and FTP servers, however, a secure, external server doesn't need it. If you disable inetd, you can ensure that both telnet and FTP capabilities are still there by installing ProFTPD (a public domain FTP server) and sshd (secure remote access software).

ProFTPD's strength is that the FTP daemon runs like a daemon—for example, it doesn't require another program running in the background. sshd's strength is that it uses encrypted sessions for remote management, whereas telnet transmits passwords in plain text.

Hackers frequently exploit applications through odd packet types sent to them in their native form. This introduces another scenario, called IP stack smashing. IP stack smashing is the practice of sending some type of IP packet to a host that causes the host's OS to either crash the network kernel module, or run a program encapsulated in the packets, or make a change through a buffer overflow. You can sidestep stack smashing by making some additions to your ipf.conf file (see Example 4).

This last line in the configuration file indicates that packets with any extra options set should be blocked and logged. Per the IP specification, IP packets have header data that include option fields for the packets themselves. For instance, applications can set the transmission priority of various packets by modifying the option fields. These option fields were originally meant for flexibility in a developing protocol. In practice, they remain unused and can be blocked without great side effects.


To protect your site against a growing community of black-hat hackers, you have to think like they do and see the same information. By following a few short steps, you can fortify what was once an insecure machine, making it secure from the inside.

Protecting your system is often about appearances. In the same way that a blinking car alarm light and a sticker on the window notifies potential thieves that the automobile is protected, a server with few available applications and an OS that can't be detected shows attackers that your site is secure. You're increasing the chances the attacker will move on. But if there's plenty of available data, the attacker will be that much more dedicated to breaking the site.

(Get the source code for this article here.)

John is the director of systems engineering and security for Digital Island. You can reach him at

Copyright © 2003 CMP Media LLC