I recently wrote a script that runs croned and port scans all of our servers daily.  It saves the output and diffs it compared to the previous days and emails me as new ports open up.

I think this will be a good way to detect new services and potential malware infection, but what about machines that are already infected?

To fix that I wrote in a function that parses the output for known malware ports.  The only problem is that I cant find a definitive list of known malware ports.  Does anyone know of such a resource?

Arpwatch is an amazingly useful tool that promiscuously listens on a specified interface for arp broadcasts.  It takes what it learns and saves the the output in a database for later reference in the following format.

mac_address ip unix_date/time hostname

It will take any changes/additions and log them to /var/log/messages as well as optionally emailing them.

This functionality is useful for detecting

• Man-in-the-middle attacks
• Arp spoofing/poisoning
• Session hijacking attacks
• New hosts introduced onto your network

Set up and configuration is easy.  Just download and compile arpwatch from lbnl’s site, create an arpwatch user (unless you want it to run as root… which you don’t), create an empty arpwatch database (touch/home/arpwatch/arp.dat) and run it.

The command line arguments you run will differ depending on how your network is set up, so check out the man page to be safe. The following should work for most situations.

/usr/sbin/arpwatch -i eth0 -u arpwatch -f /home/arpwatch/arp.dat -n x.x.x.x/21 -e -

-i eth0 tells it to listen on /dev/eth0 only.  You can run multiple instances of arpwatch for each nic/network if you are multihomed.

-u arpwatch tell it to run as the user ‘arpwatch’ instead of root.

-f /home/arpwatch/arp.dat tells it to save the arp database in that file instead of the default location

-n x.x.x.x/21 tells it that an additional address range is in use on this interface.  If you have IPs outside of those defined on your monitor nic it will report them as bogon.

-e – tells it not to email you with every thing it discovers.  You will want to run it this way the first time to avoid flooding your mail box.

At work I am mapping out our network.  Instead of visio I am using omni graffle and very happy with the asthetics and ease of use.

I am setting it up so that it is separated into both logical (firewall security zone) and physical (rack number with list of computers that are inside it).  I am running into problems doing it this way, as I have already found racks that have machines that exist it multiple security zones.

Another problem I encountered is the one that brought me to the title of tonights post: We have a number of virtual machines that don’t really (physically)  live anywhere.  The OS may consist of file systems mounted from multiple SANs in multiple racks and being run from a hypter-visor that exists in yet another rack. So, what rack does that vm belong in?

How have you guys dealt with this soft of thing?