• Surviving a 20000+ node botnet Attack

    My web server has been under attack since early this month.  This is a dedicated server that I have leased for years.  It only hosts a couple of sites for me, my family and a few select friends.  Nothing of any real importance or sensitivity exists on it.  Why this insignificant little server attracted the attention of someone who has access a 20,000+ node, worldwide bonnet is beyond me.

    It started when I noticed that sites weren’t loading.  I shelled into the box and found the load hovering around 30+.  ps and top showed that apache was the culprit.  I combed through some logs and found that my wife’s site, messymissy.net, was being hammered.  Hundreds of POST requests per second to her index page.  I tcpdumped some of it and found that it was garbage or encrypted payloads destined for gryphn.com.  She has owned gryphn.com for almost 10 years and has it parked on top of messymissy.net.

    We unparked the domain and removed the DNS zone file and apache started working again.

    A couple of hours later we noticed that nothing on our server was resolving.  I shelled back in and found that DNS was now being hammered with queries to cached zone files for gryphn.com (which didn’t exist).  This log excerpt represents a tenth of a second worth of traffic.

    Feb  3 20:35:55 host named[3235]: client 103.8.44.8#23376: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 196.43.54.190#13041: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 193.2.1.102#39491: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.4.108#59071: query (cache) ‘grYPhN.cOM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.156.227.34#44924: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 89.95.242.180#56873: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 213.228.58.145#5210: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.159.76#44010: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.189.16#47278: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 194.90.2.4#63342: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 202.216.229.12#25343: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.208.3.18#34990: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.208.3.17#48741: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 61.153.81.123#30836: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 68.105.29.237#30849: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.134.4#28981: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 83.206.226.34#10582: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 110.164.252.215#39831: query (cache) ‘gryphn.com/NS/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 196.43.54.190#17049: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 208.69.32.21#36506: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 173.194.96.19#58355: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.156.227.34#38061: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 80.10.201.97#21826: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 164.124.101.49#16876: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.16.215#54383: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.18.35.114#2426: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.4.108#29276: query (cache) ‘grYPhN.cOM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.178.16#54930: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 193.2.1.102#6891: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.1.173#39050: query (cache) ‘GryPhn.coM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 80.10.201.33#27523: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.151.75#65393: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.20.253.11#53176: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 210.94.72.122#58224: query (cache) ‘gryphn.com/A/IN’ denied

     I logged into my DNS provider and enabled the use of their DNS servers.  We awaiting propagation of the new authoritative name servers and load returned to normal.

    Immediately following that we started receiving distributed brute force login attacks to multiple email accounts (that don’t exist) associated with multiple domains that we host.  I configured my firewall scripts to monitor for this sort of thing and block them.  As the firewall block list grew, the amount of invalid login attempt notifications shrunk.  Eventually a large part of the botnet was being blocked by my firewall.

    I guess they still had some nodes that weren’t blocked yet (and some fight left in them), because the most recent activity involves distributed brute force login attempts against WordPress sites.  I added a mod_security signature to catch it and modified my firewall scripts to block IPs that trigger the rule too many times.

    It’s a really fun cat and mouse game of changing attack methods on a massive scale (world-wide bonnet of 20,000+ zombies).  I’m working on scripts that will mine my logs for multiple block events and send automated x-ARF notifications to abuse@contacts for the zombies.

    I have no idea what it is they are after, but I’m having fun playing.

    If you are responsible, use the contact form on my site to send me an idea of what it is you want.  I won’t give it you, but the suspense is killing me.  :)

  • tcp/2550 and the Chinese

    While investigating an unrelated issue and digging through firewall logs I noticed a decent amount of traffic destined for tcp/2550 on one of my work servers.

    The traffic mostly (82 of the 84 events today) originates from sequential IPs out of China. This immediately raises alarms with me.

    [Read More...]
  • Hey Mac Users… The Honeymoon is Over.

    I know, its sad.  I too am a die hard mac user.

    Today alone I have received 4 copies of an email with the subject line “2 Populaar Myths About Female Orgasms -  How to Become an Irresistible Lover” containing an attachment named “Preview.app Document”.

    I haven’t had a chance to analyse the .app yet, but I think its safe to assume that its malware of some sort.

    The good news is that OS X is still built well.  If I double click it thinking its a document its going to tell me “Hey stupid!  This is an app that was downloaded from the Internet.  Are you sure you want to run it?”.  Maybe not in those exact words.  At that point if I say – “I thought I was opening an document, but sure, lets run this app-like-document” – then I deserve to be infected.

    For all the detail oriented folks here are the headers (bold are items changed to protect my info):

    Return-path: <efflrescent@aperfectmix.com>
    Envelope-to: MY_ADDRESS
    Delivery-date: Fri, 01 May 2009 09:39:27 -0400
    Received: from [87.18.181.177] (helo=ksecb.telecomitalia.it)
    by myserver.mydomain.com with smtp (MyMail Dameon)
    (envelope-from <efflrescent@aperfectmix.com>)
    id 1LzsxZ-0000Ib-JG
    for MY_ADDRESS; Fri, 01 May 2009 09:39:27 -0400
    Message-ID: <49FAF79E.9745295@aperfectmix.com>
    Date: Fri, 01 May 2009 13:39:25 -0100
    From: Chesner <efflrescent@aperfectmix.com>
    MIME-Version: 1.0
    To: MY_ADDRESS
    Subject: 2 Populaar Myths About Female Orgasms -  How to Become an Irresistible Lover
    Content-Type: multipart/mixed;
    boundary=”————32D524EA4E2E67F07C94899F”
    X-Spam-Status: No, score=3.8
    X-Spam-Score: 38
    X-Spam-Bar: +++
    X-Spam-Flag: NO

    The body contains no data.

  • VM Escape

    Whenever anyone speaks of virtual machine security the absolute worst case scenario is the dreaded “VM Escape”.  That is the ability of a malicious user to escape a virtual machines encapsulation and reach the host (or hypervisor).  This class of attack could potentially expose all other virtual machines running on this host.

    In the VM world this type of vulnerability is an absolute worst case, but are very rare.

    On April 10th CVE-2009-1244 was released stating that a number of VMWare products are vulnerable to VM escapes.

    You should patch as soon as possible if you are running:

    • VMware Workstation 6.5.1 and earlier
    • VMware Player 2.5.1 and earlier
    • VMware ACE 2.5.1 and earlier
    • VMware Server 1.x before 1.0.9 build 156507
    • VMware Server 2.x before 2.0.1 build 156745
    • VMware Fusion before 2.0.4 build 159196
    • VMware ESXi 3.5
    • VMware ESX 3.0.2, 3.0.3, and 3.5

    Per the CVE this vulnerability:

    allows guest OS users to execute arbitrary code on the host OS via unknown vectors, a different vulnerability than CVE-2008-4916.

    This also validates why all of the best practice documents recommend that VMs of different sensitivity levels be run on physically separate hosts and/or clusters.

    SANs Internet Storm Center reports that an exploit is available ‘in the wild’ for a fee.  They also provide a link to the following video of someone allegedly leveraging this exploit.

  • Mining Ports for Malware

    I recently wrote a script that runs croned and port scans all of our servers daily.  It saves the output and diffs it compared to the previous days and emails me as new ports open up.

    I think this will be a good way to detect new services and potential malware infection, but what about machines that are already infected?

    To fix that I wrote in a function that parses the output for known malware ports.  The only problem is that I cant find a definitive list of known malware ports.  Does anyone know of such a resource?

  • Loaded C:\WINNT\system32\KERNEL32.dll differs from file image

    I have recently been updating my Windows Forensics First Responder script and have noticed a number of servers reporting the following when using Sysinternals/Microsoft’s listdlls.exe.

    *** Loaded C:\WINNT\system32\KERNEL32.dll differs from file image:
    *** File timestamp:         Wed Apr 18 12:25:36 2007
    *** Loaded image timestamp: Wed Apr 18 12:25:37 2007
    *** 0x77e40000  0×102000  5.02.3790.4062  C:\WINNT\system32\KERNEL32.dll

    Now I can think of lots of malicious reasons why this would be.  In fact I recently wrote on one of these reasons.   But I cant think of any legitimate reasons.

    I’m not one to jump to conclusions without having evaluated all possibilities but my research is turning up almost nothing.

    Can anyone think of a legitimate reason why windows would load kernel32.dll and then something alter it as its going into memory?

    Thanks guys.

  • Why Won’t Dell Stop Sucking?!

    For some reason people keep buying Dells.

    I remember a couple of years ago all the small form factor optiplex’s I had suffered from a bad cap on the motherboard.  Eventually all of them just die.

    My whole team at work have the same model workstation and the PSU went on each of them, one by one.

    I have a service tag – the “serial number” unique to each computer – and type it into their site looking for drivers.  You would think, being that this tag is unique, that they could look up your computer and give you your network card drivers, your video driver etc.  NO!  Instead they give you the choice to download every driver for every chipset that was ever used on that given model.  Why do I have this service tag?!  Why don’t I just type in the model?!  Its the same results!

    After all that people still buy these pieces of crap.  They never even question why that is.

  • A Very Righteous Hack

    A roadside traffic sign in Austin, Texas was hacked into so that it displayed a message warning passing motorists of zombies ahead. Police are investigating the incident, and if they are caught, the perpetrators could face misdemeanor road sign tampering charges.  The vandals broke a lock on the sign and then managed to gain access to the computer that controls its readout because it was using a default password.  They also changed the password, so city employees had to wait for the manufacturer to reset the password before the sign could be changed.  A city spokesperson acknowledged that while “the sign’s content was humorous, … the act of changing it wasn’t.”

    http://www.dallasnews.com/sharedcontent/dws/news/localnews/transportation/stories/013009dnmetzombies.1595f453.html

    I have an issue of 2600 magazine from about 5 years ago that contains that default password.  I had always thought it would be funny if did something like this.  They even changed the default password.  How perfect.

    DISCLAIMER: I do no endorse the “hacking” of morons who don’t change default passwords.

  • Take THAT IE Fan Boy

    Bruce Schneier just posted an interesting article on his blog entitled “Interview with an Adware Developer”.

    This article reinforces many of the things I have been telling people for a very long time, but for whatever reason never sinks in.

    [Read More...]
  • More on Heartland

    Many experts continue to speculate on why it took so long for Heartland to identify and disclose the breach. According to the Storefront Backtalk report, the payment processor revealed the breach was first discovered in late October or early November, whereas previous statements indicated that it was only in the fall. The company has had two outside forensics teams and the Secret Service working on the problem for more than two months, and yet the “sniffer” software used to collect the data was located only last week.

    [Read More...]
%d bloggers like this: