• Surviving a 20000+ node botnet Attack

    My web server has been under attack since early this month.  This is a dedicated server that I have leased for years.  It only hosts a couple of sites for me, my family and a few select friends.  Nothing of any real importance or sensitivity exists on it.  Why this insignificant little server attracted the attention of someone who has access a 20,000+ node, worldwide bonnet is beyond me.

    It started when I noticed that sites weren’t loading.  I shelled into the box and found the load hovering around 30+.  ps and top showed that apache was the culprit.  I combed through some logs and found that my wife’s site, messymissy.net, was being hammered.  Hundreds of POST requests per second to her index page.  I tcpdumped some of it and found that it was garbage or encrypted payloads destined for gryphn.com.  She has owned gryphn.com for almost 10 years and has it parked on top of messymissy.net.

    We unparked the domain and removed the DNS zone file and apache started working again.

    A couple of hours later we noticed that nothing on our server was resolving.  I shelled back in and found that DNS was now being hammered with queries to cached zone files for gryphn.com (which didn’t exist).  This log excerpt represents a tenth of a second worth of traffic.

    Feb  3 20:35:55 host named[3235]: client 103.8.44.8#23376: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 196.43.54.190#13041: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 193.2.1.102#39491: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.4.108#59071: query (cache) ‘grYPhN.cOM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.156.227.34#44924: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 89.95.242.180#56873: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 213.228.58.145#5210: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.159.76#44010: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.189.16#47278: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 194.90.2.4#63342: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 202.216.229.12#25343: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.208.3.18#34990: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.208.3.17#48741: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 61.153.81.123#30836: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 68.105.29.237#30849: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.134.4#28981: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 83.206.226.34#10582: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 110.164.252.215#39831: query (cache) ‘gryphn.com/NS/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 196.43.54.190#17049: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 208.69.32.21#36506: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 173.194.96.19#58355: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.156.227.34#38061: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 80.10.201.97#21826: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 164.124.101.49#16876: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.16.215#54383: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 209.18.35.114#2426: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.4.108#29276: query (cache) ‘grYPhN.cOM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 74.125.178.16#54930: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 193.2.1.102#6891: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.186.1.173#39050: query (cache) ‘GryPhn.coM/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 80.10.201.33#27523: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 192.221.151.75#65393: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 195.20.253.11#53176: query (cache) ‘gryphn.com/A/IN’ denied
    Feb  3 20:35:55 host named[3235]: client 210.94.72.122#58224: query (cache) ‘gryphn.com/A/IN’ denied

     I logged into my DNS provider and enabled the use of their DNS servers.  We awaiting propagation of the new authoritative name servers and load returned to normal.

    Immediately following that we started receiving distributed brute force login attacks to multiple email accounts (that don’t exist) associated with multiple domains that we host.  I configured my firewall scripts to monitor for this sort of thing and block them.  As the firewall block list grew, the amount of invalid login attempt notifications shrunk.  Eventually a large part of the botnet was being blocked by my firewall.

    I guess they still had some nodes that weren’t blocked yet (and some fight left in them), because the most recent activity involves distributed brute force login attempts against WordPress sites.  I added a mod_security signature to catch it and modified my firewall scripts to block IPs that trigger the rule too many times.

    It’s a really fun cat and mouse game of changing attack methods on a massive scale (world-wide bonnet of 20,000+ zombies).  I’m working on scripts that will mine my logs for multiple block events and send automated x-ARF notifications to abuse@contacts for the zombies.

    I have no idea what it is they are after, but I’m having fun playing.

    If you are responsible, use the contact form on my site to send me an idea of what it is you want.  I won’t give it you, but the suspense is killing me.  :)

  • Dark Horse Stout (Batch 7)

    Dark Horse 7I have been working on ‘the perfect stout’ for a long time.  In fact one of the first batches I ever brewed was a stout.

    A while back I create “The Stout Experiment” in an effort to try multiple additions in a single batch.  Oddly enough the control (without any additions) ended up being the best.

    I have since done three other bathes (including this one) that has tweaked the recipe slightly each time, based on my tasting notes.

    Appearance: Dark, no light gets through. Tan (khaki) head with tight bubbles. Alcohol clings to the side of the glass with lacing from the head.
    Mouthfeel: silky smooth, but not thick or heavy. The rolled oats contributed to this.
    Flavor: Sweet Carmel at first gives way to burnt coffee, dark chocolate and a hint of grapefruit from the cascade hops.
    Aroma: Carmel and noble hops.

    All in all it’s the best stout I have ever had but I can identify at least two or three places that show room for improvement.

    Next batch I think I will add more roasted barley to turn up that burnt coffee flavor a little and possibly change the aroma hop.

    5stars

  • Fall Cometh

    I’m going to brew a big, warming beer for my next batch.  Something that will toast your innards with alcohol burn and warm you up.

    I am thinking either a Scottish wee heavy of Russian imperial stout.

    The only problem is that these types of beers require extensive aging.  a RIS would be barely drinkable by Christmas.

    I saw on a brewing tv episode that a mead maker used a staggered yeast addition to give his mead a cellared flavor right from the carboy.

    Has anyone tried this with beer?  I would rather not experiment on a huge expensive beer like a RIS, but will if I don’t hear from anyone… for science!

    At the same time I wanted my next batch to be the one I tried the “Brew in a Bag” method.  Do the full mash in my boil kettle is appealing.  To raise the temperature I just add or adjust flame.  After the mash I just lift the bag out, rinse/sparge and start my boil.  Plus it will be less clean up without having to rinse my mash tun.

    The problem with my next batch being a big beer and my first bib batch is the risk of overflowing my kettle.  15-20 lbs of grain, 7 gallons for the boil plus however much I need to figure in for absorption during the mash… than re adding for sparge.  None of my software will do the math for brew in a bag so this is all going to have to be done manually.. yup.. on my fingers.

  • Superior Australian Lager Yeast

    I used “Superior” brand Australian Lager yeast for the first time on a black lager. I was able to find very little information on it online so I pretty much threw caution to the wind and used it.

    I pitched at about 70 degrees and stuck in my lager area at about 50 degrees. After a week of not checking on it I took a gravity reading and it was still at its original gravity. No fermentation had happened at all.

    I transferred it off its yeast cake into a 5gal carboy and stuck it in my ale closet to warm up so I could pitch another yeast.

    After one day at 70 (even after transferring it off its yeast cake) it went crazy! Lava lamp style active fermentation. Apparently this is a lager yeast that has to ferment at ale temps? After about 10 days the gravity was at its expected terminal gravity reading. I transferred it again and stuck it in the lager closet at 50. I will let you guys know how it turns out.

  • Breaking the Silence

    Wow has it been a long time!  I was maintaining radio silence during my security clearance background investigation.  Now that its all over expect me to post more (honest).

    For those of you not close to me, I was hired by the DoD (Department of Defense).  What does this mean?  Well you will never again hear the words “today at work…” uttered again.  That does not mean I will lacking topics to post about.

    My security research at home is picking up again.  Expect posts on the topics of forensics, anti-forensics, malware and possibly a new pentest tool or two.

    It’s nice to be back and for those of you reading this, thank you for sticking it out and visiting again.

  • tcp/2550 and the Chinese

    While investigating an unrelated issue and digging through firewall logs I noticed a decent amount of traffic destined for tcp/2550 on one of my work servers.

    The traffic mostly (82 of the 84 events today) originates from sequential IPs out of China. This immediately raises alarms with me.

    [Read More...]
  • Hey Mac Users… The Honeymoon is Over.

    I know, its sad.  I too am a die hard mac user.

    Today alone I have received 4 copies of an email with the subject line “2 Populaar Myths About Female Orgasms –  How to Become an Irresistible Lover” containing an attachment named “Preview.app Document”.

    I haven’t had a chance to analyse the .app yet, but I think its safe to assume that its malware of some sort.

    The good news is that OS X is still built well.  If I double click it thinking its a document its going to tell me “Hey stupid!  This is an app that was downloaded from the Internet.  Are you sure you want to run it?”.  Maybe not in those exact words.  At that point if I say – “I thought I was opening an document, but sure, lets run this app-like-document” – then I deserve to be infected.

    For all the detail oriented folks here are the headers (bold are items changed to protect my info):

    Return-path: <efflrescent@aperfectmix.com>
    Envelope-to: MY_ADDRESS
    Delivery-date: Fri, 01 May 2009 09:39:27 -0400
    Received: from [87.18.181.177] (helo=ksecb.telecomitalia.it)
    by myserver.mydomain.com with smtp (MyMail Dameon)
    (envelope-from <efflrescent@aperfectmix.com>)
    id 1LzsxZ-0000Ib-JG
    for MY_ADDRESS; Fri, 01 May 2009 09:39:27 -0400
    Message-ID: <49FAF79E.9745295@aperfectmix.com>
    Date: Fri, 01 May 2009 13:39:25 -0100
    From: Chesner <efflrescent@aperfectmix.com>
    MIME-Version: 1.0
    To: MY_ADDRESS
    Subject: 2 Populaar Myths About Female Orgasms –  How to Become an Irresistible Lover
    Content-Type: multipart/mixed;
    boundary=”————32D524EA4E2E67F07C94899F”
    X-Spam-Status: No, score=3.8
    X-Spam-Score: 38
    X-Spam-Bar: +++
    X-Spam-Flag: NO

    The body contains no data.

  • VM Escape

    Whenever anyone speaks of virtual machine security the absolute worst case scenario is the dreaded “VM Escape”.  That is the ability of a malicious user to escape a virtual machines encapsulation and reach the host (or hypervisor).  This class of attack could potentially expose all other virtual machines running on this host.

    In the VM world this type of vulnerability is an absolute worst case, but are very rare.

    On April 10th CVE-2009-1244 was released stating that a number of VMWare products are vulnerable to VM escapes.

    You should patch as soon as possible if you are running:

    • VMware Workstation 6.5.1 and earlier
    • VMware Player 2.5.1 and earlier
    • VMware ACE 2.5.1 and earlier
    • VMware Server 1.x before 1.0.9 build 156507
    • VMware Server 2.x before 2.0.1 build 156745
    • VMware Fusion before 2.0.4 build 159196
    • VMware ESXi 3.5
    • VMware ESX 3.0.2, 3.0.3, and 3.5

    Per the CVE this vulnerability:

    allows guest OS users to execute arbitrary code on the host OS via unknown vectors, a different vulnerability than CVE-2008-4916.

    This also validates why all of the best practice documents recommend that VMs of different sensitivity levels be run on physically separate hosts and/or clusters.

    SANs Internet Storm Center reports that an exploit is available ‘in the wild’ for a fee.  They also provide a link to the following video of someone allegedly leveraging this exploit.

  • Mining Ports for Malware

    I recently wrote a script that runs croned and port scans all of our servers daily.  It saves the output and diffs it compared to the previous days and emails me as new ports open up.

    I think this will be a good way to detect new services and potential malware infection, but what about machines that are already infected?

    To fix that I wrote in a function that parses the output for known malware ports.  The only problem is that I cant find a definitive list of known malware ports.  Does anyone know of such a resource?

  • Loaded C:\WINNT\system32\KERNEL32.dll differs from file image

    I have recently been updating my Windows Forensics First Responder script and have noticed a number of servers reporting the following when using Sysinternals/Microsoft’s listdlls.exe.

    *** Loaded C:\WINNT\system32\KERNEL32.dll differs from file image:
    *** File timestamp:         Wed Apr 18 12:25:36 2007
    *** Loaded image timestamp: Wed Apr 18 12:25:37 2007
    *** 0x77e40000  0x102000  5.02.3790.4062  C:\WINNT\system32\KERNEL32.dll

    Now I can think of lots of malicious reasons why this would be.  In fact I recently wrote on one of these reasons.   But I cant think of any legitimate reasons.

    I’m not one to jump to conclusions without having evaluated all possibilities but my research is turning up almost nothing.

    Can anyone think of a legitimate reason why windows would load kernel32.dll and then something alter it as its going into memory?

    Thanks guys.

%d bloggers like this: