How to stop this attack?

Portal Home > Knowledgebase > Articles Database > How to stop this attack?

Posted by JJeet, 08-30-2010, 09:21 AM
My sites have been extremely slow this morning and when I contacted the hosting support, they replied that there is a DDoS attack in the server. They put my site IP under cisco guard and suggested me to install Mod_dosevasive and (D)DoS Deflate. But it has been more than 12 hours since that, but my server is still too slow. Both Mod_dosevasive and (D)DoS Deflate have been installed. I dont know how to stop this attack. The load on the server is quite normal (around 1.0-2.0 for a quadcore CPU). But the sites are opening very slow. The output of netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n is, The process which takes most of the CPU is [ksoftirqd/2] The iptables log shows lots of messages like these in 2 seconds interval. And here is my strange bandwidth graph. Almost constant 100Mbps incoming connection during all this time. (Please see the attachment) Any help would be greatly appreciated. Thanks Attached Thumbnails
Posted by garmanonline, 08-30-2010, 09:44 AM
If your under a DDoS attack you may want to just have your IP(s) null routed temporarily. It would bring your site(s) down completely though
Posted by Techbrace, 08-30-2010, 10:00 AM
Yes, you're under an incoming DDoS attack. You'll have to determine whether this is a UDP or TCP attack. Most probably a UDP attack as your server load is still low. If so, you can control incoming UDP packets through firewall.
Posted by LynxUser, 08-30-2010, 10:22 AM
Going of those IP's by netstat your not under any attack, I been under a botnet attack before and trust me you see hunderes with thousands of connections. Dos_delfate is pretty usless if you have not configured it to work with iptables, You say your server is slow, Have you checked your connection / Traceroute / Ping to the server ?
Posted by JJeet, 08-30-2010, 10:51 AM
Thanks for the replies. Will changing my site's dedicated IP address work? @LynxUser, Yes, the server status shows load around 1-2. But the process called [ksoftirqd/2] is taking up most of the CPU usage. root@host [~]# top -c top - 19:58:04 up 7:58, 1 user, load average: 1.21, 1.29, 1.29 Tasks: 177 total, 2 running, 175 sleeping, 0 stopped, 0 zombie Cpu(s): 3.5%us, 1.8%sy, 0.0%ni, 69.2%id, 0.9%wa, 0.0%hi, 24.5%si, 0.0%st Mem: 8181348k total, 3571656k used, 4609692k free, 317160k buffers Swap: 1052248k total, 0k used, 1052248k free, 1787108k cached
Posted by khunj, 08-30-2010, 10:56 AM
Check your network stats to see what is going on :
Posted by JJeet, 08-30-2010, 11:00 AM
khunj, This is the output of netsat -s
Posted by netmar, 08-30-2010, 11:03 AM
If it is a UDP attack, then you could maybe offload your DNS servers if you haven't got too many domains. Use a DNS slave service, copy the zones, add the slave servers to the registrar entries, and then just block UDP traffic. Edit: Hmm, wait just saw your netstat numbers. There are an awful lot of UDP packets, but a good fraction of those look to be ICMP. If you haven't already, you may just want to bock all incoming UDP traffic that isn't on port 53. Last edited by netmar; 08-30-2010 at 11:07 AM.
Posted by khunj, 08-30-2010, 11:27 AM
You should try to make a UDP capture with tcpdump, because you have a huge amount of incoming UDP and outgoing ICMP packets and, in most cases, it is the result of an UDP flood.
Posted by JJeet, 08-30-2010, 11:47 AM
netmar, I asked server management guys to do that and they just did it, but still doesn't seem to be much improvement. @khunj, How do I do that? Sorry, not good in these stuffs
Posted by Techbrace, 08-30-2010, 11:54 AM
Have you blocked unwanted UDP ports for incoming connection? You should also try limiting incoming UDP connections.
Posted by netmar, 08-30-2010, 12:39 PM
Can you post the output from iptables -L I'm just curious to see exactly what has been done.
Posted by JJeet, 08-30-2010, 03:24 PM
netmar, Its a long output, here is it. There are still a lot more lines, please let me know if I should copy/paste all.
Posted by badboyx, 08-30-2010, 10:30 PM
pm me with the info i'll try to help you
Posted by netmar, 08-30-2010, 10:49 PM
Actually, that looks like enough. While it looks like they've opened ports for DNS and FTP (and explicitly for google and opendns?), they've also left the firewall open ICMP packets. Ask them if they'll close off ICMP traffic to the server. Judging by the previous packet counts, that could well be your problem.
Posted by server4sale, 08-31-2010, 02:41 AM
You may want to try some professional ddos mitigation companies here. They may help you out.
Posted by tchryan, 09-01-2010, 12:53 AM
At the end of the day, you are an end point device, the attack/traffic must first arrive to your server before you can do anything to it. Once the traffic has arrived at the server, all you are really doing is telling the kernel through iptables to ignore said traffic. As an end point device, in order to achieve any real form of DoS mitigation, you will need to work with your DC provider to have the addresses filtered upstream on there router or firewalling devices or identify specific characteristics of the attack and have them apply a filter accordingly (i.e: specific length udp packets targeted at a specific port). The reason you are seeing ksoftirqd eat up resources is that your CPU's are receiving more interrupt requests than can be handled by them, as such soft interrupts are being triggered at a rate faster than can be dealt with by the system. In this case, it would be the fact you are probably receiving an obscene amount of pps for the system to process. Again, you are an end-point device, anything you do on the server is simply telling the kernel to ignore traffic that has already arrived and with a 100mbit attack really you cant just expect filters on the end point device to achieve any margin of success. Either get with your DC provider to work the attack with you or drop the IP's being attacked so the local router stops routing the packets to your system.
Posted by FastServ, 09-01-2010, 01:18 PM
VERY strange traffic profile. It looks as if you got capped at 100Mbps and have saturated the port.

Add to Favourites Print this Article