We have a very similar issue.
OS: CentOS release 5.8 (Final), plesk psa-9.5.2-cos5.build95100504.10
Running litespeed, qmail, php, multiple vhosts (some have wordpress but we are not sure if this is the exploit path).
The server was bogged down with the email spam blast. I stopped qmail so the server could do useful work.
I did:
http://kb.parallels.com/en/1711
But added one line: printenv >> /var/tmp/mail.send
Result:
best regards
SHELL=/bin/sh
USER=apache
PATH=/usr/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/printenv
PWD=/var/www
SHLVL=3
HOME=/var/www
LOGNAME=apache
X-Additional-Header: /var/www
From: root@localhost
To:
[email protected]
Subject: Test mail 1924820051
Bla-bla-bla
From here I ran:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www
The ‘crond’ and ‘perl’ (and below) do not exist on disk. They are memory mapped files.
Via
http://serverfault.com/questions/173999/dump-a-linux-processs-memory-to-file I wrote a test.sh script:
#!/bin/bash
grep rw-p /proc/$1/maps | sed -n 's/^\([0-9a-f]*\)-\([0-9a-f]*\) .*$/\1 \2/p' | while read start stop;
do gdb --batch --pid $1 -ex "dump memory $1-$start-$stop.dump 0x$start 0x$stop"; done
I then ran this ‘test.sh’ script like so ./test.sh 29xxx where 29xxx is the pid from lsof command above.
This dumped ~20files of mapped memory. I looked around and found some pearl script but to be honest I don’t know how to interpret this mess. The highlights include:
1) /tmp/sess_f652da7dd28dce7baeeae54a46ae4092
2) wget -q -O - -t 1 -T 60 --no-check-certificate "https://accounts.google.com/ServiceLogin?service=mail" | grep -ci '<html'
3) lsif ($ki8n =~ /^START SENDMAIL$/)
{ `service sendmail start`
; next; }
elsif
($ki8n =~ /^STOP IPTABLES$/)
{ `service iptables stop`;next;}
I still have these dump files ~5MB if anyone knows a useful place to send them for analysis?
At this point I decided to kill those PIDs:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www
$ kill -9 29037
$ kill -9 29050
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13040 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13041 root cwd DIR 8,5 4096 21561622 /var/www
As of this writing I have no signs of the email spam issue but the server cannot be trusted for production and will be decommissioned shortly. Any thoughts?