New vulnerability? (plesk 9.5.4)

[mooiesite]

Hi,

This is a complete up2date box (until Micro Update 13)
Today we were hacked by an individual who remotely added a cronjob (using plesk rpc-xml). This cronjob downloaded a rootkit and our server was rooted.
I have the httpsaccesslogs which shows the actions. Is this vulnerabilty already known?

 

[zeroday]

And that server is mine. I just got a message of the datacenter. I am running a full updated 10.4.4 plesk installation and I am wondering where I can find leads how they entered my system...

 

[Blake@Parallels]

Hi,

This is a complete up2date box (until Micro Update 13)
Today we were hacked by an individual who remotely added a cronjob (using plesk rpc-xml). This cronjob downloaded a rootkit and our server was rooted.
I have the httpsaccesslogs which shows the actions. Is this vulnerabilty already known?

Can you PM me details?

 

[zeroday]

After scanning my server I found out that there was indeed a possible rootkit running
causing a ssh server on port 9999, which was there already for more than 18 months or so. (upon delivery the server was already compromised when I was running Plesk 9.x. I am unable to delete the files, (many) even chattr does not give me any rights to remove such files :-(.. I managed to find out what shell script was running and I disabled it in the startup script including adding a firewall rule for not allowing the port 9999 to be open)

any other wierd behaviour through websites was not detected.

 

[Alex Plakhtyrya]

Hello, confirmed.
Part of the perl script by trojan http://pastebin.com/gAs4EkyG

Try this:
- Scan with ps aux |grep ".pl" suspicious name of scripts, and lock this domain.
- After lock, see user Task (with Plesk GUI or in /var/spool/cron), and delete any task created this bot.
- After, delete any files in /tmp dir with name apachectrl.lock.
- Drop any incoming\outgoing connection on tcp 5432 with iptables

Good luck.

 

[Blake@Parallels]

http://forum.parallels.com/showpost.php?p=617183&postcount=33

 

[Matt Grant]

I have the same issue going on and I followed the direction posted here on how to remove it. How do I make sure it will not come back? Is there some sort of patch or update I need to do?

Thanks in advance!

 

[Blake@Parallels]

I have the same issue going on and I followed the direction posted here on how to remove it. How do I make sure it will not come back? Is there some sort of patch or update I need to do?

Thanks in advance!

http://kb.parallels.com/en/113321

 

[onsightdata]

I have applied the hot fix to my 9.5 server and run the password change script. Do i need to worry about any files left behind?

My server must have been used for ddos yesterday as it went through 3 terrabytes on gigabit port in a very very short amount of time all outgoing bandwidth.

So my questions are, if files were left behind can the person still run the files without having access to the server any more though the user accounts and/or vulnerabilty?

Does anyone have a list of common files that the hacker(s) have been dropping on servers to give an idea of what to look for? From reading other posts i see the files are primarly .php files.

I

 

[Matt Grant]

I installed the 9.5.4 MU14 updates and it appears to be fixed... But i have one issue that I am not sure if it is a problem or not. Whenever a domain is suspended or unsuspended, a 0 byte file with the ftp username gets created in the /var/spool/cron directory. My other Plesk 9.5.4 server does not do this when I suspend or unsuspend domains. Is this normal or am I still in trouble?

Thanks in advance!

 

[onsightdata]

Interesting,

In /var/spool/cron I to have the zero byte files of domains that i suspended. The active domains also have files in there and when i cat one of the files this is what was displayed

* * * * * /usr/bin/perl /var/www/vhosts/domain.com/cgi-bin/countertug.pl detach >/dev/null 2>&1
When i open in a text editior the file countertug.pl
This is the first couple of lines!!!!

#!/usr/bin/perl

#part of the Gootkit ddos system
use Fcntl qwflock EFAULT);

I am going to rm the countertug.pl can anyone see anythign else i should do? Does anyone have any other files that are created that i should be deleting?

 

[onsightdata]

congestion.pl homophthalic.pl mucronate.pl prewarns.pl shnooks.pl taillie.pl
dissuasive.pl intelligent.pl nitraniline.pl putties.pl shotten.pl
focuser.cgi lemmings.pl precessed.cgi sesames.pl sideswipes.pl troparion.pl

That is a list of the files i found in my /var/www/vhosts/domain.com/cgi-bin of a few domains. Each file is the same but appears to have different targets.

If you want to see what one of the file contains.

http://pastebin.com/z2zb1eYL

 

[Matt Grant]

That is the same thing I was dealing with before I got the server cleaned up(I think). So far none of the random files in the cgi-bin dirs have come back. Just the 0 byte files in the cron dir. I locked the server down to only the few ports that need to be open and I disabled all sorts of programs that I do not need from starting up with the server as well as disabled cgi and perl scrips on all of the domains.

 

[onsightdata]

Before doing anything check to see if the proper patch was applied to the server

Login as root
wget http://kb.parallels.com/Attachments/19203/Attachments/plesk_remote_vulnerability_checker.php
Then run command: php -d safe_mode=0 plesk_remote_vulnerability_checker.php
Output Should Be: The patch has been successfully applied.

If you are not patched then visit

http://kb.parallels.com/en/113321

Before doing anything else ensure that the web server is shut down
/etc/init.d/httpd stop.

Then ensure you change ALL passwords. The exploit stole ALL usernames and passwords for your server. Even if you patch the server the person can come back in at any time unless you change:

All Users
Root Password
FTP Passwords
Mail Passwords
If you have quite a few accounts Parallels has created a handy script to mass change everything.
http://kb.parallels.com/en/113391


Then remove all the files located in cgi-bin that have the extension .pl
They will all have random names that are generated from dictionary names some examples are:
countertug.pl dactyloscopic.pl magnetitic.pl unlikable.pl vermiculose.pl

If you are not sure if the files should be deleted. Open them with a text editor like nano or vi and the top header of the file will look like this:

#!/usr/bin/perl
#part of the Gootkit ddos system
use Fcntl qwflock EFAULT);

So once the cgi-bin folders are clean remove all cron jobs referencing them (run "crontab -u username -r -i" on each user until they're all gone)

If you want to see what users have cron jobs go to /var/spool/cron and check the contents of each file there. You can just type cat filename and it will echo the contents which correspond with cron jobs entries look for entries that reference the files in the cgi-bin directories

You could also login to plesk control panel and click on domains and under Additional Tools – Scheduled Tasks you will see any cron entries for that domain. Be careful because you may have cron jobs that were not put there buy the exploit.

Go to /tmp directory

Remove all /tmp/apachectrl* files
Remove id" and "ua" files

Restart Web Services /etc/init.d/httpd start

 

[madsere]

Same problem here on two different Plesk 9.5.4 servers that we haven't yet converted to Cpanel. Both are updated with the latest micro patches last week.

The vulnerability checker says:

# php -d safe_mode=0 plesk_remote_vulnerability_checker.php
The patch has been successfully applied.

Yet I still get this in the PSA admin log today:

202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:07:27 +0000] "POST /plesk/client@4/domain@9/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:07:44 +0000] "POST /plesk/client@4/domain@9/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:08:23 +0000] "POST /plesk/client@4/domain@10/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:08:38 +0000] "POST /plesk/client@4/domain@10/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:09:14 +0000] "POST /plesk/client@4/domain@11/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:09:24 +0000] "POST /plesk/client@4/domain@11/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:09:58 +0000] "POST /plesk/client@4/domain@12/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:10:05 +0000] "POST /plesk/client@4/domain@12/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:10:37 +0000] "POST /plesk/client@4/domain@13/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:10:56 +0000] "POST /plesk/client@4/domain@13/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
202.14.92.132 xx.xx.xx.xx:8443 - [07/Mar/2012:02:11:56 +0000] "POST /plesk/client@4/domain@14/hosting/file-manager/create-file/ HTTP/1.1" 303 0 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:11:57 +0000] "POST /plesk/client@4/domain@14/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:03 +0000] "POST /plesk/client@4/domain@15/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:04 +0000] "POST /plesk/client@4/domain@15/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:10 +0000] "POST /plesk/client@4/domain@16/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:12 +0000] "POST /plesk/client@4/domain@16/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:17 +0000] "POST /plesk/client@4/domain@19/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:19 +0000] "POST /plesk/client@4/domain@19/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:24 +0000] "POST /plesk/client@4/domain@20/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"
88.156.171.25 xx.xx.xx.xx:8443 - [07/Mar/2012:02:12:26 +0000] "POST /plesk/client@4/domain@20/hosting/file-manager/create-file/ HTTP/1.1" 200 976 "-" "Opera/7.21 (Windows NT 5.2; U)"

That would indicate the problem has NOT been solved.

 

[philb@]

Stop the bleeding

Here's a quick one liner that will remove all the pl scripts in one whack..

/bin/ls /var/www/vhosts/*/cgi-bin/*.pl | /usr/bin/xargs /bin/grep -li "Gootkit" | /usr/bin/xargs /bin/rm

You can run that from a cron if you want. It's got all the paths.

I'm working on something better but this is a good start. I'm down to where they can't live on my machine for more than one minute.

I'll post more when I get it safe for everyone.

 

[madsere]

I did something similar with a little less overhead:

rm -f `grep -ls 'Gootkit' /var/www/vhosts/*.*/cgi-bin/*.pl`

but essentially it would seem that if you patched Plesk and changed all passwords the hole ought to have been closed.

 

[philb@]

Bash Script gootkiller.sh

Below is a first pass at a script that will stop the bleeding within one minute of when you are attacked by this nasty little thing. Which gives you time to work on a real solution.

You can run this from a cron job. I append it to the /var/spool/cron/root to run it once a minute.

* * * * * /root/gootkiller.sh

Check /var/log/gootkiller.log for entries to see that the cron is running it.

Please note that it's a bit heavy handed at killing perl scripts, so if you have perl scripts running that are not gootkit you'll need to deal with that a different way. I'm open to suggestions on how to improve this so it only kills tasks that are running gootkit.

#!/bin/bash
# gootkiller.sh

cd /var/spool/cron

if [ ! -d "/root/badstuff" ]
then
echo "Making Save Directory"
mkdir /root/badstuff
fi

echo $(date) >> /var/log/gootkiller.log

# Let's assume that we don't ever want a cron job running something in cgi-bin. I think that's a pretty safe bet
count=`/usr/bin/find /var/spool/cron -type f | /usr/bin/xargs /bin/grep -li "cgi-bin" | wc -l`

if [ $count -gt 0 ]
then

# save a copy in case it is actually an important file
/bin/cp * /root/badstuff

# rm the file
/usr/bin/find /var/spool/cron/* -type f | /usr/bin/xargs /bin/grep -li "cgi-bin" | /usr/bin/xargs /bin/rm

# restart crond so it forgets the task
/sbin/service crond restart

# This needs to be improved, but for now, kill any perl jobs that are running.
# This really needs to be only killing specific jobs, but I don' thave any long running perl jobs anyway.
/usr/bin/killall /usr/bin/perl

fi

# Whack the Gootkit file from all of the cgi-bin directories if found
if [ -f /var/www/vhosts/*/cgi-bin/*.pl ]
then
/bin/ls /var/www/vhosts/*/cgi-bin/*.pl | /usr/bin/xargs /bin/grep -li "Gootkit" | /usr/bin/xargs /bin/rm
fi

 

[philb@]

I did something similar with a little less overhead:

rm -f `grep -ls 'Gootkit' /var/www/vhosts/*.*/cgi-bin/*.pl`

but essentially it would seem that if you patched Plesk and changed all passwords the hole ought to have been closed.

Agree, but that can take time. What I've provided keeps it whacked off at the knees and chops off it's head every time it rises up again. In some ways I take a perverse pleasure in that.

Anyway, the obvious final fix is to patch the vulnerability.

 

[Darkfly2]

Hi all!

Same problem here! with plesk 9.5.4.

About the password change, we need to change it for emails account to? Because in my case the hackers have used only 4 customer user account. but maybe the hackers have the other password to.... but email account?

Do you know where we can find more informations about this vulnerability, Parallels don't create a bug ticket or something like this?