qmail spam sitting in queue

[MikkyX]

Hi,
I've got a server running Plesk 7.5. Reloaded with as up to date a QMail as you can get via Plesk itself (I'm not using ART updates or anything)

My server seems to be accepting e-mail which is CLEARLY not intended for users of the server - e.g. spam. Once accepted though, it sits clogging up the queue until the message times out and it gets removed.

The last time I flushed the queue I had over 150,000 messages in it because of this - how can I configure QMail to REJECT messages instead of just sticking them in the queue to go stale?

 

[wagnerch]

The last time I flushed the queue I had over 150,000 messages in it because of this - how can I configure QMail to REJECT messages instead of just sticking them in the queue to go stale?

Are you certain one of your domains was not hacked? Try running this command "ps -fuapache" and see if there is any unusual processes, also all of the httpd should have the same PPID -- if they don't then that may mean one of your domains was hacked.


It is _very_ unusual that you would have 150K worth of e-mail in your mail queue unless you were hacked or deliberately attacked. A deliberate attack is very very unusual. I would tend to believe one of your domains was hacked due to an XML-RPC or awstats worm.

Qmail, by default, is setup in a non-relay configuration. I would be suprised if this is the problem.

 

[MikkyX]

I'll try that, thanks.

In the meantime I've noticed some specific domains seem to be getting stuck in the queue for LEGITIMATE e-mails as well (e.g. forum reply notifications) - these mails then bounce regardless of how full the queue is.

Is this because of the spam problem so these ISPs might've blacklisted us?

 

[MikkyX]

I ran "ps -fu apache" and found this line at the top:

apache 29103 1 0 Feb13 ? 00:00:00 /usr/sbin/atd d ? ? ?

All the other lines were httpd processes with the same PPID.

 

[wagnerch]

Originally posted by MikkyX
I ran "ps -fu apache" and found this line at the top:

apache 29103 1 0 Feb13 ? 00:00:00 /usr/sbin/atd d ? ? ?

All the other lines were httpd processes with the same PPID.

I would say that is very suspicious. atd never runs as apache.

You can do the following to find out what this executable really is:

readlink /proc/29103/exe

It should puke out a path (more than likely /tmp and some file that is gone) and file. If your really interested then you can download The Coroner's Toolkit and compile the pcat program:

pcat 29103 >/tmp/29103.dmp

and then run:

strings </tmp/29103.dmp |less


I don't know if there is any RPMs out there for TCT, but at the same token this thing is probably dumping tons of spam into your mail queue. Your big problem is figuring out which domain this thing came in on, usually pcat can reveal some evidence on where it came in by looking at the memory dump.

 

[MikkyX]

Thanks for that - I ran the first command there and got the following result:

/home/httpd/vhosts/<DOMAIN>.com/tmp/lang/ps (deleted)

The domain being one which has a cron script which runs every 5 minutes, if that's at all relevant. I'll run The Coroner's Toolkit on it as soon as I can and see what else it turns up - thanks for all your help!

 

[wagnerch]

Originally posted by MikkyX
Thanks for that - I ran the first command there and got the following result:

/home/httpd/vhosts/<DOMAIN>.com/tmp/lang/ps (deleted)

The domain being one which has a cron script which runs every 5 minutes, if that's at all relevant. I'll run The Coroner's Toolkit on it as soon as I can and see what else it turns up - thanks for all your help!

Interesting, most worms dump executables in /tmp or /var/tmp. In any event I wouldn't be suprised if this isn't a deliberate attack against one of the many web application vulnerabilities.

Is there any particular web applications running on this domain? Such as phpBB2, awstats.pl, phpAdsNew, phpNuke, PostNuke, Mambo, etc... All of those have remote vulnerabilities.

 

[MikkyX]

I'm running phpCOIN on the domain but the vulns. for that program don't normally allow it to send mail or anything. I'm swapping out phpCOIN for something better soon.

Unfortunately I couldn't install TCT due to a compile error. Thanks for all your help though, it's much appreciated.

 

[wagnerch]

Most vulnerabilities do not allow mail, but they allow remote execution. Which phpCOIN _is_ vulnerable to a remote execution bug. Typically these guys leverage the vulnerabilities to pull down additional applications and those applications may feed mail into your mail queue.

I would suggest apply all of the patches for phpCOIN. BTW this patch will get tct to compile:


diff -wur tct-1.15/src/fstools/mylseek.c tct-1.15.new/src/fstools/mylseek.c
--- tct-1.15/src/fstools/mylseek.c 2000-07-30 19:39:20.000000000 -0400
+++ tct-1.15.new/src/fstools/mylseek.c 2006-02-14 19:13:17.778842092 -0500
@@ -24,6 +24,7 @@
#ifdef HAVE_LLSEEK
#include <errno.h>
#include <syscall.h>
+#include <linux/unistd.h>

/*
* This is LINUX, live on the bleeding edge and watch your software break

 

[LiveChatService]

running the command: ps -fuapache

The below was shown... is that first "./php" one a problem?


apache 26368 1 0 Apr10 ? 00:00:00 ./php
apache 29982 4615 0 Apr12 ? 00:00:04 /usr/sbin/httpd
apache 30222 4615 0 Apr12 ? 00:00:03 /usr/sbin/httpd
apache 30223 4615 0 Apr12 ? 00:00:03 /usr/sbin/httpd
apache 30229 4615 0 Apr12 ? 00:00:02 /usr/sbin/httpd
apache 10836 4615 0 00:06 ? 00:00:02 /usr/sbin/httpd
apache 23428 4615 0 00:17 ? 00:00:01 /usr/sbin/httpd
apache 9289 4615 0 00:35 ? 00:00:02 /usr/sbin/httpd
apache 2159 4615 0 00:58 ? 00:00:00 /usr/sbin/httpd
apache 7206 4615 0 01:03 ? 00:00:00 /usr/sbin/httpd
apache 19319 4615 0 01:15 ? 00:00:00 /usr/sbin/httpd
apache 21924 4615 0 01:18 ? 00:00:00 /usr/sbin/httpd
apache 24133 4615 0 01:20 ? 00:00:00 /usr/sbin/httpd
apache 24138 4615 0 01:20 ? 00:00:00 /usr/sbin/httpd
apache 24139 4615 0 01:20 ? 00:00:00 /usr/sbin/httpd
apache 24891 4615 0 01:21 ? 00:00:00 /usr/sbin/httpd
apache 24894 4615 0 01:21 ? 00:00:00 /usr/sbin/httpd
apache 24897 4615 0 01:21 ? 00:00:00 /usr/sbin/httpd
apache 24898 4615 0 01:21 ? 00:00:00 /usr/sbin/httpd

 

[wagnerch]

I would say it is a bit suspicious, did you do a:

readlink /proc/26368/exe

If it isn't pointing to /usr/bin/php, then it is definately funky. But php shouldn't be a) running detached from httpd (notice it is attached to parent pid of 1, which means it is detached or backgrounded), and b) running more than a few seconds (notice that yours has been running for 2 days).