SpamAssassin spam filtering problem

[cmaxwell]

Hi all,

SpamAssassin was working flawlessly until we updated it through Plesk Updater. As far as I'm aware this was a minor update, as we were already running Plesk 7.5.4 so it wasn't a full blown Plesk upgrade (just an SA update).

Since installing the update, only about 5% of spam is flagged as spam and the rest is flagged as clean messages.

We have a 20,000 message spam database trained through Plesk, so it's not as if we haven't trained enough messages for SA to work effectively.

Does anyone have any ideas what we could do to restore what used to be a flawless service?

Many thanks.

- Chris

 

[pgeorge1983]

I'm having the same problem after upgrading to the latest version of Plesk 7.5.4 and SpamAssassin (which appears to be 2.63). The bayesian system is accepting entries just fine, however, when the mail is scanned, its like the bayesian stuff isn't taking into account. I'm not receiving the BAYES_XX markings in the spam header....

I'll post here when I figure out what happened.

Thanks,
Phillip

 

[pgeorge1983]

SOLUTION:

Manually downgrade the PSA SpamAssassin package. I downgraded to the 7.5.2 version, though the 7.5.3 version may work as well. For me, it was located at:
/root/psa/PSA_7.5.2/rpm_RedHat_el3/opt/mail/psa-spamassassin-7.5.2-rhel3.build75050130.17.i586.rpm

I used this command:
rpm -Uvh psa-spamassassin-7.5.2-rhel3.build75050130.17.i586.rpm --force

Please take care when you issue this command. I cannot guarantee that it will work with your server and will not mess things up. Theoretically, all should be well and the worse thing that can be messed up is SpamAssassin.

Also note that I had SWsoft log into my server for over 6 hours (this was NOT paid for by me, thank God). They were not able to fix the problem. Their last recommendaton was to reinstall the latest build of PSA 7.5.4 (availale from their website). I didn't want to do that

Thanks,
Phillip

 

[rommer]

I'm trying to get a handle on this since it def. seems to be broken in 7.5.4

When I look at the qmail directories I see 2 bayes_seen & bayes_tok files

/var/qmail/mailnames/domain.com/username/.spamassassin/bayes_seen

AND

/var/qmail/mailnames/domain.com/username/.spamassassin/.spamassassin/bayes_seen

The timestamps are different on the files. Is this a bug? Could it be that the training system is using a DIFFERENT directory path than spamassassin is using?

Not an expert in this area but this seems wrong to me. Anybody else see the 2 directory levels? Maybe I should just try to symbolicly link them and see if things get better?

 

[cmaxwell]

Hi rommer,

I had a look and I see there are two sets of bayes_xxx files on our RH9 server, in the same directory layout as you mentioned.

It seems one of the bayes_seen files is much larger than the other. One is 2.5MB in size, in contrast to the other which is just 90KB.

I am noticing the spam going down again as we continue to train the spam filter but it is not going down quickly and we are still getting maybe 50-60 spams a day where we used to get 0.

I may try downgrading the psa-spamassassin package but will contact [email protected] to see what they say.

I guess this is a rare problem otherwise there would be lots of noise from everyone with a Plesk server in these forums...

- Chris

 

[rommer]

I tried a symlink but that didn't work as it seemed to make all messages appear with a "learned" mark next to them. So it would appear that the 2 locations are needed to keep things straight.

If that is the case then shouldn't they BOTH get updated when the training is down? Seems training only updates the bayes_xxx files in .spamassasin directory. The bayes_xxx files in .spamassain/.spamassain only seem to get updated when there is incoming mail.

Maybe I'll try a downgrade to 7.5.3 on one of my servers to see if the behavoir changes....

 

[rommer]

hmmm, unable to susccessfully downgrade...

 

[cmaxwell]

What was the problem?

 

[rommer]

I could not get psa-spamassassin to restart. Something about spamng not being found. (It's an active server so I didn't want to take to much time)

For what it's worth, there is a significate reduction in file size of psa-spamassassin from 7.5.3 to 7.5.4 so something has changed dramtically

# ls -l /root/psa/PSA_7.5.3/rpm_RedHat_7.3/opt/mail/psa-spamassassin-7.5.3-rh7.3.build75050506.13.i586.rpm
-rw-r--r-- 1 root root 136748 Nov 8 12:47 /root/psa/PSA_7.5.3/rpm_RedHat_7.3/opt/mail/psa-spamassassin-7.5.3-rh7.3.build75050506.13.i586.rpm

# ls -l /root/psa/PSA_7.5.4/rpm_RedHat_7.3/opt/mail/psa-spamassassin-7.5.4-rh7.3.build75050927.15.i586.rpm
-rw-r--r-- 1 root root 50458 Nov 8 14:05 /root/psa/PSA_7.5.4/rpm_RedHat_7.3/opt/mail/psa-spamassassin-7.5.4-rh7.3.build75050927.15.i586.rpm

 

[pgeorge1983]

Try running it again.

-Phillip

 

[rommer]

Still trying to understand how plesk works with spamassassin.

I'm making an assumption that there are supposed to be 2 .spamassassin directories. One that spamassassin uses and one that gets modified by the plesk pannel to do retraining and learning.

If that is in fact the case then I would expect that after training the control panel would aslo copy the bayes_xxx files to the .spamassassin/.spamassassin/ directory.

With 7.5.4 it does NOT. I'm hopeing someone that has 7.5.3 or less can check to see if the timestamps change on both sets of bayes files after a training session.

For now I'm manually copying it for one test subject to see if it helps.

 

[pgeorge1983]

Hmm..I have those two directories as well.... I do notice that the spam training is empty now (thru the WWW interface), but my bayes DB is still in tact.

-Phillip

 

[rommer]

As far as I can tell this is broken on both a Mandrake system and a Redhat 7.3 system. The bayes_xxx files ARE NOT being copied back to the place that Spamassassin needs them to be after a traning session. I have been manually copying them and the amount of missed spam has gone down dramtically for my test subject.

I would assume it is probally broken for ALL versions of 7.5.4 though most people probally don't even notice.

How do we report this bug to Plesk so that they will fix it?

 

[cmaxwell]

[email protected] is the address to send bug reports. Don't expect an immediate response because they get a lot of reports which need investigated and sometimes it takes a little time.

 

[cmaxwell]

Also just to confirm, it is broken on Plesk 7.5.4 on RH9 and also RHEL by the looks of previous posts from pgeorge1983.

I'd therefore say it is broken on every OS release, and if it is not broken on every single Plesk 7.5.4 server then there must be something that occurs on a select number of servers during installation of 7.5.4 which causes SpamAssassin to fall apart.

 

[rommer]

I sent a bug report. ShouId I start holding my breath?

 

[pgeorge1983]

Yes...it was RHEL.

Heh...how long can you hold your breath?

-Phillip

 

[cmaxwell]

Well I have submitted a ticket to them as this is quite important. They have already responded and are looking at it which is reassuring.

I will update here if there is any good news.

- Chris

 

[cmaxwell]

Solution to the problem

Good news from SWsoft.

By the sounds of things they were unaware of the problem and they have thanked us for bringing it to their attention.

As far as a solid fix for the problem is concerned they said that this should be implemented in a future update.

In the meantime these steps can be taken to fix the problem:

1. Stop SpamAssassin
2. Download the new module from SWsoft which I have placed on one of our servers:

cd /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/
wget [url]http://apricot.host-serve.net/CmdManage.pm.new[/url]
chmod +x CmdManage.pm.new

Note the location of the Perl directory above may be different depending on your Perl installation. You can find the correct location by doing a "locate CmdManage.pm" which will show the right directory to CD to.
3. Make backup of existing CmdManage.pm module:

cp CmdManage.pm CmdManage.pm.bak

4. Replace with new module:

cp CmdManage.pm.new CmdManage.pm

3. Run mchk to generate 'user_prefs' files in the correct location:

/usr/local/psa/admin/sbin/mchk --spam-only

4. Start SpamAssassin
5. It should work now.

 

[rommer]

Anybody good enough to write a shell script to automate 100's or 1000's of mv commands. I appreciate the quick fix but the last part is going to be painful.

Would running mchk after getting the fix recreate the user_prefs files?