is there a log for sa-learn?

[Sven L.]

hello,

title sais it all.

is there a log somewhere (if yes: where?) to see what spamassassin daemon learned or proccesed?

basically, the daemon should learn automatically from every users .spam folder and i want to check if this is really working, because after 2 weeks of marking the same mails over and over again*** as spam, they are still not classified as spam with a bayesan value of 50% when it should be 99% already.

*** clarification: i am receiving new mails on a daily basis that are basically all the same: some pornographic subject, a link to a pornographic website followed up by a large parragraph of random unrelated text, which makes the mail pass with a lower bayesan value.

 

[InderS]

You can check the logs in : /usr/local/psa/var/log/maillog file.

 

[Sven L.]

hm, nope. in maillog i can only see the filtering of mails (as in: "this mail is spam, this mail is not spam")

i want to see a log on what happens when users move mails to the .spam folder (imap, webmail) and the sa daemon does the sa-learn on that folder

when you do a sa-learn manually, you see "50 mails scanned, 25 new mails learned" i want to see this but in a log. is there any?

 

[Alexey.Plotnitsky]

Hi Sven L,

I've asked Plesk devs about this log file. Will revert on this later.

 

[Sven L.]

Hello Alexey.Plotnitsky

thx for your reply. I'll await further info.

 

[Alexey.Plotnitsky]

Sven L.,
Currently there is no such log. However, one can redirect standard output to a file in the Plesk utility:
/usr/local/psa/admin/sbin/spamtrain

spam_learn() function should modified: look for "learn_messages" string

 

[Sven L.]

Sven L.,
Currently there is no such log. However, one can redirect standard output to a file in the Plesk utility:
/usr/local/psa/admin/sbin/spamtrain

spam_learn() function should modified: look for "learn_messages" string

thx for the answer.
i'd rather not touch such things myself, i will probably break it.

besides, this is the kind of stuff that undoes itself after an udate, right?

i'd prefer if plesk could put this change into a future version and maybe have an option in a config file to enable/disable it.

 

[Alexey.Plotnitsky]

Correct, such changes are reverted after Plesk update.
We will consider this small feature as an enhancement for SA support.

 

[Sven L.]

Ok, so I finally got a real production server running with centos 6.4 and plesk 11.5.30#14 to test out some of the stuff we've been talking now for about six month.


several times, I have asked a simple question:

when spam is moved to the spam folder by an user, is there any daemon that will actually sa-learn that folder?

response from parallels: yes, that is how it works.
I actually got their internal reference for this issue: TP137135 (http://forum.parallels.com/showthre...-on-this-roundcube-fixes-i-mentioned-long-ago)

I suspected this was not true and I asked for log files in this post. sadly there don't seem to be such log files.


so, please EXPLAIN, how is this actually suposed to be working? it can't be a scheduled task, I have checked for all scheduled task and I could not find anything related with sa-learn
is it triggered when the user selects an email (in roundcube) and moves the email to "junk"? no. doesn't look like that either.


I have had the system running for 12h. and received 59 spam emails. from those 59 spam, I had to manually mark/move some to the junk folder. a few i did within roundcube, others i did with the smartphone through imap

now i just tried this:

[root@vps01 bin]# sa-learn --spam /var/qmail/mailnames/domain.tld/user/Maildir/.Spam/cur/*
Learned tokens from 56 message(s) (57 message(s) examined)
[root@vps01 bin]# sa-learn --spam /var/qmail/mailnames/domain.tld/user/Maildir/.Spam/cur/*
Learned tokens from 0 message(s) (57 message(s) examined)
[root@vps01 bin]#

as you can see, the first time it learns from 56 messages. 57 have been examined (59 were in the junk folder)
and the second time, nothing was learned. as expected when sa-learn already checked those mails.
how comes it learned from all those mails, when parallels claims sa-learn is already being applied to the junk folder? beats me!


so, can anyone from parallels please clarify if and HOW spamassassin is suposed to learn from the junk folders of ALL MAILBOXES on the server? thank you



Sven L. / QWERTY

 

[Alexey.Plotnitsky]

Sven L.,

sa-learn executed as a Plesk internal task from the daily cron:
/etc/cron.daily/50plesk-daily

One can see the daily cron script output that includes also sa-learn output. In order to do this:
1) Plesk logging must be set to verbose:
/usr/local/psa/admin/conf/panel.ini
[log]
priority=7

2) Daily maintenance script output must be redirected to a file OR the script simply started manually:
/usr/local/psa/bin/sw-engine-pleskrun /usr/local/psa/admin/plib/DailyMaintainance/script.php

you should see there execution of sa-learn

 

[Sven L.]

Sven L.,

sa-learn executed as a Plesk internal task from the daily cron:
/etc/cron.daily/50plesk-daily

One can see the daily cron script output that includes also sa-learn output. In order to do this:
1) Plesk logging must be set to verbose:
/usr/local/psa/admin/conf/panel.ini
[log]
priority=7

2) Daily maintenance script output must be redirected to a file OR the script simply started manually:
/usr/local/psa/bin/sw-engine-pleskrun /usr/local/psa/admin/plib/DailyMaintainance/script.php

you should see there execution of sa-learn

I am very glad to get more insight on this issue. I am out of the office right now so I can't access to my shell, however I'll check it when I get home this afternoon.

so, if I understand it right:

this is executed on a daily basis. every 24h. would there be a way to schedule this 2-4 times a day or even every 15min.? reason is that if it's executed once per day on all mailboxes, that could result in a heavy resource usage peak and maybe it's more interesting to spread this process in several times per day, which would also result in a more frequent spam training

 

[Alexey.Plotnitsky]

Current Plesk daily maintenance script have to be executed every 24h, since it contains connected, time-oriented tasks.

As an option, sa-learn task could be executed individually, using this script:
/usr/local/psa/admin/sbin/spamtrain

This script could be executed as frequent as it is required.

 

[Sven L.]

Current Plesk daily maintenance script have to be executed every 24h, since it contains connected, time-oriented tasks.

As an option, sa-learn task could be executed individually, using this script:
/usr/local/psa/admin/sbin/spamtrain

This script could be executed as frequent as it is required.

awesome. this is exactly what i needed. can't wait to get home to work on it.

 

[Sven L.]

@Alexey.Plotnitsky

I have done some test last days and also looked more closely at those scripts.

all in all it looks all great, but there is ONE thing that could lead to problems.

your script learns HAM from folders like inbox, which under ideal circumstances is great...

however, there do exist a lot of users that just receive their email through pop3 (and leaving copy on the server to be able to read them on the smartphone too), never accessing the webmail or imap, thus not moving emails into the appropiate folder
and by doing this, the script would constantly learn bad -notmoved- emails from the inbox and give them a higher HAM value than they should.
in the long term, this would screw the database and the user would receive more and more spam badly clasified

so, my question is: is there a way, to disable the script for certain users, without totally disabling the antispam feature?





PS: i do encourage my users to use imap, at least on the smartphone... but not everyone will do that

 

[Alexey.Plotnitsky]

Hi Sven L.,

Re: "disable the script for certain users, without totally disabling the antispam feature?"
This is not possible right now.
We will revert on this.

QUESTION: Is it acceptable if SA will not learn on a mail-user's emails in case if Anti-SPAM is OFF for his/her emails?

 

[Sven L.]

QUESTION: Is it acceptable if SA will not learn on a mail-user's emails in case if Anti-SPAM is OFF for his/her emails?

wait wait wait... are you actually telling me that your sa-learn script learns even if the user has the antispam setting OFF?

if i turn the antispam options off, i would expect to everything being disabled, including the sa-learn script for that user.

 

[Alexey.Plotnitsky]

wait wait wait... are you actually telling me that your sa-learn script learns even if the user has the antispam setting OFF?

Yes, it seems that sa-learn script executed regardless of current mail-user's antispam setting value.
It will be resolved in next Plesk updates, here is internal id: PPP-4291

In order to turn of sa-learn script for certain mail-users with antispam = ON, one should manually adjust the script
/usr/local/psa/admin/sbin/spamtrain

 

[Sven L.]

Yes, it seems that sa-learn script executed regardless of current mail-user's antispam setting value.
It will be resolved in next Plesk updates, here is internal id: PPP-4291

In order to turn of sa-learn script for certain mail-users with antispam = ON, one should manually adjust the script
/usr/local/psa/admin/sbin/spamtrain

well, if it will be fixed in a (near) future update, i will not edit the script. it's not that urgent

 

[schlimpf]

Has this issue already been fixed?

 

[Alexey.Plotnitsky]

Hi schlimpf,

if you've meant "sa-learn script executed regardless of current mail-user's antispam setting value.", then - NO.
It is still in work.