Question Possible to use DNSBLs for moving identified email as spam to spam folder (not blocking)?

[tramvainqueur]

Server operating system version

Ubuntu 22.04 x86_64

Plesk version and microupdate number

Plesk Obsidian 18.0.62.2

Is it possible to use one or more DNSBLs, which does not block listed email as spam but having it moving to spam folder solely?

So I and the other users do not have trust to the dnsbl-creators fully and we all can look from time to time if a ham was marked falsely as spam. This would be perfect - so I would not have the fear of sometimes blocking an important email without knowing this -, but I do not know if spamassassin or plesk can make this possible. Is it? And if yes, an advice or solution how to implement this?

 

[danami]

@tramvainqueur Yes this is possible. Take a look at the "Enabling DNSBLs at the content filter level" section from the KB article below. The article was written for our Warden product but it should work the same for the default spamassassin that Plesk provides:

How can I add custom DNSBLs for use in Warden Anti-spam and Virus Protection? - Knowledgebase - Danami

 

[tramvainqueur]

@tramvainqueur Yes this is possible. Take a look at the "Enabling DNSBLs at the content filter level" section from the KB article below. The article was written for our Warden product but it should work the same for the default spamassassin that Plesk provides:

How can I add custom DNSBLs for use in Warden Anti-spam and Virus Protection? - Knowledgebase - Danami

Ok, I tried to implement changes by means of your suggestions and now I have to wait some daysto see if my change works as expected. Btw. I also implemeted because of a suggestion by another site the change of the score of a dnsbl used in spamassassin because several newsletters from the well known german news site get because of spamhaus a much to high score and in several real spams I did not find this dnsbl helping to identify it. It was sad becase I had to set the required score to 2.6 only for this german newsletters. Else the requiring score 0.7 helped to eliminate almost all spams.

If it works as expected, I will write down in net post the details what I have done. It would be great, if Plesk would make it possible to add the possibility to add different dnsbl, dnswl and other or to change scores of some lists and not to block solely.

Btw. @danami : For the advanced edit of spamassassin you recommend to edit /etc/mail/spamassassin/local.cf. But there is a probem with some systems (one type: systems using Plesk). That is a reason why Plesk makes me sometimes some headaches. After updates of Plesk and/or some things on the server, this local.cf can be overwritten (thankfully a warning in the text file itself advices to not edit this file manually). So it is better to make and edit /etc/mail/spamassassin/custom.cf or whatever name is better, so this file is permanent. I hope this is in conformance to Plesk, sufficient and does not require to include additional directive(s) in another place(s). Several pages I found think this is sufficient. At least command 'spamassassin --lint -D all 2>&1 | less' tells that the new cf-file seems to be read.

I hope all I changed is good, but as I am not sure, I wait and will analyze some days later some spams, so I can ensure at least empirically that this shall work now and in future. If I find in some spam-email headers my own directives, then I know it worked and I will tell here what I have done to achieve this.

Btw. @danami & @Kaspar@Plesk & @pleskpanel : As read on several sites for spamassassin ist shall be better to use bb.barracudacentral.org instead of b.barracudacentral.org. Plesk uses spamassassin, so I think this is more appropriate.

 

[tramvainqueur]

[...] Btw. I also implemeted because of a suggestion by another site the change of the score of a dnsbl used in spamassassin because several newsletters from the well known german news site get because of spamhaus a much to high score and in several real spams I did not find this dnsbl helping to identify it.[...]

Addendum: Interesting in this case someone can change the scores of ratings which seem to come from DNSBL's, this showed me that spamassassin has per default already some DNSBL's configured (they do not block, just move to spam folder). That is why spamassassin at first without manually configuring it works relatively good (not perfect, but not bad either).

After a half day some spams & hams indicate that my implementation seems to work. But I will test it more to ensure good working and then tell what I have done. For example to edit local.cf is bad as this is not permanent for a server with Parallels Plesk. It is sufficient to create a file with endning '.cf' like 'custom.cf'. No include is needed. Source codes of spams showed me that my implementation of directives with DNSBL's for moving are obeyed.

 

[danami]

@tramvainqueur Plesk does not overwrite /etc/mail/spamassassin/local.cf.

 

[tramvainqueur]

@tramvainqueur Plesk does not overwrite /etc/mail/spamassassin/local.cf.

Good to know. Is this warning useless? Or is the reason a distribution upgrade of Ubuntu/Debian overwrites this?

But nevertheless imho it is cleaner using a separated config file for custom changes, or?

 

[tramvainqueur]

As promised I try to report what I have done and worked, but first some preliminary informations.

I am owner of several personal domains with different TLD's. One has been registered in the year 2000 and the mail-address became in about 2015 round about 180 spams per day (yes, in the year 2000 spams were no problem in germany, so I wrote the address several times in the clear net). But such things like DMARC or components had not been configured and did not exist in these early times.

Some data about the email daemon on the server I will talk: It runs the newest Parallels Plesk Management System. There I installed this Plesk Secure Mail Extension (for integration with spamassassin), but I have not an abonnement activ (is a 'little' private server with some customers) and use just only the free features of this extension (one reason: it installs amavis and makes automatically a connection with spamassassin). This is why some network configurations for email are not the same and perhaps it is the reason, why spamassassin did not learn from new spams until I integrated this manually as cron job, but here it does not matter anything.

My realization with help of @danami what helped "my dream" to come true, that DNSBL's do not just block possible spams but move them to the spam folder, so anyone can control if spam filter worked and are able to move false positives back to an ham folder/inbox or folder in inbox without spams (but from my experience with good configuration there is never a false positive in the spam folder ... only sometimes spam in inbox but not much):

I realized enabling the so called DNSBL's (normally wrong word as 'BL' means 'block list' but with spamassassin possible spams are not blocked ... just moved to spam folder and perhaps learned by the spam filter ...) at the content filter level what website calls 'command line interface'. Yes, it is not that easy like with Plesks Server Management per website, but it gives you really much more options, so one may even deactivate a default 'DNSBL' (for example I reduced the score of 'URIBL_DBL_SPAM' of spamhaus.org, because it produced really many false positives, although some test sites allege it produces no false positives but I think it is because of bad test data ... I am sure their test data does not include mails in different anguages ... my false positives arre all not english (german, spanish, french, italian, portuguese and more to be detailed) ...). That is why I heavily recommend to do this per shell on the server in a terminal (for windows users called 'command line interface' although not powerfull ... microsoft power shell is more complex and better than the often used shell bash in the POSIX systems like unix, linux and so on, but that complicated that almost no one uses this ) directly or with SSH, if possible and not damned to use only the Plesk Server Management Interface.

Step 1: Login with SSH in to the server (I assume server runs with a linux or unix distribution ... with windows servers I do not have experience).

For linux, unix or posix desktops reaching linux, unix or posix servers for example following in terminal (for windows users it shall be similar with putty possible):

ssh [email protected]

Step 2: Go to folder '/etc/mail/spamassassin/'
Step 3: Create an empty text file with the ending '.cf' like 'custom.cf' to add your own rules. I do not recomend to edit 'local.cf' a in this file is written

ATTENTION!
#
#DO NOT MODIFY THIS FILE BECAUSE IT WAS GENERATED AUTOMATICALLY,
#SO ALL YOUR CHANGES WILL BE LOST THE NEXT TIME THE FILE IS GENERATED.

. @danami says this file is not ovewritten by plesk, but I assume this text is not senseless. I think with an update or upgrade of spamassassin or the distribution itself this file will be overwritten and the own changes will be lost. By the way it is imho cleaner to separate custom modifications from other modifiers.

Step 5: Write in 'custom.cf' DNSBL's with text like

header          C_RCVD_IN_BARRACUDACENTRAL  eval:check_rbl('barracudacentral-lastexternal','bb.barracudacentral.org.')
describe        C_RCVD_IN_BARRACUDACENTRAL  Listed in bb.barracudacentral.org DNSBL
tflags          C_RCVD_IN_BARRACUDACENTRAL  net
score           C_RCVD_IN_BARRACUDACENTRAL  0 # please adjust the score value

.

Little explanations: First column is a must (not always the rows as when only a score shall be different ... later more ...). From statistics of some dnsbl's with spamassassin description in details I take the original names like 'RCVD_IN_BARRACUDACENTRAL' and can be modified without problems (I add 'C_' at the beginning for indicating 'custom' and this change has to happen in every row at the same column. In the last row one can set the wished score (0 deactivates this 'dnsbl').

Step 4: Execute in the shell '/usr/bin/systemctl restart amavis' on ubuntu server with amavis installed and configured (on other sever something similar that restart spamassassin). Now the new '.cf'-file is respected by spamassassin and no special includes in other config files are needed. With 'spamassassin --lint' one can test in forwars if rule is accepted, correct the rule if needed and the restart amavis or spamassassin.

Helpful for emails with foreign language so spamassassin has much less or no false positives: Just add

score URIBL_DBL_SPAM 1

(or lower score like 0.5 or higher score like 1.5) to 'custom.cf'

 

[tramvainqueur]

addendum:

I have reread the posting and found some mistakes. But as I see, they shall not hinder to understand my realization. As I had to end earlier as thought, I missed to correct some things and I did not write what the result is.

Result is generally spoken good, so without paying 'email experts' one can achieve that only every pair of days a spam appears in the inbox and no ham/false positive appears in the spam folder.. And the best is, no full trust in these 'dnsbl's is needed, as with spamassassin no email is blocked without informing someone.



Details (now since one week - spamassassin not to my wish perfect configured yet ... will take perhaps some weeks for testing new 'dnsbl's and fine adjustments ...):

usecase 1 (me): has many email addresses on several own domains for over a decade; trys to be always correct (useless subscripted emails never moved to spam but derigestered or simply deleted (else spamassassin will move many legitim emails to the spam folder); real spam moved to spam folder and learned by spamasssassin; on many mailing lists and registered many news letters (some because of a good reason I will not tell here are very old and do not even remember all places); has several account with intel, oracle, lenovo, ibm, affinity, maple, mathworks, matlab, several shops and much more

usecase 2 (account from another one where I am allowed to have full rights and I am their private sys admin): uses account conservatively (account on server newly, therefore spamassassin learned from spams only some years); registered address on some news letters and mailing lists; has some accounts in several shops

usecase 3 (account from another one where I am allowed to have full rights and I am their private sys admin): uses account very freely and has no fear to announce it publicly (account on server newly, too, therefore spamassassin learned from spams only some years); registered address on many news letters and maling lists; has several accounts (most of them not known by the user itself ... ... ok, is not tech affine and has a little aversity against these things, but trys to use them ...); I think address is posted in the clear net on several sites

Results:

usecase 1: round about after a pair of days a spam appears in inbox and no false positive in spam folder (although with all email accounts I get more about 80 spams on main address and about 120 spams all other addresses counted together, but most spam is filtered)

usecase 2: round about 1-2 spams per day (there are days with 3 and more spams, but also many days with 1 and some even with 0 spams)

usecase 3: round about 2-5 spams per day (here back again some days more and on some days less ... I think days with 0 spams I did not find yet ...)

Indeed usecase 3 is my little problem child yet, but the 'dnsbl' is not fully configured yet and surely more fine adjustments are needed. And perhaps some spam fighting technics are easily addable. Sadly the user itself is a little problem, too. Some spams in inbox have been learned as ham accidentaly, as user does not strictly sort out spam from inbox. This user thinks it should just work and does not try much to keep the inbox clean. But as this configuration type is relatively fresh, there is much hope, yet, that these problems will vanish or at least diminish significantly.

Now it is late and I have to go to sleep, so I hope this text has not too many mistakes which hinder to understand this post. If something is not understandable, then please ask me and I will try to explain it in another way.

 

[tramvainqueur]

Little update: The first 3 days indicate that my little updates in 'custom.cf' and score/'fine' adjustments have been already sufficient to achieve that usecase 3 and 2 are at least that good as usecase 1, or perhaps even better, but no complete week after the week with first configuration has past yet.

 

[tramvainqueur]

Results (which will vary/surely get better in the future if spammers are not faster getting better):

usecase 1: Now after third week I get about 0.28 spams per week in inbox. About 0.28 hams appear in the spam folder, but spamassassin learns with my rules from time to time (so hams in spam folder are in next week not in spam folder anymore). Not perfect but I can live with it. With my adjustments I reduced the false positives almost completely - before I had several false positives per day(!). How good spamassassin works seems to depend from user. Although I have about 5 'private' (also owner admin-c, but not used solely privately!) domains and use about 7 email accounts (and much more with aliases ... about 70 as I see ... yes, a lot are not actively used and many are for administrative reasons created, but about a third I use them actively ...) I am the usecase with the least spams in inbox and the least false positives. The other usecases have only one account, one domain and only some alias (mostly administrative reasons).

usecase 2: It becomes now about 3 spams per week in the inbox. It became better because of my modifications (spamassassin sets per default scores some lists and to email properties and it seems to be empirically tested with a spam database, but this database seems not to have not much international emails).

uscase 3: It becomes 1-2 spams per day. False positves are about 0.28-5 per day. Why 0? From usecase 3 I naturally do not know in which mailing lists, newletters or organizations it registers. Analyzing the so called spam some appear superficially spam but further analysis told me they are indeed hams. But some of the legal emails contain only ads which do not have to do much with the organization. So for these emails I do not costumize the configuration for spamassassin as they are not worth it. Usecase 3 can contact me and tell me if they are worth indeed but this did not happen yet (I even asked usecase 3 and it told me they are not important or needed). At least looking in spam folder can help. And I am sure if some of these strange false positives are moved to inbox, in near futere they will not appear again as false positives.

Lastly, result is not perfect (but 'almost'!) and has potentials to get better yet but it is very much better now (at least for international users). Phantastical is now that you can 'play' with spamassassin configuration and one does not have to have fear some important emails are blocked radically without being informed. If false positves happen they appear in spam folder now and can be retrieved if in less than 30 days this spam folder is examined. Back again I thank @danami for poking me to this method (which I extended a bit) because now I could reduce really many spams. I hope the team of Parallels Plesk makes it possible to change 'local.cf' and to be able to edit with a custom CF-file easily within the server management web interface.

Some usernames to hopefully become additionnal attention for this: @admin @Aleksei Stepanov @Bitpalast @alapshin

 

[tramvainqueur]

Little update:

Perhaps because of my additionnal cronjobs with sa-learn in folders '.spam' with command '--spam' and in folder '.archive' with command '--ham' spamassassin seemed to have learned fast (and the usecases followed my advice moving false positives to '.archive'). Last week I got even 0 spam in inbox and 1 ham in spam folder, whereas this unique false positive is from a not important mailing list (and perhaps spamassassin learns in future these shall not go to spam).

This was usecase 1. Usecase 2 did not change much. Usecase 3 got better, too. There seem to be some false positives in '.spam', whereas from some of the some I do not know if they are spam, indeed, because I do not know in which mailing lists usecase 3 registered or typical spam that alleges to be from a mailing list and the not working 'remove link' just tells the spammer this email box lives.

The possibility of avoiding all blocking and then being able of fine tuning the configuration of spamassassin, was a really great game changer!