is SPF working as intended?

[Sven L.]

plesk 11.5.30#17 centos 6.4

* my domains all have the proper DNS TXT SPF record ending with -all (hardfail)

* in plesk SPF antispam protection i have it set to hardfail

yet i keep receiving fake emails from [email protected] to [email protected] and the email headers say "softfail" instead of "hardfail"

what's going on here?

 

[IgorG]

Have you checked Plesk maillog? Do you have record like "Error code: (2) Could not find a valid SPF record" there?

 

[Sven L.]

plesk maillog all i see is:

Oct 8 20:00:34 vps01 spf filter[25756]: Starting spf filter...
Oct 8 20:00:34 vps01 spf filter[25756]: SPF result: softfail
Oct 8 20:00:34 vps01 spf filter[25756]: SPF status: PASS

and a bit later:

Oct 8 20:00:34 vps01 spamd[23548]: spamd: result: . 6 - BAYES_60,CK_HELO_DYNAMIC_SPLIT_IP,HELO_MISC_IP,RCVD_IN_BRBL_LASTEXT,RCVD_IN_XBL,RDNS_NONE,SPF_SOFTFAIL,TVD_RCVD_IP scantime=0.2,size=16824,[email protected],uid=30,required_score=7.0,rhost=localhost,raddr=127.0.0.1,rport=50691,mid=<0WX1B47HOSGRA2JXDR6NOHHJ2EQ17XG7@857-SN2MPN2-819.828d.mgd.msft.net>,bayes=0.708091,autolearn=no


i also wanted to note about this post: http://forum.parallels.com/showthread.php?290998-sned-spam-to-spam-folder-no-headers
where Alexey.Plotnitsky confirmed that there is a bug that when you use the option to "send spam to spam folder" is enabled, the spam headers are not written properly.

as in this case this option is on, maybe this bug is causing the SPF problem too?


Sven L. / QWERTY

 

[IgorG]

I suggest you check this article too - http://kb.parallels.com/11152

 

[Sven L.]

I suggest you check this article too - http://kb.parallels.com/11152

i will follow those steps and post results, but meanwhile, so you see there is NO problem with my headers:

checked with mxtoolbox.com:

spf:MYDOMAIN.COM
Prefix Type Value PrefixDesc Description
+ a Pass Match if IP has a DNS 'A' record in given domain
+ mx Pass Match if IP is one of the MX hosts for given domain name
- all Fail Always matches. It goes at the end of your record.


also, from the headers of the emails in question:

Received-SPF: softfail (vps01.qwerty-solutions.com: transitioning domain of aexp.com does not designate 69.168.245.33 as permitted sender) client-ip=69.168.245.33; [email protected]; helo=69-168-245-33.brainerd.net;


i see NO reason whatsoever why those mails should be considered as softfail instead of hardfail. only thing i can suggest is that the bug that i mentioned above is affecting the spf antispam too

 

[Sven L.]

here are the results of the tests you asked for:

1) DNS entry in plesk panel on that domain:

mydomain.com.	TXT	v=spf1 +a +mx -all

2) first shell test: (the IP used is the IP from the real server that sent me those spams)

[root@vps01 ~]# /usr/bin/spfquery_static -ip 69.168.245.33 -sender [email protected] -rcpt-to [email protected]
failneutral
Please see http://www.openspf.org/Why?id=fakemail%40mydomain.com&ip=69.168.245.33&receiver=spfquery : Reason: default
spfquery: 69.168.245.33 is neither permitted nor denied by domain of mydomain.com
Received-SPF: neutral (spfquery: 69.168.245.33 is neither permitted nor denied by domain of mydomain.com) client-ip=69.168.245.33; [email protected];
[root@vps01 ~]#

3) following the link generated by that test brings me to the openspf webpage explaining why mydomain.com rejected a mail from 69.168.245.33

spfquery rejected a message that claimed an envelope sender address of [email protected].
spfquery received a message from 69-168-245-33.brainerd.net (69.168.245.33) that claimed an envelope sender address of [email protected].

However, the domain mydomain.com has declared using SPF that it does not send mail through 69-168-245-33.brainerd.net (69.168.245.33). That is why the message was rejected.

4) next test suggested in your KB link:

[root@vps01 ~]# dig -t TXT mydomain.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> -t TXT mydomain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21764
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;mydomain.com.                 IN      TXT

;; ANSWER SECTION:
mydomain.com.          85840   IN      TXT     "v=spf1 +a +mx -all"

<rest deleted for privacy>

and next:

[root@vps01 ~]# dig -t SPF mydomain.com

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>> -t SPF mydomain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61054
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;mydomain.com.                 IN      SPF

;; AUTHORITY SECTION:
mydomain.com.          10123   IN      SOA     ns.mydomain.com. mydomain.gmail.com. 1379887287 10800 3600 604800 10800

ok what do we see here? we see 2 things
A) dig doest really recognize my TXT record? bu www.mxtoolbox.com does?
B) what is this entry: mydomain.gmail.com and where does it come from? certainly not from my DNS records

and then there is this final text in your KB

There is no SPF or SOA. This means that the DNS server is claiming it knows nothing about mydomain.com. Since this is the DNS server that is responsible for the domain name, there is no one else to ask. Therefore, it is considered a DNS query error, and libspf2 handles it as such.

so...
- my DNS TXT entries are correct
- external tools like mxtoolbox.com seem to detect my SPF properly


what's going on here, what's wrong?

 

[IgorG]

You can try to debug this problem more deeply. Try to do following:

1. /usr/local/psa/admin/bin/mail_handlers_control --remove --name=spf --type=global --queue=before-queue
2. /usr/local/psa/admin/bin/mail_handlers_control --add --name=spf --type=global --queue=before-queue --executable='/usr/local/psa/handlers/hooks/spf' --context='debug' --enabled --priority=10

SPF handler will be in debug mode.
After that try to find a reason in maillog. There will be a lot of debug information.

 

[IgorG]

You can try to debug this problem more deeply. Try to do following:

1. /usr/local/psa/admin/bin/mail_handlers_control --remove --name=spf --type=global --queue=before-queue
2. /usr/local/psa/admin/bin/mail_handlers_control --add --name=spf --type=global --queue=before-queue --executable='/usr/local/psa/handlers/hooks/spf' --context='debug' --enabled --priority=10

SPF handler will be in debug mode.
After that try to find a reason in maillog. There will be a lot of debug information.

 

[Sven L.]

before i enable debug mode i'd like to ask 2 things

1) can you please tell me how to turn it off again for when i am done testing?

2) do you know a way so i can actually test this? spf fail mails are rare and several days my pass until i get another one... how could i make one myself?

 

[Sven L.]

also: i have now used around 6 different external SPF checkers, all say the same:

SPF Fail

69.168.245.33 is not allowed to send in the name of the domain.

so, if everyone in the world aknowledges that that IP is not allowed on my domain, why does my own server let it through?

 

[IgorG]

before i enable debug mode i'd like to ask 2 things

1) can you please tell me how to turn it off again for when i am done testing?

1. /usr/local/psa/admin/bin/mail_handlers_control --remove --name=spf --type=global --queue=before-queue
2. /usr/local/psa/admin/bin/mail_handlers_control --add --name=spf --type=global --queue=before-queue --executable='/usr/local/psa/handlers/hooks/spf' --enabled --priority=10 --dont-preserve-on-restore

2) do you know a way so i can actually test this? spf fail mails are rare and several days my pass until i get another one... how could i make one myself?

It is not so easy to simulate necessary SMTP connection manually. Much more easier just to wait next spf fail mail.

 

[Sven L.]

ok, debugging enabled. fingers crossed it doesnt break anything in the process...

 

[Sven L.]

@IgorG

I just sent myself an email from gmail, knowing that this should be a SPF PASS, but I wanted to see the maillog and check how the debug info is showing.

as you said, there is a lot of stuff in there now...

this one, even being an SPF PASS, called my attention. could it be related?

Oct  9 10:41:11 vps01 /usr/lib64/plesk-9.0/psa-pc-remote[1539]: handlers_stderr: Mail handler call failed. Error occured during execv(/usr/local/psa/handlers/hooks/drweb):#012No such file or directory
Oct  9 10:41:11 vps01 /usr/lib64/plesk-9.0/psa-pc-remote[1539]: handlers_stderr: .
Oct  9 10:41:11 vps01 /usr/lib64/plesk-9.0/psa-pc-remote[1539]: Error during 'drweb' handler

 

[IgorG]

Yes, it could be related. Try to rebiuld mailhandlers with help of article http://kb.parallels.com/113664

 

[Sven L.]

Hello IgorG

so finally I got another spoofed email, this time with the debugging ON.

the origin is [email protected]
the sending server seems to be aexp.com
and NONE of all his MX are in my SPF (DNS: v=spf1 +a +mx -all)

so, why on earth does this email clasify as SOFTFAIL when it should be HARDFAIL, which is what my plesk spf antispam is configured to filter?


this is what I got in maillog:

EDIT: jesus is this forum software HORRIBLE at times...
1) need to split the log into 3 to fit
2) detects tags when there are none.


so, HERE: get the full log from this link: [url]https://dl.dropboxusercontent.com/u/21326194/maillog%20extract.txt[/url]

 

[IgorG]

According to this log:

v% dig +short -t TXT aexp.com
"spf2.0/pra a:phxamgw01.aexp.com a:phxamgw02.aexp.com a:sppim501.aexp.com a:sppim502.aexp.com ~all"
"v=spf1 ip4:12.10.219.0/24 ip4:148.173.91.0/24 ip4:203.19.215.67 ip4:192.102.253.34 ip4:192.102.253.35 ip4:192.102.253.36 a mx a:sppim502.aexp.com a:sppim501.aexp.com a:phxamgw01.aexp.com a:phxamgw02.aexp.com ~all"

rules from remote domain aexp.com are taken and applied consistently. At the end this domain of sender is matched to softfail ~all

 

[Sven L.]

IgorG,

the thing is, the mail orginally is delivered from [email protected]

why SPF is asking aexp.com I have no idea.

MYDOMAIN.TLD has this SPF: v=spf1 +a +mx -all

 

[IgorG]

You are wrong. It is not FROM, it is TO. Look carefully at this log:

Oct 16 02:50:27 vps01 postfix/qmgr[25857]: 4995F4FC65: from=<[email protected]>, size=15257, nrcpt=1 (queue active)
Oct 16 02:50:27 vps01 postfix-local[11882]: postfix-local: [email protected], [email protected], dirname=/var/qmail/mailnames
Oct 16 02:50:27 vps01 spamd[5329]: spamd: connection from localhost [127.0.0.1] at port 60483
Oct 16 02:50:27 vps01 spamd[5329]: spamd: using default config for [email protected]: /var/qmail/mailnames/MYDOMAIN.TLD/MYMAIL/.spamassassin/user_prefs
Oct 16 02:50:28 vps01 spamd[5329]: spamd: processing message <[email protected]> for [email protected]:30
Oct 16 02:50:28 vps01 postfix/smtpd[11874]: disconnect from a242-smmi-901.blocka-154.stargate.ca[64.253.154.242]

 

[Sven L.]

IgorG,

I am no expert when it comes to analyze maillog, but I do understand the email headers a bit more...

Return-Path: <[email protected]>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
vps01.myserver.com
X-Spam-Level: *****
X-Spam-Status: No, score=6.0 required=7.0 tests=BAYES_99,RCVD_IN_BRBL_LASTEXT,
RCVD_IN_XBL,SPF_SOFTFAIL autolearn=no version=3.3.1
X-Original-To: [email protected]
Delivered-To: [email protected]
X-No-Auth: unauthenticated sender
Received-SPF: softfail (vps01.myserver.com: transitioning domain of aexp.com does not designate 64.253.154.242 as permitted sender) client-ip=64.253.154.242; [email protected]; helo=a242-smmi-901.blocka-154.stargate.ca;
X-No-Relay: not in my network
Received: from a242-smmi-901.blocka-154.stargate.ca (a242-smmi-901.blocka-154.stargate.ca [64.253.154.242])
by vps01.myserver.com (Postfix) with ESMTP id 4995F4FC65
for <[email protected]>; Wed, 16 Oct 2013 02:50:27 +0200 (CEST)
Received: from Xerox.Device743.MYDOMAIN.TLD (10.0.0.138) by MYDOMAIN.TLD (10.0.0.69) with Microsoft SMTP Server (TLS) id 2RKO89CB; Tue, 15 Oct 2013 16:50:31 -0800
Received: from Xerox5042.MYDOMAIN.TLD (10.37.177.185) by smtp.MYDOMAIN.TLD (10.0.0.181) with Microsoft SMTP Server id E29KC3A3; Tue, 15 Oct 2013 16:50:31 -0800
Date: Tue, 15 Oct 2013 16:50:31 -0800
From: "Incoming Fax" <[email protected]>
X-MS-Has-Attach: yes
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator: <[email protected]>
X-MS-Exchange-Organization-AuthSource: [email protected]
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 02
X-MS-Exchange-Organization-AVStamp-Mailbox: MSFTFF;0;0;0 0 0
X-Priority: 3 (Normal)
Message-ID: <[email protected]>
To: <[email protected]>
Subject: Scanned Image from a Xerox WorkCentre

as you can see, about EVERYTHING in that email is spoofed. please care to explain WHY this is a SOFTFAIL and not a HARDFAIL just because the RETURN-PATH is spoofed and does resolve to a valid server?
SPF should NEVER use the return-path as it can be spoofed and customized by regular users easily in their email software (like outlook)
I think the problem here is that SPF is checking on the spoofed return-path, instead of checking the FROM and SENDING IP

The theory is just so easy:
1) from which domain does it come? MYDOMAIN.TLD
2) from which IP does it come? 10.0.0.138
3) check DNS SPF on MYDOMAIN.TLD, is 10.0.0.138 allowed? NO "-all" = HARDFAIL
4) end of story

obviously, this theory is NOT working. why?

 

[IgorG]

No. Email is really from [email protected]. You can find this fact in maillog a bit above SPF session. Additionally you can see it there too:

Oct 16 02:50:27 vps01 postfix/qmgr[25857]: 4995F4FC65: from=<[email protected]>, size=15257, nrcpt=1 (queue active)

It is not SPF but Postfix thinks that FROM is another.

Oct 16 02:50:27 vps01 postfix/qmgr[25857]: 4995F4FC65: from=<[email protected]>, size=15257, nrcpt=1 (queue active)
Oct 16 02:50:27 vps01 postfix-local[11882]: postfix-local: [email protected], [email protected], dirname=/var/qmail/mailnames

Postfix takes email address from SMTP session and inserts it to the return-path. FROM address in the message header may be different from MAIL FROM: in SMTP session.