451 Error: queue file write error (only when using postfix)

[IgorG]

There are a lot of possible reasons of this issues. Support can perform deep investigation and can try to find a reason at least.

 

[philippe henneau]

since 24h not a single error,all email services OK, 7 containers without any problems! How to explain those 'waves'?

 

[EdwardG]

I think I found it!

For several years, the Parallels Plesk Panel, running Postfix, has been plagued with a problem where the server suddently starts sending out a rather useless error message:

-----------------------------------------------------------------
Transcript of session follows.

Out: 220 xxx.yyy.net Postfix (Ubuntu)
In: EHLO exprod7mo102.postini.com
Out: 250-xxx.yyyt.net
---snip---
Out: 250 DSN
In: MAIL FROM:<xxxxxxxxxxxxxxxxxx> SIZE=1038
Out: 250 2.1.0 Ok
In: RCPT TO:<xxxxxxxxxxxxxxxx>
Out: 250 2.1.5 Ok
In: DATA
Out: 354 End data with <CR><LF>.<CR><LF>
Out: 451 4.3.0 Error: queue file write error
In: QUIT
Out: 221 2.0.0 Bye

Session aborted, reason: lost connection
-----------------------------------------------------------------

My observations indicate that this is coming from the Postfix "Before Queue Filter."
http://www.postfix.org/SMTPD_PROXY_README.html states, among other things, "For privacy reasons, all the remote SMTP client sees is "451 Error: queue file write error". It would not be right to disclose internal details to strangers."

The Postfix before-queue content filter, when running under Plesk, is /usr/lib/plesk-9.0/postfix-queue. This is true even in Plesk 10. postfix-queue is a compiled binary and offers no help. It's job though is pretty well outlned in the SMTPD_PROXY_README file mentioned above.

In this case, the filter is returning 451 queue file write error, which is what postfix specifies in the readme, but the definition of a 400 series error is that it's temporary, and that the sender should retry. So it does.

I removed the before-filter from one of my servers, and one of the messages that had been retrying over and over again then delivered. I was able to get a look at it. What I saw was a little message with a binary mime attachment, clearly spam or malware. The content-filter was quite right in refusing it.

This would be cured if the postfix-queue program would return a 500 series error on the message. After all, the content filter isn't going to accept it next time around, so why retry it.

Anyway, the only solution that I see for us at the system administration level, is to remove the before-content-filter. This is accomplished by editing /etc/postfix/master.cf as follows:

Find this line: smtp inet n - - - - smtpd -o smtpd_proxy_filter=127.0.0.1:10025
remove everything after smtpd. (remove the -o and what follows)

Then restart postfix with /etc/init.d/postfix restart (or similar)

 

[Kewler]

I can't wait for no more 451 Error: queue file write error

EdwardG,

I used to know a very bright stealthy EddyG. Nice disassembly. I have been close, but the proxy trail was a brilliant find.

 

[philippe henneau]

by far the most interesting analysis, great job!
What would be the impact of removing stmpd_proxy_filter? Will spamassassin and Drweb still be in the loop?

NB: do you see any explanation to our observations: we had a dozen of containers running plesk for almost 1 Year, we upgraded Virtuozzo 4 to 4.6 and suddenly each VPS running postfix start sending out this messages. This did last for about 5 days and it suddenly stopped 3 days ago, all SMTP servers silent (and running OK). I wonder if there is not a link with Virtuozzo and I would be much interested to know if someone running Plesk directly on an HW node has ever experienced this issue with Postfix.

Regards,

Philippe

 

[EdwardG]

Hi Philippe,

As a parallels customer, I can only speculate: I think that parallels uses their postfix-queue filter to insert such things as spamassassin into the processing of mail, so removing postfix-queue would probably break those things.

Of course, many non-Parallels installations run spamassassin, clamav, DrWeb, etc, so they should all come with their own instructions on how to integrate them to postfix (without using postfix-queue).

In my case, all my servers are behind Google Postini, so I don't rely on spamassassin or any mail prefiltering on the server. The original document discussing the use of the before-filter states that servers with good pre-processing don't really need any pre-filtering.

Regarding the change happening with Virtuozzo, I don't think there is a connection. The problem of mail that triggers the prefiltering does indeed come and go, usually when certain bulk emails get into the wild. I believe that our error message flood would have abated at the same time yours did. When something new comes along, you'll have another flood, as the same messages are rejected over and over until they time out five days later.

Best,
Ed G.

 

[RitardoB]

kb.odin.com/en/6074
It has some fixes you need to apply to the box. This might be whats going on if this was happening to messages with attachments.