• Halloween is coming! To celebrate the occasion, we dusted off a couple of old skins, and even created a brand new one (Pixel).
    To get into the festive spirit, install the Skins extension.
  • The new Python extension is now available. It allows customers to deploy and manage WSGI-based Python applications on their websites directly from Plesk.
  • The Plesk Mobile apps for iOS and Android and the Plesk Mobile Center extension have been deprecated, and are no longer available for new installations. In Plesk Obsidian 18.0.84, the Plesk Mobile Center extension will be removed from servers during the upgrade, and the Plesk Mobile apps will lose their server management features.
  • The Social Login extension has been deprecated and will be removed in Plesk Obsidian 18.0.84. After that, it will no longer be possible to log in to Plesk with a Google, Facebook, GitHub account, or with a passkey. Plesk usernames and passwords, as well as logging in with a WebPros Account, are not affected.

Issue Incoming Zoho Campaigns emails fail DKIM: psa-pc-remote removes a space from a folded Content-Type header

ollosolucions

New Pleskian
Server operating system version
AlmaLinux 8.10 x86_64
Plesk version and microupdate number
18.0.82.1
Hello,

I am investigating an issue where incoming emails from Zoho Campaigns consistently end up in the Spam folder on a Plesk server.

After checking DNS, verifying the signatures locally, and capturing the Postfix–milter communication, I traced a whitespace change to the processing performed by psa-pc-remote or one of the mail handlers it invokes.

Has anyone encountered this behavior, or is there a known bug or supported fix?

Environment​

Code:
OS: AlmaLinux 8.10 x86_64
Plesk Obsidian: 18.0.82.1

plesk-mail-pc-driver:
18.0-2.redhat.8+p18.0.82.0+t261015.0936.x86_64

psa-mail-driver-common:
18.0-2.redhat.8+p18.0.82.0+t261015.0936.x86_64

Postfix:
3.5.28-2.redhat.8+p18.0.82.0+t261013.0859.x86_64

sendmail-milter:
8.15.2-34.el8.x86_64

Incoming mail follows this pipeline:

Code:
External SMTP sender
↓
Postfix / Plesk milter (psa-pc-remote, TCP 12768)
    ↓
Amavis (TCP 10024)
    ↓
Postfix reinjection (TCP 10025)
    ↓
Plesk DKIM and DMARC delivery handlers
    ↓
Dovecot

Symptoms​

Affected messages contain two DKIM signatures: one for the sender’s domain and one for a Zoho domain. Both use:

Code:
a=rsa-sha256
c=simple/simple

On the VPS, both signatures fail:

DKIM verification (...) failed: signature verification failed

SPF passes for the Zoho envelope sender domain, but it is not aligned with the visible From domain. As a result, the DKIM failure also causes DMARC to fail:

Code:
align_dkim=fail
align_spf=fail
spfres=pass
dkimres=fail
dmarccheck=DMARC_POLICY_QUARANTINE

Dovecot subsequently records:

stored mail into mailbox 'INBOX.Spam'

In the original message investigated, Amavis allowed the message through a whitelist, but the subsequent Plesk DMARC handler still caused delivery to Spam.

A separate recipient copy of the same campaign, delivered directly from Zoho to Microsoft 365 without passing through this VPS, passed both DKIM signatures and DMARC. This was a different recipient-specific message, not an identical copy of the VPS message.

Local cryptographic verification​

For an affected message stored on the VPS:
  1. Both calculated DKIM body hashes match the corresponding bh= values.
  2. The sender-domain DKIM TXT record matches between the local resolver, a public resolver, and both authoritative DNS servers. The Zoho DKIM record also matches between the local and public resolvers.
  3. Both signatures fail when verifying the stored message.
  4. Restoring exactly one space in the folded Content-Type header makes both original signatures pass using Mail::DKIM.
The stored header has this structure:

b'Content-Type: multipart/alternative;\r\n\tboundary="..."'

Both signatures validate when the header is changed, in memory only, to:

b'Content-Type: multipart/alternative; \r\n\tboundary="..."'

The only difference is the single space after the semicolon, immediately before the folded line break.

No body changes, DNS changes, replacement signatures, or other header changes are required.

Packet capture: the space is present when Postfix calls the milter​

I captured the loopback TCP conversation between Postfix and psa-pc-remote on port 12768 during a new Zoho Campaigns test delivery.

The relevant connection contained exactly one SMTP MAIL transaction.

After reconstructing the TCP streams and parsing the binary milter frames, the Content-Type value passed from Postfix to the milter was:

b'multipart/alternative; \n\tboundary="..."'

The space is present at this point.

The milter then returned:

Code:
Command: SMFIR_CHGHEADER ('m')
Header: Content-Type
Index: 1
Value: empty

This deletes the existing header.

Later in the same response stream, it returned:

Code:
Command: SMFIR_ADDHEADER ('h')
Header: Content-Type

With this value:

b'multipart/alternative;\n\tboundary="..."'

The space has disappeared.

The milter deleted and re-added multiple headers during this transaction. It also issued a body-replacement operation, although that operation alone does not establish that the body content changed.

The \n sequences shown above are the line breaks observed inside the milter header values. The stored-message comparison uses reconstructed SMTP \r\n line endings.

Amavis input/output comparison​

In a preceding test delivery, I also captured the message on:

Code:
TCP 10024: Postfix → Amavis
TCP 10025: Amavis → Postfix

For the same test Message-ID on those two ports, Content-Type already lacked the space when entering Amavis and retained the same format on output.

Therefore, Amavis was not the component removing that space in that test.

The later milter capture located the change more precisely: Postfix supplied the space to psa-pc-remote, and the milter requested that the header be re-added without it.

Relevant configuration checks​

The following global settings are empty:

Code:
header_checks =
body_checks =
milter_header_checks =
smtp_header_checks =
smtp_mime_header_checks =
smtp_nested_header_checks =
smtp_body_checks =
canonical_maps =
sender_canonical_maps =
smtp_generic_maps =

The incoming SMTP milter is:

smtpd_milters = , inet:127.0.0.1:12768

The listener on that port was confirmed to be psa-pc-remote.

The Plesk handler list includes the incoming DKIM verification handler followed by the DMARC handler in the before-local queue. No obvious custom header-rewriting handler was found.

Current conclusion​

The captures show that the space is removed during processing by psa-pc-remote and/or a mail handler it invokes.

For the historical message examined, restoring that one space is sufficient for both DKIM signatures to validate.

I have not yet identified whether the normalization occurs in the milter’s message reconstruction code or in a specific Plesk mail handler. I have also not established when this behavior started, so I cannot attribute it to a particular Plesk update.

No mail configuration changes, service restarts, or handler disabling have been performed during the investigation.

Questions​

  1. Is this whitespace normalization a known issue in psa-pc-remote or the Plesk mail-handler framework?
  2. Is there an existing bug ID, supported hotfix, or release that addresses it?
  3. Is there a supported way to preserve the original whitespace in signed headers without disabling the Plesk milter or globally weakening SPF, DKIM, or DMARC checks?
  4. If further diagnostics are needed, which targeted logs or traces would help identify the responsible internal component?
This is a production server hosting multiple domains, so I would prefer a targeted, supported solution rather than disabling mail security globally or adding a blanket header-rewriting rule.

Packet captures are available privately, but they may contain sensitive email data and will not be posted publicly.

Thank you.
 
What does the actual full header of the email look like?

And since you indicate you're using amavis, can your confirm using the Plesk Email Security extension?
 
Hi ollosolucions,

very nice analysis, the packet capture is exactly the kind of evidence support needs. A few thoughts, and please take them as hints, not as confirmed facts:

1) Why only some mails are hit
Zoho signs with c=simple/simple, and with "simple" the header must stay byte-identical. That includes a space before a folded line break. With c=relaxed (what Gmail, Microsoft and most others use) the whitespace is normalized before hashing, so "; \r\n\tboundary" and ";\r\n\tboundary" produce the same result and the signature survives. So you will only see this with senders that use c=simple AND a Content-Type that is long enough to be folded. That fits your symptoms perfectly, and it would explain why it went unnoticed for a long time. It would be interesting to know if other c=simple senders with folded headers fail the same way on your server.

2) Possible relation to an older bug
There is an old thread here about forwarded mails failing DKIM when the attachment had a long (wrapped) Content-Type, while attachments with a short one like "application/pdf;" were fine. According to that thread it was fixed in Plesk Obsidian 18.0.59 (ticket PPPM-14166, only listed in the changelog). Same symptom class, header re-written during processing. I can't tell if it is the same code path, but if it is, you are on 18.0.82.1 and this could be a regression. Mention that ticket number when you contact support.

The older thread I mentioned is this one:

The post saying it was fixed in Plesk Obsidian 18.0.59 (ticket PPPM-14166) is here:

Note that the ticket number is only listed in the public changelog, and I can't say for sure that it's the same root cause as yours. It just looks like the same symptom class (a wrapped Content-Type header getting re-written and breaking DKIM), so it's worth mentioning to support.

3) Kaspar's questions
Kaspar asks for the full header and whether you use the Email Security extension. For the first one it would help to post the raw header block of an affected mail (anonymized), especially the Content-Type lines, both DKIM-Signature headers (c=, h=, bh=) and the Received lines. For checking what happens to one message on your server, take the queue ID of a test mail and look at it in the log:
Code:
grep "<queue-id>" /var/log/maillog

and to see which handlers are active on the server:
Code:
plesk bin mailhandler --list

4) What I would do now
- Open a Plesk support ticket with your two captures (support can look at them privately, so no need to post them here), the exact package versions you posted, and the reference to PPPM-14166. A reproducible case with exact byte offsets is rare, that should speed things up a lot.
- As a temporary and targeted measure, there is a KB article about excluding a single domain from the DMARC check ("exclude a domain from DMARC check in Plesk for Linux"). You would only do that for the sender domains you trust, so the global DKIM/DMARC protection stays on. I haven't tried it on your version, and one user reported in another thread that it didn't work for him on 18.0.56, so please test it with one domain first and keep the old setting noted down.
- Please don't disable "Verify incoming mail" for DKIM as a workaround: DMARC would then see no DKIM result at all and the mails would still land in Spam (or get dropped, depending on the sender's policy).

Please keep us posted, I'm curious what support says.

Regards
Tessa
 
Back
Top