FTPS remote backups fail after upgrade to Plesk 18.0.80 due to TLS session not being resumed on the FTP data connection (pmm-ras regression)

[treenet]

Username:

TITLE

FTPS remote backups fail after upgrade to Plesk 18.0.80 due to TLS session not being resumed on the FTP data connection (pmm-ras regression)

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

Product: Plesk Obsidian
Current version: 18.0.80 Update 3
Initial affected version: 18.0.80 Update 1
Last known working version: 18.0.79 Update 5

Operating System: Ubuntu 22.04.5 LTS
Architecture: x86_64 / amd64

Plesk backup package currently installed:

plesk-backup-utilities
18.0-v.ubuntu.22.04+p18.0.80.3+t260813.2046

Remote FTPS server:

FileZilla Server 1.12.6
Implicit FTPS
TCP port 990
Passive mode enabled

The same issue is also occurring on another Plesk server running Windows, although the detailed A/B diagnostics described below were performed on the Ubuntu server.

PROBLEM DESCRIPTION

Remote backups to an external FTPS server stopped working immediately after upgrading Plesk from 18.0.79 Update 5 to 18.0.80.

The FTPS control connection works correctly:

• connection to the server succeeds;
• authentication succeeds;
• PBSZ 0 succeeds;
• PROT P succeeds;
• remote directory access succeeds;
• EPSV succeeds.

The failure occurs when Plesk opens the protected FTP data connection for the LIST command.

FileZilla Server reports:

150 Starting data transfer.
TLS session of data connection not resumed.
425 Unable to build data connection: TLS session of data connection not resumed.
The same FTPS server, account, directory, firewall and passive port configuration works correctly when tested manually from the same Ubuntu/Plesk server with the system curl using implicit FTPS:

curl -vk --ftp-pasv \
-u USER : PASSWORD \
'ftps://FTPS-SERVER:990/REMOTE-DIRECTORY/'

System curl version:

curl 7.81.0
libcurl/7.81.0
OpenSSL/3.0.2
During the successful manual connection, curl explicitly reports:

> LIST
< 150 Starting data transfer.

* Doing the SSL/TLS handshake on the data stream
* SSL re-using session ID

< 226 Operation successful

Therefore, the FTPS server and network configuration are able to correctly perform TLS session reuse on the FTP data connection.

Further investigation isolated the regression to the pmm-ras binary supplied with plesk-backup-utilities.

The following pmm-ras versions were extracted from the original Plesk packages:

Plesk 18.0.79.5
pmm-ras size: 4,299,992 bytes
SHA256:
75c96b8eaf4f89db47b0f2f68d62a356cc906670da0745761ab47e6b3b3939b1

Plesk 18.0.80.0
pmm-ras size: 4,320,600 bytes
SHA256:
acb6fc266f06cb508079ada6c029cbde06b9eff2319ca1b29940134100404e72

Plesk 18.0.80.2
SHA256:
acb6fc266f06cb508079ada6c029cbde06b9eff2319ca1b29940134100404e72

Plesk 18.0.80.3
SHA256:
acb6fc266f06cb508079ada6c029cbde06b9eff2319ca1b29940134100404e72
Therefore, pmm-ras changed between 18.0.79.5 and 18.0.80.0, while the pmm-ras binaries included in 18.0.80.0, 18.0.80.2 and 18.0.80.3 are bit-for-bit identical.

A controlled A/B test was then performed.

The complete Plesk installation remained on 18.0.80 Update 3, with all current packages and libraries unchanged.

Only:

/opt/psa/admin/sbin/pmm-ras
was temporarily replaced with the binary extracted from plesk-backup-utilities 18.0.79.5.

Result: FTPS immediately started working again.

After restoring the original 18.0.80.3 pmm-ras binary:

FTPS immediately failed again with the same TLS session resumption error.

Therefore:

Plesk 18.0.80.3
+ pmm-ras from 18.0.80.x
= FTPS FAILS

Plesk 18.0.80.3
+ pmm-ras from 18.0.79.5
= FTPS WORKS
No other configuration, package, library, firewall setting, certificate or FTPS server setting was changed during this A/B test.

This appears to isolate the regression to pmm-ras / the FTPS implementation embedded in plesk-backup-utilities starting with version 18.0.80.0.

STEPS TO REPRODUCE

1. Install or upgrade a Plesk server from:

Plesk Obsidian 18.0.79 Update 5
to:

Plesk Obsidian 18.0.80
The issue is still present with:

Plesk Obsidian 18.0.80 Update 3

1. Configure an external FileZilla Server 1.12.x with:

Implicit FTPS
TCP port 990
Passive mode
PROT P / encrypted data connections

1. Configure the external FTPS storage in:

Tools & Settings
> Backup Manager
> Remote Storage Settings

1. Test the FTPS remote storage or start a remote backup.
2. The FTPS control connection is successfully established and FileZilla logs:

USER
PASS
230 Login successful

PBSZ 0
200 PBSZ=0

PROT P
200 Protection level set to P

PWD
257 "/" is current directory

CWD <directory>
250 CWD command successful

EPSV
229 Entering Extended Passive Mode (...)

TYPE A
200 Type set to A

LIST
150 Starting data transfer

1. The TLS data connection then fails because the TLS session is not resumed:

TLS session of data connection not resumed.

425 Unable to build data connection:
TLS session of data connection not resumed.

1. From the same Plesk server, execute:

curl -vk --ftp-pasv \
-u USER:PASSWORD \
'ftps://FTPS-SERVER:990/REMOTE-DIRECTORY/'

The directory listing succeeds and curl reports:

SSL re-using session ID
...
226 Operation successful

1. As an additional controlled reproduction test, on Plesk 18.0.80.3 temporarily replace only:

/opt/psa/admin/sbin/pmm-ras
with the binary extracted from:

plesk-backup-utilities 18.0.79.5
without changing any other Plesk component.

The same FTPS storage immediately starts working.

1. Restore the original 18.0.80.3 pmm-ras.

The same FTPS storage immediately fails again.

ACTUAL RESULT

Plesk Backup Manager cannot access the FTPS remote storage.

Plesk reports:

Error: Unable to access the storage:
Transport error: unable to list directory:
Curl error: (19) FTP: could not retrieve (RETR failed) the specified file:
The last FTP request: LIST:
The last FTP response: 425 Use PORT, PASV or EPSV first
However, the FileZilla server log shows that EPSV was successfully issued and accepted.

The actual server-side error is:

150 Starting data transfer.

TLS session of data connection not resumed.

425 Unable to build data connection:
TLS session of data connection not resumed.
Therefore, the Plesk error mentioning PORT, PASV or EPSV appears to be misleading. The passive connection has already been successfully negotiated.

The actual failure occurs during TLS establishment of the FTP data connection because the TLS session from the control connection is not resumed/reused.

EXPECTED RESULT

Plesk Backup Manager should correctly reuse/resume the TLS session from the FTPS control connection when opening a protected FTP data connection after PROT P.

Expected sequence:

EPSV
229 Entering Extended Passive Mode

LIST
150 Starting data transfer

TLS session successfully resumed/reused on data channel

226 Operation successful
The behavior should be equivalent to Plesk 18.0.79 Update 5 and to the system libcurl test, both of which work correctly with the same FTPS server.

ANY ADDITIONAL INFORMATION

I already reported this problem in detail on the Plesk forum shortly after upgrading to 18.0.80, but unfortunately the original thread has not yet received any response from Plesk staff.

The original forum report contains the initial troubleshooting, screenshots and logs.

Since then I have performed additional diagnostics specifically to isolate the regression.

Important additional findings are:

1. TLS 1.2 does not solve the problem

The following setting was tested:

[pmm]
ftpMaxTlsVersion = 12

The same error occurs, therefore the problem does not appear to be specific to TLS 1.3.

2. System curl works correctly

The system curl on exactly the same Ubuntu server correctly performs FTPS implicit on TCP/990 and reports:

SSL re-using session ID
followed by:

226 Operation successful
3. pmm-ras appears to contain an embedded/static libcurl implementation

readelf does not show a dynamically linked libcurl.so dependency, while strings from the binary contains many libcurl/OpenSSL related symbols/messages, including:

This file was generated by libcurl!
SSL_SESSION_free
SSL_set_session
SSL session not peer verified, not reusing
SSL: SSL_set_session not accepted, continuing without
This suggests that Plesk Backup Manager is not using the system /usr/bin/curl implementation for this operation.

4. The A/B test isolates the regression to pmm-ras

This is the most important diagnostic result:

Plesk 18.0.80.3 + pmm-ras 18.0.80.x
= FTPS FAILS

Plesk 18.0.80.3 + pmm-ras 18.0.79.5
= FTPS WORKS
Restoring the 18.0.80.3 binary immediately restores the failure.

The pmm-ras binary changed between 18.0.79.5 and 18.0.80.0, while 18.0.80.0, 18.0.80.2 and 18.0.80.3 contain the exact same pmm-ras binary.

Based on these results, please investigate the changes made to the FTPS/libcurl implementation inside pmm-ras between:

plesk-backup-utilities 18.0.79.5
and
plesk-backup-utilities 18.0.80.0
especially the handling of TLS session reuse/resumption on FTP data connections after PROT P.

I am available to provide:

• complete FileZilla logs;
• complete Plesk logs;
• SHA256 hashes;
• binaries/package version information;
• additional network/TLS traces;
• equivalent diagnostics from the affected Windows Plesk server, if required.

A temporary official hotfix or a supported workaround restoring the FTPS behavior of pmm-ras 18.0.79.5 would also be appreciated, since the current alternative is to disable FTPS and use unencrypted FTP for remote backups.

YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Confirm bug

 

[Sebahat.hadzhi]

Thank you for the report, @treenet . I will attempt to replicate the behavior and follow up with more details tomorrow.