Forwarded to devs Changing the host's default certificate does not update cert file references in the web server config files that are using the host cert

[Bitpalast]

Username:

TITLE

Changing the host's default certificate does not update cert file references in the web server config files that are using the host cert

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

CentOS 7.9
Obsidian 18.41 with latest MU

PROBLEM DESCRIPTION

When changing the general server (hostname) SSL certificate from one default to another, the webserver configuration files are not updated with the new certificate file names automatically.

STEPS TO REPRODUCE

Switch the default server certificate.

ACTUAL RESULT

The certificate file references in the web server configuration files, that are using the host certificate, are not updated to the files of the new default certificate unless a configuration file reconfiguration is run manually.

EXPECTED RESULT

The certificate file references in the web server configuration files where the default server certificate is used instead of an individual certificate ought to be updated to the new default certificate.

ANY ADDITIONAL INFORMATION

Impact: When the "old" default certificate is removed that is actually displayed with "zero" users, the web server won't restart or reload, because all configuration files that previously used the old certificate are now pointing to non-existent certificate files.

YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Confirm bug

 

[Bitpalast]

The references in the Postfix and Dovecot configuration files are updated properly.

 

[moswak]

I can confirm. that was already the case 2 years ago when the certificate was changed. Now that the certificates are only valid for 1 year, this problem comes up every year.
a certificate is often displayed as used even though ssl is deactivated for the user. but since the certificate was selected before ssl was disabled, it is still shown as used.

 

[IgorG]

From developers:

When changing the certificate using the Make Default button under Tools & Settings > SSL/TLS Certificates, the newly selected certificate is applied for all websites using it. Please provide the exact steps to reproduce the issue.

 

[Bitpalast]

The new certificate file names are written to the psa database, the certificate is issued, but the webserver configuration files are not updated with the new filenames of the cert file and the chain file. What is missing is the update of the webserver configuration files. They keep using the old certificate filenames unless a manual reconfiguration is done.

 

[IgorG]

From developer:

Have not reproduced this on Ubuntu 20.04 and CentOS 7 according to the following steps:

1. Under Tools & Settings > SSL/TLS Certificates > Add, created a self-signed certificate.
2. Created two domains, set Not selected and Default certificate (other repository) under domains' Hosting Settings, correspondingly.
3. Under Tools & Settings > SSL/TLS Certificates > List of certificates in server pool, selected the new certificate and clicked Make Default.

Configuration files for both domains were rebuilt.
Please provide the exact steps to reproduce the issue + Plesk/OS versions.

 

[Bitpalast]

First let me say: I swear that the bug occured I spent two hours after an SSL certificate renewal to fix it and to "log" it here. Here is my order of events:

CentOS 7.9, SSLIt extension, previously upgraded from Let's Encrypt extension, but many months ago.

1.
On one host I noticed that the host's Let's encrypt SSL certificate had not been auto-renewed. The reason remained unclear as it has auto-renewed before and not changes to IP or host environment were done. The Let's encrypt certificate was selected for host web server and mail server, but it was not set as "default" certificate at that time. "Default" was a long expired wildcard DV-certificate originally obtained from GoDaddy. That had expired in 2019. Since that time we have been using Let's Encrypt certificates for our hosts. Domains that had not explicitely configured a Let's Encrypt certificate were configured to use that old, expired and defined-as-"default" certificate.

2.
I manually renewed the host's certificate. It was correctly applied to the hostname on the Nginx, Apache and Plesk own Nginx (sw-engine) webserver.

3.
I noticed that many domains on the server were using the old, expired GoDaddy certificate and that this was set as "default". So I went ahead and checked the host's "Let's Encrypt" certificate and clicked the "Make Default" button. Plesk changed the default to that certificate. It also displayed in the back office that zero domains were using the expired, old (previous) default cert. Now in the "Used" column, only on the Let's Encrypt certificate that just became the default I could see a high number of domains, actually all that previously were on the old cert. So in the Plesk GUI it all looked correct.

By that time, the file names in the web server configuration files should have been updated from the old, expired certificate to the Let's Encrypt certificate that I made "default" by clicking "Make Default". But that has not happened. Looking into the web server configuration files these were still pointing to the old, expired certificate. I did examine it with "# openssl x509 -in <cert filename> -text" and am sure that the certificate files that were mentioned in all web server configuration files where the domains had the "default" certificate listed were using the old, outdated GoDaddy certificate. There is no doubt.

4.
I removed the now no longer used old certificate from the certificate list in Plesk GUI. Removal was possible, because the "Used" number was zero, so Plesk at this time must have thought that no configuration is still using the old cert files.

5.
Next thing was that on a customer configuration change Web Server Configurations trouble shooter reported hundreds of issues: It could not find the certificate files (because they were removed in step (4)).

I did fix the situation by running "repair broken" in the Web Server Configurations Trouble Shooter. Only this has updated all web server files that were still using the old certificates. This proves that Plesk internally must have known at this time which files have not been updated, else it could not have marked them as "broken". But on the host's SSL configuration page it had displayed that all previous "default" configurations were updated to the new default. For that reason I am confident that there must be a bug.

The server has a support contract and you are welcome to look into the server yourself. However, at the moment no issues are visible, all seems to be well again.

 

[Bitpalast]

Adding this thread, because it is at least linked to the initial situation where we had to manually update the certificate that then was made "default" certificate: Issue - [extension/sslit] Failed to renew Panel certificate

 

[Bitpalast]

Maybe also linked to Issue - SSL certificates not working after changing Certificate for securing Plesk setting

 

[IgorG]

Here is the developer's reply:

Hello, the issue described in the thread can be a side-effect of PPPM-12005 (Empty apache vhost dir after changing of default certificate on server with big amount of domains (more, than 100))
If there are indeed 100+ domains the race condition can occur:

• webserver not changing all domains' configuration files instantly, it needs some time to recreate all of them (several minutes in case the certificate has changed for all domains)
• at the same time, someone (might be admin) performs some operation that requires webserver restart (change something in Apache & nginx settings of domain, for example).

As a result, webserver restarts while new records in configuration files have not been created yet. And it fails.

 

[Bitpalast]

Hello, the issue described in the thread can be a side-effect of PPPM-12005 (Empty apache vhost dir after changing of default certificate on server with big amount of domains (more, than 100))
If there are indeed 100+ domains the race condition can occur:

We're talking anywhere between 600 and 1,200 domains per server.

• webserver not changing all domains' configuration files instantly, it needs some time to recreate all of them (several minutes in case the certificate has changed for all domains)
• at the same time, someone (might be admin) performs some operation that requires webserver restart (change something in Apache & nginx settings of domain, for example).

As a result, webserver restarts while new records in configuration files have not been created yet. And it fails.

Sounds about right. Suggestion for solving this:
Run an sed replacement for the certificate filenames on /var/www/vhosts/system/*/conf/* files. This would complete within seconds. A full webserver reconfiguration could still be done later, but a simple replacement would immediately be effective and protect the server against outages.