Resolved Renewing Let's Encrypt Manually....Causes Configuration File Errors

[learning_curve]

We are starting to receive "Let's Encrypt certificate expiration notice for domain xyz" notices via e-mail which is handy and a good reminder service. Our limited Plesk knowledge is/was that these would (should!) all renew automatically with our specific setup (sig) but we thought we should attempt one manually, just in case this was NOT actually the case...

To do that appeared to be quite simple and now even has the addition of Secure webmail on this domain as well, all done via the plesk panel and /modules/letsencrypt/index.php/index/install/return/domain?dom_id=2&site_id=2 in this case.

Well, we did this domain manually, but now have an issue on the domain concerned as a result. The error is this:

Error: New configuration files for the Apache web server were not created due to the errors in configuration templates: nginx: [emerg] SSL_CTX_load_verify_locations("/usr/local/psa/var/certificates/cert-B3rl8Z") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/usr/local/psa/var/certificates/cert-B3rl8Z','r') error:2006D080:BIO routines:BIO_new_file:no such file error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib) nginx: configuration file /etc/nginx/nginx.conf test failed

The affected files shown in Webserver Configurations Troubleshooter are:

/var/www/vhosts/system/domain_name/conf/nginx.conf
/etc/nginx/plesk.conf.d/webmails/domain_name_webmail.conf

Running plesk repair all -n shows no errors anywhere and whilst we have read other quite similar errors here on the forum, we are ONLY wanting to correct the error on this one specific domain, as all the others are obviously fine and have no errors at all. We may possibly need to run Plesk Repair Utility: Web to solve this, but ONLY for this domain and so we need to understand the command line required to do specifically that, in order to carry out the repair.

The final questions are....

1) Did we need to do this? (probably not...) because; yes, the auto renewal process would have taken place without causing these errors and without any manual intervention being required. In which case the "Let's Encrypt certificate expiration notice for domain xyz" notices via e-mail become awareness only status really.

2) Why did these errors occur? Our own sequence of events during the manual process or a bug / glitch with Let's Encrypt?

 

[Bitpalast]

1) ... the auto renewal process would have taken place without causing these errors

I'd rather think that the automated process will fail if manual renewal fails. We've had countless outages over the Let's Encrypt renewal process. It is not yet stable. Developers are currently looking into an issue where the web server is restarted before a new certificate is in place - leading to the reconfiguration failure that you are describing in your post. Renewal is a bit like casino play at the moment. Hopefully this will be fixed, soon.

2) Why did these errors occur? Our own sequence of events during the manual process or a bug / glitch with Let's Encrypt?

The certificate file name listed in the web server configuration files does not match the actual file name of the certificate in the certificate directory. This can have many reasons. For example the web server configuration files were not reconfigured after the new SSL certificate was installed, so they still reflect the old certificate name, while the new certificate has a new file name. Or the certificate file might not have been created at all, so that the file name in the web server configuration is indeed correct, but the file itself is missing.

To reconfigure a single domain from the console, run
# /usr/local/psa/admin/sbin/httpdmng --reconfigure-domain <domain-name>

 

[learning_curve]

That's very helpful. Thank you Peter. As per your post, we've

...run
# /usr/local/psa/admin/sbin/httpdmng --reconfigure-domain <domain-name>

But this was the result:

# /usr/local/psa/admin/sbin/httpdmng --reconfigure-domain <*** actual domain name>
[2017-05-31 19:50:32] ERR [util_exec] proc_close() failed ['/usr/local/psa/admin/bin/nginx-config' '-t'] with exit code [1]
[2017-05-31 19:50:39] ERR [util_exec] proc_close() failed ['/usr/local/psa/admin/bin/nginx-config' '-t'] with exit code [1]
[2017-05-31 19:50:43] ERR [panel] Apache config (14962566310.82843500) generation failed: Template_Exception: nginx: [emerg] SSL_CTX_load_verify_locations("/usr/local/psa/var/certificates/cert-B3rl8Z") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/usr/local/psa/var/certificates/cert-B3rl8Z','r') error:2006D080:BIO routines:BIO_new_file:no such file error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib)
nginx: configuration file /etc/nginx/nginx.conf test failed

file: /usr/local/psa/admin/plib/Template/Writer/Webserver/Abstract.php
line: 75
code: 0
nginx: [emerg] SSL_CTX_load_verify_locations("/usr/local/psa/var/certificates/cert-B3rl8Z") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/usr/local/psa/var/certificates/cert-B3rl8Z','r') error:2006D080:BIO routines:BIO_new_file:no such file error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib)
nginx: configuration file /etc/nginx/nginx.conf test failed

Hmmmm Still a challenge awaits us then

 

[Bitpalast]

Do you have many certificates in /usr/local/psa/var/certificates? My suggestion is to find the correct certificates (private key and certificate) for the domain you are trying to protect by looking into the files. If you have renewed a certificate, a new one should be in place. You need to find out what the new name is.

# openssl x509 -in <filename.cert> -text

As an alternative, you can look at the files' datetime to find the files by date (you'll probably know when you ran the renewal).

After you have identified the correct file, copy it to the location that Nginx and Apache are expecting, e.g.

# cp <filename.cert> cert-B3rl8Z

Then check nginx and Apache configuration with
# apachectl -t
# nginx -t
They should then both be o.k. Then you can restart the services.

 

[learning_curve]

Again thanks for the very clear instuctions. We're closer, but no cigar...yet...

Do you have many certificates in /usr/local/psa/var/certificates?

19 files (certs

My suggestion is to find the correct certificates (private key and certificate) for the domain you are trying to protect by looking into the files. If you have renewed a certificate, a new one should be in place. You need to find out what the new name is.
# openssl x509 -in <filename.cert> -text
As an alternative, you can look at the files' datetime to find the files by date (you'll probably know when you ran the renewal).
After you have identified the correct file, copy it to the location that Nginx and Apache are expecting, e.g.
# cp <filename.cert> cert-B3rl8Z

Follwed this ^^ to the letter

Then check nginx and Apache configuration with
# apachectl -t
# nginx -t
They should then both be o.k. Then you can restart the services.

With Apache:
# apachectl -t
Syntax OK

With Nginx:
nginx -t

nginx: [emerg] SSL_CTX_use_PrivateKey_file("/usr/local/psa/var/certificates/cert-B3rl8Z") failed (SSL: error:0906D06C: PEM routines: PEM_read_bio:no start line:Expecting: ANY PRIVATE KEY error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file: PEM lib)
nginx: configuration file /etc/nginx/nginx.conf test failed

I had to add a space between : and PEM in each case otherwise we had a replacement emoji in my text but the rest of the code is as produced in terminal.

As a result we haven't restarted either yet as NGINX is still unhappy, despite the previous additions / corrections

 

[Bitpalast]

The mistake now is that the file must be the private key, but is a certificate. They both look the same in the file system, but they look different when you look into the content. You need to find the private key for the certificate and put it into the right file name (copy). It's really only this simple thing.

 

[learning_curve]

Ha! If only life was simple.....
We've done this:

...find the private key for the certificate and put it into the right file name...

and then via terminal:

# apachectl -t
Syntax OK
# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
#

Success! We then re-started the server via the Plesk Control Panel and.... within the Websites & Domains screen, we're right back at the start of this thread again i.e. the error report (in Plesk) is still:

Error: New configuration files for the Apache web server were not created due to the errors in configuration templates: nginx: [emerg] SSL_CTX_load_verify_locations("/usr/local/psa/var/certificates/cert-B3rl8Z") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/usr/local/psa/var/certificates/cert-B3rl8Z','r') error:2006D080:BIO routines:BIO_new_file:no such file error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib) nginx: configuration file /etc/nginx/nginx.conf test failed

and the two afffected files shown in the Webserver Configurations Troubleshooter are the same:
/var/www/vhosts/system/domain_name/conf/nginx.conf
/etc/nginx/plesk.conf.d/webmails/domain_name_webmail.conf
So it looks like your initial comment:

...Renewal is a bit like casino play at the moment

was correct!

Any further thoughts?

 

[Bitpalast]

That is perfectly o.k. You can restart httpd and nginx services for the settings to take effect, e.g.
# service nginx restart
# service httpd restart

The final step is to use Troubleshooter and reconfigure the erroneous entries. It will actually not do anything dramatic, but only recreate the same web server configuration that is already present, then it will remove the error message from the screen. The message in this stage is a bogus message, a leftover from the previous state. After reconfiguration of the erroneous web server configuration entries via Troubleshooter things should all be o.k. This step is only required to remove the message. It most likely won't change anything else.

 

[learning_curve]

Tremendous We're always cautious and proceed slowly, but your clear instructions run very well with our approach so that's everything done.
No doubt this may be a repeat loop for all the other forthcoming domains as / when the Let's Encrypt renewals arrive UNTIL it's de-bugged
Thanks again Peter