Resolved Suddenly started with odd behavior and 401 errors last week

[Another_Omeka_User]

Hi, I've written here before - I'm not great with server admin, so please be kind. I'm at a loss as to what the problem is, so here's a run-down of what has happened:

1. Beginning last week, the main domain on our website (which has always been protected by htpasswd until we're ready to roll it out live) began showing odd behavior. Specifically:

- the CSS and many thumbnails don't show on the first load, you have to refresh to see them
- occasionally the screen flashes a brief second like a broken page before then displaying the page
- some (not all) of the PDFs no longer load in the Reader viewer; it shows black/not loading and after a refresh just continually spins. However, they can be downloaded with no errors and viewed outside the browser. (There seems to be no reason why some load and others do not; they are of various sizes.)
... etc

2. A quick check from the client side - multiple browsers - reveals 403 Forbidden errors -- but only sometimes. I can post an image if that helps.

3. Looking at the logs for this particular domain I see sporadic 401 errors, like the following:

---
2017-05-16 15:26:07 Error [IP] 401 GET /omeka/items/show/1733 HTTP/1.1
2017-05-16 15:26:07 Error [IP] 401 GET /omeka/themes/seasons/css/style.css HTTP/1.1
2017-05-16 15:26:07 Error [IP] 401 GET /omeka/application/views/scripts/css/admin-bar.css HTTP/1.1
2017-05-16 15:26:07 Error [IP] 401 GET /omeka/application/views/scripts/css/iconfonts.css HTTP/1.1
2017-05-16 15:26:08 Error [IP] 401 GET /omeka/plugins/NeatlineFeatures/views/shared/javascripts/featureswidget.js HTTP/1.1
2017-05-16 15:26:08 Error [IP] 401 GET /omeka/plugins/NeatlineFeatures/views/shared/javascripts/nlfeatures.js HTTP/1.1
2017-05-16 15:26:09 Error [IP] 401 GET /omeka/themes/seasons/javascripts/jquery.zglossary.js
2017-05-16 15:26:09 Error [IP] 401 GET /omeka/themes/seasons/javascripts/jquery.touchSwipe.js HTTP/1.1
2017-05-16 15:26:10 Access [IP] 200 GET /omeka/themes/seasons/javascripts/globals.js HTTP/1.1
2017-05-16 15:26:11 Access [IP] GET /omeka/web/viewer.html?file=/omeka/files/original/45d8a736c75e0106262563ed5c52627e.pdf HTTP/1.1
2017-05-16 15:26:12 Access [IP] 200 GET /omeka/web/viewer.js HTTP/1.1
2017-05-16 15:26:17 Access [IP] 206 GET /omeka/files/original/45d8a736c75e0106262563ed5c52627e.pdf
2017-05-16 15:26:17 Access [IP] 206 GET /omeka/files/original/45d8a736c75e0106262563ed5c52627e.pdf HTTP/1.1
---

I do notice that under "Source" in the log files that the system seems to go back and forth between "Apache access" and "Apache SSL access" although there is no correlation with the errors here; some list one source and some another. However, if I try to access my domain with https:// in front of then I get the Warning about certificates and safety and when I continue to the page it then refuses to display the css. I suppose it's possible this is not helping the issue, but I'm not sure it's the cause. How do I fix this?

Here's the info from the Plesk panel home page, perhaps it is relevant:

OS ‪CentOS Linux 7.1.1503 (Core)‬
Plesk version 12.5.30 Update #65, last updated at May 16, 2017 05:42 AM
The system is up-to-date; last checked at May 9, 2017 07:41 AM

We have our own virtual server and run most of our things in just one domain. I want to note that I have personally changed nothing on the server or on the domain/main website in this time. but that since we do have the virtual server through a third party company we pay, they run updates automatically for us (such as Plesk change log updates). Is it possible that one of these automatic updates caused the issue? Also, why would it say it was last updated May 16 (for the Plesk version) but the last check for the system on May 9? Not sure how these two are related, or if these lines matter.

Any advice or suggestions on what to check would be helpful. I am fairly comfortable in the Plesk panel, but not so much at the command line beyond changing permission settings, setting up an htpasswd or htaccess file, and doing a few other basic things. So if this requires checking anything else, I would be grateful if you could provide commands I need to run.

Thanks for any help.

 

[UFHH01]

Hi Another_Omeka_User,

- the CSS and many thumbnails don't show on the first load, you have to refresh to see them

This indicates the usage of "apache" in combination with "nginx" and you missed to add "Additional NGINX directives" for "nginx". Pls. note, that "nginx" is not able to read ".htaccess" - files, so pls. convert possible existent ".htaccess" files with for example the help of the free Plesk Extension "htaccess to nginx" ( => htaccess to nginx - Plesk Extensions ), a converter, which is able to do the work for you.

Any advice or suggestions on what to check would be helpful.

If you don't use "nginx", then it could help to repair the permissions, with the example command ( logged in as user "root" over SSH ):

plesk repair fs -y -v

More possible REPAIR command options can be seen at the official Plesk 12.5 documentation at: => Plesk Repair Utility​

if I try to access my domain with https:// in front of then I get the Warning about certificates and safety and when I continue to the page it then refuses to display the css. I suppose it's possible this is not helping the issue, but I'm not sure it's the cause. How do I fix this?

Pls. consider to adjust your "Hosting Settings" and you may be interested in reading, how you are able to setup "SSL" and "SEO REDIRECTS" over the command line:

=> site: Sites​

... and pls. inform yourself about the Plesk Let's Encrypt Extension ( => Let's Encrypt - Plesk Extensions ), with which you are able to install free certificates for your (sub)domains and you could then use a "real" certificate, instead of the default "self-signed" certificate, which will certainly result in issues/errors/problems, when you use it for your (sub)domains. ​

 

[Another_Omeka_User]

Hi UFHH01, thank you for the prompt response.

I am going to try the first step re: htaccess files to see if that helps, but is there any reason this would suddenly stop working when it worked fine last week? I'm just confused because I did not personally change anything... we rent the VS from STRATO (we're in Freiburg and given you're in Hamburg maybe you're familiar with what they do on the backend for their customers?). I have no idea if nginx was set up recently in an update and this is what caused the issue?

In any case, I guess I can just install the extension and convert my single htaccess file then without problems, I hope!

 

[UFHH01]

Hi Another_Omeka_User,

maybe you're familiar with what they do on the backend for their customers?). I have no idea if nginx was set up recently in an update and this is what caused the issue?

Sorry, but pls. consider to ask such questions directly to your hosting provider, because we can not answer STRATO - related questions in this Plesk Community Forum.

Plesk itself doesn't install "nginx" during updates/upgrades/patches, you have to tell Plesk, that you would like to install it ( pls. have a closer look at the current installed components by visiting: => HOME > Tools & Settings > Server Components ).


Pls. report back with your step-by-step processes to solve the issue, if you experience further errors/problems - guessings and thoughts might not help to guide you in case of un- or misconfigurations.

 

[Another_Omeka_User]

Ah, OK. It's not under Server Components, so I guess that that's not the issue then... I'll try the other things!

 

[Another_Omeka_User]

I used the Let's Encrypt extension to create a certificate for the domain (thanks for that tip!) and it seems like things are running more smoothly now, significantly fewer errors in the logs, and the PDFs are now loading... I will repair permissions if I find more problems, but for now I think that was a big cause of the problems (still not sure why it wasn't before but hey whatever works!).

 

[Another_Omeka_User]

Hi - just a quick follow-up. How can i use Let's Encrypt to create an SSL for accessing the Plesk panel? I'm also getting a NET::ERR_CERT_AUTHORITY_INVALID every time I try to go to the panel (https://domain:8443).

 

[UFHH01]

Hi Another_Omeka_User,

How can i use Let's Encrypt to create an SSL for accessing the Plesk panel?

Actually, your question is as easy to be answered, as installing a Let's Encrypt certificate for a (sub)domain. Just create a subdomain, which matches your "/etc/hostname" - definition for your server and go to:

=> HOME > SSL/TLS Certificates​

... to add the newly created certificate ( which you are able to download as "*.pem" - file from "HOME > Subscriptions > YOUR-DOMAIN.COM > SSL/TLS Certificates" and choose to secure the Plesk Control Panel with it.

 

[Another_Omeka_User]

Thanks for this - just getting to making the change now. Can I ask a follow-up? When you say create a subdomain with "/etc/hostname" do you mean go to "Domains" and choose "Add a subdomain"? It then forces me to add it under the only domain I have on this server... or am I missing a step? Or is the parent domain the hostname of my VPS (what I see under system settings)?

Also, for "/etc/hostname" I am guessing you just mean the VPS hostname in the settings too.

EDIT: Ok I tried the steps according to this page, which is similar to what you told me to do... the first option to secure Plesk directly didn't work, so I tried the second, which is basically the same steps as above. However, when I click "Install" I get the following error message:

"Error: Let's Encrypt SSL certificate installation failed: Invalid response from https://acme-v01.api.letsencrypt.org/acme/new-cert: Error creating new cert :: too many certificates already issued for: stratoserver.net.
Type: urn:acme:error:rateLimited."

Our VPS space is hosted on stratoserver.net, but it is a unique name, e.g., serverspace.stratoserver.net.

Any advice?

 

[UFHH01]

Hi Another_Omeka_User,

the recommended way is always to create a unique servername.YOUR-DOMAIN.COM - hostname, instead of using the pre-configured one, that your hosting provider ( automatically ) creates. With this recommendation, you are as well able to create a unique subscription for "servername.YOUR-DOMAIN.COM", instead of creating a subdomain.

EDIT: Ok I tried the steps according to this page, which is similar to what you told me to do... the first option to secure Plesk directly didn't work, so I tried the second, which is basically the same steps as above. However, when I click "Install" I get the following error message:

"Error: Let's Encrypt SSL certificate installation failed: Invalid response from https://acme-v01.api.letsencrypt.org/acme/new-cert: Error creating new cert :: too many certificates already issued for: stratoserver.net.
Type: urn:acme:error:rateLimited."

Our VPS space is hosted on stratoserver.net, but it is a unique name, e.g., serverspace.stratoserver.net.

... and such an error/problem is one of the reasons, why it is a good idea to change the pre-configured hostname to a unique one for your very own ( registered ) domain for your server, to avoid the restrictions from Let's Encrypt ( which you are not able to change or modify in any way ).

 

[Another_Omeka_User]

I appreciate your help, it fixed the first problem, but I am very confused now.

So, we do have a unique domain name, it's the one I mentioned earlier in this thread. I am able to secure that through Let's Encrypt without a problem. It's just the Plesk panel I can't figure out (unique-domain.com:8443). So I created a new "servername.unique-domain.com" where the domain name is the one we own. I then used Let's Encrypt with that and though it appeared to have worked - no error like above and the SSL certificate is there - but the Plesk panel still is still unsecured even after logging out, refreshing page, logging back in (any of the domains with :8443 ... ). Am I missing a step?

 

[UFHH01]

Hi Another_Omeka_User,

Am I missing a step?

The missing step is:

=> HOME > Tools & Settings > SSL/TLS Certificates​

... where you are able to choose, WHICH certificate should be used to secure the Plesk Control Panel.

 

[Another_Omeka_User]

Ah ha. That seems be the problem. There are no certificates to choose from besides default (which doesn't work). Are they automatically supposed to be here? (And yes SSL support is enabled in the domains) I see them within the domain hosting settings, so the certificate is there but not in the => HOME > Tools & Settings > SSLCertificates:

 

[UFHH01]

Hi Another_Omeka_User,

pls. note the difference between ADMIN - ( server repository ) certificates and domain specific certificates. Actually, you should consider to upgrade to Plesk Onyx, as this options have been improoved very much there and it is easier to understand and to handle it over the Plesk Control Panel.


You are able to EXPORT certificates, with the help of the "green arrow" on the very right of each listed certificate.
To be able to use the ( newly ) created certificate in the server reporitory ( ADMIN ), to secure your Plesk Control Panel, pls. create/import a new certificate, with the needed credentials from your domain - specific certificate.

 

[Another_Omeka_User]

I forgot to mention that I did try that - I downloaded and tried to upload the certificate another time. However, when I go to upload I run into this error:
"Unable to find the appropriate private key for the certificate."

...I agree that we should eventually upgrade to Onyx. Unfortunately I don't have the time right now to deal with any errors that might occur when I do that. But it's on the to-do list for the future. If I can't get this security certificate to work now, then I will add it to the list for the future too.

Thanks!

 

[UFHH01]

Hi Another_Omeka_User,

pls. consider to OPEN the downloaded certificate ( for example with "notepad.exe" on windows - based systems ), as it is easier to add a new certificate with the provided boxes. Just copy the key:

-----BEGIN PRIVATE KEY-----
...
...
-----END PRIVATE KEY-----

... and insert it..., afterwards copy the *.crt ( certificate ):

-----BEGIN CERTIFICATE-----
...
...
-----END CERTIFICATE-----

... and insert it... and finally, copy the root - ca:

-----BEGIN CERTIFICATE-----
...
...
-----END CERTIFICATE-----

... and insert it.

Pls. be informed, that all Let's Encrypt certificates are stored at => "/usr/local/psa/var/modules/letsencrypt/etc/live". You are able to find each relevant parts there as well - note, that it is recommended to use "fullchain.pem", as root - ca, as it contains as well the intermediate certificate from Let's Encrypt.

 

[Another_Omeka_User]

Hi, finally have another chance to work on this issue. I tried the above and not sure why, but it didn't work. First, when I open the downloaded certificate and copy the contents to create an SSL to secure Plesk, it saves it using the domain initially associated with it, and not the one I specify, i.e. servername.your-domain.com instead of just your-domain.com. Which domain should I be entering when I create the certificate per your instructions above?

Then the other issue is that when I select the certificate and choose "Secure Plesk Panel" it simply does nothing. When I logout and leave the browser then try again, I still get the unsecured message, so I know it did not take effect.

Finally, I looked on the server and see the two certificates in the folder you mention above (live), but what does this mean in your message:

"You are able to find each relevant parts there as well - note, that it is recommended to use "fullchain.pem", as root - ca, as it contains as well the intermediate certificate from Let's Encrypt."

I am not sure what to do for that?

 

[UFHH01]

Hi Another_Omeka_User,

overlooking my suggestions and explanations, they are still clear and if you follow them step-by-step, you should be able to solve your issue. If you need help with these tasks, pls. consider to contact the official Plesk Support, where experienced system engineers will assist you by doing the steps for you on your server.

=> Contact Plesk Support

=> How to get support directly from Plesk?​

 

[Another_Omeka_User]

Well, it's not a huge deal, so I think I won't do anything about it right now. Could you just explain to me where the fullchain.pem could be found?

 

[UFHH01]

Could you just explain to me where the fullchain.pem could be found?

As already mentioned, you will find such files ( IF existent! ), at the domain specific folder

Pls. be informed, that all Let's Encrypt certificates are stored at => "/usr/local/psa/var/modules/letsencrypt/etc/live".

( Additional informations:

You should find domain specific folders at the above mentioned folder, which leads to for example:

=> /usr/local/psa/var/modules/letsencrypt/etc/live/YOUR-DOMAIN.COM/fullchain.pem )​