Issue Webserver unstable after introducing new subdomain

[vic666]

This is on Plesk for Linux, 18.0.29 update 2

Today, I created a new subdomain wiki.example.com and installed DocuWiki on it. For those of you who are not familiar with it, DocuWiki is a Wiki software that does not rely on a database, everything is stored in text files. That's why it's actually supposed to be very webadmin friendly as you can install it on almost any host that offers basic PHP.

I'm in the process of configuring DocuWiki. Basically, I'm just clicking around a bit, creating new wiki users, creating new pages... this may be totally unrelated, of course, but just doing these very simple tasks, already 3 times today the action I took timed out and I was no longer able to access the wiki, at all.

As it turned out, not just the wiki was affected, I could no longer access the admin panel or any other domain. Which is a problem as I have client hosts running on it. I tried access from various networks, too, to make sure that it's not just my IP that's making problems.

I looked at journalctl and through practically every file in /var/log, I couldn't find any such issue to stand out. And clearly, the network access to the server still works, as proven by my ssh access. It's just web that seems affected. There is no problem with resources either, plenty of cpu, memory and space left. The following extract of the syslog are the only lines that I'm not sure what to do with/about:

Aug 26 17:17:32 plesk.example.com named[197]: listening on IPv4 interface eth0, xxx.xx.xx.xx#53
Aug 26 17:17:32 plesk.example.com named[197]: listening on IPv4 interface lo, 127.0.0.1#53
Aug 26 17:17:32 plesk.example.com named[197]: generating session key for dynamic DNS
Aug 26 17:17:32 plesk.example.com named[197]: couldn't mkdir '//run': Permission denied
Aug 26 17:17:32 plesk.example.com named[197]: failed to generate session key for dynamic DNS: permission denied
Aug 26 17:17:32 plesk.example.com named[197]: could not create //run/named/session.key

All 3 times, I could "easily" solve the situation by just rebooting the Plesk machine. Still, this has me worried. How can I go about identifying the source of the problem?

 

[IgorG]

I would advise you to research your domain and web server logs. You can find them using the KB article  Plesk for Linux services logs and configuration files
Also, running the following commands won't hurt:

# plesk repair web
# plesk repair fs

 

[vic666]

Hi Igor,

Thanks for replying. The web server logs are actually the first thing I checked. If I understand the setup correctly, my system uses nginx as a reverse proxy, logs in /var/log/nginx, and apache that's actually serving the sites, logs in /var/log/apache2.

In the apache error log, the only thing that pops up are messages after the reboot, no errors or warnings before. The access log is empty. I guess that's expected since nginx is handling the access.

Thus, as expected, the nginx access log is full with GET and POST messages but the error log is empty.

So I guess all that's left to check is DNS. If I understand correctly, there is no separate log file and all error messages are part of the system log as retrieved with journalctl. If I look at the system logs, there seems to be nothing out of the ordinary before the reboot. Lots of failed authentication messages from obvious brute force attacks, especially for the mail service, and some system messages that seem to be related, such as:

Aug 26 10:18:39 plesk.example.com postfix/smtpd[20208]: warning: dict_nis_init: NIS domain name not set - NIS lookups disabled
Aug 26 11:02:47 plesk.example.com systemd-resolved[26980]: Server returned error NXDOMAIN, mitigating potential DNS violation DVE-2018-0001, retrying transaction with reduced feature level UDP.

The other log fragment I mentioned then happens at server reload.

Aug 26 10:40:42 plesk.example.com systemd[1]: Reloading BIND Domain Name Server.
Aug 26 10:40:42 plesk.example.com rndc[20707]: WARNING: key file (/etc/bind/rndc.key) exists, but using default configuration file (/etc/bind/rndc.conf)
Aug 26 10:40:42 plesk.example.com named[31729]: received control channel command 'reload'
Aug 26 10:40:42 plesk.example.com named[31729]: loading configuration from '/etc/named.conf'
Aug 26 10:40:42 plesk.example.com named[31729]: unable to open '/etc/bind/bind.keys'; using built-in keys instead
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Country (IPv4) (type 1) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Country (IPv6) (type 12) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP City (IPv4) (type 2) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP City (IPv4) (type 6) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP City (IPv6) (type 30) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP City (IPv6) (type 31) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Region (type 3) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Region (type 7) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP ISP (type 4) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Org (type 5) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP AS (type 9) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP Domain (type 11) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: GeoIP NetSpeed (type 10) DB not available
Aug 26 10:40:42 plesk.example.com named[31729]: using default UDP/IPv4 port range: [1024, 65535]
Aug 26 10:40:42 plesk.example.com named[31729]: using default UDP/IPv6 port range: [1024, 65535]
Aug 26 10:40:42 plesk.example.com named[31729]: generating session key for dynamic DNS
Aug 26 10:40:42 plesk.example.com named[31729]: couldn't mkdir '//run': Permission denied
Aug 26 10:40:42 plesk.example.com named[31729]: could not create //run/named/session.key
Aug 26 10:40:42 plesk.example.com named[31729]: failed to generate session key for dynamic DNS: permission denied
Aug 26 10:40:42 plesk.example.com named[31729]: sizing zone task pool based on 5 zones
Aug 26 10:40:42 plesk.example.com named[31729]: none:103: 'max-cache-size 90%' - setting to 14372MB (out of 15969MB)

Anyway, I ran the commands you mentioned. For plesk repair web, I have one warning:

Checking for nginx ULIMIT value
  8 domains have been found on server. Problem with nginx default     
  limit for open files is possible                                    
  Please check https://support.plesk.com/hc/articles/213938485 for    
  details ........................................................... [WARNING]

If I understand correctly, we would see something in the nginx error log if this warning would have produced a meaningful issue. Just in case, I also checked journalctl for any nginx entries, no errors or even warnings there. Anyway, I increased the number to 512.

I also ran plesk repair fs. Apparently, httpdocs for one customer domain had a wrong group ownership (psacln instead of psaserv).

All of this is corrected it just seems a bit unlikely that I would experience a full site crash because of these small issues. I'll continue working on the system and will get back if it should happen again.