502 Bad Gateway on all websites suddenly

[Chris1]

Hello,

I am running Plesk 12.5.30 Update #5 on CloudLinux 7.1, all of a sudden today every website on the server started getting a 502 Bad Gateway Nginx error. The Plesk panel itself is fine though, this is loading ok.

I tried restarting nginx and Apache but this didn't help.

I tried re-generating the servers configuration using the "Webserver Configurations Troubleshooter" extension, but this didn't help either.

My last yum updates were these:

Oct 11 03:33:08 Updated: psa-php5-configurator-1.7.0-cos7.build1205151001.13.x86_64
Oct 11 03:33:13 Updated: psa-locale-base-en-US-12.5.30-cos7.build1205150929.16.noarch
Oct 11 03:34:02 Updated: plesk-l10n-12.5.30-cos7.build1205150929.16.noarch
Oct 11 03:35:56 Updated: 1:fail2ban-0.9.2-centos7.15093018.noarch
Oct 11 03:35:59 Updated: 1:psa-mod_fcgid-2.0.0-cos7.build1205151001.13.x86_64

I noticed the following error in /var/log/nginx/error.log:

2015/10/14 12:48:31 [error] 871965#0: *24 connect() failed (111: Connection refused) while connecting to upstream, client: <client_ip>, server: , request: "GET / HTTP/1.1", upstream: "https://<server_ip>:7081/", host: "<server_ip>"
2015/10/14 12:49:26 [alert] 871965#0: *26 open socket #9 left in connection 4
2015/10/14 12:49:26 [alert] 871965#0: aborting

Nothing in the individual vhost error logs that would explain what is going on.

My only way to fix the issue for the time being is to disable nginx.

Any ideas on what is happening here?

 

[SibProgrammer]

Upstream means PHP in your case. Did you try to restart PHP-FPM service (Tools & Settings -> Services Management -> PHP-FPM)? You may also check /var/log/php5-fpm.log, /var/log/plesk-php*-fpm/error.log or /var/log/php-fpm/error.log files for errors.

 

[Chris1]

Hello thank you for your reply.

I'm not currently using FPM on any of the vhosts at the moment, using the Plesk provided PHP 5.6 with FastCGI handler.

...however the following seems to be in /var/log/php-fpm though:

[14-Oct-2015 12:52:41] ERROR: No pool defined. at least one pool section must be specified in config file
[14-Oct-2015 12:52:41] ERROR: failed to post process the configuration
[14-Oct-2015 12:52:41] ERROR: FPM initialization failed

Is it possible that the configurations for all/some of the vhosts are still trying to use FPM, even though when I go to "Tools & Settings > PHP Settings" it shows all of the domains using 5.6.14 FastCGI application?

 

[SibProgrammer]

Ok, got it. So you're only using nginx as a proxy, right? One more idea: maybe fail2ban is a root cause of your problems. Could you try to disable fail2ban temporary and check the results?

 

[Chris1]

Yep that's correct, just a front-end proxy. Oh ok I can try that. How does Fail2Ban interfere in this case? I do see that Fail2Ban had an update via yum a few days ago.

 

[vlikhtanskiy]

Hi @Chris1!

Maybe fail2ban banned IP address of your server? Try to add it in a list of trusted IPs.

 

[Chris1]

Ahhh you are correct!

My IP was caught in the ModSecurity jail, thank you for your assistance @SibProgrammer & @vlikhtanskiy !

I have since re-enabled nginx and it seems to be working ok now.