Resolved 502, 504 Bad Gateway

[Kingsley]

Hello;

My server has been working fine for 2 weeks now suddenly today all websites are now showing 504 error gateway, then i followed this tutorial Nginx: 504 Gateway Time-out after about 1hr all websites are showing 502 Bad Gateway.

2017-07-05 14:24:37    Error    97.123.27.126        29081#0: *14368 peer closed connection in SSL handshake (104: Connection reset by peer) while SSL handshaking to upstream                nginx error

2017-07-05 14:24:48    Error    210.186.58.103        29081#0: *13946 upstream timed out (110: Connection timed out) while reading response header from upstream                nginx error
2017-07-05 14:25:21    Error    66.249.64.15        29081#0: *14479 peer closed connection in SSL handshake (104: Connection reset by peer) while SSL handshaking to upstream                nginx error
2017-07-05 14:25:22    Error    77.107.123.132        29081#0: *14477 peer closed connection in SSL handshake (104: Connection reset by peer) while SSL handshaking to upstream

 

[Bitpalast]

So what changes were made to the SSL configuration of Apache?

 

[Kingsley]

So what changes were made to the SSL configuration of Apache?

i did not do any changes... both apache and nginx were online when the 504 issue started

 

[Bitpalast]

Are public IP and localhost (127.0.0.1) excluded from Fail2Ban?

 

[Kingsley]

i have switched apache to prefork and increased the time to 90sec

 

[Kingsley]

Are public IP and localhost (127.0.0.1) excluded from Fail2Ban?

All IPS are whitelisted

 

[Kingsley]

The sites are all offline now

 

[Bitpalast]

Did you find anything useful in the corresponding Apache log files? For example the error log files of a virtual host or the general /var/log/httpd error logs? Is the Apache service running at all? Have you tried restarting it? Does ist show any error messages?

 

[Kingsley]

/var/log/httpd

[Wed Jul 05 16:32:48.295856 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for webmail.dearmama360.com:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.295998 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for webmail.coupondig.com:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296142 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for webmail.awoofhost.com:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296289 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for lists:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296434 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-2607_5300_120_b15__:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296576 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-66_70_160_87:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296717 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-66_70_160_86:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.296859 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-66_70_160_85:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.297004 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-66_70_160_84:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.297148 2017] [ssl:warn] [pid 9420] AH01909: RSA certificate configured for default-158_69_251_21:443 does NOT include an ID which matches the server name
[Wed Jul 05 16:32:48.297387 2017] [ssl:warn] [pid 9420] AH02292: Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Wed Jul 05 16:32:48.301394 2017] [mpm_prefork:notice] [pid 9420] AH00163: Apache/2.4.6 () OpenSSL/1.0.1e-fips Apache mod_fcgid/2.3.9 configured -- resuming normal operations
[Wed Jul 05 16:32:48.301418 2017] [core:notice] [pid 9420] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

 

[Kingsley]

From one of the domain error log.

[Wed Jul 05 15:10:12.544581 2017] [proxy_fcgi:error] [pid 18548] [client 97.75.177.149:55420] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n', referer: https://www.healthable.org/login/
[Wed Jul 05 15:19:30.911256 2017] [proxy_fcgi:error] [pid 23222] [client 110.171.172.12:60738] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n'
[Wed Jul 05 15:19:37.588601 2017] [proxy_fcgi:error] [pid 23525] [client 178.158.69.21:60724] AH01071: Got error 'PHP message: PHP Warning:  session_start(): open(/var/lib/php/session/sess_qeafn0klm11f0lghugdf29tv35, O_RDWR) failed: No such file or directory (2) in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/users-ultra/xooclasses/xoo.userultra.register.php on line 517\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n', referer: https://www.healthable.org/register/
[Wed Jul 05 15:19:37.590465 2017] [proxy_fcgi:error] [pid 23525] [client 178.158.69.21:60724] AH01071: Got error 'PHP message: PHP Warning:  Unknown: open(/var/lib/php/session/sess_qeafn0klm11f0lghugdf29tv35, O_RDWR) failed: No such file or directory (2) in Unknown on line 0\nPHP message: PHP Warning:  Unknown: Failed to write session data (files). Please verify that the current setting of session.save_path is correct (/var/lib/php/session) in Unknown on line 0\n', referer: https://www.healthable.org/register/
[Wed Jul 05 15:20:15.621181 2017] [proxy_fcgi:error] [pid 24044] [client 201.43.175.213:33098] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n'
[Wed Jul 05 15:34:11.926965 2017] [proxy_fcgi:error] [pid 31153] [client 178.158.69.21:41446] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n', referer: https://www.healthable.org/
[Wed Jul 05 15:34:13.487821 2017] [proxy_fcgi:error] [pid 31119] [client 178.158.69.21:41468] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n', referer: https://www.healthable.org/login/
[Wed Jul 05 15:35:55.127625 2017] [proxy_fcgi:error] [pid 31153] [client 39.55.103.1:42296] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n'
[Wed Jul 05 16:25:59.365608 2017] [proxy_fcgi:error] [pid 18296] [client 191.101.93.11:50032] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n', referer: https://www.healthable.org/wp-login.php?action=register
[Wed Jul 05 16:33:21.890981 2017] [proxy_fcgi:error] [pid 10629] [client 54.208.213.138:49396] AH01068: Got bogus version 99
[Wed Jul 05 16:33:21.891010 2017] [proxy_fcgi:error] [pid 10629] (22)Invalid argument: [client 54.208.213.138:49396] AH01075: Error dispatching request to :
[Wed Jul 05 16:36:31.928439 2017] [proxy_fcgi:error] [pid 10626] [client 217.182.132.5:50228] AH01071: Got error 'PHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\nPHP message: PHP Warning:  DOMDocument::loadHTML(): Empty string supplied as input in /var/www/vhosts/healthable.org/httpdocs/wp-content/plugins/amazon-associates-link-builder/includes/aalb_content_filter.php on line 56\n'

 

[Kingsley]

Proxy error log

2017/07/05 16:32:27 [error] 4745#0: *16 connect() failed (111: Connection refused) while connecting to upstream, client: 100.43.85.1, server: healthable.org, request: "GET /wp-content/uploads/2014/06/Hand-Washing.jpg HTTP/1.1", upstream: "http://66.70.160.87:7080/wp-content/uploads/2014/06/Hand-Washing.jpg", host: "www.healthable.org"
2017/07/05 16:32:27 [error] 4745#0: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 66.249.64.15, server: healthable.org, request: "GET /chinese-remedies-for-memory-loss/ HTTP/1.1", upstream: "https://66.70.160.87:7081/chinese-remedies-for-memory-loss/", host: "www.healthable.org"
2017/07/05 16:32:27 [error] 4745#0: *30 connect() failed (111: Connection refused) while connecting to upstream, client: 66.249.64.21, server: healthable.org, request: "POST /wp-admin/admin-ajax.php?td_theme_name=Newspaper&v=7.4 HTTP/1.1", upstream: "http://66.70.160.87:7080/wp-admin/admin-ajax.php?td_theme_name=Newspaper&v=7.4", host: "www.healthable.org", referrer: "http://www.healthable.org/mommy-makeovers-plastic-surgery-options-make-look-young/"
2017/07/05 16:32:28 [error] 4745#0: *15 connect() failed (111: Connection refused) while connecting to upstream, client: 34.211.0.215, server: healthable.org, request: "GET /wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg HTTP/1.1", upstream: "https://66.70.160.87:7081/wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg", host: "www.healthable.org", referrer: "http://coverdaily.com/"
2017/07/05 16:32:29 [error] 4745#0: *37 connect() failed (111: Connection refused) while connecting to upstream, client: 181.143.213.202, server: healthable.org, request: "GET /delicious-homemade-chili-10-easy/ HTTP/1.1", upstream: "https://66.70.160.87:7081/delicious-homemade-chili-10-easy/", host: "www.healthable.org", referrer: "https://www.healthable.org/"
2017/07/05 16:32:30 [error] 4745#0: *41 connect() failed (111: Connection refused) while connecting to upstream, client: 91.153.52.60, server: healthable.org, request: "GET /wp-login.php HTTP/1.1", upstream: "http://66.70.160.87:7080/wp-login.php", host: "www.healthable.org"
2017/07/05 16:32:30 [error] 4745#0: *41 connect() failed (111: Connection refused) while connecting to upstream, client: 91.153.52.60, server: healthable.org, request: "GET / HTTP/1.1", upstream: "http://66.70.160.87:7080/", host: "www.healthable.org"
2017/07/05 16:32:30 [error] 4745#0: *13 connect() failed (111: Connection refused) while connecting to upstream, client: 41.102.214.129, server: healthable.org, request: "GET /wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg HTTP/1.1", upstream: "https://66.70.160.87:7081/wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg", host: "www.healthable.org", referrer: "http://coverdaily.com/"
2017/07/05 16:32:30 [error] 4745#0: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 41.102.214.129, server: healthable.org, request: "GET /wp-content/uploads/2017/07/Occupational-Health-Assessment.jpg HTTP/1.1", upstream: "https://66.70.160.87:7081/wp-content/uploads/2017/07/Occupational-Health-Assessment.jpg", host: "www.healthable.org", referrer: "http://coverdaily.com/"
2017/07/05 16:32:31 [error] 4745#0: *48 connect() failed (111: Connection refused) while connecting to upstream, client: 138.201.84.108, server: healthable.org, request: "GET / HTTP/1.1", upstream: "http://66.70.160.87:7080/", host: "www.healthable.org", referrer: "http://healthable.org/"
2017/07/05 16:32:34 [error] 4745#0: *56 connect() failed (111: Connection refused) while connecting to upstream, client: 66.249.64.19, server: healthable.org, request: "GET /?s=benzoic HTTP/1.1", upstream: "http://66.70.160.87:7080/?s=benzoic", host: "www.healthable.org"
2017/07/05 16:32:35 [error] 4745#0: *15 connect() failed (111: Connection refused) while connecting to upstream, client: 34.211.0.215, server: healthable.org, request: "GET /wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg HTTP/1.1", upstream: "https://66.70.160.87:7081/wp-content/uploads/2017/07/Waterproof-Fitness-Tracker1.jpg", host: "www.healthable.org", referrer: "http://coverdaily.com/"
2017/07/05 16:32:37 [error] 4745#0: *15 connect() failed (111: Connection refused) while connecting to upstream, client: 34.211.0.215, server: healthable.org, request: "GET /wp-content/uploads/2017/07/Occupational-Health-Assessment.jpg HTTP/1.1", upstream: "https://66.70.160.87:7081/wp-content/uploads/2017/07/Occupational-Health-Assessment.jpg", host: "www.healthable.org", referrer: "http://coverdaily.com/"
2017/07/05 16:32:40 [error] 4745#0: *56 connect() failed (111: Connection refused) while connecting to upstream, client: 66.249.64.19, server: healthable.org, request: "GET /wp-content/uploads/2015/12/heroin-218x150.jpg HTTP/1.1", upstream: "http://66.70.160.87:7080/wp-content/uploads/2015/12/heroin-218x150.jpg", host: "www.healthable.org"
2017/07/05 16:32:45 [error] 4745#0: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 66.249.64.15, server: healthable.org, request: "GET /six-pack-abs-truths-facts/ HTTP/1.1", upstream: "https://66.70.160.87:7081/six-pack-abs-truths-facts/", host: "www.healthable.org"

 

[iceman333]

I have the same, it only works PHP 7.1.2. At Nginix. Apache has errors.

Unable to generate the web server configuration file on the host <rv1797.1blu.de> because of the following errors:



Template_Exception: AH00526: Syntax error on line 107 of /etc/apache2/plesk.conf.d/vhosts/xxxxxxxx.de.conf:

Invalid command 'DirectoryIndex', perhaps misspelled or defined by a module not included in the server configuration



file: /opt/psa/admin/plib/Template/Writer/Webserver/Abstract.php

line: 75

code: 0



Please resolve the errors in web server configuration templates and generate the file again

Is on 2 servers so, error has been around 2 hours since. Think has come with # 12.

2. Server is reinstall and same errors.

 

[Bitpalast]

@iceman333: This is a totally different issue. Please open a new thread for new issues. Solution is to enable mod_dir.

 

[Bitpalast]

@Kingsley: What PHP setting are the sites using? Is the corresponding PHP-FPM service running? Can you please restart all PHP-FPM services on your system, for example
# service plesk-php54-fpm restart
# service plesk-php56-fpm restart
# service plesk-php70-fpm restart
# service plesk-php71-fpm restart
Do these restarts show any errors?

 

[LaurentR2D2]

Hello,
I have the same problem of 502 Bad gateway error Nginx for all the websites on my server. I haven't changed any configuration file or add any component before the problem occurred. The only thing I've done yesterday was to upgrade some system components through the Plesk admin panel. Plesk admin panel is running fine. I have this configuration :

OS ‪CentOS Linux 7.3.1611 (Core)‬
Produit Plesk Onyx
Version 17.5.3 Update n° 16, last upgrade : august 3rd 2017 03:13

I've tried to relaunch httpd, and nginx services, then I've tried to reboot the system, without success.

I've looked in the error logs of httpd and nginx, I have nothing.

 

[Bitpalast]

What are the errors in the error and proxy log?

 

[LaurentR2D2]

What are the errors in the error and proxy log?

Are you talking of Nginx logs ? I have this :

error.log :
2017/08/05 10:56:35 [error] 3280#0: *420 connect() failed (111: Connection refused) while connecting to upstream, client: 184.105.139.*, server: , request: "GET / HTTP/1.1", upstream: "https://195.154.174.*:7081/", host: "195.154.174.*"

access.log :
84.105.139.* - - [05/Aug/2017:10:56:35 +0200] "GET / HTTP/1.1" 502 166 "-" "-"

 

[Bitpalast]

Can you exclude other reasons for it, e.g. a block by mod_security? It is not saying that there is an issue with PHP.

 

[LaurentR2D2]

Can you exclude other reasons for it, e.g. a block by mod_security? It is not saying that there is an issue with PHP.

After trying "plesk repair --all" without success, I've checked Centos firewall. I haven't thought at that at first time, since it's the first thing I disable after installing the system and before installing Plesk, replacing it by the firewall integrated in Plesk. I don't know what happened, but the last upgrade must have reenabled it, since after having issued the commands "systemctl disable firewalls / systemctl stop firewalls", my websites are accessibles again. I also had to reactivate Plesk firewall.

I haven't marked the thread as solved since I didn't start it

 

[D4NY]

Same problem here! I'm totally in panic. All wordpress websites are down, first to go down a big website some days ago with random 504 gateway error. Now all. Without a reason. Who can explain me what's going on and help me? PLEASE!