Nginx 502 Bad Gateway

[Thomas Becker]

Dear Paralles-Team,
does Paralles work on the issue that from time to time customers do get 502 Bad Gateway from Nginx?

1st reason: The reason is that Apache is restarted by Plesk if a customer does make some changes on a domain.
See post: http://forum.parallels.com/showthread.php?261367-Nginx-BAD-GATEWAY&p=727815&viewfull=1#post727815
2nd reason: Fail2Ban blocked the local IP of the hosting for some minutes.
See my post: http://forum.parallels.com/showthread.php?310956-Local-IP-address-blocked-by-feature-fail2ban

In proxy_error_log you can find:
connect() failed (111 Connection refused) while connecting to upstream

This is serious issue because customers do think that server is faulty or misconfigured if they get 502 randomly.

Please forward to developers if they are not working on it! Thanks

Regards
Thomas

 

[daniiii_e]

Any Update?

 

[danielmt2k]

Any update on this???

 

[ASTOKES]

Thanks a bunch for highlighting the fail2ban problem. My server was after several days suddenly serving 502 bad gateway errors on all vhosts and nothing worked except a reboot. After reading this post, I checked the fail2ban logs and saw that it was adding a rule (plesk-apache-badbot) banning it's own IP address! I have added the localhost IP to the fail2ban whitelist and hope that this will solve the problem...

 

[ffischer]

hi,
same Problem on my Server.
Nothing Changed today i wonder that monitoring gets a 502 bad gateway.
Reboot not working
Restart Apache .. not working
Look into Fail2Ban BanList i see my own IP.
how i can find out what exactly happens - the fail2ban Log does not shown enough .

i want to see wich site doing this Problem.

 

[UFHH01]

Hi ffischer,

to investigate issues it it very wise to add some log - entries from your logs ( Fail2ban, apache/nginx - logs AND domain specific apache/nginx - logs ), so that investigations are easier and people willing to help don't have to guess scenarios. If you experience issues with Fail2ban, it is as well usefull to provide configuration files, so that we can see, what filters you use, what jails are setup and so on..... the more informations you provide and add, the better will be the suggestions and work-arounds to solve your issue.

Parallels Plesk Panel for Linux services logs and configuration files ( KB - article: 111 283 )​

Fail2ban - files: /etc/fail2ban/ ( jail.conf - jail.local - files from /etc/fail2ban/jails.d/ - filters from /etc/fail2ban/filter.d/ )
Domain - specific logs: /var/www/vhosts/system/YOURDOMAIN_OR_SUBDOMAIN.COM/logs/

 

[ASTOKES]

I have had no problem since adding my own IP to fail2ban whitelist - I dunno what was going wrong but this fixes it - simple!

 

[UFHH01]

The Fail2ban filter "apache-noscript" for example checks as well for clients, which request files, which are non-existent. If for some resaons your web-content has wrong or missing favicon definitions, wrong or missing image definitions, the filter will recognize, that a client request files, which return the answer "File does not exist" from the webserver. If you defined in the Fail2ban - jail - rules, that after 5 or 6 tries with such an answer, the client should be banned, then you will see, that Fail2ban will certainly ban the IP, which fit the filter rules.
Often enough this issue will as well appear, when a webserver configuration has wrong read permissions to some files or folders and then the webserver answer is a "Permission denied" in the logs.

 

[Thomas Becker]

Hello,
the fail2ban rules are the default one delivered with Plesk and are not changed. I understand what you wrote, UFHH01.

@parallels
Please think about adding by default all local IP addresses to fail2ban whitelist.

 

[ASTOKES]

am I right in thinking that this is caused because apache is getting requests from nginx which it is not treating as local? in other words if this was just apache there would be no problem, but fail2ban is picking up dodgy requests from nginx which it is perceiving as being from outside and trip it's jail rules? I have never had to whitelist localhost on other apache-only fail2ban installations...

 

[UFHH01]

First of all, Plesk didn't invent Fail2ban and just adds Fail2ban as an option / additional feature - so to Thomas.Becker: Fail to ban is not configured by Plesk, but provides some Plesk - jails, which you might want to use, or don't. All these Plesk - jails have a separate configuration file, located at: /etc/fail2ban/jail.d/plesk.conf. - the original untouched config -file from Plesk is named: plesk.conf.dpkg-dist.
Depending on your own specific wish, you may use pre-configured Plesk jails, or you stick to the initial jails, which are shipped by your vendor, but then you have to add some own configuration settings as well. The initial jail "apache-noscript" for example is not a jail, that Parallels suggests and therefore Plesk didn't pre-configure this jail in his very own Plesk - configuration - file for Fail2ban. If it is your wish, to add specific jails, either self-configured or pre-configured by your vendor package, please be so kind and read the installation instruction and manuals for it, because they will inform you, that in some cases it is wise to add your localhost and IPs in order to prevent banning your own server. The pre-configured Plesk - jails should not need any additional configuration settings, if... and here comes another clue, your server configurations are well configured.

No, ASTOKES, this is not the case. Fail2ban looks for configured rules in defined logs. These logs are either from apache and/or nginx, when it comes to an initial standard jail like "apache-noscript". It depends on your very own webserver / domain configuration and depends on the specific web-content, if an exclusion to your locahost and it's IPs is necessary, in order to prevent banning your own server IPs. Some initial jails from your vendor package are defined for the global linux - server, but some operating systems and or /additional apache and/or nginx configurations ( or their addons ), makes it necessary to change the global standard rules.
Let's assume, that you use caching and mod-pagespeed, you will see in your logs, that mod-pagespeed sometimes requests files, which are probably non-existent... for example here: a previous cached site, a renamed file because cache was deleted and new cache - file was created / and so on.... mod-pagespeed runs with the IP of your server and so you will see log - entries with 403 - errors with your IP. If you now configured Fail2ban to use "apache-noscript", you will see that the standard configuration is to ban hosts with 403 errors after 3 attempts during the find-time of 600 seconds - and *schwups*... your server IP is banned for 3600 seconds on all defined http(s) - ports.

 

[ASTOKES]

Thank-you that is very clear...