Forwarded to devs Block repeated 403 Forbidden requests with Fail2Ban as included plesk-jail - Feature Request

[Azurel]

Username: Azurel

TITLE

Block repeated 403 Forbidden requests with Fail2Ban as included plesk-jail - Feature Request

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

CentOS Linux 8.2.2004 (Core)
Plesk Obsidian Version 18.0.31 Update #1

PROBLEM DESCRIPTION

I currently have tens of thousands of views of 403 in my access_ssl_log. My project has many crawler queries and my server is sometimes completely overloaded. That's why I implemented in Apache that UserAgents are blocked with 403 http status code by using obvious crawler scripts.

RewriteCond %{HTTP_USER_AGENT} ^(crawler_name1|crawler_name2|....) [NC]
RewriteRule .* - [F]

This helps, but it would be optimal here if after a certain number of 403 entries in the access_ssl_log the IP would be banned for a period of time, so that it does not generate any traffic at all. Currently its write huge access_ssl_log files.

In some times I have 50 requests per second. This is not a single IP, these are apps/plugins where users request data and the app then polls various projects in the background, including us. The more users the apps/plugins have, the worse for us.

Alternatively an support article would be nice, how to automatically monitor all vhost access_ssl_log and if an IP gets the 403 several times it will be banned for a period of time.
Like (I need this only in SSL websites):
action = iptables-multiport[name="JAIL_NAME", port="http,https,7080,7081"]
logpath = /var/www/vhosts/system/*/logs/access_ssl_log
maxretry = 5

And maybe this failregex?
failregex = ^<HOST>.* "GET .* HTTP/[0-9.]+" 403

I have created this Jail with ban period of 48 hours.

STEPS TO REPRODUCE

See description

ACTUAL RESULT

See description

EXPECTED RESULT

See description

ANY ADDITIONAL INFORMATION



YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Help with sorting out

 

[IgorG]

Please check plesk-apache-badbot jail instead. It is designed to do exactly what you're attempting to do but in a more straightforward way. You may either use it directly or customize it. To customize:

1. either copy the jail and filter and adjust the filter regex,
2. or add custom badbotscustom regex in jail definition: filter = apache-badbots[badbotscustom=your_custom_regex].

 

[Ehud]

**First file**

~~~
# cd /etc/fail2ban/filter.d/
~~~

create file:

~~~
# touch httpd-forbidden.conf
~~~

edit file content:

~~~
# vi httpd-forbidden.conf
~~~

~~~
[Definition]
failregex = ^<HOST> - - .*HTTP/[0-9]+(.[0-9]+)?" 403
^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"/[0-9]+(.[0-9]+)?" 403
~~~


*****************************************************************

**Second file**

~~~
# cd /etc/fail2ban/jail.d
~~~

create file:

~~~
# touch httpd-forbidden.conf
~~~

edit file content:

~~~
# vi httpd-forbidden.conf
~~~

To be:

~~~
[httpd-forbidden]
enabled = true
filter = httpd-forbidden
backend = polling
logpath = /var/log/apache2/*error.log
/var/www/vhosts/system/currenge.com/logs/access_ssl_log

bantime = 48h
maxretry = 2
findtime = 4800
port = http,https,7080,7081
banaction = iptables-multiport
action = iptables-multiport[name=apache, port="http,https,7080,7081"]
~~~



*****************************************************************


test tool via special fail2ban testing command:

~~~
# fail2ban-regex /var/www/vhosts/system/currenge.com/logs/access_ssl_log httpd-forbidden.conf --print-all-matched
~~~



*****************************************************************

Via Plesk GUI on Service Management restart Fail2Ban


*****************************************************************


If you lock yourself out, via SSH:

~~~
# fail2ban-client set httpd-forbidden unbanip your-ip-address-here
~~~




See links:

Example:

Block repeated 403 Forbidden requests with Fail2Ban

You’ve already blocked access and return HTTP 403 Forbidden. Here’s how to make Fail2Ban block users and bots still hammering your server with repeated requests.

www.ctrl.blog

Fail2Ban:

GitHub - fail2ban/fail2ban: Daemon to ban hosts that cause multiple authentication errors

Daemon to ban hosts that cause multiple authentication errors - fail2ban/fail2ban

www.fail2ban.org

Regex:

7.2. re — Regular expression operations — Python 2.7.18 documentation

docs.python.org