Question Fail2ban: Cloudflare API

[omexlu]

Hello,

I found this on internet and it is very interesting to ban in cloudflare for to many requests maked:
How to Integrate fail2ban with CloudFlare API v4 Guide •

I use apache + nginx in plesk.

How can i implement this trougth the Plesk GUI?
Or maybe a functional example of iptables only (fail2ban = http-dos) and then i can modify it

Thanks in advance

 

[omexlu]

Can that be working? ( @UFHH01 , @IgorG )

Can i enter this so in the GUI?

Creating a new jail:

[cloudflare]
enabled = true
filter = cloudflare
port = http,https

actionstart =
actionstop =
actioncheck =
actionban = curl -s -X POST "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules" \
            -H "X-Auth-Email: <cfuser>" \
            -H "X-Auth-Key: <cftoken>" \
            -H "Content-Type: application/json" \
            --data '{"mode":"block","configuration":{"target":"ip","value":"<ip>"},"notes":"Fail2ban"}'

actionunban = curl -s -X DELETE "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(
              curl -s -X GET "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1&match=all" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json" | awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'id'\042/){print $(i+1)}}}' | tr -d '"' | head -n 1)" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json"

[Init]
cfuser = put-your-cloudflare-email-here
cftoken = put-your-API-key-here

logpath = /var/www/vhosts/*/logs/access_ssl_log /var/www/vhosts/*/logs/access_log
maxretry = 200
findtime = 600
bantime = 600

Create a new filter:

[Definition]
failregex = ^<HOST> -.*"(GET|POST).*
ignoreregex =

In this case the are no iptables generated to not flood the iptables but the IP get blocked/unblocked trought cloudflare where is more effective for this jail.

Any suggestion or improvements?

EDIT: The API Calls from my local maschine (ubuntu) works so i need only to implement that correctly in fail2ban (Plesk GUI) , and at this point i need help

I tried to enter this part on Action-Part but get an error:

actionstart =
actionstop =
actioncheck =
actionban = curl -s -X POST "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules" \
            -H "X-Auth-Email: <cfuser>" \
            -H "X-Auth-Key: <cftoken>" \
            -H "Content-Type: application/json" \
            --data '{"mode":"block","configuration":{"target":"ip","value":"<ip>"},"notes":"Fail2ban"}'

actionunban = curl -s -X DELETE "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(
              curl -s -X GET "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1&match=all" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json" | awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'id'\042/){print $(i+1)}}}' | tr -d '"' | head -n 1)" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json"

[Init]
cfuser = put-your-cloudflare-email-here
cftoken = put-your-API-key-here

Thanks

 

[tomer628]

i will glud to know too..

 

[omexlu]

Found a way to do it:

Create a file under /etc/fail2ban/action.d named cloudflare.conf

#
# Author: Mike Andreasen from https://guides.wp-bullet.com
# Adapted Source: https://github.com/fail2ban/fail2ban/blob/master/config/action.d/cloudflare.conf
# Referenced from: https://www.normyee.net/blog/2012/02/02/adding-cloudflare-support-to-fail2ban by NORM YEE
#
# To get your Cloudflare API key: https://www.cloudflare.com/my-account
#

[Definition]

# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart =

# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop =

# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck =

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:      IP address
#            number of failures
#            unix timestamp of the ban time
# Values:  CMD

actionban = curl -s -X POST "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules" \
            -H "X-Auth-Email: <cfuser>" \
            -H "X-Auth-Key: <cftoken>" \
            -H "Content-Type: application/json" \
            --data '{"mode":"block","configuration":{"target":"ip","value":"<ip>"},"notes":"Fail2ban"}'

# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:      IP address
#            number of failures
#            unix timestamp of the ban time
# Values:  CMD
#

actionunban = curl -s -X DELETE "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(
              curl -s -X GET "https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1&match=all" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json" | awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'id'\042/){print $(i+1)}}}' | tr -d '"' | head -n 1)" \
              -H "X-Auth-Email: <cfuser>" \
              -H "X-Auth-Key: <cftoken>" \
              -H "Content-Type: application/json"

[Init]

# Option: cfuser
# Notes.: Replaces <cfuser> in actionban and actionunban with cfuser value below
# Values: Your CloudFlare user account

cfuser = put-your-cloudflare-email-here

# Option: cftoken
# Notes.: Replaces <cftoken> in actionban and actionunban with cftoken value below
# Values: Your CloudFlare API key
cftoken = put-your-API-key-here

FILTER:

[Definition]
failregex = ^<HOST> -.*"(GET|POST).*
ignoreregex =

JAIL:

[http-ddos-custom]
enabled = true
filter = http-ddos-custom
action = cloudflare
logpath = /var/www/vhosts/*/logs/access_ssl_log
/var/www/vhosts/*/logs/access_log
maxretry = 300

Works like a charm

 

[trialotto]

@omexlu

You stated

Works like a charm

and I must say: no, OR, stated differently, yes, to some extent.

After all, for this action to work, one has to use

- CloudFlare DNS, (and/or)
- CloudFlare paid-for plans OR the CloudFlare ServerShield extension for Plesk.

The issue with the proposed action is and/or can be concerning the facts that

1 - the rulesets are limited in any CloudFlare free plan and/or when (only) using CloudFlare DNS, (and)

2 - specific conflicts can arise when using the CloudFlare ServerShield extension for Plesk, (and)

3 - there is no need to define a DDoS related jail when using CloudFlare, since CloudFlare is "in front of" the server, making jails on the server rather obsolete, when taking into account that any DDoS is already handled (and handled better) by CloudFlare,

and I am not even emphasizing the peculiar disadvantages of CloudFlare, such as the facts that

- they often are the intended target of DDoS attacks (read: using CloudFlare means that your sites are more prone to DDoS occurring at the CloudFlare "level"),
- free plans are having less priority (read: in the case of DDoS attacks, sites associated with free CloudFlare plans can become slow or even unresponsive: CloudFlare paid-for plans are getting very high priority over sites under free plans),
- CloudFlare is rather "dominant" (read: changes in DNS records on the relevant nameservers can be less persistent when using CloudFlare)
- bandwith limiting is (always) a paid-for functionality in CloudFlare (read: the essential tool for handling DDoS attacks costs money)

and so on.

In short, CloudFlare is a very advanced proxy based upon Nginx, but it has to be paid for to function as a proper proxy.

In conclusion, the following has to be noted:

a) if and when using CloudFlare, there is no need to have a Fail2Ban jail or action (since CloudFlare does that job), AND

b) the native Nginx proxy on any Plesk instance is a free and good alternative for preventing DDoS attacks (and even then Fail2Ban jails or actions are not needed).


I would strongly recommend to deactivate (or remove) the jail, since it has no function and/or since Fail2Ban itself can be somewhat "resource hungry".

Hope the above explains a bit of the background information.

Regards...........

 

[omexlu]

I only have this because i have a http get flood from time to time and so the ip gets blocked, because the attacker have found any way to bypass cloudflare in this case.

With this i can control it a little bit. If there will be difficults i deactivate them again

 

[trialotto]

@omexlu

It is quite hard "to bypass CloudFlare", so the "attacker" is probably doing something else that you do not take into account.

Nevertheless, one can also do the following:

- add the one (or more) bad IPs to a permanent ban list and load that as active Nginx config (read: hence creating a permanent block, no need for Fail2Ban)
- apply bandwith throttling via the native Nginx directives (read: bad requests do not reach Apache and bad IPs are blocked to some extent, even if they are not yet on the permanent ban list loaded as active Nginx config)

To be honest, the above two tweaks are to be preferred: they are very clean, effective and do not impose any usage of resources.

Regards........

 

[omexlu]

@trialotto

And how i do that?

The problem is that when very many ips from the attacker its a lot of work to put them into that blacklist.

Maybe roll back to fail2ban iptables bann BUT i don't blocks the ips correctly, some test with recidive (
iptables-allports[name=recidive]) was possitive there the bann was correctly effected but why not with multiport?

What about 'limit_req' for nginx?

Maybe something like that is sufficent:
in /etc/nginx/nginx.conf adding this on http block:

limit_req_zone $binary_remote_addr zone=flood:10m rate=1r/s;

And then in every nginx directive in plesk

location ~ \.php {
    limit_req zone=flood burst=5 nodelay;
}

Is that not enought to handle that problem with GET-Flood? Maybe some improvments?

 

[trialotto]

@omexlu

Your last post consist of a number of explicit AND implicit remarks and questions.

With respect to

Maybe roll back to fail2ban iptables bann BUT i don't blocks the ips correctly, some test with recidive (
iptables-allports[name=recidive]) was possitive there the bann was correctly effected but why not with multiport?

it is pretty safe to assume that something is non-default or wrong with the multiport action (inspect corresponding file in the action.d directory).

With respect to

The problem is that when very many ips from the attacker its a lot of work to put them into that blacklist.

I can give

- an answer to the (explicit) remark: it is not needed to do that manually, Fail2Ban can be used to automate that
- a comment to the (implicit) question: send me a private message and I will elaborate on this

With respect to

What about 'limit_req' for nginx?

I can give you the following hints:

- rather not, use the limit_conn directive (ngx_http_limit_conn_module) IF you are going to use a static module (and compile it into a Nginx custom binary)
- the native Nginx in Plesk allows for dynamic modules
- consider to use the limit_conn directive (ngx_stream_limit_conn_module, being part of the dynamic "stream" module)

and all of the above requires recompiling Nginx and/or creating a custom Nginx binary.

Have a look at

- Restricting Access | NGINX (AND)
- Introducing Dynamic Modules in NGINX 1.9.11 - NGINX

and be sure to note the difference between static and dynamic modules, also note the "--with-stream=dynamic" flag required at compilation time for dynamic modules.

However, I do want to point out that the native Nginx (as shipped with Plesk) contains the (static) module called ngx_http_v2_module, allowing for throttling via

- the http2_max_requests directive
- the http2_max_concurrent_streams directive

and a good combination of the above two directives gives you a lot of control over DDoS attempts aimed at the web server, if you did activate HTTP/2.

In short and in summary, one cannot really say "yes" or "no" to a specific Nginx directive, it all depends on the base structure, the stack and configuration applied.

Nevertheless, I hope that I gave you enough information to have an idea about all the (excellent) possibilities that Nginx provides.

Regards.......

 

[omexlu]

@trialotto

Thank you very much for you good explanation and infos about this thread.
I have make you a PN about the block IP part of nginx automation in combination with fail2ban.

For the other limition i can use it without recompiling in this way:
in /etc/nginx/nginx.conf -> http-block:

limit_conn_zone $binary_remote_address zone=addr:10m;

In nginx additional part in Plesk:

location ~ \.php {
limit_conn addr 1;
}

This will work or not? But i preffer you nginx block methode i think this is more than enought for that problem.

 

[TurabG]

There is already a Cloudflare module (not extension) loaded in Plesk so that your server could catch "real IPs" of the requesters. That means you already ban actual IPs with fail2ban wihout needing a Cloudflare action.

Secondly, as @trialotto stated, it's kind of obsolete to create a Cloudflare action while you are already behind the Cloudflare firewall. If Cloudflare already manages to ban malicious IPs, why would you try to detect them and ban again while they are already stopped before they reach you. If you're saying that you consider some of actions malicious while Clodflare doesn't; then again as stated; free service of Cloudflare has a very limited number of rule sets available. That means your action would actually never work and will do nothing but consuming system resources for doing nothing.

 

[omexlu]

Thats the problem that cloudflare dosn't catch this attack, so we are working on it
@trialotto will explain me how to automatize this with nginx deny in combination with fail2ban, in this time i use that API call.

Maybe something like this can be make with nginx deny and fail2ban:
create a action that make actionban:

echo "deny <ip>;" >> /etc/nginx/blockips.conf

in additional nginx directive from plesk gui enter the following:

include /etc/nginx/blockips.conf;

Here the ips would be blocked automaticly if fail2ban triggers, can that works that way? @trialotto

and for example to delete on actionunban:

sed -i '/deny <ip>;/d' /etc/nginx/blockips.conf

The problem here is that nginx need a reload to take affect what can be critical if many entrys get done.

EDIT: I have tested now the part above, that shine to work the problem is that how to reload nginx after that because fail2ban is exectuted by its own user and this don't work:

echo "deny <ip>;" >> /etc/nginx/blockips.conf && service nginx reload
sed -i '/deny <ip>;/d' /etc/nginx/blockips.conf && service nginx reload

How can i do that else?

 

[trialotto]

@omexlu

Some important remarks with respect to your last post:

1 - it is not needed to use the include directive, just place the blockips.conf file in the /etc/nginx/conf.d/ directory (all config files in this directory are loaded)

2 - Nginx does need a reload, but it is not recommended to do an automatic reload: you will end up blocking a lot of IPs, including "good" IPs (not acceptable!)

3 - the IP blacklist in a .conf file generated by Fail2Ban should (only) be used as a "panic button": when under attack, reload nginx and all bad IPs are persistently blocked

4 - create a Fail2Ban filter that allows to store bad IPs as long as possible: for instance, take a duration of one year

5 - the "service nginx reload" will never work in a Fail2Ban action: just remove that (when keeping in mind points 2 and 3)

6 - the echo and sed commands in your action will work, but they are not using Fail2Ban optimally.........you can get more out of Fail2Ban variables!

Regards........

 

[omexlu]

@omexlu

Some important remarks with respect to your last post:

1 - it is not needed to use the include directive, just place the blockips.conf file in the /etc/nginx/conf.d/ directory (all config files in this directory are loaded)

2 - Nginx does need a reload, but it is not recommended to do an automatic reload: you will end up blocking a lot of IPs, including "good" IPs (not acceptable!)

3 - the IP blacklist in a .conf file generated by Fail2Ban should (only) be used as a "panic button": when under attack, reload nginx and all bad IPs are persistently blocked

4 - create a Fail2Ban filter that allows to store bad IPs as long as possible: for instance, take a duration of one year

5 - the "service nginx reload" will never work in a Fail2Ban action: just remove that (when keeping in mind points 2 and 3)

6 - the echo and sed commands in your action will work, but they are not using Fail2Ban optimally.........you can get more out of Fail2Ban variables!

Regards........

Hi,

Thanks for you reply

1 - i have done so and move blockips.conf to /etc/nginx/conf.d/ so it get loaded automaticly, this works.

2 - yes it still needs a nginx reload but i want to automise this because i m a lot of time not at home, so i want these attacks banned after fail2ban get triggered.

3 - 4 - 5 Okay i understand

6 - maybe i gonna improve that if needed.

Can i maybe create a cronjob who reload the nginx all few minutes?

Or maybe better create a cronjob who checks if a fail2ban has been done and then it reloads the service, so it don't get reloaded constantly.

 

[WP Bullet]

Hello Plesk users,

I just wanted to chime in here to clear up some concerns about using the jail that I wrote. While it is true that the Cloudflare firewall can be very useful, it is far from perfect. I have seen on several occasions where Cloudflare sends traffic through which is malicious but not detected. When this happens, even if you have fail2ban or any other firewall rule on the server level to ban these requests, Cloudflare will continue to send them (since your server will see it as coming from CF's IPs and won't block them) and potentially very quickly which can have a DoS effect. Cloudflare continues to send them because it has no firewall rule for these IPs which is why the action I made is ideal since it allows for nuanced filters unique to any attacks on your server and then blocks them on Cloudflare so the traffic will not be sent to you.

The Cloudflare firewall rules are pretty liberal How many IPs can I add to rules in the IP Firewall? so there is not much to worry about since fail2ban unbans these after the ban limit expires as well and deletes the IP rule.

These are based on my experience and therefore I would not agree with these statements by @trialotto as they do not apply based on my experience with paid or free plans. I would strongly urge that anybody running Cloudflare run tests to determine the best setup for you.

'3 - there is no need to define a DDoS related jail when using CloudFlare, since CloudFlare is "in front of" the server, making jails on the server rather obsolete, when taking into account that any DDoS is already handled (and handled better) by CloudFlare,

a) if and when using CloudFlare, there is no need to have a Fail2Ban jail or action (since CloudFlare does that job).'

As @trialotto says, fail2ban can be resource hungry if your error logs are huge (this is usually a sign to fix any plugins or code) since python is used to parse the logs with the defined filters so you should be cognizant of this as well before implementing fail2ban. It can also be resource hungry if you have many log files because of a lot of sites. Personally I prefer the one site per VPS/container philosophy but I can appreciate many people do not run this way so I will add a caveat to the guide making sure users are well-informed.