Bot protection blocks a bot I need

[wl-mh]

Server operating system version

Ubuntu 20.04

Plesk version and microupdate number

18.0.62 Update #1

Hello

When the bot protection is enable (Wordpress -> Security -> Bot protection) the following lines are added to /var/www/vhosts/system/*/conf/nginx.conf

if ($http_user_agent ~* "(?:acunetix|BLEXBot|domaincrawler\.com|LinkpadBot|MJ12bot/v|majestic12\.co\.uk|AhrefsBot|TwengaBot|SemrushBot|nikto|winhttp|Xenu\s+Link\s+Sleuth|Baiduspider|HTTrack|clshttp|harvest|extract|grab|miner|python-requests)") {
return 403;
}

I would like to grand access MJ12bot, because this is the data bot for my SEO tool Mangools, but still block all of the other bots. How can I do that? Where is the list of "bad bots" defined? I haven't found it in the database and the file system till now.

Thanks!

 

[Kaspar@Plesk]

There currently isn't any method to adjust the list of bots of the WP toolkit "Bot protection" option.

There is an request on our UserVoice page to implement this feature, which you can vote for. Features that become popular might be considered for implementation into Plesk.

Bot protection in Wordpress Toolkit should include the option to exclude some bots

It should be possible to exclude some bots from "bot protection" option in Wordpress Toolkit, for example, Semrushbot is used for seo in sites, and it should remain active, or acunetix is a security scanner and users may want to use it, whlie blocking other unwanted bots.

plesk.uservoice.com

 

[wl-mh]

Ok, I can understand, that it is not customizable at the moment. But the current list must be defined on some place. Directly in the code?

 

[Kaspar@Plesk]

Unfortunately it's not possible to adjust the list because it's hard coded.

What you can try as a workaround is to disabled the Bot protection option and instead add the same nginx directive to the "Additional nginx directives" of the domain (which can be found at the Hosting & DNS tab > Apache & nginx).

if ($http_user_agent ~* "(?:acunetix|BLEXBot|domaincrawler\.com|LinkpadBot|majestic12\.co\.uk|AhrefsBot|TwengaBot|SemrushBot|nikto|winhttp|Xenu\s+Link\s+Sleuth|Baiduspider|HTTrack|clshttp|harvest|extract|grab|miner|python-requests)") {
return 403;
}

 

[wl-mh]

Sure, this will work. But then, I need to add this code to all Websites (a lot) of the server per hand and Plesk will show the Wordpress instance as unsecure, because the Bot protection is disabled.

An alternative is, to write a small script which do a search and replace in all /var/www/vhosts/system/*/conf/nginx.conf files and (in case at least one file was changed) do a NGINX reload. This script needs to run very 10 minutes or so via cron. This should work but it is a dirty hack.