• Introducing WebPros Cloud - a fully managed infrastructure platform purpose-built to simplify the deployment of WebPros products !  WebPros Cloud enables you to easily deliver WebPros solutions — without the complexity of managing the infrastructure.
    Join the pilot program today!
  • The Horde component is removed from Plesk Installer. We recommend switching to another webmail software supported in Plesk.
  • The BIND DNS server has already been deprecated and removed from Plesk for Windows.
    If a Plesk for Windows server is still using BIND, the upgrade to Plesk Obsidian 18.0.70 will be unavailable until the administrator switches the DNS server to Microsoft DNS. We strongly recommend transitioning to Microsoft DNS within the next 6 weeks, before the Plesk 18.0.70 release.

Question Best way to block bad user agents in global Nginx configuration?

goomba

New Pleskian
Server operating system version
CentOS 7
Plesk version and microupdate number
18.0.58
I have tried the following in a file like /etc/nginx/conf.d/bad_bots.conf, but it still allows successful HTTP response.

NGINX:
map $http_user_agent $bad_user_agent {
    default             0;
    ~*MJ12bot           1;
    ~*coccocbot-image   1;
    ~*Baiduspider       1;
}

server {
    if ($bad_user_agent) {
        return 403;
    }
}

I've been able to successfully block them for individual domains through the Plesk interface with an if ($http_user_agent ... ) approach. How can this be done globally? Also open to a global Apache method.
 
You'd rather want a rule lik
Code:
if ($http_user_agent ~* "(?:acunetix|BLEXBot|domaincrawler\.com|LinkpadBot|MJ12bot/v|majestic12\.co\.uk|AhrefsBot|TwengaBot|SemrushBot|nikto|winhttp|Xenu\s+Link\s+Sleuth|Baiduspider|HTTrack|clshttp|harvest|extract|grab|miner|python-requests)") {
    return 403;
}

However, why mess with Nginx if you have Fail2ban. I recommend reading this new article where we address the issue of bad bots and a lot more:
 
Hi Peter,

I have tried adding the above rules server side to nginx include meanwhile i got the below error. Do you have some work around for the same?

nginx: [emerg] "if" directive is not allowed here in /etc/nginx/conf.d/bad_bot_block.conf:1
nginx: configuration file /etc/nginx/nginx.conf test failed
 
Back
Top