Resolved Where to put apache and nginx global rewrite rules?

[tkalfaoglu]

I have this code for our main site to stop the pesky crawlers in our main site's hosting settings:

Apache & nginx Settings for kalfaoglu.net :

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (SemrushBot|PetalBot|DotBot|gumgum-bot|postmanruntime|ag_dm_spider|scrapy|chimebot|trendkite-akashic-crawler|ZoominfoBot|Sogou|ALittle|Keybot|Buck|curl|webpr
osbot|RestSharp|Snap|SemrushBot|AhrefsBot|DataForSeoBot|Sogou|trendkite-akashic-crawler|cortex|ALittle|Go-http-client|RestSharp) [NC]
RewriteRule .* - [F,L]

and for nginx:

if ($http_user_agent ~ (SemrushBot|PetalBot|DotBot|gumgum-bot|postmanruntime|ag_dm_spider|scrapy|chimebot|trendkite-akashic-crawler|ZoominfoBot|Sogou|ALittle|Keybot|Buck|curl|webprosbot|RestSharp|Snap|SemrushBot|AhrefsBot|Da
taForSeoBot|Sogou|trendkite-akashic-crawler|cortex|ALittle|Go-http-client|RestSharp|MJ12bot)) {
        return 403;
}

I would like to make this applied to all sites we host. How do I go about making this globally for every site we host/will host?

Many thanks, -t

 

[Monty]

You could put those directives in your own file in /etc/httpd/conf.d and /etc/nginx/conf.d
Use a filename like "zzz_mytweaks.conf" or similar to make sure it is included as the last file

 

[Nik G]

what about additional apache and nginx directives in domain settings ?

 

[tkalfaoglu]

Many thanks! I created that zzz_mytweaks.conf, did a httpdmng --reconfigure-domain, but I do not see my changes in the vhost.conf of that domain. Am I suppose to see them there? I mean, how can I tell if my conf is included in the httpd conf?

 

[tkalfaoglu]

what about additional apache and nginx directives in domain settings ?

I was trying to do that globally for all the domains.

 

[tkalfaoglu]

Many thanks! I created that zzz_mytweaks.conf, did a httpdmng --reconfigure-domain, but I do not see my changes in the vhost.conf of that domain. Am I suppose to see them there? I mean, how can I tell if my conf is included in the httpd conf?

In fact, I don't see it anywhere in the /etc/httpd (recursively searched) or in /var/www/vhosts/kalfaoglu.net/conf directories.. I only see it my zzz_mytweaks file. Shouldn't I see it in my kalfaoglu.net conf files too?

Thanks!

 

[Monty]

Shouldn't I see it in my kalfaoglu.net conf files too?

Nope, you won't see those files anywhere except where you've created them (/etc/httpd/conf.d/zzz_mytweaks.conf and/or /etc/nginx/conf.d/zzz_mytweaks.conf)
Once you've created them, a simple reload of Apache + nginx is enough

Note: If you're on Debian or Ubuntu, the Apache global config is in /etc/apache2/conf.d

See also: Apache and Nginx Configuration Files

 

[tkalfaoglu]

many thanks!