Resolved mod_security issue with facebook share?

[L. Hagen]

Hi,

today I've noticed a problem and canot find a solution. I have a "normal Facebook" Share button on my websites. This button simple calls "Facebook". On my old server (not self maintained) this worked great, but now I get a ERROR 403.

I already checkes the mod_security log but this log is empty. Even all other los are not show any useful information. Until now I tried:

• disabled mod_security
• disabled firewall
• checked htaccess

I'm pretty sure that it is a configuration issue, but I have no idea what it could be. I hope that someone you you can help me or can give me a hint how this can be fixed.

Thank you and best regards,
Lars

 

[WebHostingAce]

I checked your website.

Facebook sharing button works fine for me.

 

[UFHH01]

Hi L. Hagen,

I can confirm your issue:

​

You have either a wordpress plugin, which might interfere, or you have missing or misconfigured nginx directives.
Pls. keep in mind, that some wordpress themes use as well ".htaccess" - files, which should be converted into nginx - directives and then added to the "Additional nginx directives" - textbox at your domain - specific "Apache & nginx Settings".

 

[G J Piper]

You may want to check and see if your server has implemented any kind of "image hot-linking protection." This is often a rewrite-style directive setting in nginx or apache that prevents websites not hosted on your server from displaying images found on your website(s).

If this is the case, your sharing plugin may in fact work, but the image that Facebook finds to display with the link will show a forbidden error like that, or even a different default image in some cases.

The setting may be found in an .htaccess file as a mod_rewrite directive or in any config file for your web services. Sorry I can't be more specific for you.

 

[L. Hagen]

The strange is that I have moved the complete installation to the new server without changing htaccess directives etc. On the old system it was working perfectly. I'm sure that the theme (completely written by myself) or any plugins are responsible for the blocking. There are no plugins that wrote something to the htaccess.

In the log I've found:

69.63.188.207 - [16/Apr/2017:23:55 +0200] "GET /de/results/?eventID=32264 HTTP/1.0 403 861 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"

I checked htaccess again and again. There is no directive to block the "facebookexternalhit nor the IP.

 

[UFHH01]

Hi L. Hagen,

did you check as well the suggestion from @G J Piper ?

 

[L. Hagen]

Hi UFHH01,

yes I did, but there is no such directive. I really guess that it has something to do with the nginx. When I try:

curl -i -H 'Accept: */*' -H 'Range: bytes=0-40960' -H 'Connection: close' -A 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' https://www.speedskatingnews.info

i received:

HTTP/1.1 403 Forbidden
Server: nginx
Date: Mon, 17 Apr 2017 07:53:02 GMT
Content-Type: text/html; charset=iso-8859-1
Content-Length: 406
Connection: close
Vary: Accept-Encoding

but I really don't know how I can fix this (I'm still not so familar with nginx and I have to learn to deal with it very soon).

 

[UFHH01]

Hi L. Hagen,

pls. share some more informations about

• Your current operating system
• Your current Plesk version ( incl. #MU )
• Your current PHP - Handler in use for the specific domain with issues
• Do you use Web Application Firewall ( ModSecurity ) on your server? If yes, with which rules and settings?

 

[L. Hagen]

Hi UFHH01,

well the settings are:

• ‪Ubuntu 14.04.5 LTS
• Plesk Onyx Version 17.0.17 Update #23
• PHP 7.1.3 / 7.1.4
  
• mod_security is activated with "Atomic Basic ModSecurity" (no additional or manual rules)
• apache webserver with nginx as proxy (normal package installation via Plesk - no manual compiled etc.)

thank you for your help!

 

[UFHH01]

Hi L. Hagen,

mod_security is activated with "Atomic Basic ModSecurity" (no additional or manual rules)

Pls. consider to DE-activate "ModSecurity" completely and test again, if this is your root cause. ( even if you already tested it... we should just be able to test it as well! And pls. check your corresponding ModSecurity log - file for errors/issues/problems. )

 

[L. Hagen]

Hi UFHH01,

nothing changed. I turned off "ModSecurity" again and have also checked the log-files. The log is empty and I also get a 403 with mod_security "off".

 

[UFHH01]

Hi L. Hagen,

are you using Fail2Ban ( with for example the "badbot" - jail ) ?

 

[L. Hagen]

no, it isn't installed

 

[UFHH01]

Hi L. Hagen,

well... the last suggestion from me would then be to insert an "Additional nginx directive" as for example:

       if ($http_user_agent ~* (facebookexternalhit|twitterbot|developers\.google\.com|googlebot|yahoo|bingbot)) {
           proxy_pass https://XXX.XXX.XXX.XXX:7081;
       }

... where XXX.XXX.XXX.XXX has to be replaced with the depending IP for the corresponding (sub)domain.
( I just used some common bots here - pls. consider to delete/add/modify the suggestion to your very own desire. )

 

[L. Hagen]

Hi UFHH01,

this what I tried recently (found on google). I entered:

gzip_proxied any;
if ($http_user_agent ~* (facebookexternalhit|twitterbot|developers\.google\.com|googlebot|yahoo|bingbot)) {
        proxy_pass https://85.214.248.107:7081;
 }

I get the following error message:

Invalid nginx-configuration: nginx: [emerg] "proxy_pass" directive is not allowed here in /var/www/vhosts/system/speedskatingnews.info/conf/vhost_nginx.conf:3 nginx: configuration file /etc/nginx/nginx.conf test failed

The vhost_nginx.conf just contains one line of code "gzip_proxied any;" and the "nginx.conf" is the standard installed version. So I never have touched them.

As mentioned before: I'm not yet very familar with nginx :-(

I also tried to put the directive into a block like:

location ~* {
if ($http_user_agent ~* (facebookexternalhit|twitterbot|developers\.google\.com|googlebot|yahoo|bingbot)) {
        proxy_pass https://85.214.248.107:7081;
      }
}

Then the proxy_pass seems to be accepted, but I get a 403 on every single page I triy to access. In other words the webpage is not accessible...

 

[UFHH01]

Hi L. Hagen,

this what I tried recently (found on google). I entered:

Thanx for providing the information AFTER I thought about a suggestion for you...

existent ( as standart ) is the nginx directive:

    location / {
        proxy_pass https://XXX.XXX.XXX.XXX:7081;
        proxy_set_header Host             $host;
        proxy_set_header X-Real-IP        $remote_addr;
        proxy_set_header X-Forwarded-For  $proxy_add_x_forwarded_for;
        proxy_set_header X-Accel-Internal /internal-nginx-static-location;
        access_log off;
    }

... and "gzip_proxied any;" can be found in your global "nginx.conf".

If you need any help for creating special nginx directives, pls. consider to visit the NGINX community forum: => Nginx Forum

 

[L. Hagen]

Well, of course I tried several thing before writing here in the forum ...but I thought I made mistakes. However it doesn't worked for me. I will check the nginx forum. I'm still hoping for a solution

However, I had to add "gzip_proxied any;, because the gzip is disabled by default in my nginx.conf...

#user  nginx;
worker_processes  1;

#error_log  /var/log/nginx/error.log;
#error_log  /var/log/nginx/error.log  notice;
#error_log  /var/log/nginx/error.log  info;

#pid        /var/run/nginx.pid;

include /etc/nginx/modules.conf.d/*.conf;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    #tcp_nodelay        on;

    #gzip  on;
    #gzip_disable "MSIE [1-6]\.(?!.*SV1)";

    server_tokens off;

    include /etc/nginx/conf.d/*.conf;
}

 

[L. Hagen]

Well, I have to ask again today, 'caus I don't find a solution for the orign problem. Until today I made a break a started new to discover the error. Some news:

• disabled the nginx proxy but the error (403) still happens. So (according to my first thoughts) it has nothing to do with a misconfigurred nginx
• then I disabled all additional security features (mod_security, firewall) - problem still exists
• removed htaccess - problem still exists
• the problem happens on ALL websites hosted on my server

Then I played a bit with the user agent. Usually facebook uses this UA "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)". The string that seems to be checked anywhere is "facebookexternal" (I shortened the UA until it was working):

curl -i -H 'Accept: */*' -H 'Range: bytes=0-40960' -H 'Connection: close' -A 'facebookexternalhit' https://www.speedskatingnews.info

Then I tried to find this string anywhere in a conf with:

grep -rnw '/etc/' -e "facebookexternal"

The result was empty. So no such string was found. However it doesn't work and I alsways get a 403 permission denied

HTTP/1.1 403 Forbidden
Server: nginx
Date: Mon, 24 Apr 2017 14:04:59 GMT
Content-Type: text/html; charset=iso-8859-1
Content-Length: 397
Connection: close
Vary: Accept-Encoding

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /
on this server.</p>
<p>Additionally, a 403 Forbidden
error was encountered while trying to use an ErrorDocument to handle the request.</p>
<hr>
<address>Apache Server at www.speedskatingnews.info Port 443</address>
</body></html>

where to hell does it come from? Please does anybody has an idea...

 

[L. Hagen]

Despite from my last post I could find the problem. My "grep" was wrong and there was a global apache2 rewrite rule in /etc/apache2/conf.d/rewrite.conf that has blocked the
facebookexternalhit.

Now it is working again.

 

[UFHH01]

Hi L. Hagen,

... and you used as well a search without including domain - specific configuration files and content search at: => /var/www/vhosts