Question Apache .htaccess redirecting problem with nginx

[Matt M]

I have the following code in my .htaccess:

RewriteEngine on
RewriteBase /

RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/$ /cgi-bin/adm/constructor.pl?d=$1  [T=application/x-httpd-cgi,L]
RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/index\.html$ /cgi-bin/adm/constructor.pl?d=$1  [T=application/x-httpd-cgi,L]
RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/index_(.+)\.html$ /cgi-bin/adm/constructor.pl?d=$1&l=$2  [T=application/x-httpd-cgi,L]
RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/(.*)\.html$ /cgi-bin/adm/constructor.pl?d=$1&p=$2  [T=application/x-httpd-cgi,L]

Every part of this works correctly, except the first rule:

RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/$ /cgi-bin/adm/constructor.pl?d=$1  [T=application/x-httpd-cgi,L]

For URLs matching that rule, it looks like nginx never passes the request to Apache and returns 404.

If I open nginx.conf for the domain, I find the following code:

        location ~ /$ {
                index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";
        }

Commenting this code out makes everything work like it's supposed to. However, this is not a good long-term solution, because the file is automatically generated.

Any idea as to how I can resolve this issue?

(I have "Smart static files processing" turned off in Plesk)

 

[Bitpalast]

For URLs matching that rule, it looks like nginx never passes the request to Apache and returns 404.

Is this what the proxy access and/or error logs state?

 

[Matt M]

Is this what the proxy access and/or error logs state?

Yes. The proxy_error_log contains the following message:

2018/05/25 21:57:46 [error] 23082#0: *456 "/var/www/vhosts/redacted.com/httpdocs/video/index.html" is not found (2: No such file or directory), client: XX.XX.XX.XX, server: redacted.com, request: "GET /video/ HTTP/1.1", host: "www.redacted.com", referrer: "http://www.redacted.com/"

The error_log contains no message (as I would expect since the request is not being passed). I probably should've also mentioned that the 404 page that is returned is the default nginx 404 page.

 

[Bitpalast]

- Make sure that Nginx is ineed enabled. Maybe web server configuration files are not being generated correctly, because some basic setting on the system is wrong.
# /usr/local/psa/admin/bin/nginxmng -s
should return "Enabled".

- You wrote that you turned smart static files processing off, but did you leave proxy mode on? If not, turn proy mode on.

- Make sure that your Index file is named in all lower case letters. A very frequent mistake is that one of the letters is an uppercase letter, such as "Index.html" instead of "index.html". This will lead exactly to the symptom that you have described.

- PHP must NOT be set to "FPM served by nginx", but must be "by Apache". This is another frequently done mistake. If set to "by nginx", requests are not passed to Apache.

- Check whether the issue only appears when the website is opened through https://. In that case, it is an SSL configuration issue. Sometimes with this symptom the website will open when opened by http:// instead of https://. If this is the case, you'll need to look into the basic SSL configuration of the system.

 

[Matt M]

I think you are confused about the problem. There is no file called "index.html".
When a user accesses http://www.redacted.com/video/
I want the user redirected (hidden redirect) to http://www.redacted.com/cgi-bin/adm/constructor.pl?d=video
That is what I have set up the Apache rewrite rules for (code shown in my OP, but repeated here):

RewriteCond %{REQUEST_URI} !/blog/
RewriteCond %{REQUEST_URI} !/sendmail/
RewriteRule  ^(.*)/$ /cgi-bin/adm/constructor.pl?d=$1  [T=application/x-httpd-cgi,L]

However, it looks like the following nginx code is interfering with the Apache rewrite rules:

        location ~ /$ {
                index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";
        }

How can I get around this?

EDITED TO ADD:
The following redirects should happen:

1. http://www.redacted.com/video/ -> http://www.redacted.com/cgi-bin/adm/constructor.pl?d=video
   
2. http://www.redacted.com/video/index.html -> http://www.redacted.com/cgi-bin/adm/constructor.pl?d=video
3. http://www.redacted.com/video/index_2.html -> http://www.redacted.com/cgi-bin/adm/constructor.pl?d=video&l=2
4. http://www.redacted.com/video/test.html -> http://www.redacted.com/cgi-bin/adm/constructor.pl?d=video&p=test

All of these redirects work correctly, EXCEPT #1.

The folder /var/www/vhosts/redacted.com/httpdocs/video DOES NOT exist. It's not supposed to.

 

[Matt M]

I have tried running the following to disable nginx and have Apache handle everything:

plesk bin domain -u redacted.com  -nginx-transparent-mode true

Same problem, nginx returns a 404 error. This is not expected behavior.

 

[Bitpalast]

Maybe you can try to translate the .htaccess rules to Nginx rules such as

location /blog/ {
}
location /sendmail/ {
}
location / {
rewrite ^/(.*)/$ /cgi-bin/adm/constructor.pl?d=$1;
}

Regarding the Nginx "transparency": It may be necessary to reload the configuration with
# service nginx reload
for it to take effect.

 

[Matt M]

I have been looking at the template at /usr/local/psa/admin/conf/templates/default/domain/nginxDomainVirtualHost.php

The block that is causing the problem, this block:

        location ~ /$ {
                index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";
        }

Is coded by the following code in the template:

        <?php if ($VAR->domain->physicalHosting->directoryIndex): ?>
    location ~ /$ {
        index <?=$VAR->quote($VAR->domain->physicalHosting->directoryIndex)?>;
    }
        <?php endif ?>

So clearing directoryIndex should help... except that I'm not sure how to do this, and it's also used in the Apache vhost configuration

Can anybody explain to me what in the heck this code does in the first place? Commenting it out has no effect that I can see.

        location ~ /$ {
                index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";
        }

Shouldn't just putting the line

index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";

under "location / {" achieve the same thing?

 

[Matt M]

Maybe you can try to translate the .htaccess rules to Nginx rules such as

location /blog/ {
}
location /sendmail/ {
}
location / {
rewrite ^/(.*)/$ /cgi-bin/adm/constructor.pl?d=$1;
}

Regarding the Nginx "transparency": It may be necessary to reload the configuration with
# service nginx reload
for it to take effect.

When I add these directives to "Additional nginx directives" in Plesk, I get:

Invalid nginx configuration: nginx: [emerg] duplicate location "/" in /var/www/vhosts/system/annalevinson.com/conf/vhost_nginx.conf:5 nginx: configuration file /etc/nginx/nginx.conf test failed

I tried "service nginx reload" to get transparency mode to activate. It made no difference.

 

[Bitpalast]

In this case I think the last resort is to create custom templates from
/usr/local/psa/admin/conf/templates/default/domain/nginxDomainVirtualHost.php
into
/usr/local/psa/admin/conf/templates/custom/domain/nginxDomainVirtualHost.php
so that on re-generation they will be used instead of the default templates. Do not overwrite or change the default templates.

 

[Matt M]

However, I am concerned about breaking other websites on this server. It seems silly to break global templates to fix 1 domain out of about 60 on this server.

 

[Yan D. Alaury]

Hi Peter,
Really sorry to disturb you ;-)
I have a real trouble with Google and a huge mistake I've done when writing the urls of the menu:
I have written:

https://mydomainname.com/sorry.php/

Goggle indexed this site and these urls with a ending slash.
So my site is now on google and as you imagine the pages does not show up correctly.
I tried to modify the urls into the sitemap and send it to google last december but it always show errors
My question is:
How must I make a rewrite from https://mydomainname.com/sorry.php/ to

https://mydomainname.com/sorry.php

I use also NGINX and APACHE and I work under Plesk 17.8.11 Centos 7.9
Into plesk I add for Apache https:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_URI} ^/sorry.php/ [NC]
RewriteRule .* - [L]
RewriteRule ^(.*)$ https://mydomainname.com/sorry.php$1 [L,R=301]
</IfModule>
and for NGINX I add:
rewrite ^/https://mydomainname.com/sorry.php/$ mydomainname.com/sorry.php redirect;
And this doesn't work
I really need some help
Thanks a lot and Have a good day
Yan