Resolved NGiNX - browser caching for static resources

[Eleshar]

Hi,

On Plesk 12.0 I used the following rules under additional nginx directives to enable browser caching for static resources:

location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff|svg)$ {
expires 30d;
add_header Pragma public;
add_header Cache-Control “public”;
try_files $uri @fallback;
}

but since upgrading to Plesk 12.5 it is no longer working. No matter what I've tried so far has worked to get browser caching for static resources working again.

Does anyone have any ideas?

 

[UFHH01]

Hi Eleshar,

to find issues/problems or misconfigurations, we really need more informations. You should consider to add depending configuration files from "/var/www/vhosts/system/YOUR_DOMAIN.COM/conf" and you should add informations, HOW you set up serving static files in your domain hosting configuration.

The additional directive you provided is correct, but depend on your hosting settings for the domain.

 

[rbstern]

I'm having the same problem as Eleshar, in Plesk 12.0.18 (with current update) while trying to have a domain add browser caching to the header for static files served by nginx.

Web server settings for a domain I am trying to get working with browser caching:

Serve static files directly by nginx (checked, with the file types: ac3 avi bmp css cue dat doc docx dts exe flv gif gz htm html ico img iso jpeg jpg js mkv mp3 mp4 mpeg mpg ogg pdf png ppt pptx txt wav xls xlsx zip)

Additional Nginxi directives for domain:

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
expires 7d;
access_log off;
add_header Cache-Control "public";
}

I've verified that the above directive was saved by Plesk in the /var/www/vhosts/system/testeddomain.com/conf/vhost_nginx.conf and made sure nginx reloaded without any errors.

Testing with :

curl -I http://testeddomain.com/test.jpg file

returns the following header:

HTTP/1.1 200 OK
Server: nginx
Date: Fri, 02 Oct 2015 17:28:45 GMT
Content-Type: image/jpeg
Content-Length: 50342
Last-Modified: Wed, 09 Sep 2015 21:17:12 GMT
Connection: keep-alive
ETag: "55f0a1d8-c4a6"
Accept-Ranges: bytes

No caching instructions in the header.

No nginx errors in the error log.

Mystifying.

 

[Janko1000]

Try to delete "jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc" from the "Serve static files directly by nginx" List.

 

[bulent]

Try to delete "jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc" from the "Serve static files directly by nginx" List.

Same problem with me,
removed these file types but did not help.
If I remove the tick from "Smart static files processing" it sets correct headers, but then my WP install do not want to open posts.

Any other sugesstions?

 

[bulent]

So, after some digging, I found solution to my problem.
In Plesk 12.0 in additional nginx directive, I had these directives:

//for Wordpress permalinks
if (!-e $request_filename){
rewrite ^(.*)$ /index.php break;
}

//For gzip
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml application/x-font-ttf font/opentype;

//And Cache Control
location ~* \.(?:ico|css|js|gif|jpe?g|png|svg)$ {
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
}
All with tick on "Smart static files processing ", "Serve static files directly by nginx " and "Process PHP by nginx"

But in Plesk 12.5 I left only these directives

gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml application/x-font-ttf font/opentype;

location ~* \.(?:ico|css|js|gif|jpe?g|png|svg|woff)$ {
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
}

Nginx and php-fpm 5.6 in php settins are selected, tick from "Smart static files processing" and "Serve static files directly by nginx " is removed.
Reset once permalinks in WordPress as http://example.com/sample-post/ and it works like a charm.

 

[trialotto]

@Eleshar,

In response to your post (and other posts)

Hi,

On Plesk 12.0 I used the following rules under additional nginx directives to enable browser caching for static resources:

location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff|svg)$ {
expires 30d;
add_header Pragma public;
add_header Cache-Control “public”;
try_files $uri @fallback;
}

but since upgrading to Plesk 12.5 it is no longer working. No matter what I've tried so far has worked to get browser caching for static resources working again.

Does anyone have any ideas?

the following has to be considered:

a) the simple solution:

- manually removing js css png jpg gif ico woff and svg from the textbox, below the "serve static files with nginx..." checkbox.
- click on OK and everything should work

and, after reloading nginx, the custom directives should be present in /var/www/vhosts/system/<domain>/conf/vhost_nginx.conf,

b) the root cause of the problem: the current nginx template creates the file /var/www/vhosts/system/<domain>/conf/nginx.conf, which contains the lines (in the server block):

location ~ ^/(.*\.(<all files in the textblock below the "serve static files with nginx ..." checkbox>))$ {
try_files $uri @fallback;
}

include "/var/www/vhosts/system/<domain>/conf/vhost_nginx.conf";

and these lines are, in essence, misplaced: it is better to have the include directive before the location directive, mentioned above.

What happens is the following: nginx does not read your custom directives, involving js css etc, since it finds the before mentioned location directive first.

As such, nginx will execute "try_files $uri @fallback" and never comes to any custom specification of directives, as specified and stored in vhost_nginx.conf.

This is why the "simple solution" from point a) will work: specific file extentions (removed from the textblock) will not cause nginx to execute the before mentioned location directive, but will allow nginx to proceed to the directives in vhost_nginx.conf and execute them.

Sure, one can also change the /var/www/vhosts/system/<domain>/conf/nginx.conf file, in order to have the include directive before the location directive.


Finally, note that, in my humble opinion, one does not have to mess with custom nginx templates, so I have notified Odin Team and requested to change the default templates.

Hope that Odin Team will implement my suggestions (with respect to nginx) very soon.

Anyway, hope that all of the above will help you (and others) to get proper nginx config settings.

Regards....

PS @Eleshar, also remove the line "try_files $uri @fallback;" from your custom location directive. This line is not necessary at all.

 

[trialotto]

@rbstern,

No, not mystifying.

You used:

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$

and all of the files behind the ?: should be excluded from the textblock below the "serve static files with nginx ... " checkbox.

See my previous post, for an explanation.

Also note that it is better to remove the ?: in your location directive (it actually has little effect, but can impact performance heavily, when a lot of requests occur).

Regards....

 

[trialotto]

@bulent,

The issue on hand was

//for Wordpress permalinks
if (!-e $request_filename){
rewrite ^(.*)$ /index.php break;
}

In essence, this should never be implemented as such. It is a dynamic redirection with nginx, on top of the already existing redirects in WordPress.

The problem was that you redirected ALL traffic to the home (i.e. index) page, implying that (for instance) logins, posts (and so on) are redirected to the wrong page (i.e. php files).

The whole issue has nothing to do with php, php-fpm, nginx or the "smart processing..." and "serve static..." settings.

I am pretty sure that you can try to (re-)activate these specific settings, they should work on your server (and are working on my servers).

Finally, it is not really necessary to define the "add_header Pragma public;" in the location directive, since "Pragma" has to be considered "old and deprecated".

Another tip: I assume that you have activated the gzip directives on the WordPress domain only, while it can be considered valuable to have this configuration active for all sites.

In order to allow gzip for all sites, just

- move your gzip related lines (they are fine, by the way) to /etc/nginx/conf.d/<custom-name>.conf
- execute: nginx -t && service nginx reload

and all files will have gzip compression activated, with the settings you specified.

Hope that all of the above helps!

Regards......

 

[bulent]

@trialotto,

I did exactly what you said.

Left the tick on "Smart static files processing"

Left the tick on "Serve static files directly by nginx"
and removed ico|css|js|gif|jpe?g|png|svg

in Additional nginx directives left only:
location ~* \.(?:ico|css|js|gif|jpe?g|png|svg)$ {
expires 30d;
add_header Cache-Control "public";
}

Put gzip related lines in /etc/nginx/conf.d/custom.conf file.
/meanwhile that's one is a good idea, thank's/

In PHP settings I have PHP 5.16, run PHP as "FPM application served by nginx"

When I set Wordpress permalinks like http://example.com/sample-post and when I try to open some post I get 404 not foud / nginx

When I have in custom nginx directives:
if (!-e $request_filename){
rewrite ^(.*)$ /index.php break;
}
I can use this kind of permalinks http://example.com/sample-post, but without that line I can use only http://example.com/?p=123 like permalinks

Also with or without this rewrite rule I do not have expire headers set in files

1. This issue with permalinks and rewrite rule is only related to Wordpress installs ony with pretty URLs, it work perfect without that rule if you use ugly ULRs.
2. The explanation in my previous post is not OK, because I foud that it relies on .htaccess file for WordPress rewrite rules, which means it falls back to Apache

 

[trialotto]

@bulent,

In response to your (last) post, first some questions:

a) did you regenerate the permalinks after activating the "FPM application served by nginx"?

b) did you use a Plesk-based install of WordPress (i.e. an APS package), or a custom/manual installation of a WordPress version?

c) do you use some type of caching within WordPress (a caching plugin of any kind) and did you flush the cache?

I suppose that permalink regeneration would offer some comfort, but that would not be a solution to the issue on hand.

In general, the "smart static file processing" is

- not necessary for a WordPress installation, with "Serve static files directly by nginx" activated,
- probably the root cause of the problem you encountered (even though I cannot pinpoint the exact cause),

and, therefore, you could try to deactivate the "smart static file processing".

Note that it will not really decrease performance: WordPress with "serve static files directly by nginx" activated is already "fast enough".

Also note that you can considerable performance boosts by using

- a "self-hosted CDN" for static files (i.e. a subdomain on the same server, or a sub-/domain on an external server)
- Redis Cache (incredible speed!)

but that are not really the solutions to the problems you encountered.

Let´s focus on the statements made.

You stated:

The explanation in my previous post is not OK, because I foud that it relies on .htaccess file for WordPress rewrite rules, which means it falls back to Apache

I am not sure what you are stating with respect to the relevance of .htaccess files, but Nginx does not "read" them.

In essence, with Nginx serving as a proxy, the .htaccess file (and the content thereof) becomes irrelevant.

You stated:

This issue with permalinks and rewrite rule is only related to Wordpress installs ony with pretty URLs, it work perfect without that rule if you use ugly ULRs.

The result with ugly URLs implies that pretty URLs are not generated properly and/or have to be regenerated: simply regenerate the permalinks.

A persistent problem can exist, if you use some kind of caching mechanism, such as a "caching plugin".

Note that most of the "caching plugins" within WordPress are altering (i.e. messing up) .htaccess files, but that is not really relevant, since they can be safely ignored.

However, when using a "caching plugin" within WordPress, cache can be messed up by parts of the .htaccess file (I know, poor coding from the plugin authors, but it exists).

Anyway: flush cache and disable all caching plugins for the time being.

Finally, you stated:

Also with or without this rewrite rule I do not have expire headers set in files

I can safely assume that you try to state that expires headers are not set, despite of the "expires 30d;" in the location directive.

If I am not mistaken and do recall it correctly, the solution is to remove the "?:" part. Try that, it should then work.


As a final tip: try to prevent regex match pattern (e.g. .*) AND try to prevent if statements, wherever possible (note that "if is evil" in nginx).

Kind regards....

 

[bulent]

@trialotto

So it is standart, clean plesk 12.5 install on centos 7.1, wordpress is installed from plesk, without any plugin. PHP is set to FPM with nginx and permalinks in WorPress are reset.

First and main problem is when Smart static files processing is set and Serve static files directly by nginx is set /with removed jpg, png, css ... from the text box/ it doesn't set Expire parameter as it was in Plesk 12.0. /This config doesnt work without if statment with pretty URLs/

When I remove the tick form "Smart static files processing " and remove the " if " statement from "Additional nginx directives", The Expire parameter is set correctly, but like I said there is a catch. I found that when you reset the Permalinks in WordPress it places .htaccess file with standart WordPress Apache rewrite rules in the root www folder. What does it mean is somewhat it uses Apache, somewhere in the backend to set the permalinks correctly. With this config from "Tools and Stiings" -- "Service management" if you stop Apache, you will get 502 bad gateway when you try to open the page. So, even in php settings you have set php-fpm with nginx, it still uses Apache somwhere in the backend.

When Smart static files processing & Serve static files directly by nginx are set, with "if" statement in "Additional nginx directives" even with Apache service is stopped, the pages are served correctly with pretty URLs /except Expire headers /

So Odin Team please check why Nginx do not respect expire parameters in "Additional nginx directives" in Plesk 12.5

A bit off topic:
I can guess that "is" evil but I coudn't find any other working solution. Even in this post http://talk.plesk.com/threads/plesk-11-5-linux-php-fpm-and-nginx-index-html-error.287417/page-2 @EugeneKazakov advices so. There is possible config without "if" but then it's not possible to to open /wp-admin/ page. @deepsleep has anoter advice:

rewrite !\.(js|ico|gif|jpg|png|css|pdf|mov|mp3|eot|svg|ttf|woff|otf|txt)$ /index.php break;
rewrite /wp-admin/$ /wp-admin/index.php break;
rewrite /$ /index.php break;

That's one work pretty well, except in WooCommerce store sites. When you press "complete the order" button in checkout page , you get URL like this http://example.com/checkout/order-received/6721?key=wc_order_55486faf1ae71 and the browser opens Not found page, but the order is ok /complete/. So I guless I will use this method for most of the sites except WooCommerce ones.

 

[trialotto]

@bulent,

My advice: back to basics.

The short summary is: Nginx with fpm is "fancy stuff", getting you into trouble.

In fact, Nginx with fpm is actually implying that you are using the Nginx server as a (partial) web server, whilst still having Apache behind it.

Even when not taking into account the "trouble" caused by Nginx with fpm, the use of Nginx as a (partial) web server is inefficient to a high degree.

Just use Apache with fpm and use Nginx as a true proxy.

In general, despite what others say, you cannot expect that WordPress, i.e. a database-driven site running on php AND primarily focussed on Apache, will do well on Nginx.

Sure, there are some elaborate methods of running WordPress on Nginx, functioning as a FULL web server.

But that requires some advanced compiling of Nginx, as such not being "good practice" in a Plesk installation.

In conclusion, focus on the Apache with fpm + Nginx (proxy).

Regards....

 

[bulent]

Yepp,
may be you are right, but at the end Expire header parameter in "Additional nginx directives" must be fixed.
So Odin Team please check that issue Idon't want to go back to Plesk 12.0

 

[trialotto]

@bulent,

I already reported the issue to Odin Team, let´s wait and see.

For your information, read my response to @Eleshar, in particular point b.

The issue can be resolved by implementing the "simple solution" from point a: removing the extensions, for which the expires header has to be set, from the textbox below the "serve static files with nginx" checkbox.

Regards....

 

[cookkoo]

Hello Every Plesk Expert,

As i am not a programmer. and i lose a lot of time MORE THAN A MONTH to trying to config my site for stable by PLESK. my site have PLESK 12.5.30, run nginx as reverse proxy , Wordpress , the database size about 33 GB.

I wish go to work , to build the content , marketing , etc

Please help to write the efficiency code or standard code for put in to Apache & nginx Settings -
.htaccess content or Additional nginx directives too.

Please help, i need go to build my web site. don't need to lose my time.

Thank you very much

 

[cookkoo]

Hello, My last test. i've only put the simple .htaccess code and click "Convert to nginx" as below:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

And then my site is Error 504 Gateway time-out

 

[UFHH01]

Hi cookkoo,

please note, that tweaking a server, tweaking configurations and as well tweaking your domain content is not a "general configuration issue" - it is a process, which can take days / weeks and sometimes months. Especially in your case ( you use wordpress, with a non-provided number of additional plugins ), there can't be any satisfactory response, because we do not know, what plugins you use and if these plugins will need additional rewrite rules.
We can guide you, and we can point to possible issues. We can try to find possible solutions and work-arounds together with you, but all this is based on your very own provided informations and therefore we are really limited in our answers. If you leave out informations, leave out errors from log - files, leave out current domain - specific configuration files and leave out informations about PHP - handler in use ( and possible "phpinfo" for it ), etc..... we are only able to guess - which we very rarely do, because this guessing procedure is not only time-consuming, it might as well confuse and irritate forum-users, because the guessing suggestions may be completely wrong and won't fit.

If you would like help with a current issue/problem, please don't forget informations, because you will otherwise experience, that no one will answer to your post or thread, or you will experience, that you will only get vague, superficial answers.

 

[trialotto]

@cookkoo,

In addition to the post of UFHH01, also try to explain something about the intended purpose of your site.

After all, any WordPress installation with a database of 33GB (!!!) will get "lost" in the database requests and/or database querying, implying that the benefit of performance tweaking with Nginx, php-fpm or any other "old trick in the book" will be completely absent.

In short, be more specific about the intended purpose for your site: performance optimization is probably also to be obtained with other kinds of software, applications and so on.

 

[cookkoo]

Hello UFHH01 and trialotto

Thank you for your reply and try to help, Your answer have a good reason.

1. I've Enable plugins the details as below:

- akismet
- bbpress
- bp-profile-search
- buddypress
- BuddyPress Avatar Bubble
- buddypress-cover-photo
- Contact Form 7
- Google XML Sitemaps
- K Elements
- MailChimp for WordPress
- Nav Menu Roles
- Paid Memberships Pro
- rtMedia for WordPress, BuddyPress and bbPress
- Shortcode Widget
- Skt NURCaptcha
- Slider Revolution
- Taxonomy Metadata
- Wordfence Security
- WPBakery Visual Composer

2. I've Disable plugins the details as below: (As i used to use it and have plan to use it again after Server are stable)

- Broken Link Checker
- BuddyMenu
- BuddyPress Follow
- Buddypress Friend of a Friend (FOAF)
- BuddyPress Message Attachment
- BuddyPress Wall
- Custom Facebook Feed
- Email Login
- GEO my WP
- GMW Add-on - Kleo Geolocation
- Hello Dolly
- Instagram Feed
- Lingotek Translation
- Nginx Cache Controller
- Nginx Helper
- P3 (Plugin Performance Profiler)
- Polylang
- Postman SMTP
- Quick Chat
- Quick Flag
- Social Articles
- TinyChat Room Spy
- TinyMCE Advanced
- UpdraftPlus - Backup/Restore
- Visitor Maps and Who's Online
- W3 Total Cache
- Wordpress Video Chat Advanced - Only On GitHub!
- WP fail2ban

3. And the errors log as below:

- 404 GET /members/Sohel_P/ HTTP/1.1
- 500 GET /members/ellward/ HTTP/1.1
- 503 GET / HTTP/1.1
- 404 GET /groups/the-many-effective-speed-reading-technique-you-never-heard-about/ HTTP/1.1
- Premature end of script headers: index.php
- mod_fcgid: read data timeout in 120 seconds
- Warning mod_fcgid: stderr: WordPress database error Lock wait timeout exceeded; try restarting transaction for query UPDATE `wp_3tvcbc_wp_options` SET `option_value` = 'a:5:{s:5:\\"today\\";i:971;s:8:\\"thisdate\\";s:10:\\"2015-04-12\\";s:5:\\"month\\";i:3659;s:9:\\"thismonth\\";s:2:\\"12\\";s:7:\\"alltime\\";i:54353;}' WHERE `option_name` = 'pmpro_views' made by require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/themes/kleo/404.php'), get_header, locate_template, load_template, require_once('/themes/kleo/header.php'), wp_head, do_action('wp_head'), call_user_func_array, pmpro_report_login_wp_views, update_option
- Warning mod_fcgid: stderr: [limit_reached] => , referer: http://cookkoo.com/wp-admin/plugins.php?plugin_status=all&paged=1&s

This is the example and it have a TON of Warning and Warning but a lot of it nearly the same.

4. Hosting settings for cookkoo.com
- Preferred domain - None
- SSL support
- SSI support
- PHP support PHP version 5.6.15 Run PHP as FastCGI application
(Now i change to FastCGI, Because before this time i use Reverse Proxy Server (nginx) & PHP-FPM 5.6.15 , but my site have only link of "home" , and i can not login to wp-admin)

5. And you can check my "phpinfo" at cookkoo.com/phpinfo.php

6. Now my site have 33GB of database excluded with 2,100,000 users profile image on web server.
But in the really target,i have 18,000,000 users need to import to database with total 250GB. And if included with users profile image, the Grand Total space is about 5.6TB

7. My site cookkoo.com run behide CDN as Cloudflare > Cloudproxy > cookkoo.com

8. UFHH01 used to help to suggestion and successul to install memcached on my site but i can not choose it on W3 Total Cache. (But i never consul UFHH01 again in this case because i headache with others issue)

9. My server is 32GB RAM , HHD 6TB SATA with Raid 1.

I used to test run my site on ipage, dreamhost, google cloud, amazon cloud and testing all of them more than a year.but every service not work for my site. and now i hope PLESK may can fix this issue.

Thank you