Browser caching with nginx not working

[trialotto]

@danami,

You made some interesting points in your response, some of which were already brought to the attention of Plesk Team by me.

With respect to your comments about gzip, I already offered some (essentially) identical instructions in my section C, but your comments are probably more clear, so thanks.

With respect to your comments about the headers issue, that is not true: headers settings can be included in global configuration files, but you really should know what to do.

Furthermore, one interesting point you made was

In its current form I don't see any other options without Plesk making the changes themselves to allow for this sort of thing. Maybe some sort of global include for nginx gets included in everyones virtualhost that fixes this sort of thing?

and I already discussed the associated issues with and/or even made some suggestions for solutions to the Plesk Team.

It is good to see that the topic pops up again.

It is also good to emphasize that it is not the task of Plesk Team to optimize Nginx configuration, which is essentially the core of the before mentioned topic.

I considered to develop an extension for that, i.e. an extension for easy Nginx configuration, but actual feedback from Plesk sysadmins really remains absent.

I will ask again: what do you (all) want or need, when it concerns the configuration of Nginx.

Please describe the desired functionality as clear as possible.

Regards.........

 

[G J Piper]

I will ask again: what do you (all) want or need, when it concerns the configuration of Nginx.
Please describe the desired functionality as clear as possible.

I want to be able to add custom nginx settings within the "location" directive brackets that are created from the Plesk UI's "Serve static files directly by nginx" setting, without having to do it from the command line and without having to modify the files outside the UI. (so a system update won't revert my settings)
(see post #9)

 

[bulent]

I want to be able to add custom nginx settings within the "location" directive brackets that are created from the Plesk UI's "Serve static files directly by nginx" setting, without having to do it from the command line and without having to modify the files outside the UI. (so a system update won't revert my settings)
(see post #9)

Exactly.
Previously with Plesk 12.0 it was performing that way.

 

[trialotto]

@G J Piper and @bulent

Point taken, regarding the statement

I want to be able to add custom nginx settings within the "location" directive brackets that are created from the Plesk UI's "Serve static files directly by nginx" setting, without having to do it from the command line and without having to modify the files outside the UI. (so a system update won't revert my settings)

and to that statement I must add the following notes:

a) technically already possible, via the "Domain > [ domain ] > Apache & nginx Settings > Additional Nginx directives" part of the Plesk Panel, with

- Nginx directives ending up in the vhost_nginx.conf file
- the vhost_nginx.conf file not being overwritten,

BUT with the disadvantage that some of the "traditional" location directives will NOT result in a pattern match (and application of the Nginx directives specified),

b) the limitations, as mentioned under point a (see the disadvantage), are inherent to

- the method in which the vhost_nginx.conf file is included,
- the structure of the nginx.conf file,
- the method of proxying (read: request forwarding to Apache) in a default Nginx + Apache setup, as provided with Plesk.


Given the two notes above, there probably is the necessity to work-around the default Nginx template (as reflected in nginx.conf) or use a custom template (that has to be designed).

Both options are under investigation.

Also note that I tried to report the limitations of the current Nginx setup to Plesk Team, but this report did not yield any result or response by Plesk Team.

I hope that this feedback on this topic thread will renew the attention of Plesk Team, in order to allow more flexibility in custom configuration of Nginx.

However, you should be aware of the fact that the current Nginx setup is proper, in the sense that it is not wise to change a lot.

The latter implies that there is some necessity to find a proper balance between (on the one hand) maintaining the current default setup (which is very secure and stable) and (on the other hand) introducing some flexibility, in order to allow customizations (without the need of creating custom templates).

In short, a challenging task (and it will take some time, but I am confident that a solution will arise).

Regards.......

 

[Lloyd_mcse]

Hi guys,
I've been doing some testing today, and found you can add the standard block eg..

location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff|svg)$ {
       expires 30d;
       add_header Pragma public;
       add_header Cache-Control "public";
       try_files $uri @fallback;
}

into the same template file (nginxDomainVirtualHost.php) I mentioned previously like so...

location ~* \.(js|css|png|jpg|jpeg|gif|ico|woff|svg)$ {
    expires 365d;
    add_header Pragma public;
    add_header Cache-Control "public";
    try_files $uri @fallback;
   }

    location ~ ^/(.*\.(<?php echo $VAR->domain->physicalHosting->proxySettings['nginxStaticExtensions'] ?>))$ {
        try_files $uri @fallback;
    }

Obviously this is just an example, set the file types and duration as you wish.
I tried adding it after the "nginxStaticExtensions" block as a test, but that didn't work as we suspected.

Anyway, that's another way to go with this until we can add it in the Plesk Panel.
Regards

Lloyd

 

[trialotto]

@Lloyd_mcse

You stated

I tried adding it after the "nginxStaticExtensions" block as a test, but that didn't work as we suspected.

and I am not sure whether I am reading this correctly.

After all, the section D in my post

https://talk.plesk.com/threads/browser-caching-with-nginx-not-working.337315/#post-798858

contains the Nginx directives that one CAN add to vhost_nginx.conf, via the Plesk Panel (!).

The only thing I forget to mention (probably due to the late hour at the time of writing) is that you have the remove file extensions, for which you specify custom configuration, from the text box below the check box "serve static files with Nginx ...".

To illustrate, if one has a custom configuration involving .css files, one has to remove the css part in the string in the text box below the check box "serve static files ...".

Maybe I am interpreting it wrong, but you should be able to use section D to add "expire headers" (and other types of Nginx directives).

Regards....

 

[trialotto]

@Everyone,

Really, I must state formally that I dislike the idea that there is (more or less) a common opinion that the best answer is to be found in

https://talk.plesk.com/threads/browser-caching-with-nginx-not-working.337315/#post-798883

No personal offense intended, but it is not a good idea to use that specific approach, for many reasons, being (amongst others)

- expires headers should differ across the various types of files,
- security issues arise if all file extensions, specified in the Nginx "file extension textbox", are treated the same,
- most of the basic edits of the default template (to form a custom template) are still associated with the limitations, inherent to the default template
- the specific customization in question will give problems with WordPress and similar applications
- the specific customization in question will give problems when having some cache method configure.

To be honest, most of the above mentioned "reasons" are "problems" that are not really related to the customization in question.

In fact, the majority of the above mentioned "problems" are the result of the problematic default template, which problems are continued with the customization of that default template.

Nevertheless, if the majority finds or thinks that there is "a best answer", it is fine by me and I suggest to end the discussion.

Regards......

 

[xberg]

OK, so if I understand correctly I can either:
- implement browser caching
OR
- use nginx to serve the file.

But I cannot do both on a same file type, as to allow for browser caching I need to remove the file type from the box "serve static files with nginx".

And if my understanding is right, which is the better option in terms of performance: serve static JPGs and CSSs thru nginx or cache them on the end user's browser served by apache?

 

[trialotto]

@xberg

It is just the other way around:

1 - you can implement browser caching by using Nginx directives, AND
2 - separate the delivery of static files from delivery of dynamic files,

and the second point mainly implies that specific files (of the static type) are not being served by Apache, hence increasing performance by reducing the workload on Apache.

In general, it is good practice to have Nginx set up to serve (almost) all static files (i.e. performance improvement) AND allow browser to cache.

It might be best illustrated with some practical example.

Suppose you have one php file (a dynamic file) rendering html in which a picture (a static file) is presented according to some design from a css stylesheet (a static file).

Normally, you would have Apache serving the requests for all dynamic and static files, which has a relative high impact on performance, when taking into consideration that the static files do not really demand any rendering by Apache and, as such, the requests should not be handled Apache.

Now we introduce Nginx, any browser caching is still off.

In this case, only the request for the php (dynamic) file is handled by Apache, while all other requests for static files are handled by the light-weight superfast Nginx.

On an average site, the use of Nginx can increase performance significantly.

However, every time the page is loaded, there still are three requests, for three files.

In essence, that is a little bit ineffective, certainly when taking into consideration that, as a result of each request, some files are download to the PC on which the browser resides.

Now consider the browser caching, so we have Apache handling the request for the dynamic php file and Nginx handling the requests for the static files.

However, the static files are only requested on first load: afterwards, the browser will not pass a request to Nginx, as long as the (static) files are still in (local) cache.

The request for the (dynamic) php file will be executed each time, but the endresult is that a whole lot of passes (requests for files) are not necessary anymore.

And that is it..........it is simply about reducing the number of requests:

- Nginx to serve static files: a method to reduce the number of requests passed to Apache
- Browser cache: a method to reduce the number of requests to either Apache or Nginx

Hope the above helps a bit!

Regards......

 

[xberg]

@trialotto
Thanks for your detailed answer. I understand the concept that both together are better, but this whole thread points out that with plesk you have to select either one or the other.
That said, I tried last night and I was able to have both a file extension set in the "serve static files with nginx" box and browser caching working! So I finally have the best of both worlds.
thanks.

 

[trialotto]

@xberg,

No problem, thanks for the like and I am glad that the explanation helped.

I would like to point out that there is a general misconception about the interaction between or the possibilities of Nginx and Apache.

A good way to illustrate this is to talk about the well-known W3TC plugin for WordPress, that has many nice features, but an awful lot of code to make these features work.

In essence, allmost all caching features of the W3TC plugin can be achieved via configuration of Nginx directives, implying that one does not really need W3TC for certain features.

Nevertheless, people tend to use W3TC and do not think about the following: W3TC is dealing with Apache.

Instead of using the performant Nginx, one introduces Apache into the equation (again) by using W3TC.

In short, if and whenever somebody has to optimize the process of handling requests and caching of requests, one first has to think of the general design: when to use Apache or Nginx.

Only a good request handling structure will get the most out of your web server, you can imagine that by now.

Hope the above helps...

Regards....

 

[xberg]

This is still not fixed...
As I wrote above if I have the file extension (ex: jpg) in the box "use nginx to serve file" then I cannot use the cache control in the extra nginx directives...
Could I please hear from Plesk when they plan on fixing this?

 

[trialotto]

@xberg,

In response to your latest post

As I wrote above if I have the file extension (ex: jpg) in the box "use nginx to serve file" then I cannot use the cache control in the extra nginx directives...

the answer should be: "yes, you can".

A simple directive like: add_header Cache-Control public; would be sufficient to enable this header.

Nevertheless, it is a header used by browsers and a lot has changed with respect to browsers and how they handle http requests.

For instance, Cache-Control header is a HTTP/1.1 header (!!!) that replaces the Expires header (read: it is not really necessary to specify an Expires header in Nginx config).

Moreover, the introduction of HTTP/2 is a game changer.

In fact, one has to work with the Nginx "expires" directive in order to get some browser caching done.

I suggest that you have a look at the following page: http://nginx.org/en/docs/http/ngx_http_headers_module.html#expires

If you still have some problems with your nginx configuration, just post the relevant Nginx config, so we can all have a look.

Regards.....

PS By the way, if you are testing caching with Google Chrome browser, then there can be a major pitfall: a fallback to HTTP/1.1 protocol can be present, even in the case that a HTTP/2 based Nginx configuration is active for a site or an application. Be aware of that pitfall.

 

[xberg]

Hi,
Unfortunately no progress has been made on this matter and none of these answers is helping me.
What I would like to do is simple:
1. serve static files thru nginx
2. set expiry header
3. set this all up through plesk.

So far I can only have 2 out of 3 of these:
- tick "serve static files with nginx" but then the expires no longer work
- OR remove those files (eg: jpg, css...) from the plesk box and then my expire headers will work
- OR customise nginx.conf out of plesk (I do not have access to this).

I am not sure why you are mentionning things like http2 or a link to the nginx expire header documentation...

thanks

 

[xberg]

I've found that it seems there is currently only one way to get nginx proxy and expires to work together on static files — and it involves modifying Plesk UI controlled files in the command line.
(of course, as a result any changes to Plesk's nginx settings will overwrite these settings)

Go into settings in the Domain -> Apache & nginx Settings and make sure these are checked:
[on] Smart static files processing
[on] Serve static files directly by nginx

Then go into the server files and find this file if you have more than just the default domain configured:
/var/www/vhosts/system/domain.ext/conf/nginx.conf

or this file if you only use the default domain:
/var/www/vhosts/system/defaultdomain.ext/conf/nginx_ip_default.conf

In one of these files (whichever one applies to you) look for the two sections that look like this:
location ~ ^/(.*\.(ac3|avi|bmp|bz2|css|cue|dat|doc|docx|dts|eot|exe|flv|gif|gz|htm|html|ico|img|iso|jpeg|jpg|js|mkv|mp3|mp4|mpeg|mpg|ogg|pdf|png|ppt|pptx|qt|rar|rm|svg|swf|tar|tgz|ttf|txt|wav|woff|woff2|xls|xlsx|zip))$ {
try_files $uri @fallback;
}

Make them both look like this:
location ~ ^/(.*\.(ac3|avi|bmp|bz2|css|cue|dat|doc|docx|dts|eot|exe|flv|gif|gz|htm|html|ico|img|iso|jpeg|jpg|js|mkv|mp3|mp4|mpeg|mpg|ogg|pdf|png|ppt|pptx|qt|rar|rm|svg|swf|tar|tgz|ttf|txt|wav|woff|woff2|xls|xlsx|zip))$ {
expires 7d;
try_files $uri @fallback;
}

There are two of these sections in the file because one is for SSL vhost settings and one is for regular vhost settings.

Then restart nginx by:
service nginx restart

Voila! You have nginx reverse-proxy and expires on any of the static files listed for the domain you modified.

Keep in mind -- if you change any settings in your "Apache & nginx Settings" for that domain you will have to redo these customizations manually, again.
I know it isn't an easy way to do it, especially when you have a lot of domains, but it seems to be the only way to get everse-proxy static files served with expires tags included.

The reason these settings have to be manually added is that once a "location" directive is made on a certain group of files, if those files are found in another "location" directive they are ignored by nginx.

Seems like it would be extremely simple for Plesk to include another setting next to the static files preferences in the UI that would allow enabling or disabling expires at least.

Hope this helps.

This seems to be the ONLY answer which solves my problem... provided I can get access to the file you mention. Has there been no progress on the plesk side since you wrote this 9 months ago?

 

[xberg]

@paWELL (and @everyone),

In order to answer your question

(...)

D - Allow for (browser) caching

Well, in general, there are many rules that we have to take into consideration.

I decided to keep it simple and provide a sufficient example to set some (relevant) headers for specific files:
(...)

Hi,
You are not addressing our problem: we simply cannot enable through plesk 1: serving of static files though nginx and 2: setting expiry header on the same files.

regards

 

[888]

Hope this helps.

Thank you so much, you've saved my day