Resolved Smart static files processing is only working for lowercase file extensions

[Hangover2]

Username:

TITLE

Smart static files processing is only working for lowercase file extensions

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

Plesk Obsidian 18.0.59 Update #1, Debian 11.9, x86-64,

PROBLEM DESCRIPTION

The "Smart static files processing" under "Apache & nginx Settings" of the hosting panel is only working for lowercase file extensions. Files with e.g. uppercase letters in the file extension are not processed by Nginx alone but Apache instead.

STEPS TO REPRODUCE

- activate "Smart static files processing" under "Apache & nginx Settings" for a domain
- create two files in the public folder:
a) lowercase_file_extension.txt
b) uppercase_file_extension.TXT
- call the files directly in your browser

ACTUAL RESULT

- the request for lowercase_file_extension.txt will be handled by Nginx (proxy_access_ssl_log if SSL is active)
- the request for uppercase_file_extension.TXT is proxied to the Apache (access_ssl_log if SSL is active)

EXPECTED RESULT

- both files should be handled by Nginx alone

ANY ADDITIONAL INFORMATION

- suggested fix for the nginx.conf:
- current implementation

location ~ ^/(.*\.(ac3|avi|bmp|bz2|css|cue|dat|doc|docx|dts|eot|exe|flv|gif|gz|htm|html|ico|img|iso|jpeg|jpg|js|mkv|mp3|mp4|mpeg|mpg|ogg|pdf|png|ppt|pptx|qt|rar|rm|svg|swf|tar|tgz|ttf|txt|wav|w
off|woff2|xls|xlsx|zip|webp))$

- new implementation

# ~*: The tilde followed by asterisk sign is used for case insensitive regular expression match against a requested URI.
location ~* ^/(.*\.(ac3|avi|bmp|bz2|css|cue|dat|doc|docx|dts|eot|exe|flv|gif|gz|htm|html|ico|img|iso|jpeg|jpg|js|mkv|mp3|mp4|mpeg|mpg|ogg|pdf|png|ppt|pptx|qt|rar|rm|svg|swf|tar|tgz|ttf|txt|wav|w
off|woff2|xls|xlsx|zip|webp))$

YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Confirm bug

 

[Peter Debik]

Thank you! Forwarded to a test engineer as ID PPS-15683.

 

[Peter Debik]

Response from developers:

Typically on Linux files are treated in case-sensitive manner. And if enable case-insensitivity for file extensions they might be processed incorrectly by other services as well.

It is unclear reason for naming files with mixed case like MyStaticFile.Txt. How many webspaces has such names inside? Are there some specific reasons for such naming? Most popular CMS using lowercase file naming in Unix.

If there are several users with such names in webspace it is possible to define for them list of file extensions to handle:

plesk bin subscription --update-web-server-settings $NEW_DOMAIN_NAME -nginx-serve-static true -nginx-static-extensions "ac3 avi bmp bz2 css cue dat doc docx dts eot exe flv gif gz htm html ico img iso jpeg jpg js mkv mp3 mp4 mpeg mpg ogg pdf png ppt pptx qt rar rm svg swf tar tgz ttf txt Txt TXT wav woff woff2 xls xlsx zip"

Also, it is possible to create script based on command above and then create event handler in Plesk on domain creation event. Then list of the file extensions will be applied for newly created domain.

 

[Hangover2]

I think there is a misunderstanding of the problem. Files like:

- MyStaticFileWithUppercaseLetters.txt are working already with the current location rule

Only files with uppercase letters in the extension are handed over to Apache:

- MyStaticFileWithUppercaseLetters_InTheExtension.TxT

If you do not like the ~* solution, it can also be solved without it. Just update the extension part of the regular expression by adding "?i:":

location ~ ^/(.*\.(?i:ac3|avi|bmp|bz2|css|cue|dat|doc|docx|dts|eot|exe|flv|gif|gz|htm|html|ico|img|iso|jpeg|jpg|js|mkv|mp3|mp4|mpeg|mpg|ogg|pdf|png|ppt|pptx|qt|rar|rm|svg|swf|tar|tgz|ttf|txt|wav|w
off|woff2|xls|xlsx|zip|webp))$

I think this approach is even better.

And yes, we have a lot of clients that upload files with capital letters to their websites, also in the extension. Your workaround is quite hard as we cannot inject the "?i:" via the given command. So basically we need to add for every extension all possible variants (amount of letters factorial), e.g. "webp" = 4! = 24 variants.

 

[Hangover2]

Response from developers:

Typically on Linux files are treated in case-sensitive manner. And if enable case-insensitivity for file extensions they might be processed incorrectly by other services as well.

Maybe as a little addition: the location directive does not change the filename that will be processed. So it doesn't matter what file system is used.
Right now try_files is not even called because of the strict regular expression for capital letters in the extension part that forbids any match. My two possible solutions will just allow the request to be checked by try_files. I see no negative side effects at all. Especially because capital letters are already allowed in the main part of the filename via .*.

 

[Peter Debik]

I think the reasoning of developers is rather that you already have an option to set any extension you like to have, so they argue that there is no need to invest into updating the feature.

 

[Hangover2]

OK, just in case someone needs it, here is the list (and my factorial approach was wrong ):

ac3 aC3 Ac3 AC3 avi avI aVi aVI Avi AvI AVi AVI bmp bmP bMp bMP Bmp BmP BMp BMP bz2 bZ2 Bz2 BZ2 css csS cSs cSS Css CsS CSs CSS cue cuE cUe cUE Cue CuE CUe CUE dat daT dAt dAT Dat DaT DAt DAT doc doC dOc dOC Doc DoC DOc DOC docx docX doCx doCX dOcx dOcX dOCx dOCX Docx DocX DoCx DoCX DOcx DOcX DOCx DOCX dts dtS dTs dTS Dts DtS DTs DTS eot eoT eOt eOT Eot EoT EOt EOT exe exE eXe eXE Exe ExE EXe EXE flv flV fLv fLV Flv FlV FLv FLV gif giF gIf gIF Gif GiF GIf GIF gz gZ Gz GZ htm htM hTm hTM Htm HtM HTm HTM html htmL htMl htML hTml hTmL hTMl hTML Html HtmL HtMl HtML HTml HTmL HTMl HTML ico icO iCo iCO Ico IcO ICo ICO img imG iMg iMG Img ImG IMg IMG iso isO iSo iSO Iso IsO ISo ISO jpeg jpeG jpEg jpEG jPeg jPeG jPEg jPEG Jpeg JpeG JpEg JpEG JPeg JPeG JPEg JPEG jpg jpG jPg jPG Jpg JpG JPg JPG js jS Js JS mkv mkV mKv mKV Mkv MkV MKv MKV mp3 mP3 Mp3 MP3 mp4 mP4 Mp4 MP4 mpeg mpeG mpEg mpEG mPeg mPeG mPEg mPEG Mpeg MpeG MpEg MpEG MPeg MPeG MPEg MPEG mpg mpG mPg mPG Mpg MpG MPg MPG ogg ogG oGg oGG Ogg OgG OGg OGG pdf pdF pDf pDF Pdf PdF PDf PDF png pnG pNg pNG Png PnG PNg PNG ppt ppT pPt pPT Ppt PpT PPt PPT pptx pptX ppTx ppTX pPtx pPtX pPTx pPTX Pptx PptX PpTx PpTX PPtx PPtX PPTx PPTX qt qT Qt QT rar raR rAr rAR Rar RaR RAr RAR rm rM Rm RM svg svG sVg sVG Svg SvG SVg SVG swf swF sWf sWF Swf SwF SWf SWF tar taR tAr tAR Tar TaR TAr TAR tgz tgZ tGz tGZ Tgz TgZ TGz TGZ ttf ttF tTf tTF Ttf TtF TTf TTF txt txT tXt tXT Txt TxT TXt TXT wav waV wAv wAV Wav WaV WAv WAV woff wofF woFf woFF wOff wOfF wOFf wOFF Woff WofF WoFf WoFF WOff WOfF WOFf WOFF woff2 wofF2 woFf2 woFF2 wOff2 wOfF2 wOFf2 wOFF2 Woff2 WofF2 WoFf2 WoFF2 WOff2 WOfF2 WOFf2 WOFF2 xls xlS xLs xLS Xls XlS XLs XLS xlsx xlsX xlSx xlSX xLsx xLsX xLSx xLSX Xlsx XlsX XlSx XlSX XLsx XLsX XLSx XLSX zip ziP zIp zIP Zip ZiP ZIp ZIP webp webP weBp weBP wEbp wEbP wEBp wEBP Webp WebP WeBp WeBP WEbp WEbP WEBp WEBP

 

[Kaspar]

This is actually an interesting topic. I still have @AYamshanov thread about "Site/Server Automatic Health Reports" lingering in my mind, and I feel that in some way this topic touches upon that subject a little bit as well. (I say 'little', because I don't exactly know what "Site/Server Automatic Health Reports" are supposed to be.)

From a developers perspective it seems completely logical to argue that file names are treated case sensitive in Linux. Which is why most (linux) files are lower case anyway, and most of the popular CMSes are strictly using lowercase file naming too. So the practical use case for using case insensitive file extension matching is assumed to be low. Besides, as mentioned by the developer, it is possible to manually define uppercase (or mixed case) file extensions in Plesk for a domain.

However, I do think @Hangover2 also has a valid point. Exactly because files are treated case sensitive in Linux it should not be taken for granted that there is a possibility that users upload and use files with uppercase file extensions. Which made me curious. That's why I ran a quick file search on one of my servers to see how many domains there are with files that have an uppercase file extension. Out of the 483 domains, there are 68 domains (= 14%) that have files with uppercase file extensions. Surprisingly most of them (54) are Wordpress sites. Which to me is still a substantial number of domains.

Now this is also where I feel @AYamshanov thread about "Site/Server Automatic Health Reports" comes into play. Because I believe that very few website owners and even website developers are fully aware about the fact that files are treated case sensitive. Most of them probably assume that an uppercase or lowercase file extension makes no difference in how files are handled by the web server. If (end-)users are not aware that there is a difference they may also not realize that "Smart static files processing" does not apply to files with uppercase file extension. As a result they run a less optimized website.

Which made me think that perhaps this is a useful use case for the "Site/Server Automatic Health Reports"? Maybe it can report bad health if a domain/site uses files with uppercase file extensions. (Or would the Performance Booster be a better candidate for that?).

But maybe this issue should be reconsidered all together and file extensions should be made case insensitive for "Smart static files processing".

 

[ChristophRo]

Typically on Linux files are treated in case-sensitive manner. And if enable case-insensitivity for file extensions they might be processed incorrectly by other services as well.

This does not make any sense...
There has never ever been any case since the dawn of time, where a upper/mixed case file extension would represent a different file type than it's lower case counterpart