Facebook
Twitter
Username:
TITLE
Apache web server config files all regenerated nightly since 18.0.48
PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE
18.0.48
CentOS 7.9
PROBLEM DESCRIPTION
Since the update to 18.0.48 all Apache web server configuration files
e.g. /var/www/vhosts/system/<domain>/conf/httpd.conf
and their corresponding symbolic links
e.g. /etc/httpd/conf/plesk.conf.d/vhosts/<domain>.conf --> ...
are reconfigured nightly at a random time, probably as one part of the nightly standard maintenance. The general server.conf file is also reconfigured. The nginx configuration files are not reconfigured, which is confusing. Why are all Apache config files reconfigured, but not the nginx files?
Here is why the reconfiguration is causing issues:
On servers that service many domain names the reconfiguration process can lead to web server failures if the web server is restarted while the regeneration process has not completed. The reason is that there is a latency between removal of the old files and links and provision of the new files and links. There is a noticeable latency between replacement of the physical httpd.conf file in .../system/<domain>/conf/ and its symbolic link counterpart.
On our (fast) servers we see a 2-minute long latency between regeneration of server.conf and individual domain configuraton files and a 1-minute long latency between regeneration of the physical hosting files and regeneration of their corresponding symbolic links.
If the physical hosting file has not yet been replaced, but the symbolic link still exists, web server restarts will fail for the missing symbolic link target. I have positively seen that the symbolic link in some cases leads to non-existent physical files. As the issue occurs nightly, it is likely to hit the time frame when full server backups are done so that the reconfiguration process takes a lot of time (due to the already high cpu load by PMM) increasing the risk that a web server reload/restart attempt accidentally hits the latency period where symbolic links are inconsistent with their physical counterparts.
STEPS TO REPRODUCE
Have a server with many domains, e.g. 1,000.
Wait until the nightly reconfiguration process starts, then have a script that continuously tests whether all physical Apache hosting config files exist for all the symbolic links.
ACTUAL RESULT
During the reconfiguration phase there will be symbolic links that lead to non-existent physical config files.
Current order of events:
1) Physical hosting configuration file is removed.
2) Something is done that seems to be time consuming so that it takes a while to write the new file. Then physical hosting file is recreated.
3) Symbolic link is updated.
EXPECTED RESULT
Change order of events:
1) On reconfiguration first remove the symbolic link to the file that will be rewritten. This ensures that accidental web server reloads or restarts during that phase will still work.
2) Recreate the physical config file.
3) Recreate the symbolic link.
4) After all files are in place: Reload Apache config (service httpd reload), probably done already (but very important, because if an additional reload was done between (1) and (3) that configuration was not included in the reloaded configs).
ANY ADDITIONAL INFORMATION
(DID NOT ANSWER QUESTION)
YOUR EXPECTATIONS FROM PLESK SERVICE TEAM
Help with sorting out
TITLE
Apache web server config files all regenerated nightly since 18.0.48
PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE
18.0.48
CentOS 7.9
PROBLEM DESCRIPTION
Since the update to 18.0.48 all Apache web server configuration files
e.g. /var/www/vhosts/system/<domain>/conf/httpd.conf
and their corresponding symbolic links
e.g. /etc/httpd/conf/plesk.conf.d/vhosts/<domain>.conf --> ...
are reconfigured nightly at a random time, probably as one part of the nightly standard maintenance. The general server.conf file is also reconfigured. The nginx configuration files are not reconfigured, which is confusing. Why are all Apache config files reconfigured, but not the nginx files?
Here is why the reconfiguration is causing issues:
On servers that service many domain names the reconfiguration process can lead to web server failures if the web server is restarted while the regeneration process has not completed. The reason is that there is a latency between removal of the old files and links and provision of the new files and links. There is a noticeable latency between replacement of the physical httpd.conf file in .../system/<domain>/conf/ and its symbolic link counterpart.
On our (fast) servers we see a 2-minute long latency between regeneration of server.conf and individual domain configuraton files and a 1-minute long latency between regeneration of the physical hosting files and regeneration of their corresponding symbolic links.
If the physical hosting file has not yet been replaced, but the symbolic link still exists, web server restarts will fail for the missing symbolic link target. I have positively seen that the symbolic link in some cases leads to non-existent physical files. As the issue occurs nightly, it is likely to hit the time frame when full server backups are done so that the reconfiguration process takes a lot of time (due to the already high cpu load by PMM) increasing the risk that a web server reload/restart attempt accidentally hits the latency period where symbolic links are inconsistent with their physical counterparts.
STEPS TO REPRODUCE
Have a server with many domains, e.g. 1,000.
Wait until the nightly reconfiguration process starts, then have a script that continuously tests whether all physical Apache hosting config files exist for all the symbolic links.
ACTUAL RESULT
During the reconfiguration phase there will be symbolic links that lead to non-existent physical config files.
Current order of events:
1) Physical hosting configuration file is removed.
2) Something is done that seems to be time consuming so that it takes a while to write the new file. Then physical hosting file is recreated.
3) Symbolic link is updated.
EXPECTED RESULT
Change order of events:
1) On reconfiguration first remove the symbolic link to the file that will be rewritten. This ensures that accidental web server reloads or restarts during that phase will still work.
2) Recreate the physical config file.
3) Recreate the symbolic link.
4) After all files are in place: Reload Apache config (service httpd reload), probably done already (but very important, because if an additional reload was done between (1) and (3) that configuration was not included in the reloaded configs).
ANY ADDITIONAL INFORMATION
(DID NOT ANSWER QUESTION)
YOUR EXPECTATIONS FROM PLESK SERVICE TEAM
Help with sorting out
