Very long Apache graceful restart

[MariuszB]

I have problem with Apache restart time. Usualy it takes about 2-3 minutes. I have turned on graceful restart.

When Apache starting in process list is one httpd process. It takes 100% time of core.

My system is CentOS 6 and Plesk 11.5.30#47.

 

[InderS]

How many domains you have hosted on your server ? Also check the error logs file while restarting the Apache on your server

 

[MariuszB]

Now we have 1395 domains hosted. In error logs I see nothing special.

 

[MariuszB]

Here is log after long restart when I added new subscription. Do you see something interesting? May be I should disable some Apache modules?

[Sun Aug 10 23:19:38 2014] [notice] Graceful restart requested, doing restart
[Sun Aug 10 23:23:22 2014] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `domain.com' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:28 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Sun Aug 10 23:23:28 2014] [notice] ModSecurity for Apache/2.7.7 (http://www.modsecurity.org/) configured.
[Sun Aug 10 23:23:28 2014] [notice] ModSecurity: APR compiled version="1.3.9"; loaded version="1.3.9"
[Sun Aug 10 23:23:28 2014] [notice] ModSecurity: PCRE compiled version="7.8 "; loaded version="7.8 2008-09-05"
[Sun Aug 10 23:23:28 2014] [notice] ModSecurity: LUA compiled version="Lua 5.1"
[Sun Aug 10 23:23:28 2014] [notice] ModSecurity: LIBXML compiled version="2.7.6"
[Sun Aug 10 23:23:28 2014] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status
[Sun Aug 10 23:23:30 2014] [notice] Digest: generating secret for digest authentication ...
[Sun Aug 10 23:23:31 2014] [notice] Digest: done
[Sun Aug 10 23:23:32 2014] [notice] mod_bw : Memory Allocated 0 bytes (each conf takes 48 bytes)
[Sun Aug 10 23:23:32 2014] [notice] mod_bw : Version 0.92 - Initialized [0 Confs]
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `domain.com' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Sun Aug 10 23:23:32 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Sun Aug 10 23:23:32 2014] [warn] No JkShmFile defined in httpd.conf. Using default /etc/httpd/logs/jk-runtime-status
[Sun Aug 10 23:23:32 2014] [error] python_init: Python version mismatch, expected '2.6.5', found '2.6.6'.
[Sun Aug 10 23:23:32 2014] [error] python_init: Python executable found '/usr/bin/python'.
[Sun Aug 10 23:23:32 2014] [error] python_init: Python path being used '/usr/lib64/python26.zip:/usr/lib64/python2.6/:/usr/lib64/python2.6/plat-linux2:/usr/lib64/python2.6/lib-tk:/usr/lib64/python2.6/lib-old:/usr/lib64/python2.6/lib-dynload'.
[Sun Aug 10 23:23:32 2014] [notice] mod_python: Creating 4 session mutexes based on 256 max processes and 0 max threads.
[Sun Aug 10 23:23:32 2014] [notice] mod_python: using mutex_directory /tmp
[Sun Aug 10 23:23:32 2014] [notice] Apache/2.2.15 (Unix) DAV/2 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 mod_jk/1.2.37 mod_python/3.3.1 Python/2.6.6 mod_perl/2.0.4 Perl/v5.10.1 configured -- resuming normal operations

 

[Thomas Gross]

I would guess it's due to the amount of (config-)files it needs to load for each website on force-reload - we have the same problem. Try changing to piped logs:

replace into misc (param,val) values ('apache_pipelog', 'true');

We also encounter the problem with restarting apache (even gracefully) that all fcgid procs get killed without finishing first - so - a good amount of unhappy customers

 

[MariuszB]

I have turned on piped logs. Do you have SSD? I think it may be helpful with our issue.

 

[Thomas Gross]

Just a quick note, we don't have ssd's as we're running VMware but I have installed lsapi from Litespeed, which is very good resource wise and restarts taking less time due to the fact that httpd doesn't have to wait for api's to finish etc. - here is the link: http://www.litespeedtech.com/download/litespeed-sapi-download

This is the compiling way - we're using Cloudlinux which has a yum repository for easy download and install. Maybe test lsapi - I would be interested if you see better performance. Make sure you use:

lsapi_terminate_backends_on_exit off

which is the main great feature. Please note that you can't run the standard mod_php in parallel though.