• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

Apache2 Crashes everyday at 7am

I

ilijamt

Basic Pleskian
Hello

For some reason Apache2 crashes everyday around 7am

Code: 
[Tue Oct 20 07:01:43.033135 2015] [core:notice] [pid 18913] AH00060: seg fault or similar nasty error detected in the parent process
[Wed Oct 21 06:57:37.975684 2015] [core:notice] [pid 31859] AH00060: seg fault or similar nasty error detected in the parent process
[Thu Oct 22 06:58:54.488246 2015] [core:notice] [pid 13528] AH00060: seg fault or similar nasty error detected in the parent process

And it's always preceded by a graceful restart
Code: 
[mpm_event:notice] [pid 13528:tid 140709733955456] AH00493: SIGUSR1 received.  Doing graceful restart

This happens everyday at around the same time always.

Executing this almost always results in a crash with the message above.
Code: 
for i in seq 1 10; do echo $i; sudo service apache2 reload; sleep 2; done;

Plesk version
Code: 
Product version: 12.5.30 Update #7
    Update date: 2015/10/20 21:48
     Build date: 2015/08/26 19:00
     OS version: Debian 8.2
       Revision: 344620
   Architecture: 64-bit
Wrapper version: 1.2
 
Have you checked your cronjob for this time (7am)? What is this cron task?
Do you have Plesk installed on VPS?
 
IgorG said:
Have you checked your cronjob for this time (7am)? What is this cron task?
Do you have Plesk installed on VPS?
Click to expand...

No it's a dedicated machine with Intel® Core™ i7-4770 Quad-Core Haswell and 32 GB DDR3 RAM

I'm not running any cron tasks, this started happening after upgrade to Plesk 12.5 from 12.0.18
 
Also like I said if I run

Code: 
for i in seq 1 10; do echo $i; sudo service apache2 reload; sleep 2; done;

It will crash again.

But here is my cron list
Code: 
0       1       *       *       *       /opt/backups/pflogsum_mail_daily.sh
0       1       *       *       1       /opt/psa/libexec/modules/watchdog/cp/send-report weekly
20      1       *       *       *       /opt/psa/libexec/modules/watchdog/cp/clean-events
15      1       *       *       1       /opt/psa/libexec/modules/watchdog/cp/pack-sysstats week
15      1       1       *       *       /opt/psa/libexec/modules/watchdog/cp/pack-sysstats year
0       1       *       *       *       /opt/maldet.generate.sh
0       0       *       *       *       /opt/psa/admin/bin/php -c '/opt/psa/admin/conf/php.ini' -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/servershield/scripts/servershield-version-check.php'
0       1       *       *       1       /opt/psa/libexec/modules/watchdog/cp/secur-check
15      1       *       *       *       /opt/psa/libexec/modules/watchdog/cp/pack-sysstats day
*       */1     *       *       *       /usr/local/sendmailanalyzer/sa_cache -a > /dev/null 2>&1
*/10    *       *       *       *       /opt/check_interval.sh
0       1       *       *       2       /opt/backup_from_ftp_store.sh
0       3       *       *       7       /opt/psa/libexec/modules/watchdog/cp/clean-reports
0       0       *       *       0       /opt/psa/admin/bin/php -c '/opt/psa/admin/conf/php.ini' -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/servershield/scripts/update-servershield.php'
15      1       1       *       *       /opt/psa/libexec/modules/watchdog/cp/pack-sysstats month
10      1       *       *       *       /opt/psa/libexec/modules/watchdog/cp/clean-sysstats
MAILTO=""
0       0       *       *       *       /opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/magicspam/scripts/rotate_logs.php'
MAILTO=""
*       *       *       *       *       /opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/sidekick/scripts/sidekick-ping.php'
MAILTO=""
0,5,10,15,20,25,30,35,40,45,50,55       *       *       *       *       /opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/magicspam/scripts/rl-manager.php'
MAILTO=""
0       0       *       *       *       /opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/magicspam/scripts/ms_purge_notifications.php'
MAILTO=""
43      0,6,12,18       *       *       *       /opt/psa/admin/bin/php -dauto_prepend_file=sdk.php '/opt/psa/admin/plib/modules/magicspam/scripts/magicspam-updater.php'
 
Apache2 Modules

Code: 
Compiled in modules:
  core.c
  mod_so.c
  mod_watchdog.c
  http_core.c
  mod_log_config.c
  mod_logio.c
  mod_version.c
  mod_unixd.c

Code: 
Loaded Modules:
 core_module (static)
 so_module (static)
 watchdog_module (static)
 http_module (static)
 log_config_module (static)
 logio_module (static)
 version_module (static)
 unixd_module (static)
 access_compat_module (shared)
 actions_module (shared)
 alias_module (shared)
 auth_basic_module (shared)
 auth_digest_module (shared)
 authn_core_module (shared)
 authn_file_module (shared)
 authz_core_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 bw_module (shared)
 cgi_module (shared)
 cgid_module (shared)
 cloudflare_module (shared)
 dav_module (shared)
 dav_fs_module (shared)
 dav_lock_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 expires_module (shared)
 fastcgi_module (shared)
 fcgid_module (shared)
 filter_module (shared)
 geoip_module (shared)
 headers_module (shared)
 include_module (shared)
 info_module (shared)
 mime_module (shared)
 mpm_event_module (shared)
 negotiation_module (shared)
 perl_module (shared)
 proxy_module (shared)
 proxy_fcgi_module (shared)
 python_module (shared)
 rewrite_module (shared)
 rpaf_module (shared)
 security2_module (shared)
 setenvif_module (shared)
 socache_shmcb_module (shared)
 ssl_module (shared)
 status_module (shared)
 suexec_module (shared)
 sysenv_module (shared)
 unique_id_module (shared)
 userdir_module (shared)
 
Do you have any specific PHP settings, additions? Maybe XCache installed or something else?
 
IgorG said:
It may be not yours but system crontasks. Check it.
Click to expand...

I can't find anything that crashes.
For some reason when you do graceful restart of apache2 it occasionally stops working

IgorG said:
Do you have any specific PHP settings, additions? Maybe XCache installed or something else?
Click to expand...

Code: 
php5-xcache:
  Installed: (none)
  Candidate: 3.2.0-1
  Version table:
     3.2.0-1 0
        500 http://mirror.hetzner.de/debian/packages/ jessie/main amd64 Packages
        500 http://http.debian.net/debian/ jessie/main amd64 Packages

As for specific PHP settings, I haven't touched the settings they are default as they were.
The only modifications I've done are through the PHP settings in the Panel, and usually for Memory Limits and Error Reporting (which doesn't work, no matter what I set it to, I get notices, even though PHP Info shows the correct value for error_reporting, display_errors)
 
As for PHP, I've installed them from the Plesk repository

Code: 
PHP 5.6.14 (cgi-fcgi) (built: Oct  5 2015 13:11:33)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v5.0.18, Copyright (c) 2002-2015, by ionCube Ltd.
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies
 
And the native OS binary

Code: 
PHP 5.6.13-0+deb8u1 (cgi-fcgi) (built: Sep  7 2015 13:39:12)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with the ionCube PHP Loader (enabled) + Intrusion Protection from ioncube24.com (unconfigured) v5.0.12, Copyright (c) 2002-2015, by ionCube Ltd.
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies
 
/etc/logrotate.d/apache2
Code: 
/var/log/apache2/*.log {
        daily
        missingok
        rotate 14
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if /etc/init.d/apache2 status > /dev/null ; then \
                    /etc/init.d/apache2 reload > /dev/null; \
                fi;
        endscript
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi; \
        endscript
}
 
I would suggest you contact Odin Support Team. Experienced supporters will investigate and fix this issue directly on your server.
 
So the only cron that runs around that time is cron.daily, maybe it's the log rotate script but I'm not sure, so I added a log entry in the logrotate script

Code: 
$ cat /etc/logrotate.d/apache2
/var/log/apache2/*.log {
        daily
        missingok
        rotate 14
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                logger -i -t "logrotate.apache2" "postrotate"
                if /etc/init.d/apache2 status > /dev/null ; then \
                    /etc/init.d/apache2 reload > /dev/null; \
                fi;
        endscript
        prerotate
                logger -i -t "logrotate.apache2" "prerotate"
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
                        run-parts /etc/logrotate.d/httpd-prerotate; \
                fi; \
        endscript
}

Is there nothing else that you know, that can cause this?

Is there a way to regenerate all php.ini files for all the domains ?
 
Looks like the crash is done by logrotate

Code: 
[Tue Oct 20 07:01:43.033135 2015] [core:notice] [pid 18913] AH00060: seg fault or similar nasty error detected in the parent process
[Wed Oct 21 06:57:37.975684 2015] [core:notice] [pid 31859] AH00060: seg fault or similar nasty error detected in the parent process
[Thu Oct 22 06:58:54.488246 2015] [core:notice] [pid 13528] AH00060: seg fault or similar nasty error detected in the parent process

Code: 
Oct 20 07:01:44 s1 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="19573" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Oct 21 06:57:38 s1 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="19573" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
Oct 22 06:58:55 s1 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="19573" x-info="http://www.rsyslog.com"] rsyslogd was HUPed

As you can see rsyslog was HUPed right after the crash.

After this you need to kill all the Apache2 processes

Code: 
killall apache2

And after that you can start the service right away.

Code: 
service apache2 start

And it works, but not without cleaning all the apache2 processes.

@IgorG do you know of any reason why this would happen
 
ilijamt said:
@IgorG do you know of any reason why this would happen
Click to expand...
I don't know. Therefore I suggested you contact Odin Support. Investigation directly on your server is required. Not sure what we can do it in scope of forum discussion.
 
The problem is logrotate, I've confirmed it.

One way around it is to replace the default /etc/logrotate.d/apache2, haven't had a crash since I updated the logrotate.
Looks like the problem is in the event and with some php plugins.
In my problem it was the mime module.

Code: 
/var/log/apache2/*.log {
        daily
        missingok
        rotate 24
        compress
        delaycompress
        notifempty
        create 640 root adm
        copytruncate
}

Another way to fix this is to switch from event to prefork.

If you want to stay on event you will have to disable all the modules and then enable one by one, and test if they work by issuing the following in bash

Code: 
for i in seq 1 20; do echo $i; sudo service apache2 reload; sleep 2; done;

It's a longer process but if you want to stay on event you don't have a choice. Because if you stay, you will start having segfaults with the apache processes

And you can recompile the module you found faulty, but it will be overwritten on next update, unless you create your own repository with your own packages and pin it to always install from there.

Either way problem solved.
 
You must log in or register to reply here.

Similar threads

Ming
Issue Passenger issue - boost::thread_resource_error: Resource temporarily unavailable
Replies
12
Views
3K
Ming
Ming
Rincewind
Issue Apache restart failing
Replies
2
Views
1K
Rincewind
Rincewind
E
Issue Apache sporadically shuts down
Replies
5
Views
2K
Ohya
O
H
Apache dies every night since upgrading to Debian 8
Replies
2
Views
2K
H9k
H
E
Apache crashes
Replies
1
Views
3K
UFHH01
U
Back
Top