Apache: couldn't grab the accept mutex

[faris]

I'm having an unusual problem. Apache keeps dying on me between once and 5 times a day.

/var/log/httpd/error_log has the following for each crash:
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [emerg] (43)Identifier removed: couldn't grab the accept mutex
[Wed Apr 20 00:36:41 2005] [notice] caught SIGTERM, shutting down

This is under RedHat 9, with Apache 2.0.40-21.17, Scott's grsec kernel, and with mod_security installed.

I'm getting close to 300 domains on this server (275). I'm wondering if this might be the issue? The problem is that it can happen anything from just once a day to many times a day. BUT, when it does happen more than once, they are clustered together -- they are not spread out. For example, last night, there were 4 problems like this, all within a 20 minute timeframe, from about 00:20 to 00:40.

Luckily I have 4psa's Server Assistant, which monitors Apache and re-starts it when it dies. But I would like to try and figure out what might be causing this. Does anyone have any ideas?

Faris

 

[Cranky]

Are you running any "unusual" apache modules? Do you have sufficient diskspace and memory?

 

[faris]

Other than mod_security, nope. Plenty of disk. Plenty of memory (well, only 1Meg but more than a very small amount of Swap almost never gets used so I figure it should be enough).

I've searched for this error on Google and came up with nothing useful at all.

I've not had any errors in the past 24 hours though.

Faris.

 

[killedar]

Having same issue with no load on the sever.

I am getting this same problem. plesk 8.0 Fc4.
New server and no domain installed yet. so there is no load still I get this error. Apache dies with SIGTERM will above exact same mutex entries.
Any ideas?

 

[faris]

I'm racking my brain trying to remember what happened regarding this, and why these errors went away for me.

I think it happened at about the same time that I had some suspected hardware issues which mysteriously went away, but I can't remember for sure.

This problem was very frustrating at the time.

Do you have any sort of system health monitoring software installed? Is there anything at all unusual in any log other than the apache error?

Is it possible to start from scratch on that server, given that you don't have anyone on it yet? Maybe try a different OS (e.g. Centos 4)?

Faris.

 

[killedar]

this is happenign due to watchdog

I found that this is happening due to watch dog(watchdog2 that comes default with plesk). I disabled the watchdog to monitor apache and it goes away. I enabled watchdog and it comes back.
The mutex problem happened only once. But the SIGTERM was happening periodically every 25 minutes or so.

The watchdog doesn't have any issues with other servers/resources I am monitoring. Only Apache.
Any ideas?