Too much httpd and load

[skiiiks]

I have 3.0 pentium with 2Gb RAM... It's the first time I use plesk and this is strange...

top - 23:20:00 up 2:42, 1 user, load average: 9.87, 13.60, 10.29
Tasks: 149 total, 29 running, 120 sleeping, 0 stopped, 0 zombie
Cpu(s): 93.9% us, 6.1% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2064724k total, 1208656k used, 856068k free, 46252k buffers
Swap: 522104k total, 0k used, 522104k free, 354784k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2354 mysql 20 0 290m 90m 4500 S 12.6 4.5 21:08.10 mysqld
29098 apache 20 0 44140 23m 4352 R 8.0 1.2 0:02.20 httpd
28621 apache 20 0 41560 23m 4360 R 7.3 1.1 0:06.38 httpd
28855 apache 20 0 43992 23m 4356 R 7.3 1.2 0:06.28 httpd
29252 apache 20 0 34672 15m 3344 R 7.3 0.8 0:00.22 httpd
27117 apache 20 0 44632 24m 4692 S 6.6 1.2 0:19.58 httpd
28517 apache 20 0 44196 23m 4500 R 6.6 1.2 0:10.76 httpd
28930 apache 20 0 44272 23m 4504 R 6.6 1.2 0:06.16 httpd
28994 apache 20 0 42756 22m 4396 R 6.6 1.1 0:03.12 httpd
29026 apache 20 0 43584 23m 4700 R 6.6 1.1 0:04.04 httpd
29173 apache 20 0 41212 22m 3708 R 6.6 1.1 0:01.86 httpd
29194 apache 20 0 41056 21m 3756 R 6.6 1.1 0:01.08 httpd
29229 apache 20 0 38132 18m 3668 R 6.6 0.9 0:01.04 httpd
29232 apache 20 0 42876 22m 4344 R 6.6 1.1 0:01.08 httpd
29236 apache 20 0 36708 17m 3728 R 6.6 0.9 0:00.74 httpd
28564 apache 20 0 44300 23m 4712 R 6.0 1.2 0:08.96 httpd
28926 apache 20 0 44228 23m 4408 R 6.0 1.2 0:04.44 httpd
29063 apache 20 0 43588 22m 4336 R 6.0 1.1 0:03.40 httpd
29073 apache 20 0 43944 23m 4388 R 6.0 1.2 0:03.94 httpd
29172 apache 20 0 41056 21m 3752 R 6.0 1.1 0:01.50 httpd
29192 apache 20 0 44704 24m 4408 R 6.0 1.2 0:03.20 httpd
29237 apache 20 0 37100 18m 3660 R 6.0 0.9 0:00.64 httpd
29254 apache 20 0 33432 14m 3340 R 6.0 0.7 0:00.18 httpd
29228 apache 20 0 35056 16m 3656 S 5.3 0.8 0:00.36 httpd
29105 apache 20 0 44744 24m 4452 R 4.7 1.2 0:04.74 httpd
29106 apache 20 0 43908 23m 4548 R 4.7 1.2 0:05.36 httpd
29193 apache 20 0 35124 16m 3652 R 4.7 0.8 0:00.68 httpd
29195 apache 20 0 43324 22m 4332 S 4.7 1.1 0:01.36 httpd
28551 apache 20 0 44240 23m 4596 R 4.0 1.2 0:10.64 httpd
28727 apache 20 0 44012 23m 4492 S 4.0 1.2 0:08.70 httpd
29223 apache 20 0 44048 23m 4428 S 3.3 1.2 0:01.26 httpd
29224 apache 20 0 43236 22m 4328 S 2.0 1.1 0:01.06 httpd
29219 apache 20 0 35108 16m 3648 S 1.3 0.8 0:00.70 httpd
29259 apache 20 0 28860 9816 3296 R 1.3 0.5 0:00.04 httpd
29260 apache 20 0 28464 9476 3284 R 1.3 0.5 0:00.04 httpd

I have some webs with 30K/40K visits each day (it's not too much)... A smaller CPU with 1Gb RAM with cPanel and Apache 1.3 worked perfectly, and now with a bigger computer and plesk I see this...

More info on server-status

Server Version: Apache/2.0.54 (Fedora)
Server Built: Jul 26 2006 11:00:19

Current Time: Monday, 28-Apr-2008 23:40:53 CEST
Restart Time: Monday, 28-Apr-2008 23:25:38 CEST
Parent Server Generation: 0
Server uptime: 15 minutes 14 seconds
Total accesses: 13025 - Total Traffic: 44.9 MB
CPU Usage: u1183.04 s50.79 cu.01 cs0 - 135% CPU load
14.3 requests/sec - 50.3 kB/second - 3616 B/request
34 requests currently being processed, 67 idle workers

C_C_C_____W__C_WC__WCW_W_R______C____W_W_W._W.___WW____WC.CC___W
___W__WC_W__W_________W__..WW_W.___W____._.__...................
................................................................
..........................................................


Any help, PLEASE ?

 

[Dreamforge]

Check your log and see wich script causes this problem
Stop apache and try to isolate this page / script
Such a high number of idle apache worker processes is not normal

Is the same website you had on cPanel / apache 1.3 ?
I've had this kind of problem with a malformed CGI script that was causing infinite loop

Other way of reflexion... DDOS Attack ?

 

[skiiiks]

I run netstat to see if it's dos attack... it's not

Yes, it's EXACTLY the same site I had on cpanel/apache 1.3... no changes... that's what this gets me confused...

I have turn keepalive off, and put more childs , and things is going better... but still need tunning

 

[madcat]

Have a look in /tmp (and /var/tmp and /dev/shm if you have them) for "stange" files, or files that start with "!#" in the first line (ie: "#!/usr/bin/perl" or "#!/bin/sh"), or files that are binary. It's possible one or more of your users has a Perl or PHP script in their 'httpdocs' directory that is exploitable, and which allows various miscreants on the internet to use your server to attack other servers or send spam.

Of course, tracking down which user is causing the issue is quite difficult, since Plesk doesn't provide any tools to help, and does not use suphp by default. Tools such as 'ps -ef', 'lsof', and 'netstat' can help. Trying running 'lsof' on the process ID of a "bad" script, and/or it's parent process.

Hope this helps ... Good luck.

 

[Amin Taheri]

I had a similar problem once, not sure what caused it but I had about 30 httpd graceful restarts occuring at once that were all stepping on each other, and raising load as lots of requests came in.

I just killed them all

for i in `ps -ef | grep graceful | grep -v grep | awk '{print $2}'`; do kill -9 $i;done

Thats not to say that your problem is the same, but hopefully that helps some what