• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

DDOS or hihg CPU load dedicated centos linux server

T

tenoch

New Pleskian
HI
I have been trying to identify why mi server has high load cpu, suddenly

CPU load suddenly jumps, from 1 to 40 and until 110
If I restart mysqld and apache, the cpu gets back to normal, ultil this happen again

My server is
GenuineIntel, Intel(R)Core(TM) i5-2400 CPU @ 3.10GHz
Version Parallels Plesk Panel v11.0.9_build110120608.16 os_CentOS 6
CentOS 6.4 (Final)
CentOS 6 x64 Processor: Intel Core i5 - 3.10 GH

I already installed FAIL2BAN and DDOS DEFLATE without lock
I have CloudFlare Enabled

When there is a high CPU load, I have checked netstats, and I don't indentify many ips conected, so I think it could be an internat script (or virus)

I will paste the result of
ps fax
command, in a moment of high cpu load
in the next reply


Any suggestion or idea what could be happenig?

thanks a lot
 
PART 1
Code: 
    2 ?        S      0:00 [kthreadd]
    3 ?        S      0:00  \_ [migration/0]
    4 ?        S      0:00  \_ [ksoftirqd/0]
    5 ?        S      0:00  \_ [migration/0]
    6 ?        S      0:00  \_ [watchdog/0]
    7 ?        S      0:00  \_ [migration/1]
    8 ?        S      0:00  \_ [migration/1]
    9 ?        S      0:00  \_ [ksoftirqd/1]
   10 ?        S      0:00  \_ [watchdog/1]
   11 ?        S      0:00  \_ [migration/2]
   12 ?        S      0:00  \_ [migration/2]
   13 ?        S      0:00  \_ [ksoftirqd/2]
   14 ?        S      0:00  \_ [watchdog/2]
   15 ?        S      0:00  \_ [migration/3]
   16 ?        S      0:00  \_ [migration/3]
   17 ?        S      0:00  \_ [ksoftirqd/3]
   18 ?        S      0:00  \_ [watchdog/3]
   19 ?        S      0:00  \_ [events/0]
   20 ?        S      0:00  \_ [events/1]
   21 ?        S      0:00  \_ [events/2]
   22 ?        S      0:00  \_ [events/3]
   23 ?        S      0:00  \_ [cgroup]
   24 ?        S      0:00  \_ [khelper]
   25 ?        S      0:00  \_ [netns]
   26 ?        S      0:00  \_ [async/mgr]
   27 ?        S      0:00  \_ [pm]
   28 ?        S      0:00  \_ [sync_supers]
   29 ?        S      0:00  \_ [bdi-default]
   30 ?        S      0:00  \_ [kintegrityd/0]
   31 ?        S      0:00  \_ [kintegrityd/1]
   32 ?        S      0:00  \_ [kintegrityd/2]
   33 ?        S      0:00  \_ [kintegrityd/3]
   34 ?        S      0:00  \_ [kblockd/0]
   35 ?        S      0:00  \_ [kblockd/1]
   36 ?        S      0:00  \_ [kblockd/2]
   37 ?        R      0:00  \_ [kblockd/3]
   38 ?        S      0:00  \_ [kacpid]
   39 ?        S      0:00  \_ [kacpi_notify]
   40 ?        S      0:00  \_ [kacpi_hotplug]
   41 ?        S      0:00  \_ [ata/0]
   42 ?        S      0:00  \_ [ata/1]
   43 ?        S      0:00  \_ [ata/2]
   44 ?        S      0:00  \_ [ata/3]
   45 ?        S      0:00  \_ [ata_aux]
   46 ?        S      0:00  \_ [ksuspend_usbd]
   47 ?        S      0:00  \_ [khubd]
   48 ?        S      0:00  \_ [kseriod]
   49 ?        S      0:00  \_ [md/0]
   50 ?        S      0:00  \_ [md/1]
   51 ?        S      0:00  \_ [md/2]
   52 ?        S      0:00  \_ [md/3]
   53 ?        S      0:00  \_ [md_misc/0]
   54 ?        S      0:00  \_ [md_misc/1]
   55 ?        S      0:00  \_ [md_misc/2]
   56 ?        S      0:00  \_ [md_misc/3]
   57 ?        S      0:00  \_ [khungtaskd]
   58 ?        D      0:05  \_ [kswapd0]
   59 ?        SN     0:00  \_ [ksmd]
   60 ?        SN     0:00  \_ [khugepaged]
   61 ?        S      0:00  \_ [aio/0]
   62 ?        S      0:00  \_ [aio/1]
   63 ?        S      0:00  \_ [aio/2]
   64 ?        S      0:00  \_ [aio/3]
   65 ?        S      0:00  \_ [crypto/0]
   66 ?        S      0:00  \_ [crypto/1]
   67 ?        S      0:00  \_ [crypto/2]
   68 ?        S      0:00  \_ [crypto/3]
   73 ?        S      0:00  \_ [kthrotld/0]
   74 ?        S      0:00  \_ [kthrotld/1]
   75 ?        S      0:00  \_ [kthrotld/2]
   76 ?        S      0:00  \_ [kthrotld/3]
   78 ?        S      0:00  \_ [kpsmoused]
   79 ?        S      0:00  \_ [usbhid_resumer]
  109 ?        S      0:00  \_ [kstriped]
  144 ?        S      0:00  \_ [i915]
  146 ?        S<     0:00  \_ [kslowd000]
  147 ?        S<     0:00  \_ [kslowd001]
  328 ?        S      0:00  \_ [scsi_eh_0]
  329 ?        S      0:00  \_ [scsi_eh_1]
  330 ?        S      0:00  \_ [scsi_eh_2]
  331 ?        S      0:00  \_ [scsi_eh_3]
  332 ?        S      0:00  \_ [scsi_eh_4]
  333 ?        S      0:00  \_ [scsi_eh_5]
  424 ?        D      0:00  \_ [jbd2/sda3-8]
  425 ?        S      0:00  \_ [ext4-dio-unwrit]
  426 ?        S      0:00  \_ [ext4-dio-unwrit]
  427 ?        S      0:00  \_ [ext4-dio-unwrit]
  428 ?        S      0:00  \_ [ext4-dio-unwrit]
  450 ?        D      0:00  \_ [flush-8:0]
  478 ?        S      0:02  \_ [kauditd]
  962 ?        S      0:00  \_ [jbd2/sda1-8]
  963 ?        S      0:00  \_ [ext4-dio-unwrit]
  964 ?        S      0:00  \_ [ext4-dio-unwrit]
  965 ?        S      0:00  \_ [ext4-dio-unwrit]
  966 ?        S      0:00  \_ [ext4-dio-unwrit]
  967 ?        S      0:00  \_ [jbd2/sdb1-8]
  968 ?        S      0:00  \_ [ext4-dio-unwrit]
  969 ?        S      0:00  \_ [ext4-dio-unwrit]
  970 ?        S      0:00  \_ [ext4-dio-unwrit]
  971 ?        S      0:00  \_ [ext4-dio-unwrit]
 1266 ?        S      0:00  \_ [kondemand/0]
 1267 ?        S      0:00  \_ [kondemand/1]
 1268 ?        S      0:00  \_ [kondemand/2]
 1269 ?        S      0:00  \_ [kondemand/3]
    1 ?        Ss     0:00 /sbin/init
  525 ?        S<s    0:00 /sbin/udevd -d
 2953 ?        S<     0:00  \_ /sbin/udevd -d
 2954 ?        S<     0:00  \_ /sbin/udevd -d
 1236 ?        Ss     0:00 /sbin/portreserve
 1243 ?        Sl     0:00 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
 1290 ?        Ss     0:00 irqbalance
 1540 ?        Ss     0:00 dbus-daemon --system
 1569 ?        Ss     0:00 /usr/sbin/acpid
 1578 ?        Ss     0:00 hald
 1579 ?        S      0:00  \_ hald-runner
 1607 ?        S      0:00      \_ hald-addon-input: Listening on /dev/input/event0 /dev/input/event3 /dev/input/event1
 1614 ?        S      0:00      \_ /usr/libexec/hald-addon-generic-backlight
 1620 ?        S      0:00      \_ hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
 1683 ?        S      0:00 /usr/sbin/sw-cp-serverd -f /etc/sw-cp-server/config
 3575 ?        Ss     0:00  \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 3756 ?        D      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 3757 ?        D      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 3758 ?        D      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 5701 ?        D      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 6276 ?        D      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 6848 ?        S      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 6856 ?        S      0:00      \_ /usr/bin/sw-engine-cgi -c /usr/local/psa/admin/conf/php.ini -d auto_prepend_file=auth.php3 -u psaadm
 
PART 2

Code: 
 1694 ?        Ss     0:00 /usr/sbin/sshd
 4897 ?        Ss     0:00  \_ sshd: caplogin4 [priv]
 5056 ?        S      0:00      \_ sshd: caplogin4@pts/0
 5068 pts/0    Ss     0:00          \_ -bash
 5386 pts/0    S      0:00              \_ su -
 5523 pts/0    S      0:00                  \_ -bash
 6915 pts/0    R+     0:00                      \_ ps fax
 1702 ?        Ss     0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
 6241 ?        Ds     0:00  \_ /var/qmail/bin/relaylock /var/qmail/bin/qmail-smtpd /var/qmail/bin/smtp_auth /var/qmail/bin/true /var/qmail/bin/cmd5checkpw /var/qmail/bin/true
 6507 ?        Ss     0:00  \_ proftpd: caplogin - 189.191.250.7: IDLE
 1719 ?        S      0:00 /usr/lib/courier-imap/couriertcpd -address=0 -stderrlogger=/usr/sbin/courierlogger -stderrloggername=imapd -maxprocs=40 -maxperip=4 -pid=/var/run/imapd.pid -nodnslookup -noidentlookup 143 /usr/sbin/imaplogin
 1721 ?        S      0:00 /usr/sbin/courierlogger imapd
 1729 ?        S      0:00 /usr/lib/courier-imap/couriertcpd -address=0 -stderrlogger=/usr/sbin/courierlogger -stderrloggername=imapd-ssl -maxprocs=40 -maxperip=4 -pid=/var/run/imapd-ssl.pid -nodnslookup -noidentlookup 993 /usr/bin/cou
 1731 ?        S      0:00 /usr/sbin/courierlogger imapd-ssl
 1737 ?        S      0:00 /usr/lib/courier-imap/couriertcpd -address=0 -stderrlogger=/usr/sbin/courierlogger -stderrloggername=pop3d -maxprocs=40 -maxperip=4 -pid=/var/run/pop3d.pid -nodnslookup -noidentlookup 110 /usr/sbin/pop3login
 1739 ?        S      0:00 /usr/sbin/courierlogger pop3d
 1746 ?        S      0:00 /usr/lib/courier-imap/couriertcpd -address=0 -stderrlogger=/usr/sbin/courierlogger -stderrloggername=pop3d-ssl -maxprocs=40 -maxperip=4 -pid=/var/run/pop3d-ssl.pid -nodnslookup -noidentlookup 995 /usr/bin/cou
 1748 ?        S      0:00 /usr/sbin/courierlogger pop3d-ssl
 1758 ?        S      0:00 qmail-send
 1760 ?        S      0:00  \_ splogger qmail
 1761 ?        S      0:00  \_ qmail-lspawn | /usr/bin/deliverquota ./Maildir
 1762 ?        S      0:00  \_ qmail-rspawn
 1763 ?        S      0:00  \_ qmail-clean
 3582 ?        D      0:00  \_ bin/qmail-queue
 1800 ?        Ss     0:00 /usr/sbin/abrtd
 1809 ?        Ss     0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
 1810 ?        D      0:01  \_ nginx: worker process                   
 1855 ?        S      0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid --basedir=/usr --user=mysql
 1960 ?        Sl     0:22  \_ /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
 2038 ?        Ssl    0:00 /usr/sbin/named -u named -c /etc/named.conf -u named -t /var/named/chroot
 2061 ?        Ss     0:02 /usr/bin/spamd --username=popuser --daemonize --nouser-config --helper-home-dir=/var/qmail --max-children 5 --create-prefs --virtual-config-dir=/var/qmail/mailnames/%d/%l/.spamassassin --pidfile=/var/run/spam
 2062 ?        S      0:00  \_ spamd child
 2063 ?        S      0:00  \_ spamd child
 2129 ?        Ss     0:01 /usr/sbin/httpd
 2132 ?        S      0:00  \_ /usr/sbin/httpd

************* REPEAT 10 TIMES SIMILAR DATA ***********

 2188 ?        S      0:01  \_ /usr/sbin/httpd
 6274 ?        S      0:00  |   \_ bin/qmail-inject -H --
 6275 ?        D      0:00  |       \_ bin/qmail-queue
 2190 ?        D      0:01  \_ /usr/sbin/httpd
 
 ************* REPEAT 80 TIMES SIMILAR DATA ***********

 
 6416 ?        S      0:00  |   \_ /usr/bin/perl -w? mt-comments.cgi
 2719 ?        S      0:01  \_ /usr/sbin/httpd
 2885 ?        S      0:02  \_ /usr/sbin/httpd
************* REPEAT 10 TIMES SIMILAR DATA ***********
 2889 ?        S      0:02  \_ /usr/sbin/httpd
 6147 ?        S      0:00  |   \_ bin/qmail-inject -H --
 6154 ?        D      0:00  |       \_ bin/qmail-queue
 2998 ?        S      0:00  \_ /usr/sbin/httpd
 3097 ?        S      0:01  \_ /usr/sbin/httpd

************* REPEAT 40 TIMES SIMILAR DATA ***********

 6081 ?        S      0:00  \_ /usr/sbin/httpd
 6084 ?        S      0:00  \_ /usr/sbin/httpd
 6374 ?        S      0:00  |   \_ bin/qmail-inject -H --
 6379 ?        D      0:00  |       \_ bin/qmail-queue
 6087 ?        S      0:00  \_ /usr/sbin/httpd
 6144 ?        S      0:00  \_ /usr/sbin/httpd
 6149 ?        S      0:00  \_ /usr/sbin/httpd
 6158 ?        S      0:00  \_ /usr/sbin/httpd
 6362 ?        S      0:00  \_ /usr/sbin/httpd
 ************* REPEAT 20 TIMES SIMILAR DATA ***********
 6790 ?        S      0:00  \_ /usr/sbin/httpd
 6793 ?        S      0:00  \_ /usr/sbin/httpd
 2482 ?        S      0:00 /usr/bin/sw-engine -c /usr/local/psa/admin/conf/php.ini /usr/lib64/plesk-9.0/psa-health-monitor-notification.php
 2518 ?        Ssl    0:08 /usr/sbin/sw-collectd -C /etc/sw-collectd/collectd.conf
 2631 ?        Sl     0:00 /usr/bin/python /usr/bin/fail2ban-server -b -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x
 2706 ?        Ss     0:00 /usr/sbin/atd
 2888 ?        Ss     0:00 crond
 3514 ?        S      0:00  \_ CROND
 3544 ?        Ss     0:00  |   \_ /bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
 3549 ?        S      0:00  |       \_ curl -k -s -S -w  {"http_code": %{http_code}} -X POST -H Authorization: ApplicationToken ff83792ce2f8f6ed4b021fecd1e145e2450e7053 -H Expect:  -d {"jsonrpc":"2.0","method":"runAll","params":[],"id"
 4582 ?        S      0:00  \_ CROND
 4596 ?        Ss     0:00  |   \_ /bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
 4612 ?        S      0:00  |       \_ curl -k -s -S -w  {"http_code": %{http_code}} -X POST -H Authorization: ApplicationToken ff83792ce2f8f6ed4b021fecd1e145e2450e7053 -H Expect:  -d {"jsonrpc":"2.0","method":"runAll","params":[],"id"
 4585 ?        S      0:00  \_ CROND
 5210 ?        S      0:00  |   \_ bin/qmail-inject -H -f root --
 5543 ?        D      0:00  |       \_ bin/qmail-queue
 4724 ?        S      0:00  \_ CROND
 4739 ?        Ss     0:00  |   \_ /bin/bash /usr/bin/run-parts /etc/cron.hourly
 5324 ?        D      0:00  |       \_ /bin/sh /etc/cron.hourly/plesk-php-cleanuper
 5325 ?        S      0:00  |       \_ awk -v progname=/etc/cron.hourly/plesk-php-cleanuper progname {?????   print progname ":\n"?????   progname="";????       }????       { print; }
 5640 ?        S      0:00  \_ CROND
 5670 ?        Ss     0:00  |   \_ /bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
 5671 ?        S      0:00  |       \_ curl -k -s -S -w  {"http_code": %{http_code}} -X POST -H Authorization: ApplicationToken ff83792ce2f8f6ed4b021fecd1e145e2450e7053 -H Expect:  -d {"jsonrpc":"2.0","method":"runAll","params":[],"id"
 6218 ?        S      0:00  \_ CROND
 6239 ?        Ss     0:00  |   \_ /bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
 6256 ?        S      0:00  |       \_ curl -k -s -S -w  {"http_code": %{http_code}} -X POST -H Authorization: ApplicationToken ff83792ce2f8f6ed4b021fecd1e145e2450e7053 -H Expect:  -d {"jsonrpc":"2.0","method":"runAll","params":[],"id"
 6752 ?        S      0:00  \_ CROND
 6774 ?        Ss     0:00      \_ /bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
 6797 ?        S      0:00          \_ curl -k -s -S -w  {"http_code": %{http_code}} -X POST -H Authorization: ApplicationToken ff83792ce2f8f6ed4b021fecd1e145e2450e7053 -H Expect:  -d {"jsonrpc":"2.0","method":"runAll","params":[],"id"
 2930 tty1     Ss+    0:00 /sbin/mingetty /dev/tty1
 2932 tty2     Ss+    0:00 /sbin/mingetty /dev/tty2
 2934 tty3     Ss+    0:00 /sbin/mingetty /dev/tty3
 2936 tty4     Ss+    0:00 /sbin/mingetty /dev/tty4
 2938 tty5     Ss+    0:00 /sbin/mingetty /dev/tty5
 2940 tty6     Ss+    0:00 /sbin/mingetty /dev/tty6
 3129 ?        Ss     0:00 /bin/sh -e /dev/fd/10
 3271 ?        DN     0:00  \_ /bin/bash /etc/cron.daily/readahead.cron
 3243 ?        S<sl   0:00 auditd
 4292 ?        S      0:00 bin/qmail-inject -a -- root
 4315 ?        D      0:00  \_ bin/qmail-queue
 4317 ?        S      0:00 /bin/sh /usr/local/ddos/ddos.sh
 4330 ?        S      0:00  \_ sleep 1800
 4528 ?        S      0:00 bin/qmail-inject -a -- root
 4549 ?        D      0:00  \_ bin/qmail-queue
************* REPEAT 20 TIMES SIMILAR DATA ***********
 6496 ?        S      0:00 /bin/sh /usr/local/ddos/ddos.sh
 6497 ?        S      0:00  \_ sleep 1800
 
/bin/bash /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini
Click to expand...
Do you really use CBM on your Plesk server?
 
No, I don´t
I just installed recently, wanted to see what it is
But I haven't try it

Any idea, what script could be the problem?

Thanks
 
You should contact your system administrator to check this more futher (this needs additional troubleshooting and high CPU is something that is not easy to find, where the problem is).

You have to check additional logs like apache/nginx/mysql, check mysql with command "SHOW PROCESSLIST;" when there is high CPU, list current active processes during load with:

# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -20

This will list top 20 processes that use the most CPU.

It's hard to help when you're not connected to the server :)
 
Check out the sysdig package from the atomic repo, it will let you dig very deep into what the system is doing. To install:

wget -q -O - http://www.atomicorp.com/installers/atomic |sh

yum install sysdig

Then to look at where httpd is spending most of the system I/O:

sysdig -c topfiles_time proc.name=httpd

The output will show you exactly which files are using the most I/O, and this type of query is just scratching the surface on what it can do.
 
atomicturtle said:
Check out the sysdig package from the atomic repo, it will let you dig very deep into what the system is doing. To install:

wget -q -O - http://www.atomicorp.com/installers/atomic |sh

yum install sysdig

Then to look at where httpd is spending most of the system I/O:

sysdig -c topfiles_time proc.name=httpd

The output will show you exactly which files are using the most I/O, and this type of query is just scratching the surface on what it can do.
Click to expand...

Thank you, I am installing this package and I will check logs later

Meanwhile, I have activated CloudFlare in mode "I am under attack", that delays 5 seconds any time someone enters to my websites, and check browser
https://www.cloudflare.com/ddos

And this way, my server is not crashed with this ddos attacks


Regards
 
You must log in or register to reply here.

Similar threads

E
Apache uses all CPU, server dies
Replies
1
Views
7K
Faris Raouf
F
D
High Apache CPU usage and MySQL CPU usage
Replies
3
Views
5K
Kanagaraj
K
F
Health monitor - Memory usage increase slowly but continuously
Replies
4
Views
3K
FranckL
F
1
Bug in centos 6 x64 plesk 10.3.1
Replies
15
Views
5K
Alexey_Shpak
A
Back
Top