Issue Websites down, login impossible, high average load, bot low other load metrics.

[rreimche]

Recently I've experienced a problem on my CentOS 7 Server running a couple of websites and Plesk. Suddenly the websites went offline and I could not log into the server (my login credentials were not accepted), Plesk was unresponsible was well. The problem was gone (almost) by itself after one or two reboots. Today I've finally had time to investigate the issue. For now I've look only on system logs and here's what I've found.

During the time of the problem I see high avg load but low and nearly constant CPU load, some raise in memory usage and nothing interesting in storage drive statistics:

Dropbox - File Deleted - Simplify your life

www.dropbox.com

Dropbox - File Deleted - Simplify your life

www.dropbox.com

Dropbox - File Deleted - Simplify your life

www.dropbox.com

How is this possible that load avg is high but particularly nothing is anywhere seriously loaded? What else should I look at to clarify the issue?

 

[Denis Gomes Franco]

Surely this is not related to Plesk at all. I'm no programming expert but this is almost always related to how a site was coded. I cannot provide any in depth advice but you should look at that code to try to identify what's going on. Also, your sites may be vulnerable and they may be receiving hacking attempts, getting DDOS'ed (whether with legitimate or illegitimate traffic) or someone is exploiting a flaw. For instance: a long time ago I hosted a site that was hand-coded in PHP and it had a flaw in the contact form that allowed one to send spam. I quickly dumped that client.

 

[Denis Gomes Franco]

On another note, your screenshots seem to indicate you have a large server. Please note that having a server with plenty of specs is no substitute for efficient coding/better software architecture in case of a big site with lots of traffic.

 

[randompleskuser]

Just screenshots of graphs provide 0 information, what does TOP show, etc.?

 

[Bitpalast]

I can't see any high load average. Could you please explain this in more detail what you mean by that? On the first scdreenshot I can only see the upper left corner of a chart, but that is basically empty. The other screenshots show RAM and hard disk usage, neither one is relevant.

 

[Denis Gomes Franco]

I could swear I read that graph as 100% CPU usage...

 

[randompleskuser]

I could swear I read that graph as 100% CPU usage...

yes, it is, but it does not say what processes are using CPU so not useful at all

 

[Denis Gomes Franco]

Plesk will not help you that much, it is a control panel, not a debugging tool. You can SSH into your server and then run HTOP (if it's not installed then just install it) and it MIGHT help you find out which client is causing the high load - HTOP in Linux is like Task Manager in Windows.

But if you really need to dig down exactly which scripts are causing loads and what you can do to improve the site's coding, then Im afraid you'll have to use external debugging tools, maybe even New Relic.

In any case you didn't mention if you're just hosting these websites for your clients, or if these are your own websites. Debugging high CPU usage, if this is really a case of bad programming, is the primary responsibility of the coders.