Issue High CPU After upgrading to plesk 18

[TurnKeyWebsites]

After upgrading to plesk 18 I noticed watchdog is reporting high cpu usage every day (i.e 1.3182% for a single cpu vps).
Since then I have also completed a fresh install (ubuntu 18, plesk 18) and it's like this for a fresh install with no websites or data.

Is there an issue with this new version of plesk?

 

[AYamshanov]

Hi TurnKeyWebsites,

Could you please describe your server hardware and show how it looks like in Plesk and in the reporting message?

 

[TurnKeyWebsites]

Hi,
Plesk is running on a VPS
CPU Intel Core Processor (Haswell, no TSX) (1 core(s))
Version Plesk Obsidian v18.0.23_build1800200130.12 os_Ubuntu 18.04
OS Ubuntu 18.04.4 LTS
6GB memory (8.37% usage)
25G disk (23.87% used)


This is the most recent watchdog report, every day it's about 1.3% load:

Watchdog is running since Feb 5, 2020 07:51 AM.
Watchdog is monitoring services:
Plesk Web Server
Plesk PHP Engine
Web Server (Apache)
SMTP Server (Postfix)
Dovecot IMAP and POP3 server
DNS Server (BIND)
MySQL
Plesk Postfix milter filter
Web Proxy Server (Nginx)
Fail2Ban authentication failure monitor
Watchdog is not monitoring disks.

Average CPU load: 1.3497%.
Average RAM usage: 954173.0210.
Security scans number: 1.

I've attached a graph of cpu load for 7 days, if you want anything else, let me know. Thanks.

 

[TurnKeyWebsites]

also not sure why it states "Watchdog is not monitoring disks.", watchdog IS monitoring disks.

 

[TurnKeyWebsites]

Hi TurnKeyWebsites,

Could you please describe your server hardware and show how it looks like in Plesk and in the reporting message?

Is that the info you wanted?

 

[AYamshanov]

Is that the info you wanted?

Yes.

Why do you think "watchdog is reporting high CPU usage"?
- 1.3% is not high, right? The message said about the average usage.
- It seems just as a daily reporting with brief information about the server.

 

[TurnKeyWebsites]

Prior to this version, a typical server load was Average CPU load: 0.4035% that's on a VPS with several active websites.
My understanding of 'cpu load' was that a number over 1 is equivalent to over 100% 'cpu usage' on a single cpu system.
so 1.3 would imply processes being queued which should not happen on an idle system.

Understanding Linux CPU Load - when should you be worried? | Scout APM Blog

In terms of load averages the three numbers represent averages over progressively longer periods of time (i.e. 1, 5, and 15-min. averages).

scoutapm.com

• The "Need to Look into it" Rule of Thumb: 0.70 If your load average is staying above > 0.70, it's time to investigate before things get worse.
• The "Fix this now" Rule of Thumb: 1.00. If your load average stays above 1.00, find the problem and fix it now. Otherwise, you're going to get woken up in the middle of the night, and it's not going to be fun.

.

 

[AYamshanov]

Oh, I see now.

I also have checked these numbers on my test server (where I have three WordPress sites for test purposes).

• Average CPU load - it is a CPU load between 0% and 100%. I see the same graphs from Watchdog and Advanced Monitoring extension. On my server, the Average CPU load near 2-3%.

• Load Average - it is a number (not a percent) that shows a load on a server. Looks like it can be obtained from Advanced Monitoring only. On my server, the Load Average near 0.0-0.1.​

ubuntu@panel:~$ w
04:53:39 up 33 days, 22:25,  1 user,  load average: 0.11, 0.15, 0.07

ubuntu@panel:~$ sudo plesk version
Product version: Plesk Obsidian 18.0.24.0
     OS version: Ubuntu 18.04 x86_64
     Build date: 2020/02/15 00:00
       Revision: f7e1d27512780b5609f45dcb5a501022c4d001f2

ubuntu@panel:~$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              1
On-line CPU(s) list: 0
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               63
Model name:          Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
Stepping:            2
CPU MHz:             2400.005
BogoMIPS:            4800.13
[...]

 

[TurnKeyWebsites]

Hi,
When I take a look at the graph for my CPU Load Average I am seeing daily spikes of close to 1 and you can even see one day it was over 2:


The CPU total usage seems to paint a different picture with peaks of 25%:


The rest of my numbers are similar to yours:
# w
09:02:17 up 5 days, 2:06, 1 user, load average: 0.07, 0.11, 0.05

# plesk version
Product version: Plesk Obsidian 18.0.23.3
OS version: Ubuntu 18.04 x86_64
Build date: 2020/01/30 12:00
Revision: 3f028be63f3d1a32011228166fb0039d29c56972

# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 1
On-line CPU(s) list: 0
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 60
Model name: Intel Core Processor (Haswell, no TSX)
Stepping: 1
CPU MHz: 2394.454
BogoMIPS: 4788.90


So I'm not clear if there is an actual problem with plesk or if this is a watchdog reporting problem:

1. Watchdog is reporting high cpu after moving to plesk 18 (Average CPU load: 1.3497%. vs Average CPU load: 0.3951%.).
2. I seeing daily spikes close to 1 of the cpu load average graph.

Something has definitely changed with this new version and I need to figure out what the problem is before migrating my production servers.
Thanks.

 

[AYamshanov]

1. Watchdog is reporting high cpu after moving to plesk 18 (Average CPU load: 1.3497%. vs Average CPU load: 0.3951%.).

1.3% is not a high CPU usage because it is a percentage of usage, it can be between 0% and 100%. If it close to 0%, it means a CPU almost in the idle stage. I see peaks at the same time, near 00:00, ~06:00 every day; it could be some Cron tasks, automated backups or something like this.

2. I seeing daily spikes close to 1 of the cpu load average graph.

Looks like they happened at the same time as CPU peaks.

Something happened periodically on the server. If you find what processes exactly start at this time, I sure it helps to figure out what to do next. Try to check crontab files, logs, backup settings.

 

[TurnKeyWebsites]

I had thought that the Average CPU Load was displaying the CPU Load as described above, if this is an actual percentage, then there is no issue.

I find it hard to believe that over the last several years all of my VPSes have never gone over 2-3 % average CPU usage. I suppose it's possible but if it's really a percentage I would have expected to see maybe even a 15-30% on at least one daily report, especially when I'm migrating multiple websites between servers. I wonder how accurate this metric is?

 

[AYamshanov]

I wonder how accurate this metric is?

I know how it works in the Advanced Monitoring extension: currently, their graphs are based on data from the collectd service. In general, they look similar to graphs from the watchdog.

Only one note: Advanced Monitoring shows numbers in milliseconds; watchdog calculates the usage to percentages. Unfortunately, I don't have the ability for fast comparing these numbers to each other.

 

[skopasis]

I know how it works in the Advanced Monitoring extension: currently, their graphs are based on data from the collectd service. In general, they look similar to graphs from the watchdog.

Only one note: Advanced Monitoring shows numbers in milliseconds; watchdog calculates the usage to percentages. Unfortunately, I don't have the ability for fast comparing these numbers to each other.

After switching to obsidan, there were jumps in the load on the processor and memory, the site became very slow

 

[AYamshanov]

Hi skopasis,

...there were jumps in the load on the processor and memory...

Could you please describe the screenshots a little bit more? Is there a moment when you upgrade Plesk Onyx to Plesk Obsidian or it is just the last two days? What time periods need to compare? The changes are noticeable on week graphs? Based on services' graphs, could you find what exact service has started to consume more? Did you some other changes (updating OS/services) at the same time, when Plesk was being updated? OS/Plesk version?

 

[skopasis]

Hi skopasis,



Could you please describe the screenshots a little bit more? Is there a moment when you upgrade Plesk Onyx to Plesk Obsidian or it is just the last two days? What time periods need to compare? The changes are noticeable on week graphs? Based on services' graphs, could you find what exact service has started to consume more? Did you some other changes (updating OS/services) at the same time, when Plesk was being updated? OS/Plesk version?

Hello. Apparently, I found the reason. The following requests occur from different ip addresses:

 

[skopasis]

Are they trying to hack our site? Or has it already been done?

 

[TurnKeyWebsites]

I know how it works in the Advanced Monitoring extension: currently, their graphs are based on data from the collectd service. In general, they look similar to graphs from the watchdog.

Only one note: Advanced Monitoring shows numbers in milliseconds; watchdog calculates the usage to percentages. Unfortunately, I don't have the ability for fast comparing these numbers to each other.

Thanks for your help. I guess I need to do some load testing and see these numbers change in order to gain some confidence in these numbers.

 

[AYamshanov]

skopasis,

It is hard to say based only on these two screenshots, I think you need to continue investigating the results of these repeated requests; as an example, it can be some kind of DDoS attack and you need to filter out such IP-addresses (I am not sure but maybe fail2ban can help here).

 

[skopasis]

skopasis,

It is hard to say based only on these two screenshots, I think you need to continue investigating the results of these repeated requests; as an example, it can be some kind of DDoS attack and you need to filter out such IP-addresses (I am not sure but maybe fail2ban can help here).

Thank you this is what you need