Resolved Scheduled task cause all websites to be unavailable for a few minutes

[orenk]

Server operating system version

CloudLinux 8

Plesk version and microupdate number

Plesk 18.0.73 #03

Hi.
There is probably a scheduled task, which I can't find, that cause all the wordpress websites to be unavailable around 5 minutes. (timeout or very slow response)
this happens every day between 4:00am and 4:30am (different time every day, but around this window time).
at that time, PLESK panel is very slow. the RAM and NETWORK shows normal levels, the CPU is around 20%-30% (the average is 15%-20% at noon time)

When I SSH to the server, and use "TOP" to see the active processes, I can see the PHP-FPM and LSPHP (cloud linux) are very high at this time. like 30% each

since it happens at the same time frame every day, I suppose it is a scheduled task. but I can't find which task it is.
but I can't find any task that starts around 3:30am-4:30am (except the clean-sysstats)
I looked for it in the following:
1. Plesk GUI, Scheduled Tasks
2. root # crontab -l
3. scheduled backup (which I moved to 14:00)

any other ideas where I can search scheduled tasks ?
plesk 18.0.73 + cloudlinux 8 (but it happens for a few months now, so the plesk version is less relevant

Thanks

 

[Raul A.]

Hi,

Cron task execution is logged in /var/log/cron. You will find there all the tasks that were executed.

How do you back up your data? Is it scheduled within this timeframe?

 

[Bitpalast]

It might not be caused by a real cron, but by one specific website. For example it could be a backup plugin of a specific website that is triggered by the Wordpress internal cron (which won't show up in the cron logs). To find out what the top transactions really are, run

MYSQL_PWD=`cat /etc/psa/.psa.shadow` watch "uptime && echo "\ " && ps aux | sort -nrk 3,3 | head -n 20 && echo "\ " && mysqladmin proc status -u admin"

and observe which website requests the CPU attention when the CPU load strongly increases.

 

[orenk]

Hi,

Cron task execution is logged in /var/log/cron. You will find there all the tasks that were executed.

How do you back up your data? Is it scheduled within this timeframe?

Thanks.
I viewed the file you mentioned, I have found countless CRONTAB, about 5-8 executing each minute. (I checked from 3:50 to 4:10)
so its kind of very hard to identify which cron tab causing it, unless I will execute each one of them... which I might do...
I have found many crons related to the "imunify" , and even though it is set to scan all the websites weekly on Sunday at 4am, maybe it runs daily...
So I tried to set it to run on a different hour, and I'll see if something change.

regard your question about the backup, I moved the backup time (from PLESK GUI) to noon time, and it did not solve the problem, so I suppose it is not related to the Plesk backup.
I also have an external image backup, which backs up the entire VPS drive, but that happens around 2am, and it backs up a shadow copy, so it should not affect the VM performance.

Thanks.

 

[orenk]

It might not be caused by a real cron, but by one specific website. For example it could be a backup plugin of a specific website that is triggered by the Wordpress internal cron (which won't show up in the cron logs). To find out what the top transactions really are, run

MYSQL_PWD=`cat /etc/psa/.psa.shadow` watch "uptime && echo "\ " && ps aux | sort -nrk 3,3 | head -n 20 && echo "\ " && mysqladmin proc status -u admin"

and observe which website requests the CPU attention when the CPU load strongly increases.

I will look into that, but thats exactly why I am using Cloud Linux, because of the DB Governor.
It happened to me previously, that single wordpress overloaded the DB, and since then I decided to move to cloudlinux.
I hope this is not the case right now.. but I will check it anyway. maybe I will find other clues..

 

[sGoico]

@orenk did you find what was causing it? I'm experiencing the exact same problem
WP sites will be randomly unavailable for just a few minutes near 4 am every other day. It's a bare metal server on OVH, just mention it in case it's a vendor specific configuration.
I am using pingdom for external monitoring of the sites.

 

[orenk]

@orenk did you find what was causing it? I'm experiencing the exact same problem
WP sites will be randomly unavailable for just a few minutes near 4 am every other day. It's a bare metal server on OVH, just mention it in case it's a vendor specific configuration.
I am using pingdom for external monitoring of the sites.

Hi.
No, I have not solve it yet. but I suspect the Imunify360, because I see two Imunify processes consuming 100% CPU about the same time the websites are becoming unavailable.
I am on it with cloudlinux support. I will let you know if this was the problem.

 

[orenk]

solved it.
the problem was that every day in that specific time, the DNS (9.9.9.9) (set in resolv) is becoming unavailable for about 30 minutes. so there is no DNS at this time.
we don't know why this happens but changing in the ./etc/resolv to 8.8.8.8 , and adding 2 more public DNS, solved the problem, the Imunify processes does not consuming high CPU anymore, and it looks like everything is stable now.