Issue Server down: user_beancounters - physpages problem?

[Andreas2022]

Server operating system version

AlmaLinux 8.9

Plesk version and microupdate number

Plesk Obsidian Version 18.0.58

Hi everybody,

I have a wordpress website on a VPS with 6 cores, 14 GB RAM and 600 GB SSD hard drive space. A script runs in the background, that reads Wikipedia pages throughout the day and writes excerpts of the texts into the Wordpress database. Monitoring shows me that I'm actually barely utilizing my resources. But unfortunately, my website is often down, especially recently. When I check monitoring, there are peaks in CPU and services at that time.

My provider said, my tariff is too small and sent me the attached screenshot as proof. Unfortunately I don't know what the values mean. Since I'm not a server expert, I'd like to know what's causing the problem with "physpages" in general so I can isolate the cause of the error. I don't want to book an upgrade to 8 cores and 20 GB RAM recommended by my provider without knowing whether that makes sense.

Generally asked: could it be that the server-down-problem is not my fault at all, but is caused by another customer on the server?

Thank you very much in advance for your help!
Andreas

 

[trialotto]

Hi everybody,

I have a wordpress website on a VPS with 6 cores, 14 GB RAM and 600 GB SSD hard drive space. A script runs in the background, that reads Wikipedia pages throughout the day and writes excerpts of the texts into the Wordpress database. Monitoring shows me that I'm actually barely utilizing my resources. But unfortunately, my website is often down, especially recently. When I check monitoring, there are peaks in CPU and services at that time.

My provider said, my tariff is too small and sent me the attached screenshot as proof. Unfortunately I don't know what the values mean. Since I'm not a server expert, I'd like to know what's causing the problem with "physpages" in general so I can isolate the cause of the error. I don't want to book an upgrade to 8 cores and 20 GB RAM recommended by my provider without knowing whether that makes sense.

Generally asked: could it be that the server-down-problem is not my fault at all, but is caused by another customer on the server?

Thank you very much in advance for your help!
Andreas

@Andreas2022

Your script is probably causing the overload.

I would suggest that you

1 - reboot the server (and immediately check user_beancounters afterwards .... it should not have failcnts)

2 - monitor the server before the script is run : if user_beancounters shows failcnts, then it is very likely to be another process causing issues

3 - have a thorough look at the script anyway ....... it cannot do harm to search for code that is clogging the server.

Please note that upgrading the VPS does not make any sense at all, if you do not know what causes the issues ..... they will return, if you do not solve issues!

Kind regards....

PS Also be aware of the fact that any WP database related action can and will cause issues with MySQL database, if and as far as the MySQL config is not fine-tuned to the action that has to be done. Default mysql config will be fine for standard actions, but writing huge amounts of data at once in a WP dbase is very likely to clog the server. Try to temporarily store the data on the disk and feed the data in chunks to the MySQL server (if that is needed at all).

 

[Peter Debik]

Are you using Fail2Ban to block bad bots? It will help.

How to Avoid High CPU Load & Block Bad Bots with Plesk

Learn how to effectively reduce CPU usage and keep bad bots and hackers away from your websites using Plesk Cgroups Manager and Fail2Ban.

www.plesk.com

 

[Andreas2022]

Thank you very much for your quick replies!

@trialotto I will test as you suggested. It's true, MySQL database queries are a difficult topic. And I know, that automatically scraping content from external websites can cause very quick performance problems. I'm using curl for that, but I think, there's not much to optimize. I think the thousands of queries are the problem. But will an tariff upgrade solve this problem?

@Peter Debik Yes, I already installed Fail2Ban and have a few IP addresses in my black list. The article is very interesting. How do you handle Fail2Ban? Are there public black lists that one can download and use?

 

[Peter Debik]

Fail2Ban analyzes your log files and bans activities that are offensive against the server health.

 

[trialotto]

Thank you very much for your quick replies!

@trialotto I will test as you suggested. It's true, MySQL database queries are a difficult topic. And I know, that automatically scraping content from external websites can cause very quick performance problems. I'm using curl for that, but I think, there's not much to optimize. I think the thousands of queries are the problem. But will an tariff upgrade solve this problem?

@Peter Debik Yes, I already installed Fail2Ban and have a few IP addresses in my black list. The article is very interesting. How do you handle Fail2Ban? Are there public black lists that one can download and use?

@Andreas2022

Even though Fail2Ban may impose some performance issues, it is not very likely that your server is hindered by Fail2Ban.

Curl is barely relevant in this equation.

The equation consists of a web crawl (piece of cake for the system, even though the external site might block you or slow the requests down), augmented by the MySql workload that arises from (importing) all of the results of the web crawl.

In general, the default Plesk MySql config is sufficient to run multiple sites, but not sufficient to take care of mysql workloads that are different by nature.

Since you use WP, there is already a considerable overload on the MySql database - WP is essentially a bit messy by nature.

If you actually do import the results from the web crawl into the WP databases, then a high overload of the MySql server is highly likely.

Since you use WP (read: a database based website), any WP based website will become considerably slower when the script (the web crawl) is run.

You can check that easily, without any complex monitoring of logs.

If I were you, I would make some changes to the mysql config and have a look from there, in the hope that the issue will be mitigated to some extent.

If the issue is still there (after mysql config optimization), then you are certain that the root cause of the problem lies elsewhere.

Just follow a step-by-step method, just eliminate the potential causes in order to get to the root cause of the problem.

And with any WP instance, the first logical step should simply be to have a look at /var/log/modsec_audit.log (read: are you under attack? if yes, then reduce the number of incoming requests) and then finetune mysql config!

Please note that the above is one-and-the-same-step : it is mysql config tuning, but in such a fashion that you do not facilitate malicious requests by altering and relaxing mysql config - that would be terrible, right?

I hope the above helps a bit!

Kind regards....

PS Please make very small steps when assigning a bit more resources to MySql server - be very careful, since any small change in mysql config can cause a considerable impact on server performance : resource outage is always possible.

PS2 How much memory did you assign to the WP instance that is being used to import the results from the web crawl?