Resolved php-fpm memory leak?

[Franco]

Hello,
this may be connected to or a consequence of this other thread (Issue - php7.2 php-fpm.sock file error), but I create a new one as this one seems a more specific issue.
I went back to php 7.1 and even 7.0 on some websites. However, despite this, the free swap memory keeps going down and after 1-2 hours mysql exits with failure code and I have to reboot the system. Even in presence of relatively low traffic. I have 4GB RAM and about 20 active websites running php-fpm with nginx. It was extremely stable and performing till yesterday when I have upgrade to latest MU #33 and added php7.2 (but no longer used).

When I look at the memory used (top -o %MEM) I am puzzled to see plenty of sleeping processes under those users which were upgraded to 7.2 and then downgraded back to 7.1 or 7.0. The others only have one process or none at all.

For instance:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12167 mysql 20 0 828732 252696 12476 S 0.0 6.5 0:10.48 mysqld
11480 soge.co+ 20 0 610380 45004 6664 S 0.0 1.2 0:00.57 php-fpm
11491 soge.co+ 20 0 686436 44904 7404 S 0.0 1.2 0:00.62 php-fpm
11489 soge.co+ 20 0 685864 44788 7032 S 0.0 1.2 0:00.47 php-fpm
11485 soge.co+ 20 0 610380 44524 6672 S 0.0 1.1 0:00.61 php-fpm
11490 soge.co+ 20 0 610380 44520 6668 S 0.0 1.1 0:00.61 php-fpm
11498 soge.co+ 20 0 610380 44520 6668 S 0.0 1.1 0:00.58 php-fpm
11482 soge.co+ 20 0 610380 43684 6680 S 0.0 1.1 0:00.56 php-fpm
11503 soge.co+ 20 0 610588 43668 6656 S 0.0 1.1 0:00.55 php-fpm
11504 soge.co+ 20 0 610588 43668 6656 S 0.0 1.1 0:00.53 php-fpm
11481 soge.co+ 20 0 610380 43664 6660 S 0.0 1.1 0:00.56 php-fpm
11499 soge.co+ 20 0 610380 43660 6660 S 0.0 1.1 0:00.57 php-fpm
11502 soge.co+ 20 0 610588 43660 6620 S 0.0 1.1 0:00.56 php-fpm
11483 soge.co+ 20 0 610380 43656 6644 S 0.0 1.1 0:00.57 php-fpm
11484 soge.co+ 20 0 607816 42400 6380 S 0.0 1.1 0:00.47 php-fpm
11479 soge.co+ 20 0 606916 41084 6572 S 0.0 1.1 0:00.51 php-fpm
11497 soge.co+ 20 0 606916 41068 6588 S 0.0 1.1 0:00.51 php-fpm
11500 soge.co+ 20 0 606396 41024 6484 S 0.0 1.1 0:00.53 php-fpm
11501 soge.co+ 20 0 606396 41024 6484 S 0.0 1.1 0:00.49 php-fpm
11477 soge.co+ 20 0 682364 40236 7256 S 0.0 1.0 0:00.54 php-fpm

Each of those takes 1.1% memory and does not release it; it even grows to 1.2+. I tried to killall -u [user] for those users, but they come back and stay there. Those website and very low traffic, btw.
The result is progressive memory depletion and mysql failing.

How to clean this mess? Any idea much appreciated

Regards

 

[moswak]

maybe this bug
After the installation of micro-update 32 on Plesk Onyx 17.5 server memory usage has increased

 

[Franco]

Bingo!
Thanks a lot, setting it on demand solved my huge problem and I am an happy man again

Which means that php7.2 is not the culprit, but choosing it APPLIED the php-fpm default settings that screw up my system. What a story...

 

[Niek_Beernink]

We noticed this too and are setting the default back to ondmand for servers that have memory problems. It would be very interesting to learn why the default setting was changed from ondmand to static...

 

[mr-wolf]

Which means that php7.2 is not the culprit, but choosing it APPLIED the php-fpm default settings that screw up my system. What a story...

Only reading the posts in this thread I don't understand why you come to this conclusion. PHP is having a memory leak according to you. Whatever settings are applied that should never happen.
Unfortunate of course that Plesk chooses a default that isn't working properly in practice.

I too have difficulties since the introduction of these new features. I was hoping that setting it to "ondemand" would fix it, but alas, it does not

I have a WordPress website that crashes whenever I use its menus. The funny part is that it only does this in desktop view (in mobile view it works fine). Restoring the site to a 2 months older version didn't change a thing.
It turned out to be PHP and it started on the 29th of November.


I can fix it by choosing any PHP other than 7.1.2 and 7.0.26. Which works fine is PHP 7.0.15 by OS Vendor.....

Question - PHP general protection errors in syslog since Nov 29th

 

[Franco]

Well, I have a number of cases like those. It mostly depends on theme or plugin incompatibility. I wish I could run all my sites on the latest php version, but it is unlikely. Usually (but not always) you get an indication of where it bangs in the subscription log. However, as php is interpreted you never know when and where it could crash. Sometimes the "wrong" code is in a branch rarely executed.
At the moment I have that funny behaviour with LayerSlider, on some sites and with some php versions the slider is just not displaying, in others is distorted, but no indication whatsoever, go figure...

 

[inado1]

It is now 2020, and I had the same increase in memory issue with PHP FPM operated by Nginx.
The memory didn't stop growing until my server crashed. This was due to the absence of garbage collection. This means that even if the processes are IDLE they still take space in the server memory. The memory of several PHP-FPM processes grow indefinitely until the server didn't respond anymore and I had to reboot the server.
I have found that the problem was related to the DYNAMIC mode in the PHP settings. I moved my settings to ONDEMAND and it looks that it made it stable. By the way, I thought I would lost performance by switching to ondemande and I haven't noticed anything.

 

[Franco]

Since I set it "ondemand" no more problems. I also set the maxchildren to 49. Sometimes, for some reason I ignore, those maxchildren are randomly set back to the default 5. That way I have a little script that finds 5 and sets it to 49

 

[simmons]

this is exactly what is happening to me all day.
The PHP FPM memory goes up untill the server crashes and i need to reboot.
Some minutes ago i started to use top in ssh to see the processes and seems that all the blame is for php-fpm. I've kille them all and i managed to use the server without restarting.
But i know this is not the solution.
Too bad my setting are already what you suggest. ONDEMAND.
I don't know what to do anymore.
I have a VPS with 2 vCPU and 2 GB of Ram.
Maybe is not enough for my sites?

 

[Franco]

2GB is definitely not enough, most of that will be taken by the OS and very little is left to your programs/websites. Of course, it depends on your config, number of apps, traffic, etc., but you should have at least 4GB (which will triple your net capacity), better 8GB. In the past I was running with 4GB, but after a couple of years of adding stuff I had to upgrade, but with VPSes you can't do that and you have to start a new one (and migrate). I suggest you foresee some growth and be large. After all, the cost of a VPS these days is not a factor in our business.

 

[simmons]

i came from a shader hosting, where the support was a mess and i reach the point of hating them for many reason. I switched to a VPS to be in control of the system and hoping to have more performances. But seems i'm having problems i've never had in a shared hosting. How could be this even possible?
I hope is some sort of misconfiguration by my side.

I've only 4 sites on it and only one is a little bigger. I started indeed to have problem when i migrated it in the system.

 

[Franco]

Well, I think you took the right decision, but it takes some time and experience to get a VPS running smooth. I suggest you pay an expert a few hours and after that you'll find your way and be happy. It's not only tuning and performance, but also security, such as running fail2ban, rkhunter, SFTP with keys, etc.

 

[simmons]

yes but right now my priority is to be able to use the sites without the need of rebooting the system every 2 hours.
From the top command seems it's the php-fpm (multiple processes) that overload the system. I need to understand how to block them and what is causing this.
Do you have any suggestion?

 

[Franco]

Not sure, but you may try and reduce maxchildren. Anyway, I would avoid wasting time and add memory asap. If that does not solve your problem then it will be worth investigating further.

 

[inado1]

@Franco
I didn't check but I think that there is a missing directive in the plesk php-fpm configuration.


add in the php settings of your domain or the service plan the following:

[php-fpm-pool-settings]
pm.process_idle_timeout = 5s;

I have spotted php-fpm processes that were idle and never gave back their memory.
So they stay in the memory like this, idle, and the memory grows.

 

[Franco]

Hi Inado1,
the documentation says that parameter is used in the ondemand case. As my system works using ondemand and I observe no memory issue I assume the timeout is defined (10s?) and works. There is (was) a problem for the dynamic case instead, I guess. I won't try again

Regards