Resolved PHP struggling under heavy load - "failed to prepare the stderr pipe: Too many open files (24)"

[Rob6566]

We've noticed that PHP repeatedly gives this error when our server is under heavy load. We've experienced this during peak live traffic, and also load testing on a clone of the server.

In /var/log/plesk-php56-fpm/error.log:
[07-May-2020 05:45:20] ERROR: failed to prepare the stderr pipe: Too many open files (24)
[07-May-2020 05:45:21] ERROR: failed to prepare the stderr pipe: Too many open files (24)
[07-May-2020 05:45:21] ERROR: failed to prepare the stderr pipe: Too many open files (24)
[07-May-2020 05:45:21] ERROR: failed to prepare the stderr pipe: Too many open files (24)
[07-May-2020 05:45:21] ERROR: failed to prepare the stderr pipe: Too many open files (24)
[07-May-2020 05:45:21] ERROR: failed to prepare the stderr pipe: Too many open files (24)

When we see these errors, we get a lot of 504 Gateway Timeouts.

We can see that PHP processes have a file limit of 1024, which seems too low (we've increased it for MySQL/Apache/Nginx, but can't figure out how to increase it for PHP):

In /proc/{php_process}/limits:
Max open files 1024 4096 files

We can't figure out how to increase this.

# ulimit -n
65535

In /etc/security/limits.conf:
* soft nofile 65535
* hard nofile 65535

In /proc/sys/fs/file-max:
250000

In /etc/sysctl.conf:
fs.file-max = 250000

We've restarted the server after making those changes.

Software on the server:
- Plesk 12.5
- Apache 2.4.6
- nginx 1.11.1
- PHP 5.6
- Centos 6
- MySQL (MariaDB) 5.5.60


Any thoughts on increasing PHP's max open files?

 

[IgorG]

failed to prepare the stderr pipe: Too many open files (24)

This usually means that Open file descriptor rlimit is exceeded.

To fix the issue create a directory and additional configuration file:

# mkdir /etc/systemd/system/php-fpm.d
# vi /etc/systemd/system/php-fpm.d/override.conf

Add the following in the config file:

[Service]
LimitNOFILE=65536

And restart systemd daemon and the service:

# systemctl daemon-reload
# systemctl restart php-fpm.service

 

[Rob6566]

Thanks for that fast reply Igor.

I've tried that, but it didn't work:

# systemctl restart php-fpm.service
Job for php-fpm.service failed because the control process exited with error code. See "systemctl status php-fpm.service" and "journalctl -xe" for details.

# systemctl status php-fpm.service
php-fpm.service - The PHP FastCGI Process Manager
   Loaded: loaded (/usr/lib/systemd/system/php-fpm.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2020-05-07 06:57:22 UTC; 41s ago
  Process: 5688 ExecStart=/usr/sbin/php-fpm --nodaemonize (code=exited, status=78)
 Main PID: 5688 (code=exited, status=78)
May 07 06:57:22 [snip] systemd[1]: Starting The PHP FastCGI Process Manager...
May 07 06:57:22 [snip] php-fpm[5688]: [07-May-2020 06:57:22] ERROR: No pool defined. at least one pool section must be specified in config file
May 07 06:57:22 [snip] php-fpm[5688]: [07-May-2020 06:57:22] ERROR: failed to post process the configuration
May 07 06:57:22 [snip] php-fpm[5688]: [07-May-2020 06:57:22] ERROR: FPM initialization failed
May 07 06:57:22 [snip] systemd[1]: php-fpm.service: main process exited, code=exited, status=78/n/a
May 07 06:57:22 [snip] systemd[1]: Failed to start The PHP FastCGI Process Manager.
May 07 06:57:22 [snip] systemd[1]: Unit php-fpm.service entered failed state.
May 07 06:57:22 [snip] systemd[1]: php-fpm.service failed.

However, if I try to load a PHP page on this server, it works.

I restarted the server after making that change.

When I try looking at /proc/{php_process_id}/limits, I see that max open files is still 1024.
Max open files 1024



We're on PHP 5.6.40 on "FPM Application Served By Apache", using Plesk 12.5.

 

[Rob6566]

I've tried something else that seems to have worked:
In /opt/plesk/php/5.6/etc/php-fpm.conf, I added:
rlimit_files = 65536
Then restarted PHP

 

[IgorG]

Yes, my solution was for OS vendor's PHP.
But you have Plesk PHP 5.x (in fact bad idea to use EOLed version)

 

[Rob6566]

Cheers. Yeah, we're working on upgrading to the latest Plesk/PHP.