Issue /Server/full - /usr/local/psa/handlers/spool/

[Erwan]

Server operating system version

Centos 6.9

Plesk version and microupdate number

18

Hi all,

This morning we found a server with root 100% used disk space.

# df -lh
Filesystem Size Used Avail Use% Mounted on
/dev/root 20G 20G 0 100% /
devtmpfs 16G 280K 16G 1% /dev
/dev/md4 1,8T 535G 1,2T 31% /var
tmpfs 16G 0 16G 0% /dev/shm
/dev/root 20G 20G 0 100% /var/named/chroot/etc/named
/dev/md4 1,8T 535G 1,2T 31% /var/named/chroot/var/named
/dev/root 20G 20G 0 100% /var/named/chroot/etc/named.rfc1912.zones
/dev/root 20G 20G 0 100% /var/named/chroot/etc/rndc.key
/dev/root 20G 20G 0 100% /var/named/chroot/usr/lib64/bind
/dev/root 20G 20G 0 100% /var/named/chroot/etc/named.iscdlv.key
/dev/root 20G 20G 0 100% /var/named/chroot/etc/named.root.key
/dev/root 20G 20G 0 100% /var/named/chroot/etc/services
/dev/root 20G 20G 0 100% /var/named/chroot/etc/protocols

The problem seems to come from an email in /usr/local/psa/handlers/spool/ (16Go).

-rw------- 1 popuser popuser 16G 18 nov. 02:43 /usr/local/psa/handlers/spool/messagexXi82w

It was removed but no change in /dev/root, still 100% used.
If I search for large files:

# find / -type f -size +1000000k -printf "%p %s\n"
find: "/proc/13367/task/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)
find: "/proc/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)

I don't know why this proc pops up. I suspect him to be the one who created the previously deleted email.

Do you have any idea how to find where the problem comes from? Thank you!

Erwan

 

[Erwan]

# find / -type f -size +1000000k -printf "%p %s\n"
find: "/proc/13367/task/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)
find: "/proc/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)

There are no other "big" files than the one deleted from 16G:
-rw------- 1 popuser popuser 16G 18 nov. 02:43 /usr/local/psa/handlers/spool/messagexXi82w

..as the file had not been deleted.
Is there a service to reload to refresh?

 

[Erwan]

Ok. I found deleted process to kill...

# lsof +L1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
dbus-daem 1231 dbus txt REG 9,2 343176 0 954736 /bin/dbus-daemon (deleted)
sendmail 6306 root 3r REG 9,2 16385150976 0 897812 /usr/local/psa/handlers/spool/messagexXi82w (deleted)
qmail-inj 6307 root 3r REG 9,2 16385150976 0 897812 /usr/local/psa/handlers/spool/messagexXi82w (deleted)
qmail-que 6308 root 3r REG 9,2 16385150976 0 897812 /usr/local/psa/handlers/spool/messagexXi82w (deleted)

Deleting the file /usr/local/psa/handlers/spool/messagexXi82w is not enough.

 

[mow]

# find / -type f -size +1000000k -printf "%p %s\n"
find: "/proc/13367/task/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)
find: "/proc/13367/fdinfo/5": Aucun fichier ou dossier de ce type (No such file or folder)

I don't know why this proc pops up. I suspect him to be the one who created the previously deleted email.

Use find options "-mount" or "-xdev" to stay in the same filesystem - you only want to look for big files in the maxed-out filesystem anyway. You get those errors when there is a race between the find and the printf where the process is already done when it gets to the printf (and therefor its file representation under /proc gone).