pmmcli-daemon running constantly

[Pagemakers]

For some reason pmmcli-daemon has started to run constantly at 90-100% CPU even when it's not backing up.

No error message emails are generated.

I can stop it, but then I find it running after it has completed the domain backups.

Any ideas?

 

[IgorG]

It is difficult to say something without detailed investigation of /usr/local/psa/admin/PMM/logs/*log*
Try to find something useful and related there.

 

[Pagemakers]

Here is the pmmcli-daemon log..

[2014-05-23 05:15:19] Action PmmcliLogRotateAction started
[2014-05-23 05:15:19] Action param: file_name:migration.log
[2014-05-23 05:15:19] rename log file to /usr/local/psa/PMM/logs/migration.log.2014-05-23-051519
[2014-05-23 05:15:19] Action PmmcliLogRotateAction started
[2014-05-23 05:15:19] Action param: file_name:migration_handler.log
[2014-05-23 05:15:19] daemon job delayed for 30 minutes
[2014-05-23 05:45:19] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] MailToAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] MailToAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added
[2014-05-23 05:45:20] UnsuspendTaskAction added

 

[IgorG]

Check value of DAEMON_TIMEOUT in /usr/local/psa/admin/share/pmmcli/pmmcli-rc

 

[Pagemakers]

# pmmcli daemon timeout (in minutes)
DAEMON_TIMEOUT 30

 

[Pagemakers]

It is possible to manually run the process per domain, then maybe I can see which domain it is hanging on?

 

[Pagemakers]

I have checked open resources of pmacli daemon and found

/var/PMM/sessions/2014-05-17-050630.544/migration.result

last two lines

/bin/tar: paresh/Maildir/.Deleted Messages/tmp/1400276172.M86229P11272_2524.xx-server.com,S=1825700: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/tmp/1400276371.M263P11996_64.xx-server.com,S=29325: Cannot stat: No such file or directory
/bin/tar: Error exit delayed from previous errors

 

[Pagemakers]

Each morning I have to manually cancel the process with killall pmmcli_daemon

When the process runs it is far more CPU intensive than before and I have no idea why.

 

[Pagemakers]

Once again I am having to manually kill the pmmcli_daemon process because the backups have finished yet it’s still running at 100% CPU.

I received 3 error logs last night. One being 39MB in size containing hundreds of thousands of lines like below.

The files is relates to were hundreds of thousands of emails in the paresh/Maildir/.Deleted Messages folder that were manually removed. Why is the backup still trying to locate these non-existant files? Is there a way to re-set the backup?


/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M294177P8802V0000000000000807I000000001D5504FD_1018.xxxx-server.com,S=8816\:2,S: File removed before we read it
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M459520P8802V0000000000000807I000000001D550502_1019.xxxx-server.com,S=301045\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M460174P8802V0000000000000807I000000001D550508_1020.xxxx-server.com,S=1451861\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M463217P8802V0000000000000807I000000001D55050C_1021.xxxx-server.com,S=72183\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M463440P8802V0000000000000807I000000001D55050E_1022.xxxx-server.com,S=679440\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M464456P8802V0000000000000807I000000001D550512_1023.xxxx-server.com,S=13086\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M464739P8802V0000000000000807I000000001D550517_1024.xxxx-server.com,S=394\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M464857P8802V0000000000000807I000000001D550519_1025.xxxx-server.com,S=14498\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M465122P8802V0000000000000807I000000001D55051D_1026.xxxx-server.com,S=11522\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M465364P8802V0000000000000807I000000001D550521_1027.xxxx-server.com,S=14407\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M465606P8802V0000000000000807I000000001D550525_1028.xxxx-server.com,S=11840\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M465843P8802V0000000000000807I000000001D550529_1029.xxxx-server.com,S=11466\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M466089P8802V0000000000000807I000000001D55052D_1030.xxxx-server.com,S=51046\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M466382P8802V0000000000000807I000000001D550535_1031.xxxx-server.com,S=14411\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M466624P8802V0000000000000807I000000001D55053A_1032. xxxx-server.com,S=21773\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M467026P8802V0000000000000807I000000001D55053E_1033.xxxx-server.com,S=24296\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M467278P8802V0000000000000807I000000001D550542_1034.xxxx-server.com,S=6869\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M467509P8802V0000000000000807I000000001D550546_1035.xxxx-server.com,S=134108\:2,S: Cannot stat: No such file or directory
/bin/tar: paresh/Maildir/.Deleted Messages/cur/1400253491.M467906P8802V0000000000000807I000000001D55054B_1036.xxxx-server.com,S=21848\:2,S: Cannot stat: No such file or directory

 

[IgorG]

Why is the backup still trying to locate these non-existant files? Is there a way to re-set the backup?

Supporters can't help you in scope of created ticket?

 

[Pagemakers]

Sorry igor. What do you mean?

I meant to send that last message by pm

 

[Dmitriy S]

Check the number of scheduled backups:

# plesk db
mysql> select d.id as 'Domain ID',d.name as 'Domain name',c.cname as 'Client Name',bs.obj_type as 'Backup type',bs.repository as 'Destination',bs.last as 'Last time',bs.active as 'Status',bs.backup_time as 'Time',bs.content_type as 'Type',bs.period 'Period'from domains d INNER JOIN clients c ON d.cl_id=c.id INNER JOIN BackupsScheduled bs ON d.id=bs.obj_id WHERE bs.active='true';

Performing several backup tasks at the same time could be the root cause of the high load.

 

[Pagemakers]

The problem was not that the server load was high, it was because the backup process pmmcli daemon never stopped even after the backup finished. The server has been backing up these domains at these times for years without a problem. The issue has only started a few days ago.

That said, Parallels support have investigated and the last 2 evening’s backups have been fine.

Fingers crossed it’s fixed.

 

[Pagemakers]

Server has been fine for a week.

Your help has been invaluable. Brilliant support. Thank you.