pmm-ras won't stop until it's killed

[derde0]

Hi there,

i am using Plesk 12 (last updates are installed) with centOS 6.5 (last updates installed) - and the process of backup the whole server has an annoying custom after upgrading from Plesk 11.5.30 to 12.
The pmm-ras won't stop (and customizes 100 % of a cpu-kernel) until i kill the task manually on console (kill <pid>). Alll logs are looking perfect, the backup job seems almost finished (100 %), but the backup-file itself is llieing on the server repository and is not uploaded to the personal repository (ftp-server for backups). After killing pmm-ras the backup file will be uploaded to the personal repository - so i have no clue what's going wrong with pmm-ras.

So the question is: What does pmm-ras do? And how can i debug this problem? Does anybody has the same or similar problem?

Cio derde

 

[IgorG]

pmm-ras (plesk-migration-manager repository-access-service) is responsible for organization of backup repository and all transactions with repository.
For debugging you should change FORCE_DEBUG_LOG from 0 to 1in /usr/local/psa/admin/share/pmmcli/pmmcli-rc
Then repeat backup procedure check log like /var/log/plesk/PMM/backup-2014-10-08-15-33-17-700/backup.log for any pmm-ras mentions.

 

[Pascal_Netenvie]

Hi Igor,
I have the same problem on one of my servers and in "pmmcli.log" i get this every minute :

[2015-12-09 16:16:21.594|43842] DEBUG: --> <pmmcli.GetTaskStatusAction object at 0x1d266d0>
[2015-12-09 16:16:21.595|43842] INFO: parameters: ['540']
[2015-12-09 16:16:21.601|43842] DEBUG: <pmmcli.ActionRunner object at 0x1d26710>: doActivity
[2015-12-09 16:16:21.601|43842] DEBUG: Acquired session mutex: MainThread
[2015-12-09 16:16:21.630|43842] DEBUG: Get task (540) status from migration.result...
[2015-12-09 16:16:21.630|43842] DEBUG: Session path (/opt/psa/PMM/sessions/2015-12-09-032902.428)
[2015-12-09 16:16:21.677|43842] DEBUG: Get task (540) status from migration.result...
[2015-12-09 16:16:21.677|43842] DEBUG: Session path (/opt/psa/PMM/sessions/2015-12-09-032902.428)
[2015-12-09 16:16:21.679|43842] DEBUG: Released session mutex: MainThread
[2015-12-09 16:16:21.679|43842] DEBUG: <pmmcli.GetTaskStatusAction object at 0x1d266d0>: response
[2015-12-09 16:16:21.683|43842] INFO: Outgoing packet:
<?xml version="1.0" ?><response>
    <errcode>0</errcode>
    <data>
        <task-status task-id="540">
            <finished log-location="/opt/psa/PMM/sessions/2015-12-09-032902.428/migration.result" status="error">
</finished>
        </task-status>
    </data>
</response>

How can if ix this ?

 

[Pascal_Netenvie]

I looked at the backup log and it seems it come from a connection problem to FTP backup server.
Ok but why pmm-ras continue to take so much processor time ? Does it retry to connect every minute ?
How to fix that or change this retry delay ?

 

[Pascal_Netenvie]

Hi,
This problem is still there and very annoying.
It take all processor time.
Please help ...

 

[Pascal_Netenvie]

The only error in log is that one :
ERROR: () Can't upload file 'domains/xxxxxx.com/backup_xxxxx.com_info_1512220329.xml' to ftp. Error code: 143

So ok FTP upload fail but why pmm-ras stay active and take so much processor ??

 

[Pascal_Netenvie]

Please need help on that point.

 

[trialotto]

@Pascal_Netenvie

Please be sure that you kill the pmm-ras related processes. If you did not do that, those processes can continue endlessly.

Furthermore, retry to make a backup, this time in the server repository: if that fails, you do not have a FTP related issue, but some issue on the server-side occurs.

If the local backup doesn´t fail, we can then rule out a number of potential causes and proceed with analysis of a a small subset of causes (i.e. culprits for the issue on hand).

Also, be more specific and provide information (logs, etc. etc.)

Regards....

 

[Pascal_Netenvie]

Hi,
The problem come from FTP.
But i have 10 web servers under plesk backuping to 2 backups servers and only 2 are failing.

On the 2 that are failing only 1st one get pmm-ras still running after fail.

The 2nd one return these errors :
Unable to rotate dump: The dump rotation is failed with code '126' at /opt/psa/admin/bin/plesk_agent_manager line 1038.
OR
Can't upload file 'domains/xxxx.fr/backup__xxx.fr_info_1512200418.xml' to ftp. Error code: 1

For the 1st (one with pmm-ras still runing) i kill process manually each time.
I've done what IgorG said about log but when i look log there is no log for last backup (the one that fail).

 

[trialotto]

@Pascal_Netenvie

Please be more specific.

For instance:

a) is the backup failure associated with one remote FTP server, or not?

b) what are the configs of the troublesome remote FTP server(s)? (i.e. can be one or both of them)

c) what is the output of the command "df -h" and/or is your tmpfs large enough?

d) does cleaning up the /var/lib/psa/dumps help?

and so on.

In short, we can only help if we are able to isolate the issue to the core of the underlying cause: you can help by providing adquate and sufficient information.

Regards....

 

[Pascal_Netenvie]

Hi,
Thx for reply.

a) Backup failure is associated to a remote FTP server.

b) What do you mean ? I don't understand you on this point ...

c) Output is :
Sys. fich. Taille Util. Dispo Uti% Monté sur
rootfs 8,9G 3,7G 4,8G 44% /
udev 10M 0 10M 0% /dev
tmpfs 800M 776K 800M 1% /run
/dev/sda3 8,9G 3,7G 4,8G 44% /
tmpfs 5,0M 0 5,0M 0% /run/lock
tmpfs 1,8G 12K 1,8G 1% /run/shm
/dev/sda1 89M 20M 64M 24% /boot
/dev/sda4 40G 17G 22G 44% /var

d) Done. I will tell you more tomorrow.

Anyway even if Backup fail (for any reason) :
Why this process stay active and why it take so much CPU ?
How can we prevent that ?

Regards.

 

[Pascal_Netenvie]

Hi,

B) if i understand well you want to know FTP config ?
I tried with or without passive and with or without sFTP.
It change nothing.
In all cases backup archive is password protected.

D) changed nothing.
Problem still there.

THe server with problem have only one site with bit large data (around 1G).

Only with one server. All others backuping to the same remote FTP are OK : I mean sometimes there is problems (connection, dump rotate or others) but pmm-ras dont stay active.

 

[Pascal_Netenvie]

Is there a command i can call with a cron each morning around 9 o clock to kill pmm-ras task ?
Just the time we find a solution to this problem.
This process drain too much server ressource ...

 

[Pascal_Netenvie]

For now i execute this cron task to be sure :
55 8 * * * pkill -f 'pmm-ras'

As backup is launched at 0.30 it let more than 8 hours for backup to complete.

 

[Squeeze999]

I have the same problem.
I am backing up 2 Plesk 12.5 servers to the same ftp server. One runs fine, the other keeps 2 pmm-ras processes running while uploading the file(s). Is that normal?
There is no change, whether I split backups or not. The ftp upload seems to complete, but the processes stay.