Issue plesk log grows out of control sometimes

[Linulex]

plesk 12.5.30, latest mu (47 atm)
centos 6.8

/var/log/sw-cp-server/sw-engine.log

Sometimes, for no reason the plesk webserver log starts to grow and grow at a rate of about 750 lines every minute until it takes up all available space on the server. These are the entry line from 1 minute.

The only solution i know is to monitor the size of this log and when it start growing, delete /var/log/sw-cp-server/sw-engine.log and restart sw-engine

But i don't think this is normal behavior. so my questions:

what is the reason?
how to avoid it?

is this a bug in plesk itself? if yes, please fix

an example of the lines


[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21617 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21614 exited with code 0 after 0.007705 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21618 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21615 exited with code 0 after 0.007693 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21619 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21617 exited with code 0 after 0.007108 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21621 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 21618 exited with code 0 after 0.007681 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22110 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22107 exited with code 0 after 0.007594 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22111 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22109 exited with code 0 after 0.007057 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22113 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22110 exited with code 0 after 0.007638 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22114 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22111 exited with code 0 after 0.007622 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22115 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22113 exited with code 0 after 0.007255 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22117 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22114 exited with code 0 after 0.007873 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22118 started
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22115 exited with code 0 after 0.007879 seconds from start
[22-Sep-2016 08:52:12] NOTICE: [pool plesk] child 22119 started

regards
Jan

 

[IgorG]

Maybe it is DDOS attack? Have you checked panel.log at the same time?
Also following information may be useful - http://serverfault.com/questions/535792/php-fpm-hundreds-of-seconds-in-the-log

 

[IgorG]

Tuning log_level in /etc/sw-engine/sw-engine-fpm.conf also could help with investigation.

 

[Linulex]

Maybe it is DDOS attack? Have you checked panel.log at the same time?
Also following information may be useful - http://serverfault.com/questions/535792/php-fpm-hundreds-of-seconds-in-the-log

This is not a ddos attack, this are the last lines from the pannel.log

[2016-09-06 15:40:14] ERR [panel] [Action Log] Failed login attempt with login 'boks' from IP 82.168.255.229
[2016-09-06 15:44:42] ERR [panel] [Action Log] Failed login attempt with login 'regio1.0' from IP 82.168.255.229
[2016-09-06 15:53:38] ERR [panel] [Action Log] Failed login attempt with login 'boks' from IP 82.168.255.229
[2016-09-06 15:53:56] ERR [panel] [Action Log] Failed login attempt with login 'inf@boks' from IP 82.168.255.229
[2016-09-09 17:58:26] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-09 17:58:26] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-09 17:58:28] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-09 17:58:28] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-09 17:58:30] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-09 17:58:30] ERR [panel] [Action Log] Failed login attempt with login 'admin' from IP 97.74.116.99
[2016-09-11 04:23:10] ERR [panel] Error: HTTP code: 503
[2016-09-18 23:28:46] ERR [panel] Kan de eigenschappen van het mailaccount niet bijwerken:<br />
This password is not strong enough according to the server security policy. Improve the password strength by using numbers, upper and lower-case characters, and special characters !,@,#,$,%,^,&,*,?,_,~<br><a href='http://kb.plesk.com/plesk-error/sea...+and+special+characters+!,@,#,$,%,^,&,*,?,_,~' target='_blank'>Search for related Knowledge Base articles</a>

Tuning log_level in /etc/sw-engine/sw-engine-fpm.conf also could help with investigation.

The problem is that it doesn't happen regularly, only now and then and not on the same server. The first time i saw it (and this is when i installed the size-monitor) was when i upgraded plesk 12.0.18 to 12.5.30.
since then i saw it twice on different servers, this was the second time.

regards
Jan

 

[IgorG]

This is not a ddos attack

Yes, it is bruteforce attack. As result - a lot of requests and FPM records in logs. Consider to use fail2ban for preventing this issue.

 

[Linulex]

What rule would intercept this?

We are using fail2ban, and the standard "plesk-panel" rule is active.

But im not sure how fail2ban would intercept this.
There are no ip adresses in sw-engine.log, and no entries about failed logins in panel.log.

regards
Jan

 

[Linulex]

Yes, it is bruteforce attack. As result - a lot of requests and FPM records in logs. Consider to use fail2ban for preventing this issue.

shouldn't the panel.log have entries about the failed logins? The last entries in there are from 9/9 (2 weeks ago), the entries in the sw-engine.log are from this morning.

and why does it stop after restarting sw-engine ? i never heard of a ddos attack that stopped by rebooting. If this was the solution to a ddos attack, life would be a lot easier.

regards
Jan

 

[IgorG]

The last entries in there are from 9/9 (2 weeks ago), the entries in the sw-engine.log are from this morning.

Hmm... maybe my English is not so good, but I wrote:

Have you checked panel.log at the same time?

We need to know what happens at this exact time, that could cause this issue. There are should be some kind of connections with the other logs.

 

[Linulex]

Hmm... maybe my English is not so good, but I wrote:


We need to know what happens at this exact time, that could cause this issue. There are should be some kind of connections with the other logs.

No, no, your english is fine.

These ARE the last lines from the panel.log. Nothing after this line in there after this one:

[2016-09-18 23:28:46] ERR [panel] Kan de eigenschappen van het mailaccount niet bijwerken:<br />
This password is not strong enough according to the server security policy. Improve the password strength by using numbers, upper and lower-case characters, and special characters !,@,#,$,%,^,&,*,?,_,~<br><a href='http://kb.plesk.com/plesk-error/sea...+and+special+characters+!,@,#,$,%,^,&,*,?,_,~' target='_blank'>Search for related Knowledge Base articles</a>

Not 2 weeks, (last failed attempt was from 2 weeks ago), but still, not today, not at the same moment the sw-engine.log started to grow this morning.

regards
Jan

 

[Linulex]

The panel.log is working. I have tried to update a mail password with a wrong password and now the 2 last entries in the log are

[2016-09-18 23:28:46] ERR [panel] Kan de eigenschappen van het mailaccount niet bijwerken:<br />
This password is not strong enough according to the server security policy. Improve the password strength by using numbers, upper and lower-case characters, and special characters !,@,#,$,%,^,&,*,?,_,~<br><a href='http://kb.plesk.com/plesk-error/sea...+and+special+characters+!,@,#,$,%,^,&,*,?,_,~' target='_blank'>Search for related Knowledge Base articles</a>
[2016-09-22 11:11:38] ERR [panel] Unable to update the mail account properties:<br />
This password is not strong enough according to the server security policy. Improve the password strength by using numbers, upper and lower-case characters, and special characters !,@,#,$,%,^,&,*,?,_,~<br><a href='http://kb.plesk.com/plesk-error/sea...+and+special+characters+!,@,#,$,%,^,&,*,?,_,~' target='_blank'>Search for related Knowledge Base articles</a>

regatds
Jan