Constant "Alarm level changed" messages

[Pauli Jokela]

Hey,

I've been getting constant emails from Plesk with the topic "alarm level changed.".

This started when I upgraded to Plesk 10 and I've tried various hardware configurations (running inside a VPS) to see if increasing the RAM might fix it, but the problem still persists.

The site and services don't seem slow, load's low and everything seems fine, it just seems like Plesk is reading apaches memory usage a bit too well.

On a daily basis, I might get 4-5 emails where the alarm level has gone from green -> yellow -> red and then back to green.

Here's an entire alarm email, just for reference (had to snip a bit to make it fit):

Server: <mydomain>.

Server health parameter "Services > Apache memory usage" changed its status from "yellow" to "red".

top - 15:12:16 up 4 days, 21 min, 0 users, load average: 0.01, 0.08, 0.08
Tasks: 157 total, 1 running, 156 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.4%us, 0.1%sy, 0.0%ni, 98.9%id, 0.6%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4194480k total, 4077800k used, 116680k free, 191300k buffers
Swap: 1081336k total, 516k used, 1080820k free, 3031536k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 15 0 2176 680 584 S 0.0 0.0 0:00.01 init
1632 haldaemo 18 0 5464 3488 1652 S 0.0 0.1 0:00.06 hald
1633 root 18 0 3280 1084 940 S 0.0 0.0 0:00.00 hald-runner
1655 root 21 0 2024 464 380 S 0.0 0.0 0:00.00 hidd
1686 root 18 0 4640 1268 1112 S 0.0 0.0 0:00.42 xe-daemon
1784 root 21 0 27384 1392 1080 S 0.0 0.0 0:00.00 automount
1799 snort 15 0 56536 44m 4764 S 0.0 1.1 0:01.81 snort
1804 root 15 0 2968 624 536 S 0.0 0.0 0:00.00 snortsam
1822 sw-cp-se 15 0 9776 6156 1496 S 0.0 0.1 0:00.40 sw-cp-serverd
1845 root 21 0 3884 884 612 S 0.0 0.0 0:00.01 sshd
1860 root 15 0 2852 872 696 S 0.0 0.0 0:00.00 xinetd
1892 root 18 0 4640 1308 1128 S 0.0 0.0 0:00.00 mysqld_safe
1980 mysql 18 0 140m 31m 7068 S 0.0 0.8 0:55.16 mysqld
2062 qmails 15 0 1812 428 336 S 0.0 0.0 0:00.00 qmail-send
2064 qmaill 18 0 1768 500 428 S 0.0 0.0 0:00.00 splogger
2065 root 18 0 1796 392 300 S 0.0 0.0 0:00.00 qmail-lspawn
2066 qmailr 15 0 1792 400 308 S 0.0 0.0 0:00.00 qmail-rspawn
2067 qmailq 18 0 1764 360 296 S 0.0 0.0 0:00.00 qmail-clean
2086 root 15 0 34288 28m 2404 S 0.0 0.7 0:00.62 spamd
2090 popuser 15 0 35224 29m 2160 S 0.0 0.7 0:00.19 spamd
2091 popuser 15 0 34288 27m 892 S 0.0 0.7 0:00.00 spamd
2100 root 18 0 2008 476 400 S 0.0 0.0 0:00.00 gpm
2180 named 18 0 72032 4988 2064 S 0.0 0.1 0:00.59 named
2254 postgres 18 0 21240 3356 2856 S 0.0 0.1 0:00.15 postmaster
2256 postgres 18 0 11016 876 376 S 0.0 0.0 0:00.00 postmaster
2258 postgres 15 0 21240 1084 580 S 0.0 0.0 0:00.01 postmaster
2259 postgres 18 0 12020 788 284 S 0.0 0.0 0:00.00 postmaster
2260 postgres 20 0 11200 988 380 S 0.0 0.0 0:00.00 postmaster
2448 root 18 0 44712 15m 4464 S 0.0 0.4 0:08.64 sw-engine
2456 root 18 0 71924 2432 856 S 0.0 0.1 27:44.40 sw-collectd
2537 mailman 18 0 13224 5132 960 S 0.0 0.1 0:00.00 mailmanctl
2549 root 18 0 169m 44m 4252 S 0.0 1.1 0:09.00 dropbox
2559 mailman 15 0 13124 6960 2788 S 0.0 0.2 0:00.09 python
2560 mailman 15 0 13092 6988 2788 S 0.0 0.2 0:00.10 python
2561 mailman 15 0 13068 6972 2788 S 0.0 0.2 0:00.08 python
2562 mailman 15 0 13128 6964 2788 S 0.0 0.2 0:00.08 python
2563 mailman 15 0 13112 7008 2788 S 0.0 0.2 0:00.08 python
2564 mailman 15 0 13092 7040 2788 S 0.0 0.2 0:00.11 python
2565 mailman 15 0 13128 6960 2788 S 0.0 0.2 0:00.09 python
2566 mailman 18 0 13128 6964 2788 S 0.0 0.2 0:00.07 python
2568 root 15 0 3620 444 292 S 0.0 0.0 0:00.00 smartd
2571 root 15 0 1772 512 452 S 0.0 0.0 0:00.00 agetty
2573 root 34 19 26092 10m 2144 S 0.0 0.2 0:00.03 yum-updatesd
2578 root 34 19 2676 1208 1000 S 0.0 0.0 0:00.00 gam_server
3574 root 15 0 5056 1924 1556 S 0.0 0.0 0:00.00 couriertls
3575 popuser 15 0 4148 964 768 S 0.0 0.0 0:00.00 imapd
12696 root 18 0 42476 20m 8628 S 0.0 0.5 0:00.19 httpd
12698 apache 18 0 31144 9048 576 S 0.0 0.2 0:00.00 httpd
12699 apache 18 0 62480 35m 7728 S 0.0 0.9 0:03.82 httpd
12700 apache 15 0 62688 35m 7660 S 0.0 0.9 0:04.91 httpd
12701 apache 17 0 58248 31m 7644 S 0.0 0.8 0:04.66 httpd
12702 apache 15 0 63744 36m 7896 S 0.0 0.9 0:04.57 httpd
12703 apache 15 0 63492 36m 7708 S 0.0 0.9 0:06.22 httpd
12704 apache 15 0 112m 86m 7760 S 0.0 2.1 0:05.22 httpd
12705 apache 15 0 65076 37m 7584 S 0.0 0.9 0:04.77 httpd
12706 apache 15 0 61984 34m 7668 S 0.0 0.8 0:04.41 httpd
12772 apache 15 0 62868 35m 7816 S 0.0 0.9 0:03.63 httpd
13295 apache 15 0 62684 35m 7788 S 0.0 0.9 0:03.71 httpd
14138 apache 15 0 63928 36m 7768 S 0.0 0.9 0:03.71 httpd
15523 root 15 0 0 0 0 S 0.0 0.0 0:00.40 pdflush
16566 eerie 16 0 48116 34m 7060 S 0.0 0.8 0:10.39 php-cgi
16995 paulijok 18 0 35616 21m 6472 S 0.0 0.5 0:00.34 php-cgi
19139 apache 15 0 62244 35m 7624 S 0.0 0.9 0:02.99 httpd
23611 eerie 16 0 48648 34m 6988 S 0.0 0.8 0:03.29 php-cgi
23616 eerie 16 0 47428 33m 7076 S 0.0 0.8 0:10.71 php-cgi
24802 root 22 0 0 0 0 S 0.0 0.0 0:00.03 pdflush
29527 didstopi 22 0 10352 2292 1424 S 0.0 0.1 0:00.00 crond
29532 didstopi 24 0 2512 992 872 S 0.0 0.0 0:00.00 sh
29533 didstopi 18 0 28772 14m 6136 S 0.0 0.4 0:00.09 php

 

[KirkM]

I have been getting that too since upgrade to 10.1. Would love to get a tip for fixing this.

 

[Pauli Jokela]

I've got a solution for this.

Go to "Health Monitoring" and click on "Download Configuration File", which will let you download an XML file.
Open it up in an editor of your choice and scroll down to where it says:

<service name="Web" monitor="true">
<alarm level="red" threshold="25" type="percent"/>
<alarm level="yellow" threshold="20" type="percent"/>
</service>

And modify accordingly. I set the thresholds like this:

<service name="Web" monitor="true">
<alarm level="red" threshold="75" type="percent"/>
<alarm level="yellow" threshold="50" type="percent"/>
</service>

Then just save the file and back at "Health Monitoring" click "Upload Configuration File".

So far the alarm-spam has stopped.

 

[KirkM]

Thanks for that. I tried it and it cut the warnings down to about 1 per day. So I went in and did 90/80 for the limits. I don't want to hear from the panel unless things are truly critical.

I think this should work out fine.

 

[GFEMajor]

i hvae the same Problem. I changed my config to:

<Services>
<CpuTime>
<service name="Web" monitor="false">
</service>
...

and

<MemoryUsage>
<service name="Web" monitor="false">
</service>
...

but i still got the emails with changed levels.

i also tryed it with your suggestion:

<service name="Web" monitor="true">
<alarm level="red" threshold="75" type="percent"/>
<alarm level="yellow" threshold="50" type="percent"/>
</service>

but with the same result.

Do you have any hints for me? Do i have to restart the Plesk Services to make the changes take effect?

Greets Major

 

[Pauli Jokela]

Well, just to clarify, if it's Apache memory notifications, make sure you're inside the MemoryUsage tag, not the CPU one, so:

<MemoryUsage>
<service name="Web" monitor="true">
<alarm level="red" threshold="60" type="percent"/>
<alarm level="yellow" threshold="75" type="percent"/>
</service>

After that, you shouldn't need to restart anything (though you could restarts psa, psa-health-monitor-notificationd and sw-collected if you felt like it).

 

[GFEMajor]

The Problem is solved since i updated my plesk version. But Thank you for your help

 

[iainh]

I've got a solution for this.

Thank you for a very nice solution. Down loading and examining my monitoring alert file I see the issue; alarms for Apache, Mail, MySQL and Plesk itself are all set to 20% but of course this doesn't reflect actual usage. So on my VPS (2Gb) for instance:

Service Mb memory Percentage
Apache 400.0 21.9%
Mail 18.7 1.0%
MySQL 50.6 2.8%
Plesk 9.4 0.5%
Total memory used 26.2%

So you can see settings of 20% for each service simply doesn't reflect the usage of the differing services, meanwhile I keep getting usually green to yeellow and back to green warnings about Apache, yet everything is running in RAM with no usage of swap and so things are hardly memory constrained.

The thing to do seems not just to increase the alarm level for Apache but reduce the levels for others as if Mail (Courier and Postfix) climbed from 1 to 20% something very odd would be going on! What's more with the levels redistributed the alarms will still kick-in before things go bad as otherwise your alarms could exceed 100% and so in theiry not go off even though the box was flat out. So my thinking is to try:

Service Yellow Red
Apache 35.0% 50.0%
Mail 5.0% 10.0%
MySQL 10.0% 15.0%
Plesk 5.0% 10.0%
Total 55.0% 85.0%