Issue Monitoring shows "An unexpected error occurred"

[Azurel]

Server operating system version

AlmaLinux 8.10 (Cerulean Leopard)

Plesk version and microupdate number

Plesk Obsidian 18.0.68

Since today I get in Monitoring "An unexpected error occurred".


Because all requests shows a 500 error


/var/log/grafana/grafana.log shows many lines with this only for today and only for my request in plesk

logger=authn.service t=2025-03-11T08:41:25.668405458+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"

Is here a way to fix this? Since I only access Grafana via Monitoring, I am surprised by the message.

 

[alapshin]

Hello @Azurel I suggest to apply the solution from the article Advanced monitoring extension error: {"message":"Invalid username or password"}

1. Restart Grafana in Tools & Settings > Services Management
2. Reload Grafana in Extensions > My Extensions > Grafana > Reload Status

If these steps don't work, reinstall Grafana and Monitoring: How to manage Plesk extensions
If this still not working contact Plesk support for thorough investigation How to submit a request to Plesk support?

 

[Azurel]

Grml....
Thanks, unfortunately "Reload Status" not fix this issue.
Reinstall Monitoring and Grafana result in this and click on "restart integration" shows a new error (right top)


Now the log shows lines like this

logger=alerting.evalContext t=2025-03-11T09:33:31.049884821+01:00 level=error msg="Alert Rule Result Error" ruleId=13 name="Apache & PHP-FPM memory usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc00329f200}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:33.044701953+01:00 level=error msg="Alert Rule Result Error" ruleId=14 name="nginx memory usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc00341ffb0}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:36.050985898+01:00 level=error msg="Alert Rule Result Error" ruleId=16 name="MySQL memory usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc0038106f0}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:36.054228731+01:00 level=error msg="Alert Rule Result Error" ruleId=21 name="Swap usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc003382930}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:36.057729061+01:00 level=error msg="Alert Rule Result Error" ruleId=19 name="Partition \"/boot\" utilization" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc003383fb0}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:36.057866411+01:00 level=error msg="Alert Rule Result Error" ruleId=15 name="Mail server memory usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc003fe6060}" changingstateto=keep_state
logger=alerting.evalContext t=2025-03-11T09:33:36.065577484+01:00 level=error msg="Alert Rule Result Error" ruleId=17 name="Plesk memory usage" error="request handler response error {invalid status code. status: 403 Forbidden A <nil> [] [] 0xc003fe7ef0}" changingstateto=keep_state
logger=authn.service t=2025-03-11T09:33:37.466261128+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"
logger=authn.service t=2025-03-11T09:33:37.466667898+01:00 level=warn msg="Failed to authenticate request" client=auth.client.session error="user token not found"
logger=authn.service t=2025-03-11T09:33:37.51096388+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"

 

[Azurel]

“Restart integration” button now suddenly works again out nowhere. And here we are, back at the initial error, allegedly too many login attempts and blocked.

logger=authn.service t=2025-03-11T11:10:27.038897323+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"

P.S. For a short time talk.plesk.com redirected to webpros.com.

 

[Azurel]

I restarted grafana and see this now in log:

logger=plugin.grafana-simple-json-backend-datasource t=2025-03-11T11:15:54.444211034+01:00 level=warn msg="plugin failed to exit gracefully"
logger=plugin.plesk-json-backend-datasource t=2025-03-11T11:15:54.445362281+01:00 level=error msg="plugin process exited" path=/var/lib/grafana/plugins/plesk-json-backend-datasource/dist/gpx_plesk-json-backend-datasource_linux_amd64 pid=1402611 error="signal: terminated"
logger=sqlstore t=2025-03-11T11:15:54.701284323+01:00 level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r-----
logger=plugin.signature.validator t=2025-03-11T11:15:54.778046568+01:00 level=warn msg="Permitting unsigned plugin. This is not recommended" pluginID=grafana-simple-json-backend-datasource
logger=plugin.signature.validator t=2025-03-11T11:15:54.778089239+01:00 level=warn msg="Permitting unsigned plugin. This is not recommended" pluginID=plesk-json-backend-datasource
logger=plugin.loader t=2025-03-11T11:15:54.781553179+01:00 level=error msg="Could not start plugin" pluginId=grafana-simple-json-backend-datasource err="Incompatible API version with plugin. Plugin version: 1, Client versions: [2]"
logger=plugin.grafana-simple-json-backend-datasource t=2025-03-11T11:15:54.782133597+01:00 level=error msg="plugin process exited" path=/var/lib/grafana/plugins/grafana-simple-json-backend-datasource/dist/simple-json-plugin_linux_amd64 pid=1420231 error="signal: killed"
logger=provisioning.alerting t=2025-03-11T11:15:56.438407075+01:00 level=error msg="can't read alerting provisioning files from directory" path=/etc/grafana/provisioning/alerting error="open /etc/grafana/provisioning/alerting: no such file or directory"
logger=authn.service t=2025-03-11T11:17:16.776397659+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[password-auth.failed] failed to authenticate identity: [password-auth.invalid] invalid password"
logger=authn.service t=2025-03-11T11:17:17.374805874+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"

Everything was going great until yesterday. Why now "invalid password"? I do not set a password.

 

[Azurel]

I uninstalled and reinstalled it again via shell:

# plesk bin extension --uninstall grafana
# plesk bin extension --uninstall monitoring
# yum remove grafana && rm -rf /usr/share/grafana /etc/grafana /var/lib/grafana
# plesk bin extension --install grafana
# plesk bin extension --install monitoring

Still not working with error in log

logger=authn.service t=2025-03-12T12:41:09.111067855+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[password-auth.failed] failed to authenticate identity: [password-auth.invalid] invalid password"
logger=authn.service t=2025-03-12T12:41:09.122164571+01:00 level=warn msg="Failed to authenticate request" client=auth.client.basic error="[login-attempt.blocked] too many consecutive incorrect login attempts for user - login for user temporarily blocked"

Where do you set the password for grafana? Can I correct it somewhere?

 

[Azurel]

As of today, it just works again. The only change I can see was the automatic update to 18.0.68 Update #1.