Resolved fail2ban filter apache-badbots "unknown"

[shopuser]

Server operating system version

Ubuntu 22.04.3 LTS

Plesk version and microupdate number

Plesk Obsidian v18.0.57_build1800231128.16

i try to block in fail2ban filter apache-badbots unknown boots but dosent work

badbotscustom = unknown|...

 

[Maarten]

I'm not sure what you're trying to do. Is "unknown" a word you've seen in the webserver logs?

 

[shopuser]

yes in the log, the agent:

2023-12-11 13:53:33 Error 45.128.232.56 400 POST /register/saveRegister/sTarget/account/sTargetAction/index HTTP/1.0 Unbekannt 18.8 K SSL/TLS-Zugriff für Apache

 

[Maarten]

Is that a line from /var/www/vhosts/domain.com/logs/proxy_access_ssl_log or from another source?

 

[shopuser]

/var/www/vhosts/system/*/logs/*access*log

 

[Peter Debik]

Does it really say "Unbekannt" in the log or does it say "unknown"? If the user-agent string is "Unbekannt", you must add Unbekannt as the custom bot name, too.

 

[shopuser]

Unbekannt ( in German)

 

[Kaspar]

yes in the log, the agent:

2023-12-11 13:53:33 Error 45.128.232.56 400 POST /register/saveRegister/sTarget/account/sTargetAction/index HTTP/1.0 Unbekannt 18.8 K SSL/TLS-Zugriff für Apache

I suspect that in some cases the user-agent data on a request is shown in the Plesk log viewer as unknown (or Unbekannt in German), while in the actual log the user-agent string is either absent or malformed. So the word unknown (or Unbekannt) isn't actually logged, but only shown as an indication that the user-agent data is unknown to Plesk. Which is probably the reason why the fail2ban filter cannot act on it.

If you want to be sure about the user-agent you'll have to take a look at the raw log. Which you can view from Plesk via the log viewer by clicking on the "Manage Log Files" button (I don't know what the German translation is). On the list with log files of the domain you can click on the icons on the right side to either view the raw log in a new browser window or download the raw log.

 

[shopuser]

yes this is the problem, in the logfile access_ssl_log.processed i can see only "-" There is a normal entry in the 4th line

192.144.39.104 - - [11/Dec/2023:00:45:07 +0100] "GET / HTTP/1.0" 301 5742 "" "-"
192.144.39.104 - - [11/Dec/2023:00:45:08 +0100] "GET / HTTP/1.0" 200 5489 "https://......de/" "-"
38.89.139.99 - - [10/Dec/2023:20:57:18 +0100] "GET / HTTP/1.1" 403 5752 "-" "-"
44.203.225.207 - - [11/Dec/2023:15:05:55 +0100] "GET / HTTP/1.1" 301 5695 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)

 

[JVG]

Hi,

I've seen in my server in the last few months a lot of seemingly malicious "-" "-" entries in the access logs also. I created a custom jail just for these entries, instead of trying to use the badbots jail. One caveat: it requires a really fine tuning and monitoring so to not block harmless normal accesses.

 

[Peter Debik]

I can already anticipate hackers naming their bad bots "unknown" to totally confuse users ...

 

[Peter Debik]

Hi,

I've seen in my server in the last few months a lot of seemingly malicious "-" "-" entries in the access logs also. I created a custom jail just for these entries, instead of trying to use the badbots jail. One caveat: it requires a really fine tuning and monitoring so to not block harmless normal accesses.

Actually, when no user-agent is transmitted and the source address is neither localhost, nor your own public IP, you could almost certainly block that traffic.

 

[JVG]

Actually, when no user-agent is transmitted and the source address is neither localhost, nor your own public IP, you could almost certainly block that traffic.

I'll keep that in mind, thanks!