Issue IP Address Banning fail2ban UTF-8 error

[nmiguel]

We are running a ‪CentOS 6.8 (Final)‬ VPS Server with Plesk 12.5.30 #48 and have a issue with IP Address Banning.

When it's on the logfile /var/log/fail2ban.log shows:
2016-09-27 18:44:01,060 fail2ban.filter [16304]: WARNING Error decoding line from '/var/log/maillog' with 'UTF-8'. Consider setting logencoding=utf-8 (or another appropriate encoding) for this jail. Continuing to process line ignoring invalid characters: 'Sep 26 06:28:38 xxxxx smtp_auth[6546]: FAILED: #001D\xc7... - password incorrect from (null) [xxx.xxx.xxx.xxx]\n'

and stops working (don't ban ip's).

I tried to add the:
logencoding = utf-8
(also try logencoding = auto and logencoding = us-ascii) to /etc/fail2ban/jail.conf and /etc/fail2ban/jail.local but without results.

(That is reported in here: https://talk.plesk.com/threads/fail2ban.338421/ )

Any ideias ?
Thanks for your help!!!

 

[IgorG]

What sort of output of following command:

# LC_ALL=UTF-8 fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/postfix-sasl.conf

?

 

[nmiguel]

Hi IgorG,

Thanks for your reply.

In addiction to the last post:
Installed mail server QMail
Installed IMAP/POP3 server Courier-IMAP

As asked:

LC_ALL=UTF-8 fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/postfix-sasl.conf

Running tests
=============

Use failregex filter file : postfix-sasl, basedir: /etc/fail2ban
Use log file : /var/log/maillog
Use encoding : ANSI_X3.4-1968

Results
=======

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [826517] (?AY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
`-

Lines: 826517 lines, 0 ignored, 0 matched, 826517 missed [processed in 82.47 sec]
Missed line(s): too many to print. Use --print-all-missed to print all 826517 lines

 

[IgorG]

Installed mail server QMail

In this case try to add

logencoding = utf-8

to /etc/fail2ban/filter.d/plesk-qmail.conf

and check result with

# fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/plesk-qmail.conf

 

[nmiguel]

Hi IgorG,

Done without results :-( ...

fail2ban-regex /var/log/maillog /etc/fail2ban/filter.d/plesk-qmail.conf

Running tests
=============

Use failregex filter file : plesk-qmail, basedir: /etc/fail2ban
Use log file : /var/log/maillog
Use encoding : UTF-8


Results
=======

Failregex: 45643 total
|- #) [# of hits] regular expression
| 1) [45643] ^(.*)smtp_auth(.*) FAILED: (.*) - password incorrect from (.*)\[<HOST>\]$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [842038] (?AY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
`-

Lines: 842038 lines, 0 ignored, 45643 matched, 796395 missed [processed in 367.16 sec]
Missed line(s): too many to print. Use --print-all-missed to print all 796395 lines

From /var/log/fail2ban.log :
2016-09-30 09:26:59,414 fail2ban.filter [21917]: WARNING Error decoding line from '/var/log/maillog' with 'UTF-8'. Consider setting logencoding=utf-8 (or another appropriate encoding) for this jail. Continuing to process line ignoring invalid characters: 'Sep 14 12:33:25 xxxxx smtp_auth[6523]: FAILED: d\xe9sir\xe9e - password incorrect from (null) [xxx.xxx.xxx.xxx]\n'

I don't know why appears diferente dates in the fail2ban.log line the date is today 2016-09-30 09:26:59 (correct in local time) but why appears Sep 14 12:33:25 ?!

 

[nmiguel]

Hi to all,

Any more ideias ?!

Thanks

 

[Ehud]

Hi,

Check the below link:

fail2ban-regex stops working on invalid (wrong encoded) character · Issue #1248 · fail2ban/fail2ban

I believe I found a bug in the regex of sshd jail (centos 6.7, fail2ban 0.9.3) The following user attack breaks the regex [root@main ~]# cat /var/log/secure | grep "llinco" Nov 8 00:16:12 main sshd...

github.com

At first they asked to change in:
cat /etc/fail2ban/jail.local
So this will be set:
logencoding = utf-8

I'm not sure it will solve the issue.