Question Help understanding Fail2Ban log entry

[David Jimenez]

The "normal" log entries are pretty easy to understand, they go something like this:

2017-04-15 16:33:53,497 fail2ban.filter [28873]: INFO [ssh] Found 218.0.241.177
2017-04-15 16:33:55,508 fail2ban.filter [28873]: INFO [ssh] Found 218.0.241.177
2017-04-15 16:33:57,517 fail2ban.filter [28873]: INFO [ssh] Found 218.0.241.177
2017-04-15 16:33:58,404 fail2ban.actions [28873]: NOTICE [ssh] Ban 218.0.241.177
2017-04-15 16:33:59,534 fail2ban.filter [28873]: INFO [ssh] Found 218.0.241.177
2017-04-15 16:33:59,828 fail2ban.filter [28873]: INFO [recidive] Found 218.0.241.177

However, there are a number of less obvious log entries dealing with UNBAN that I would like to ask if anyone can decipher for me:

2017-04-15 16:26:15,449 fail2ban.actions [28873]: NOTICE [ssh] Unban 67.245.245.69
2017-04-15 16:26:15,664 fail2ban.action [28873]: ERROR iptables -D f2b-SSH -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- stdout: ''
2017-04-15 16:26:15,665 fail2ban.action [28873]: ERROR iptables -D f2b-SSH -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- stderr: 'iptables: No chain/target/match by that name.\n'
2017-04-15 16:26:15,666 fail2ban.action [28873]: ERROR iptables -D f2b-SSH -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- returned 1
2017-04-15 16:26:15,666 fail2ban.actions [28873]: ERROR Failed to execute unban jail 'ssh' action 'iptables' info '{'matches': u'Apr 14 16:26:08 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2Apr 14 16:26:10 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2Apr 14 16:26:12 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2', 'ip': '67.245.245.69', 'time': 1492201575.093924, 'failures': 3}': Error unbanning 67.245.245.69

Thanks in advance for your help.

 

[Bitpalast]

The unban error means that there is no rule in iptables that is blocking (banning) what is being tried to unban. The issue in that case it not that fail2ban cannot unban the IP address, the question is rather why the IP address was not banned in the first place. You can check
# iptables --list | grep 67.245.245.69
whether the IP address is listed. From the type of entries you can normally derive the rule that was added (there is a slight chance that the removal command to iptables does not match the add command; that would really be a bad software bug, but not really a problem for the system). It is possible that the IP address is no listed in the output but rather the hostname that the IP address translates to. You can figure that out with
# host 67.245.245.69
(It delivers cpe-67-245-245-69.nyc.res.rr.com). Then run
# iptables -list | grep cpe-67-245-245-69.nyc.res.rr.com
to check whether that one is still blocked. I guess that neither IP, nor name are in the iptables list, so things should be alright.

 

[David Jimenez]

Maybe this adds more clarity on the BAN side:

2017-04-14 16:26:09,066 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:12,079 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:14,088 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:15,094 fail2ban.actions [28873]: NOTICE [ssh] Ban 67.245.245.69
2017-04-14 16:26:15,301 fail2ban.action [28873]: ERROR iptables -I f2b-SSH 1 -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- stdout: ''
2017-04-14 16:26:15,302 fail2ban.action [28873]: ERROR iptables -I f2b-SSH 1 -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- stderr: 'iptables: Memory allocation problem.\n'
2017-04-14 16:26:15,302 fail2ban.action [28873]: ERROR iptables -I f2b-SSH 1 -s 67.245.245.69 -j REJECT --reject-with icmp-port-unreachable -- returned 1
2017-04-14 16:26:15,302 fail2ban.actions [28873]: ERROR Failed to execute ban jail 'ssh' action 'iptables' info 'CallingMap({'ipjailmatches': <function <lambda> at 0x7f276ac4ba28>, 'matches': u'Apr 14 16:26:08 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2\nApr 14 16:26:10 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2\nApr 14 16:26:12 03f98ae sshd[9558]: Failed password for root from 67.245.245.69 port 37426 ssh2', 'ip': '67.245.245.69', 'ipmatches': <function <lambda> at 0x7f276ac4bc80>, 'ipfailures': <function <lambda> at 0x7f276ac4bb18>, 'time': 1492201575.093924, 'failures': 3, 'ipjailfailures': <function <lambda> at 0x7f276ac4bcf8>})': Error banning 67.245.245.69
2017-04-14 16:26:15,715 fail2ban.filter [28873]: INFO [recidive] Found 67.245.245.69
2017-04-14 16:26:16,106 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:19,115 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:21,123 fail2ban.filter [28873]: INFO [ssh] Found 67.245.245.69
2017-04-14 16:26:21,309 fail2ban.actions [28873]: NOTICE [ssh] 67.245.245.69 already banned

 

[Bitpalast]

Probably a RAM issue. Are there many websites on the server? Does it have enough memory to handle all? At least fail2ban had a problem when inserting a rule into iptables.

 

[David Jimenez]

Single web site, one subdomain, and a couple of forms using Machform using 3GB of disk space. Network Solutions VPS, but they only allocate 1GB of DRAM. Normally, the resource shows about 25% memory usage. What may be more of an issue is that they set numiptent to 128 and absolutely refuse to move it up.

 

[David Jimenez]

Here is another question. There is one IP address that is already in jail with a ban period of 1 week. Yet, while this IP address is in recidive it still pounds our server with thousands of hits that look like they are trying a phone book for usernames. I would have thought that once you are in jail, you can't even try to log in again until you are unbanned???

LOGIN FAILED, user=brain, ip=[::ffff:202.114.78.29]Apr 14 23:15:23 03f98ae courier-pop3d: LOGIN FAILED, user=brianna, ip=[::ffff:202.114.78.29]Apr 14 23:15:23 03f98ae courier-pop3d: LOGIN FAILED, user=briana, ip=[::ffff:202.114.78.29]Apr 14 23:15:24 03f98ae courier-pop3d: LOGIN FAILED, user=brid, ip=[::ffff:202.114.78.29]Apr 14 23:15:25 03f98ae courier-pop3d: LOGIN FAILED, user=bride, ip=[::ffff:202.114.78.29]Apr 14 23:15:26 03f98ae courier-pop3d: LOGIN FAILED, user=bridie, ip=[::ffff:202.114.78.29]Apr 14 23:15:27 03f98ae courier-pop3d: LOGIN FAILED, user=brien, ip=[::ffff:202.114.78.29]Apr 14 23:15:28 03f98ae courier-pop3d: LOGIN FAILED, user=briggs, ip=[::ffff:202.114.78.29]Apr 14 23:15:29 03f98ae courier-pop3d: LOGIN FAILED, user=brigid, ip=[::ffff:202.114.78.29]Apr 14 23:15:30 03f98ae courier-pop3d: LOGIN FAILED, user=brigida, ip=[::ffff:202.114.78.29]Apr 14 23:15:31 03f98ae courier-pop3d: LOGIN FAILED, user=brit, ip=[::ffff:202.114.78.29]Apr 14 23:15:31 03f98ae courier-pop3d: LOGIN FAILED, user=britney, ip=[::ffff:202.114.78.29]Apr 14 23:15:32 03f98ae courier-pop3d: LOGIN FAILED, user=britteny, ip=[::ffff:202.114.78.29]Apr 14 23:15:33 03f98ae courier-pop3d: LOGIN FAILED, user=bronwen, ip=[::ffff:202.114.78.29]Apr 14 23:15:34 03f98ae courier-pop3d: LOGIN FAILED, user=brook, ip=[::ffff:202.114.78.29]Apr 14 23:15:35 03f98ae courier-pop3d: LOGIN FAILED, user=bruce, ip=[::ffff:202.114.78.29]Apr 14 23:15:36 03f98ae courier-pop3d: LOGIN FAILED, user=brunero, ip=[::ffff:202.114.78.29]Apr 14 23:15:37 03f98ae courier-pop3d: LOGIN FAILED, user=bruno, ip=[::ffff:202.114.78.29]Apr 14 23:15:38 03f98ae courier-pop3d: LOGIN FAILED, user=bryan, ip=[::ffff:202.114.78.29]Apr 14 23:15:39 03f98ae courier-pop3d: LOGIN FAILED, user=bryce, ip=[::ffff:202.114.78.29]Apr 14 23:15:40 03f98ae courier-pop3d: LOGIN FAILED, user=bryon, ip=[::ffff:202.114.78.29]Apr 14 23:15:40 03f98ae courier-pop3d: LOGIN FAILED, user=bryn, ip=[::ffff:202.114.78.29]Apr 14 23:15:41 03f98ae courier-pop3d: LOGIN FAILED, user=bse, ip=[::ffff:202.114.78.29]Apr 14 23:15:42 03f98ae courier-pop3d: LOGIN FAILED, user=buck, ip=[::ffff:202.114.78.29]Apr 14 23:15:43 03f98ae courier-pop3d: LOGIN FAILED, user=buckminster, ip=[::ffff:202.114.78.29]Apr 14 23:15:44 03f98ae courier-pop3d: LOGIN FAILED, user=bue, ip=[::ffff:202.114.78.29]Apr 14 23:15:45 03f98ae courier-pop3d: LOGIN FAILED, user=buffy, ip=[::ffff:202.114.78.29]Apr 14 23:15:46 03f98ae courier-pop3d: LOGIN FAILED, user=building, ip=[::ffff:202.114.78.29]Apr 14 23:15:47 03f98ae courier-pop3d: LOGIN FAILED, user=bullard, ip=[::ffff:202.114.78.29]Apr 14 23

 

[Bitpalast]

These users are connecting to your host by IPv6, but the current, productive Fail2Ban only supports IPv4. It cannot detect or ban these login attempts. I fyou want to block them, you need to turn of IPv6 on your server in general and use IPv4 only. In the latest Fail2Ban beta IPv6 is supported, but it is beta, so might not perfectly work.

 

[Bitpalast]

but they only allocate 1GB of DRAM

1 GB of RAM is the minimum requirement for Onyx (500 MB RAM + 500 MB swap will do, too). It may be too low for error free operations.

 

[David Jimenez]

Thanks. Since most of the attacks are trying SSH, is setting "Access to the server over SSH" to "Forbidden" an alternative? We don't use SSH and could always turn it back on if needed.

 

[UFHH01]

Hi David Jimenez,

Since most of the attacks are trying SSH, is setting "Access to the server over SSH" to "Forbidden" an alternative?

No, sorry, it is not.

This is your ( current ) solution:

you need to turn of IPv6 on your server in general and use IPv4 only.

 

[David Jimenez]

Unfortunately, we have a number of customers that use IPv6, so shutting that down isn't an option.

Any advice on installing the Fail2Ban Beta?

 

[UFHH01]

Hi David Jimenez,

Any advice on installing the Fail2Ban Beta?

It is not guaranteed, that you might break the current compatibility with the Plesk Control Panel ( and it's provided standart jails ), if you install the beta. Pls. do it at your own risk and be aware, that it might be updated/upgraded/patched, when you install the beta in the very same folder as the Plesk Fail2Ban component.

If you still insist of using the Fail2Ban - beta, pls. keep an eye onto the changelog ( => Comparing 0.10.0a1...0.10 · fail2ban/fail2ban · GitHub ) and current issues ( => Issues · fail2ban/fail2ban · GitHub )

 

[danami]

Unfortunately, we have a number of customers that use IPv6, so shutting that down isn't an option.

Any advice on installing the Fail2Ban Beta?

Just a note that Juggernaut Firewall does support IPv6 out of the box if you don't want to wait for the fail2ban to get out of beta.