Dns recurcion problem or attack?

[Richieboydev]

Hello,

I just checked my fail2ban log and I am getting hundreds of these: The main ones I am worried about are the iptables near the bottom.

2014-09-07 03:36:27,127 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:36:35,135 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:36:41,142 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:36:47,148 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:36:54,156 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:36:59,161 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:04,167 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:09,172 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:13,177 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:18,183 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:22,188 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:26,193 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:31,198 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:35,203 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:41,210 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:46,216 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:50,221 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:37:55,227 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:38:01,234 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:38:07,241 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:38:11,246 fail2ban.actions[16003]: INFO [ssh] 50.7.229.214 already banned
2014-09-07 03:38:13,248 fail2ban.actions[16003]: WARNING [ssh] Unban 50.7.229.214
2014-09-07 03:38:13,255 fail2ban.actions.action[16003]: ERROR iptables -n -L INPUT | grep -q 'fail2ban-SSH[ \t]' returned 100
2014-09-07 03:38:13,255 fail2ban.actions.action[16003]: ERROR Invariant check failed. Trying to restore a sane environment
2014-09-07 03:38:13,263 fail2ban.actions.action[16003]: ERROR iptables -D INPUT -p tcp --dport ssh -j fail2ban-SSH

 

[UFHH01]

That sometimes happens, when there are too much script kiddies online, or botnet scripts, which try to do silly things.... nothing to worry about, because if you don't use passwords like 12345, which are very easy to guess, they just try to login but will never be able to login. This is why you should really make sure, that your passwords have a good strength and generally can't be guessed. You can set different levels for the password security in the "tools and settings" - page over Plesk.

To adjust fail2ban, please stop and restart fail2ban from time to time, because then you can be sure, that all created jails are working as they suppose to work. The iptables rules will then be renewed and current intruders will be banned immidiatly. Think of setting a jail like "recidive" as well, to make sure, that repeat offenders get longer bans.

Command line: service fail2ban restart ( or: /etc/init.d/fail2ban restart ; depending on your operating system. )​

You always have the possibilty to stop and restart jails over the Plesk Panel itself, but it takes longer, so my preferred way is always the direct way.
In my opinion it makes sense to create a daily cronjob for a fail2ban restart, because it is irritating to control the fail2ban log every day, just to see if there are failures and/or issues. This is why I restart fail2ban after the daily logrotate cronjob. - normally you could have a command inside the specific logrotate job at "/etc/logrotate.d/fail2ban" , but on Ubuntu the desired line doesn't work the way it should before the "endscript" - line, that's why I created the single cronjob to restart fail2ban at about 0:45 a.m. ^^

 

[Richieboydev]

That sometimes happens, when there are too much script kiddies online, or botnet scripts, which try to do silly things.... nothing to worry about, because if you don't use passwords like 12345, which are very easy to guess, they just try to login but will never be able to login. This is why you should really make sure, that your passwords have a good strength and generally can't be guessed. You can set different levels for the password security in the "tools and settings" - page over Plesk.

To adjust fail2ban, please stop and restart fail2ban from time to time, because then you can be sure, that all created jails are working as they suppose to work. The iptables rules will then be renewed and current intruders will be banned immidiatly. Think of setting a jail like "recidive" as well, to make sure, that repeat offenders get longer bans.

Command line: service fail2ban restart ( or: /etc/init.d/fail2ban restart ; depending on your operating system. )​

You always have the possibilty to stop and restart jails over the Plesk Panel itself, but it takes longer, so my preferred way is always the direct way.
In my opinion it makes sense to create a daily cronjob for a fail2ban restart, because it is irritating to control the fail2ban log every day, just to see if there are failures and/or issues. This is why I restart fail2ban after the daily logrotate cronjob. - normally you could have a command inside the specific logrotate job at "/etc/logrotate.d/fail2ban" , but on Ubuntu the desired line doesn't work the way it should before the "endscript" - line, that's why I created the single cronjob to restart fail2ban at about 0:45 a.m. ^^

Thanks so much!

I did make some minor changes today. I changed some of the bans to block all ports instead of specific. I also restarted fail2ban.. I added fail2ban in the first place because I have a high traffic site and am getting hit by these guys all day pretty much and every day. I was hoping by banning most of them I could save some resources.

I will change my current password which is "password" immediately.. lol just kidding.

My fail2ban is set up only partially through plesk. I make the edits in root or I get an error through Plesk.

Thanks again,
Rich

 

[Richieboydev]

fail2ban is filling up after a few days and my server is still getting hit with thousands of those probes.

When I restart fail2ban i always get the following. This is not right.

Starting fail2ban: WARNING 'ignoreregex' not defined in 'Definition'. Using default one: ''
WARNING 'ignoreregex' not defined in 'Definition'. Using default one: ''

 

[UFHH01]

Hello Richieboydev,

if you declare something to be "not right", you should first read the Fail2Ban documentation about it.

Fail2Ban has the option to declare a global string, which the filter search should ignore, if no exeption in a specific filter is defined. For example, you would like the word "honolulu" to be ignored, for what ever reason. If no exeption string in a specific filter is defined, Fail2Ban will now take the global exeption "honolulu".
Every filter should at least define the line "ignoreregex = ", even if there is no exeption, because otherwise the global exeption is used - this is just the way, how Fail2ban is coded.

But now the clue to all this... what if there is NO global exeption defined in a Fail2ban configuration? Well... nothing.... there just won't be any global exeptions at all - so NO string will be ignored in the filters, where no exeption line has being defined. Now that you know, what this "ignoreregex" - line in a filter stands for, do you still think, that this is "not right" ???

You could now say: But there is a big, fat WARNING, every time I restart Fail2ban. O.k.... and the answer is: Yes, this warning can be ignored by you... just read the documentation next time, please.

 

[Richieboydev]

yes, I did read all about this but I read different things about it every where i go. Coming here was my last alternative and I am very sorry if my post annoyed you. "Reading the documentation" is not always as easy as it sounds. I am fairly new to this script and am more used to working on the other side of the server. I came here because someone took the time to help me last time and I thought there would be a solution and I wanted to verify that this is working as it should/

My concern was not just this warning but the fact that I have to reboot this every few days with many thousands banned. Rebooting takes a few minutes and Fail2ban always ends up crashing during stop command. I assume it is timing out trying to remove thousands of banned ips. How many is ok to let build up before restarting the service? It appears to stop working after awhile which is why I end up rebooting. Is this normal?

 

[UFHH01]

You don't have to reboot at all, to restart Fail2ban - this for a first information. The command "service fail2ban restart" or "/etc/init.d/fail2ban restart" is fairly enough.

To investigate issues/problems with Fail2ban it is necessary to provide log entries, like you did in your first post, because the possibilities of what might not work as suggested are too much to guess.

If you have too much failures in your log, pointing to "sane environment" ( a few in a day could be ignored, because Fail2ban adjust such failures by restoring the configuration itself ), you could try to de- and reinstall Fail2ban, just be sure, that you don't have any strange configurations, which stop Fail2ban to work correctly. Either use the Plesk Panel ( https://YOURSERVERDOMAIN.COM:8443/admin/update/add-components/ ) to remove the extension and afterwards add it again, or use the command line for it:

/usr/local/psa/admin/bin/autoinstaller --select-product-id plesk --select-release-current --reinstall-patch --remove-component fail2ban
​

( I always include to the command "--reinstall-patch", just to be sure, that the Plesk version always has the latest patches provided by Plesk are installed as welll )

Even that Plesk is shipped with a pretty good basic configuration, please control your filters and jails. You probably don't want a global bantime of only 600 seconds - it is a very good idea to define own bantime and findtime definitions, if your server has a lot of bans during a day. After a few days / weeks you could adjust the jails again to the basic configuration.


A "normal" filter always defines exeptions, a you can see in this example ( filter: plesk-panel.conf )

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failures messages in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values:  TEXT
#

failregex = Failed login attempt with login '(.*)' from IP <HOST>$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#

ignoreregex =

Please correct filters you use, if they don't have such a "ignoreregex =" line in the filter, to avoid failure notices.

 

[Richieboydev]

Thanks, yes..I didn't mean reboot. i meant restarting the service. It takes for ever though and crashes during the stopping part. I then have to use the start command and it starts again with that warning.

I will follow your suggestions and will post logs if issues are not resolved. Thank you so much for your replies. I really, really appreciate it.