Resolved Enhance firewall to block services for certain countries

[mr-wolf]

@mr-wolf and @OverWolf

Nice to read that you are busy with implementing Plesk Firewall extension and Fail2Ban, which can be a good combination.

However, there are some common "good practices" when it concerns Plesk Firewall and Fail2Ban together.

Amongst others:

- Fail2Ban is rather resource hungry: make sure that your Fail2Ban config is proper in the sense that the performance footprint is minimal
- Fail2Ban can allow some hack attempts to go undetected: keep track of your log files to establish whether Fail2Ban misses out on specific hack attempts
- Fail2Ban customization and Fail2Ban usage with Nginx can make the system very strong in terms of security
- Plesk Firewall can be used in combination with Fail2Ban, but there are some pitfalls, which I will explain below

I am focussing on the last point, since the usage of Plesk Firewall extension and Fail2Ban can have some unexpected consequences.

In the light of the above, the following has to be mentioned:

a) Plesk Firewall extension is essentially a GUI for iptables: having Plesk Firewall and Fail2Ban both altering and creatig iptable firewall rules can result in a huge firewall set, which can result in a (relatively) huge performance penalty, even in the case where ipset is being used, (and)

b) Plesk Firewall extension is a GUI that is not showing all iptabels rulesets: as a result, it can be the case that

- manually created firewall rule sets AND rules sets created with Plesk Firewall extension AND firewall rulesets created by Fail2Ban will cause a (lot of) double firewall rules,
- Fail2Ban has to "work harder" if the Plesk Firewall extension is not used to it's full potential and/or to the full extent,

and the overall result is often a significant performance penalty, (and)

c) firewall rulesets have to be persistent (across reboots) and

- Plesk Firewall extension uses a script that results in persistence
- Fail2Ban uses python code to rescan logs at reboot, resulting in persistence
- manual iptables entries are NOT persistent by default

and the overall result is that one has to be careful and take into account that one really has to have persistent iptables rulesets,

and all of the above is non-exhaustive summary of points that have to be taken into consideration.

However, a number of general rules of thumb can be introduced to make life more easy:

1 - use Fail2Ban to detect and temporarily block bad IPs,
2 - use Fail2Ban to detect recurring attempts from bad IPs AND add these bad IPs via the Plesk Firewall extension,
3 - do not OR try to prevent usage of manual iptables entries.

All of the above will result in

- a more efficient Fail2Ban process: most of the recurring attempts from bad IPs are blocked by (persistent!) firewall rules, Fail2Ban does not have to "work hard"
- the probability of double firewall rulesets is decreased significantly
- the chance that important iptables rulesets are non-persistent is almost non-existing

Finally, note that one can use Nginx or ModSec to make the whole system less vulnerable to hack attempts and/or make the entire system more efficient.

Hope the above helps a bit.....

Regards.........

I don't think you have seen my 2 contributions which have nothing to do with fail2ban nor the Plesk firewall....
I'm just helping overwolf to apply my script ipset-country.
Please check them out first and then comment all you like...

 

[trialotto]

@mr-wolf,

The thing is that I present some basics, as an explanation for the logic behind a proper structure and setup that can ban entire CIDR blocks for specific country regions.

This logic is a "must know", since it does not make any sense to block country regions entirely.

Moreover, for every goal there is not a right tool, there are only good (but not perfect) solutions consisting of multiple tools.

Any firewall ipset-country based solution can be easily passed with simple solutions: just reroute traffic over VPSes in other countries that are not blocked at all.

A firewall is not so picky about the traffic origin, it only cares about the origin IP, the destination port and so on.

For instance, only using a firewall ipset-country based solution will give a sysadmin false confidence that a system is secure.......

........ and, in essence, the before mentioned solution is not very good at stopping MTM attacks or other types of attacks that are using some kind of sequential string of VPS or servers to obscure the actual source from which the traffic originates.

As a good example, note the following fact.

Most sysadmins really want to block the annoying CIDR ranges that China uses...........

........... and there might be some truth behind that idea, but most Chinese hackers simply use Russian VPSes or even servers in other countries.

As another good example, consider something else.

Most malicious traffic is deemed to come from specific countries like China or Russia............

............ but actually, it often comes from the unexpected countries like the United States or Great Britain.

And, really, I still have to find a sysadmin that is blocking those countries with some kind of geo-ip based firewall ruleset.


In short, there is much more than meets the eye and a simple solution will never suffice..........

........and for the interested: just have a look at actual real-time occurrences of threats!

A good example of a "threat map" can be found on: Atomicorp Threat Intelligence or http://www.atomicrbl.com/globe/

Note that you will have to let the sites with the above mentioned URLs run for a while, in order to get a good indication (i.e. the site fills with data gradually).

Regards........

 

[mr-wolf]

You don't tell me anything new
But as I'm monitoring all connections to my system I want to keep a "clear view" by blocking and not logging connections from countries that are clutering this same view so I can concentrate on the rest.

 

[OverWolf]

1 - use Fail2Ban to detect and temporarily block bad IPs,
2 - use Fail2Ban to detect recurring attempts from bad IPs AND add these bad IPs via the Plesk Firewall extension,
3 - do not OR try to prevent usage of manual iptables entries.

Hi trialotto,
do you talk about firewall-cmd (package: firewalld) to use with f2b ?

 

[mr-wolf]

@mr-wolf,

The thing is that I present some basics, as an explanation for the logic behind a proper structure and setup that can ban entire CIDR blocks for specific country regions.

This logic is a "must know", since it does not make any sense to block country regions entirely.

I would even reverse this...
There is not much logic into allowing an SSH login from any other continent than Europe (for my servers).
In fact I could cope with a handful of subnets, disallowing the rest of the world.

All this would come on top of all other security measures.
Your implied "Security through Obscurity" does not apply....

It makes good sense to properly lock your old bike with 3 chains and on top of that put it in a spot where it isn't seen that easily.

@mr-wolf,
Moreover, for every goal there is not a right tool, there are only good (but not perfect) solutions consisting of multiple tools.

Excactly!!
That's why I don't understand that this is your next sentence

@mr-wolf,Any firewall ipset-country based solution can be easily passed with simple solutions: just reroute traffic over VPSes in other countries that are not blocked at all.

A firewall is not so picky about the traffic origin, it only cares about the origin IP, the destination port and so on.

For instance, only using a firewall ipset-country based solution will give a sysadmin false confidence that a system is secure.......

No, it doesn't
If it does, you're not a sysadmin.

Most sysadmins really want to block the annoying CIDR ranges that China uses...........
........... and there might be some truth behind that idea, but most Chinese hackers simply use Russian VPSes or even servers in other countries.

As another good example, consider something else.
Most malicious traffic is deemed to come from specific countries like China or Russia............
............ but actually, it often comes from the unexpected countries like the United States or Great Britain.
And, really, I still have to find a sysadmin that is blocking those countries with some kind of geo-ip based firewall ruleset.
In short, there is much more than meets the eye and a simple solution will never suffice..........

15 years of extensive logging and monitoring has put enough on my retina to know what makes sense and what doesn't

 

[mr-wolf]

@mr-wolf and @OverWolf

All of the above will result in

- a more efficient Fail2Ban process: most of the recurring attempts from bad IPs are blocked by (persistent!) firewall rules, Fail2Ban does not have to "work hard"

In a normal firewall the first rule (ESTABLISHED) will take away 98% of all traffic.
Because the F2B chains are put in front of that rule, all those rules will need to be processed continuously even if they are already connected.
Being already a bit superfluous itself (F2B) and also too late, this is just not worth it.
These extra resources needed for the firewall will diminish the chances to survive a DoS, so don't do it....

F2B will notice a hostile connection much time after a log entry has been made.
Proper daemons would have disconnected long before.
The "ESTABLISHED" rule will be false for those...

Here's a netfilter solution for SSH.
Fail2ban doesn't come close....

iptables-save | egrep -i '(protect|established| 22 )'

:bruteprotect - [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ownservers
-A INPUT -p tcp -m tcp --dport 22 -j bruteprotect
-A INPUT -p tcp -m tcp --dport 22 -j logaccept
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A bruteprotect -m recent --set --name BRUTEFORCE --rsource
-A bruteprotect -m recent ! --update --seconds 60 --hitcount 4 --name BRUTEFORCE --rsource -j RETURN
-A bruteprotect -j LOG --log-prefix "[DROP BRUTEFORCE] : " --log-tcp-options --log-ip-options
-A bruteprotect -j DROP

 

[trialotto]

@mr-wolf

No personal or technical comments at all.

As I have told you and more or less requested you in a personal conversation, it is not the intention of the Plesk forum to start and maintain discussions.

The Plesk forum is "for Plesk users and by Plesk users": help other persons just for the sake of helping them.

I will not comment on your posts. Final point.

Regards..........

 

[trialotto]

Hi trialotto,
do you talk about firewall-cmd (package: firewalld) to use with f2b ?

No, not at all.

By the way, F2B and iptables (read: firewall) and Plesk Firewall extension (read: an easy GUI for iptables, in essence) are three separate things.

Do not install anything additional, that would make matters more complex and more error-prone (which is not desirable).

Regards.....

 

[mr-wolf]

After experimenting with firewalld I made some enhancements to the script.
It is now much better in detecting its right place in the firewall.
In firewalld for example the chain used is not INPUT but a seperate chain.

I tested the code against several iptables and it always found its best position in the firewall.
This way it will keep it efficient.

A 2nd enhancement I made is that if you use my script to whitelist a country, the original "all accept rule" will be converted to an "all drop rule". This will make it easier to use my script with a normal standard firewall.

-A INPUT -p tcp -m tcp --dport 22 -j ownservers
-A INPUT -p tcp -m tcp --dport 22 -j bruteprotect
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

-A INPUT -p tcp -m tcp --dport 22 -j ownservers
-A INPUT -p tcp -m tcp --dport 22 -j bruteprotect
-A INPUT -p tcp -m tcp --dport 22 -m set --match-set ipset-country-nl src -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m set --match-set ipset-country-be src -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j DROP
​

Here's the modified code:

cat /usr/local/sbin/ipset-country

#!/bin/bash

HEADLESS=
tty >/dev/null || HEADLESS=true

entries() {
  ipset list $1 2>/dev/null | grep -c '\..*\..*\.'
}

THISSCRIPT="`readlink -f $0`"
BASE=${THISSCRIPT##*/}
[ -z "${BASE}" ] && BASE=${0##*/}

IPDENY_PREFIX=http://ipdeny.com/ipblocks/data/aggregated
IPDENY_POSTFIX=aggregated.zone

# Find out the chain used for opening ports to services
# Using a best guess system by picking the chain with the most accepts
CHAIN="`iptables-save | egrep -i 'tcp --dport [0-9][0-9]+ .*accept' | awk '{print $2}' | sort | uniq -c | sort -n | tail -n1 | awk '{print $2}'`"

[ ${HEADLESS} ] || echo "Found out that this firewall is using the chain ${CHAIN}"

# For sanity purpose double check if the chain has at least 5 entries
ENTRIES=`iptables -nL ${CHAIN} 2>/dev/null | egrep -c 'dpt:[0-9][0-9]+'`
if [ ${ENTRIES} -lt 5 ] ; then
  [ ${HEADLESS} ] || echo "Chain ${CHAIN} has too few entries and I will revert to chain INPUT"
  CHAIN=INPUT
fi

# Sometimes a suffix is used (like in firewalld)
SUFFIX="`iptables -nL ${CHAIN} | grep -i accept | egrep -o 'dpt:[0-9][0-9]+.*' | sed 's/dpt:[0-9]*//' | sort | uniq -c | sort -n | tail -n1 | cut -c9-`"

[ ${HEADLESS} ] || echo "Most used suffix in chain ${CHAIN} is \"${SUFFIX}\""


# Using a seperate config file makes it easier to distribute the script over more servers
if grep -q "^WHITE_TCP" ${THISSCRIPT}.conf 2>/dev/null ; then
  [ ${HEADLESS} ] || echo "Use the config file ${THISSCRIPT}.conf"
else
  [ ${HEADLESS} ] || echo "Create the config file ${THISSCRIPT}.conf"
  echo 'WHITE_COUNTRIES=nl,be
BLACK_COUNTRIES=sc,cn
WHITE_TCP=
BLACK_TCP=8443,22
WHITE_UDP=
BLACK_UDP=' >${THISSCRIPT}.conf

fi

# Read the config file from same folder
. ${THISSCRIPT}.conf

# Some sanity checks first

IPTABLES_LENGTH=`iptables-save | grep -c "${CHAIN}"`
if [ ${IPTABLES_LENGTH} -lt 10 ] ; then
  echo "iptables is less than 10 rows long, there's not much use to go on" >&2
  exit 1
fi

# Check if ipset is installed
if ! ipset -v >/dev/null 2>&1 ; then
  echo "This script relies on ipset, without it this will NOT work!" >&2
  echo "Use sudo apt-get install ipset !!!" >&2
  exit 1
fi

# Check if aggregate is installed
if ! which aggregate 2>&1 >/dev/null ; then
  echo "This script relies on aggregate, without it this will NOT work!" >&2
  echo "Use sudo apt-get install aggregate !!!" >&2
  exit 1
fi

# Now define the 2 core functions

create_set() {
  # remove country IP-list when older than 30 days
  find /tmp -maxdepth 1 -type f -name ${BASE}-${i} -mtime +30 -exec rm {} \;

  if [ ! -f /tmp/${BASE}-${i} ] || [ `entries ${BASE}-${i}` -eq 0 ] ; then
    # Fetch netblocks for this country from the site http://ipdeny.com/ipblocks/data/aggregated
    if wget -qO /tmp/${BASE}-${i}.tmp ${IPDENY_PREFIX}/$i-${IPDENY_POSTFIX} ; then
      # I noticed these aren't always aggregated, so I do this myself
      egrep -o "^[0-9]+\.[0-9]+\.[0-9]+\.[0-9/]+" /tmp/${BASE}-${i}.tmp | aggregate 2>/dev/null >/tmp/${BASE}-${i}
      rm /tmp/${BASE}-${i}.tmp

      # If the file contains CIDR-addresses then continue
      if [ -s /tmp/${BASE}-${i} ] ; then
        [ ${HEADLESS} ] || echo "Create set ${BASE}-${i}"
        # Create the country-set if it does not exist yet
        if ! ipset -exist create ${BASE}-${i} hash:net ; then
          echo -e "\n\nIt seems iptables can't work with ipset, this is probably a virtual machine!!\n\n" >&2
          rm /tmp/${BASE}-${i}
          exit 1
        fi
        # Flush the addresses as we just picked up a fresh list
        ipset flush ${BASE}-${i}
        # Add all the subnets to the set
        while read IPNET ; do
          ipset add ${BASE}-${i} ${IPNET}
        done</tmp/${BASE}-${i}
      fi
    fi
  else
    [ ${HEADLESS} ] || echo "The set ${BASE}-${i} is still current"
  fi
}

add_to_firewall() {

  [ `entries ${BASE}-${i}` -eq 0 ] && return  # only useful if we have a filled set
  # loop through all the ports
  for j in ${PORT//,/ } ; do
    [ ${HEADLESS} ] || echo -e "  ${PROT}:${j} set ${BASE}-${i}"

    iptables --line-numbers -nL ${CHAIN} >${TMPDIR}/chain

    # Find the first catch-all of this peticular port in the live firewall
    ANCHOR=`egrep -i "(ACCEPT|DROP) *${PROT}.*0\.0\.0\.0/0 *0\.0\.0\.0/0 *${PROT} dpt:$j${SUFFIX}$" ${TMPDIR}/chain | \
            head -n1 | awk '{print $1}'`
    # If it fails to find the anchor, retry without suffix (making the one with suffix predominant)
    if [ -z "${ANCHOR}" ] ; then
      ANCHOR=`egrep -i "(ACCEPT|DROP) *${PROT}.*0\.0\.0\.0/0 *0\.0\.0\.0/0 *${PROT} dpt:$j$" ${TMPDIR}/chain | \
              head -n1 | awk '{print $1}'`
    fi

    # Did or didn't we find the anchor to attach our iptables lines?
    if [ -z "${ANCHOR}" ] ; then
      echo "Unable to find the anchor for ${PROT} port ${j} in the iptables chain ${CHAIN}" >&2
    else
      # Find the ipset line for this protocol / port / ipset
      INSERT=`iptables --line-numbers -nL ${CHAIN} | \
              grep "${ACTION} *${PROT}.*0\.0\.0\.0/0 *0\.0\.0\.0/0.*${PROT} dpt:$j match-set ${BASE}-${i} src" | \
              head -n1 | awk '{print $1}'`

      if [ -n "${INSERT}" ] ; then
        [ ${HEADLESS} ] || echo -e "\tA line for ${PROT} dpt:$j is already added (${BASE}-${i}) @ ${INSERT}"
      else
        # If the ANCHOR rule is an ACCEPT and so is the INSERT rule,
        # we'll replace the ACCEPT rule into a DROP rule
        if [ "${ACTION}" = "ACCEPT" ] ; then
          if egrep -qi "^${ANCHOR} +.*ACCEPT " ${TMPDIR}/chain ; then
            iptables -D ${CHAIN} ${ANCHOR}
            iptables -I ${CHAIN} ${ANCHOR} -p ${PROT} --dport ${j} -j DROP
          fi
        fi

        if iptables -I ${CHAIN} ${ANCHOR} -p ${PROT} --dport ${j} -m set --match-set ${BASE}-${i} src -j ${ACTION} ; then
          echo -e "\tAdd ${ACTION}-line for port ${j} (set ${BASE}-${i}) @ position ${ANCHOR}"
        else
          echo -e "\n\nIt seems iptables is giving some problem, abort!! " >&2
          exit 1
        fi
      fi
    fi
  done
}

# Let's get to work

TMPDIR=`mktemp -t -d ${0//*\/}.XXXXXXXXXX`

[ ${HEADLESS} ] || echo "Create the sets for whitelisting...."
for i in ${WHITE_COUNTRIES//,/ } ; do
  [ ${HEADLESS} ] || echo -n "$i... "
  create_set
done
[ ${HEADLESS} ] || echo -e "\nCreate the sets for blacklisting...."
for i in ${BLACK_COUNTRIES//,/ } ; do
  [ ${HEADLESS} ] || echo -n "$i... "
  create_set
done

[ ${HEADLESS} ] || echo -e "\n\nAdd entries to firewall"
n=1
# Loop 4 times through the firewall script with different parameters
while [ $n -le 4 ] ; do
  [ $n -eq 1 ] && PORT=${WHITE_TCP} && PROT=tcp && COUNTRIES=${WHITE_COUNTRIES} && ACTION=ACCEPT
  [ $n -eq 2 ] && PORT=${WHITE_UDP} && PROT=udp
  [ $n -eq 3 ] && PORT=${BLACK_TCP} && PROT=tcp && COUNTRIES=${BLACK_COUNTRIES} && ACTION=DROP
  [ $n -eq 4 ] && PORT=${BLACK_UDP} && PROT=udp

  for i in ${COUNTRIES//,/ } ; do
    add_to_firewall
  done
  let n+=1
done

rm -r ${TMPDIR}