Resolved ASSP integration with plesk postfix

[nisamudeen97]

Hi,

Let me know if somebody has integrated ASSP with plesk postfix ?
I am looking for installation steps.

 

[UFHH01]

Hi nisamudeen97,

"postfix" is not a Plesk components, nor is it a Plesk extension. If you desire help for third party components ( and therefore NON-Plesk-related components ! ) you should consider to open a thread at:

=> Home > Forum > General Discussion > Open Topics​

 

[mr-wolf]

@UFHH01 I invited him to create such a thread because I have a set of scripts that have Plesk specifics that will make it an as good as extension to Plesk as other commercial solutions.
I do agree that the thread should be moved.
But to "extensions" and not the dungeon....

 

[UFHH01]

Hi mr-wolf,

you are welcome to suggest a forum for a thread, but you should always keep in mind not to mix PLESK components and extensions, with other third party components and extensions.

 

[mr-wolf]

Because Plesk uses perl as well we should create an alternate perl installation in /usr/local/bin
The standard installation used by Plesk is in /usr/bin
The ASSP installation needs a multi-threaded one
Here's a recipe to create it from source:

cd /opt
wget http://www.cpan.org/src/5.0/perl-5.24.1.tar.gz
tar -xzf perl-5.24.1.tar.gz
cd perl-5.24.1
./Configure -des -Dusethreads –Dprefix=/usr/local
make
make test
make install
/usr/local/bin/perl -MCPAN -e shell
install CPAN
reload cpan
/usr/local/bin/perl -MCPAN -e "get('Mail::SPF')"
cd /root/.cpan/sources/authors/id/J/JM/JMEHNLE/mail-spf/
tar -xzvf Mail-SPF-v2.9.0.tar.gz
cd Mail-SPF-v2.9.0/

perl Build.PL
./Build
./Build test
./Build install

Some other prerequisites:

apt-get install libssl-dev

This script should be used to start the script

cat /opt/ASSP/start

#!/bin/bash
#
# 'start' - Shell script to start ASSP
#

ins_ipt_rule ()
{
  echo 'Insert PREROUTING rules in iptables'
  while read listenline ; do

      ASSP_IP=`echo ${listenline} | awk -F: '{print $1}'`
    ASSP_PORT=`echo ${listenline} | awk -F: '{print $2}'`

    if [ ! -z "${ASSP_PORT}" ] ; then
      if ! iptables-save | grep DNAT | grep 'dport 25' | grep ${ASSP_IP} | grep -q "${ASSP_PORT}" ; then
        echo "Traffic going to ${ASSP_IP}:25 will be translated to ${ASSP_IP}:${ASSP_PORT}"
        iptables -t nat -A PREROUTING  -d ${ASSP_IP} -p tcp -m tcp --dport 25 -j DNAT --to-destination ${ASSP_IP}:${ASSP_PORT}
      fi
    fi

  done <${BASE}/assp.listenlist
}


if [ -z "$1" ] ; then
  BASE=`echo $(dirname $(readlink -f $0))`
else
  BASE=$1;
fi
export BASE

if [ -z "${BASE}" ] ; then
  echo "I'm unable to determine the BASE folder where ASSP is running" >&2
  exit 1
elif [ ! -d "${BASE}" ] ; then
  echo "\"${BASE}\" is not a valid folder on this machine" >&2
  exit 1
fi

PROXY_CFG=$BASE/assp.cfg
pidfile=${BASE}/pid

if head -n1 ${BASE}/assp.pl | grep -q '/usr/bin/perl' ; then
  sed -i 's/\/usr\/bin\/perl/\/usr\/local\/bin\/perl/g' ${BASE}/assp.pl
fi

ASSP_PORT=''
# Override pidfile if a name is given in assp.cfg
if [ -f ${PROXY_CFG} ] ; then
  pidfile="`grep "^pidfile.*=" ${PROXY_CFG} | sed -e 's/pidfile.*=//g'`"

  grep -i '^ListenPort:=' ${PROXY_CFG} | awk -F:= '{print $2}' | tr -cd '0-9.|:' | tr '|' '\n' | sort -n | uniq >${BASE}/assp.listenlist
  # if another listen port is used, this will be used as a flag to go ahead with redirecting traffic.
  ASSP_PORT=`grep ':'  ${BASE}/assp.listenlist | head -n1  | awk -F: '{print $2}'`
fi

# If no PATH is given use $BASE as offset
echo "${pidfile}" | grep -q '/' || pidfile=${BASE}/${pidfile}

# create pidfile if assp.pl is already running in this directory according to 'ps' and no pidfile exists
if [ ! -f ${pidfile} ] ; then
  PROGID="`ps -ef | grep "${BASE}/[a]ssp\.pl" | awk '{print $2}'`"
  [ -z "${PROGID}" ] || echo -n "${PROGID}" >${pidfile}
fi

if [ -f ${pidfile} ] ; then

  # Sanitize content of pidfile
  PROGID=`cat ${pidfile} | tr -cd 0-9`
  NOW=`date +%s`
  PIDTIME=`stat -c%Z ${pidfile}`
  PIDAGE=`echo $((${NOW} - ${PIDTIME}))`

  if [ -z "${PROGID}" ] ; then
    echo "delete invalid pidfile" >&2
    rm -f ${pidfile}
  else
    if ps --pid ${PROGID} 2>/dev/null | tail -n+2 | grep -q "[a]ssp\.pl" ; then
      echo "ASSP already running (${PROGID})" >&2
      exit 1
    else
      # Remove PIDFILE if it's much more than a minute old
      if [ ${PIDAGE} -gt 90 ] ; then
        rm -f ${pidfile}
      else
        echo "strange pid in ${pidfile}, abort" >&2
        exit 1
      fi
    fi
  fi
fi

echo Starting ASSP Anti-SPAM Proxy server in ${BASE}
trap '' 1
LANG=
export LANG
exec ${BASE}/assp.pl ${BASE} --nointchk:=1 &

if [ ! -z "${ASSP_PORT}" ] ; then
  # initially wait 5 seconds to start and let ASSP settle
  sleep 5
  # Check every 2 seconds if ASSP has got a hold of its SMTP-port
  n=1
  while [ $n -lt 30 ] ; do
    sleep 2
    netstat -lntp | grep -q ":${ASSP_PORT} " && break
    let n+=1
  done
  sleep 1
  netstat -lntp   | grep -q ":${ASSP_PORT} " && ins_ipt_rule
fi

 

[mr-wolf]

Because ASSP is proxying (and is not transparent) all the incoming traffic will seem to come from the server's own IP for Postfix.
This would turn your setup into an open relay.
To prevent this you should let ASSP only accept domains that are in files/localdomains.txt and files/localaddresses.txt

To automate the creation of those entries I created a script.
This script should be run as a 15-minute cronjob.
It manages the file /opt/ASSP/files/localdomains.txt by reading the MySQL database and /etc/postfix/relay_domains
It also manages the file files/localaddresses.txt
Each domain will get a *@<domain> automatically in that file.

You can optionally add an email-address for a specific domain and from that moment on it will only accept that address for that domain.
The address can be put anywhere in the file.
The script will take care of a neat format. Just check it out after it ran and you will see what I mean.

There's no need to do this for the Plesk domains
Because ASSP is directly proxying for Postix you should let Postfix decide if an address is to be accepted.
Postfix knows this for all the Plesk domains.
It's only useful for the domains for which postfix is relaying (/etc/postfix/relay_domains & /etc/postfix/transport).
Because it doesn't know immediately which addresses are accepted.

cat /usr/local/sbin/assp_localdomains
ln -s /usr/local/sbin/assp_localdomains /etc/cron.15min/

#!/bin/sh

LOCALDOMAINS=/opt/ASSP/files/localdomains.txt
LOCALADDRESSES=/opt/ASSP/files/localaddresses.txt
POSTFIXDIR=/etc/postfix
RELAYFILE=${POSTFIXDIR}/relay_domains
TRANSPORTFILE=${POSTFIXDIR}/transport
HEADER="# WARNING do not EDIT!!! \n# This file will be overwritten by $0 (`readlink -f $0`)"

FGROUP=`stat -c%G ${LOCALDOMAINS}`
FUSER=`stat -c%U ${LOCALDOMAINS}`

TMP1=`mktemp`
TMP2=`mktemp`
TMP3=`mktemp`
chown ${FUSER}.${FGROUP} ${TMP1}
chown ${FUSER}.${FGROUP} ${TMP2}

echo -e "${HEADER}\n" >${TMP1}
echo -e "################################################\n# Domains coming from Plesk\n################################################"  >>${TMP1}

mysql  --skip-column-names -uadmin -p`cat /etc/psa/.psa.shadow` psa -e "select domains.name from mail,domains,accounts where mail.dom_id=domains.id and mail.account_id=accounts.id;" 2>/dev/null >${TMP2}
mysql  --skip-column-names -uadmin -p`cat /etc/psa/.psa.shadow` psa -e "select domain_aliases.name from domain_aliases where domain_aliases.mail='true';" 2>/dev/null >>${TMP2}

sort -u ${TMP2} >>${TMP1}


echo -e "\n################################################" >>${TMP1}
if [ -e "${RELAYFILE}" ] ; then
  echo -e "# Domains coming from ${RELAYFILE}\n################################################"  >>${TMP1}
  awk '{print $1}' "${RELAYFILE}"  >>${TMP1}
else
  echo -e "\n# ${RELAYFILE} does NOT exist!!\n################################################"  >>${TMP1}
fi

if ! diff ${TMP1} ${LOCALDOMAINS} >/dev/null 2>&1 ; then
  cp -p ${TMP1}  ${LOCALDOMAINS}
  killall -HUP assp.pl
fi

# All addresses without wildcards
egrep -o '^.*@[a-z0-9.]+\.[a-z]+'  ${LOCALADDRESSES} | grep -v '^\*@' | sort -u | sort -k2 -t@ >${TMP1}
# Only the domains of these addresses
awk -F@ '{print $2}' ${TMP1} | sort | uniq | sed 's/.*/^&$/g'>${TMP3}

# Start building LOCALADDRESSES
echo -e "################################################\n# Warning!\n# This file is maintained by $0 (`readlink -f $0`)\n# You can add an e-mail address for one of the local domains,\n# but this means all other addresses of that domain will get rejected.\n################################################\n\n\n################################################\n# Wildcards first (all addresses are accepted by ASSP)\n# The MTA can still reject the mail\n################################################\n" >${TMP2}
# Make wildcards for all domains except the ones that have already a full address defined
egrep '^[a-z][a-z0-9.-]+\.[a-z]+$' ${LOCALDOMAINS} | grep -vf ${TMP3} | sort -u | sed 's/.*/*@&/g' >>${TMP2}
echo -e "\n\n################################################\n# Only these addresses will be accepted by ASSP\n################################################\n"  >>${TMP2}

sed -ie 's/\^/@/g;s/\$//g' ${TMP3}

while read DOMAIN ; do
  echo -e "\n################################################" >>${TMP2}
  echo      "##    ${DOMAIN}"                                  >>${TMP2}
  echo -e   "################################################" >>${TMP2}

  grep "[a-z0-9_]${DOMAIN}" ${TMP1} >>${TMP2}
done <${TMP3}

if ! diff ${TMP2} ${LOCALADDRESSES} >/dev/null 2>&1 ; then
  cp -p ${TMP2}  ${LOCALADDRESSES}
  killall -HUP assp.pl
fi

rm -f ${TMP1}
rm -f ${TMP2}
rm -f ${TMP3}

 

[mr-wolf]

Only useful if you are using it as a relaying server as well I have another script that checks /var/log/maillog that reads if addresses are being rejected by the relayed domains.
A rejected address will get an entry in /opt/ASSP/files/rejecttheselocaladdresses.txt
This will prevent backscatter.

apt-get install swaks
apt-get install logtail

cat /usr/local/sbin/assp_rejecttheselocaladdresses
ln -s /usr/local/sbin/assp_rejecttheselocaladdresses /etc/cron.hourly/

#!/bin/sh

REJECTADDRESSES=/opt/ASSP/files/rejecttheselocaladdresses.txt
LOCALDOMAINS=/opt/ASSP/files/localdomains.txt
POSTFIXDIR=/etc/postfix
POSTFIXTRANSPORT=/etc/postfix/transport
POSTFIXLOGDIR=/usr/local/psa/var/log
POSTFIXLOG=${POSTFIXLOGDIR}/maillog
#POSTFIXLOG=${REJECTADDRESSES}

[ -e ${REJECTADDRESSES} ] || exit 1
[ -e ${LOCALDOMAINS} ] || exit 1
[ -e ${POSTFIXLOG} ] || exit 1

TIMETOKEEP=14
VERYOLD=28

HEADER="# Automatically maintained by $0 (`readlink -f $0`)"

FGROUP=`stat -c%G ${REJECTADDRESSES}`
FUSER=`stat -c%U ${REJECTADDRESSES}`

TIMESTAMP="`date --rfc-3339="seconds"`"
TIMESTAMPSECONDS="`date +%s`"

TMP1=`mktemp`
TMP2=`mktemp`
TMP3=`mktemp`
TMP4=`mktemp`

TMP5=`mktemp`
chown ${FUSER}.${FGROUP} ${TMP5}

TMPDIR=${TMP4%/*}

VALIDUSERS=${0##*/}.validusers
OFFSET=${TMPDIR}/${0##*/}.offset

HELO=`hostname -f`
# HELO=smtp.yourdomain.com


# Process current file and extract
while read LINE ; do

  if echo "${LINE}" | egrep -q '.*@.*\.[a-z]+' ; then
    COMMENTSTAMP="`echo "${LINE}" | awk -F# '{print $2}' | egrep -o ' [0-9]+-[0-9]+-[0-9]+ [0-9:+]+' | head -n1`"

    # If there's no valid timestamp it's regarded as a manual entry and kept
    if [ -z "${COMMENTSTAMP}" ] ; then
      echo "${LINE}" >>${TMP1}
    else
      # extract the date when this entry was first added
      COMMENTSTAMPORG="`echo "${LINE}" | awk -F# '{print $2}' | egrep -o ' [0-9]+-[0-9]+-[0-9]+ [0-9:+]+' | tail -n1`"
      COMMENTORGSECONDS=`date +%s -d "${COMMENTSTAMPORG}"`
      AGE_ORG_IN_SECONDS=$((${TIMESTAMPSECONDS} - ${COMMENTORGSECONDS}))
      AGE_ORG_IN_DAYS=$((${AGE_ORG_IN_SECONDS} / 86400))

      COMMENTSECONDS=`date +%s -d "${COMMENTSTAMP}"`
      AGE_IN_SECONDS=$((${TIMESTAMPSECONDS} - ${COMMENTSECONDS}))
      AGE_IN_DAYS=$((${AGE_IN_SECONDS} / 86400))
      DOMAIN="`echo "${LINE}" | awk '{print $1}' | awk -F@ '{print $2}'`"

      SERVER="`grep -m1 "^${DOMAIN}" ${POSTFIXTRANSPORT} | awk -F: '{print $2}'`"
      PORT="`grep -m1 "^${DOMAIN}" ${POSTFIXTRANSPORT} | awk -F: '{print $3}' | tr -cd '0-9'`"

      if [ ${AGE_ORG_IN_DAYS} -gt ${VERYOLD} ] || [ -z "`which swaks`" ] || [ -z "${SERVER}" ] ; then
        echo "This entry will be deleted because it's too old (${AGE_ORG_IN_DAYS} days): \"${LINE}\"" >&2
      elif [ ${AGE_IN_DAYS}  -le ${TIMETOKEEP} ] ; then
        echo "${LINE}" >>${TMP1}
      else
        [ -z "${PORT}" ] && PORT=25
        EMAIL="`echo "${LINE}" | awk '{print $1}' | tr 'A-Z' 'a-z'`"
        swaks -t "${EMAIL}" --helo ${HELO} --protocol SMTP -s ${SERVER} -p ${PORT} --quit-after RCPT  --timeout 6s 2>/dev/null >${TMP2}
        if grep -A3 'RCPT TO' ${TMP2} | grep -q '550 5\.1\.1' ; then
          echo "This entry will be kept because the server still refuses this mail: \"${EMAIL}\"" >&2

          ADDRESLINE="`echo -n "${LINE}" | awk -F# '{print $1}'`"
          echo "${ADDRESLINE}# ${TIMESTAMP} ${COMMENTSTAMPORG}" >>${TMP1}
        else
          REASON="it's too old"
          grep -q 'RCPT TO' ${TMP2} && REASON="the server now accepts this mail"
          echo "This entry will be deleted because ${REASON}: \"${LINE}\"" >&2
        fi
      fi
    fi
  fi
done< "${REJECTADDRESSES}"

# Create a file with local addresses that were accepted by the endusers MTA
if [ -z "`find ${TMPDIR} -maxdepth 1 -type f -mtime -1 -name ${VALIDUSERS}`" ] ; then
  # generate file (TMP2) with a regular expression of a local domain accept line
  egrep -v '(^ *$|^#)' ${LOCALDOMAINS} | sed 's/.*/to=<.*@&>, .*, status=sent/' >${TMP2}
  zgrep -iE -f ${TMP2} ${POSTFIXLOG}* | awk -F, '{print $1}' | awk -Fto= '{print $2}' | tr 'A-Z' 'a-z' | sort -u | tr -d '<>' | egrep '^[a-z0-9.-]+@[a-z0-9.-]+\.[a-z]+$' >${TMPDIR}/${VALIDUSERS}
fi

# generate tail of postfixlog (TMP4)
logtail -f "${POSTFIXLOG}" -o ${OFFSET} >${TMP4}

# generate file (TMP2) with a regular expression of a local domain refusal line
egrep -v '(^ *$|^#)' ${LOCALDOMAINS} | sed 's/.*/to=<.*@&>, .* said: 550 5.1.1 .* reply to RCPT TO command/' >${TMP2}
# generate file (TMP3) with emailaddresses using TMP2 as regular expression
egrep -if ${TMP2} ${TMP4} | egrep -o 'to=<.+@[a-z0-9.-]+\.[a-z]+>' | awk -F= '{print $2}' | tr 'A-Z' 'a-z' | sort -u | tr -d '<>' >${TMP3}


# extract already rejected mail from file (TMP2)
egrep -o '^[A-Za-z0-9+_&/-]+@[a-z0-9.-]+\.[a-z]+' ${TMP1} >${TMP2}

# generate file with newly
grep -v -f ${TMP2} ${TMP3} | grep -v -f ${TMPDIR}/${VALIDUSERS} | awk '{ printf "%-40s\n", $1}' | sed "s/.*/& # ${TIMESTAMP}/g" >${TMP4}
cat ${TMP1} >>${TMP4}


echo -e "${HEADER}\n" >${TMP5}
echo -e "################################################\n# Reject these addresses\n################################################"  >>${TMP5}

sort -k2 -t@ ${TMP4} >>${TMP5}

if ! diff ${TMP5} ${REJECTADDRESSES} >/dev/null 2>&1 ; then
  cp -p ${TMP5}  ${REJECTADDRESSES}
  killall -HUP assp.pl
fi

rm -f ${TMP1}
rm -f ${TMP2}
rm -f ${TMP3}
rm -f ${TMP4}
rm -f ${TMP5}

To make postfix relay for some domains you need to modify postfix a bit....

touch /etc/postfix/relay_domains
postmap /etc/postfix/relay_domains


touch /etc/postfix/transport
postmap /etc/postfix/transport

In /etc/postfix/main.cf

relay_domains = $mydestination, hash:/etc/postfix/relay_domains

transport_maps = hash:/var/spool/postfix/plesk/transport, hash:/etc/postfix/transport

 

[nisamudeen97]

Okay, Going through the update in detail.

 

[mr-wolf]

cat /opt/ASSP/stop

#!/bin/bash
del_ipt_rule ()
{
  echo 'Delete PREROUTING rules in iptables'
  while read listenline ; do
      ASSP_IP=`echo ${listenline} | awk -F: '{print $1}'`
    ASSP_PORT=`echo ${listenline} | awk -F: '{print $2}'`

    if [ ! -z "${ASSP_PORT}" ] ; then
      if iptables-save | grep DNAT | grep 'dport 25' | grep ${ASSP_IP} | grep -q "${ASSP_PORT}" ; then
        echo "Traffic going to ${ASSP_IP}:25 will NOT be translated to ${ASSP_IP}:${ASSP_PORT} anymore"
        iptables -t nat -D PREROUTING  -d ${ASSP_IP} -p tcp -m tcp --dport 25 -j DNAT --to-destination ${ASSP_IP}:${ASSP_PORT}
      fi
    fi

  done <${BASE}/assp.listenlist
}

create_pid ()
{
  PROGID="`ps -ef | grep "${BASE}" | grep "[a]ssp\.pl" | awk '{print $2}'`"
  [ -z "${PROGID}" ] || echo -n "${PROGID}" >${pidfile}
}

if [ -z "$1" ] ; then
  BASE=`echo $(dirname $(readlink -f $0))`
else
  BASE=$1;
fi
export BASE

if [ -z "${BASE}" ] ; then
  echo "I'm unable to determine the BASE folder where ASSP is running" >&2
  exit 1
elif [ ! -d "${BASE}" ] ; then
  echo "\"${BASE}\" is not a valid folder on this machine" >&2  exit 1
fi

pidfile=${BASE}/pid
PROXY_CFG=$BASE/assp.cfg

ASSP_PORT=''
ASSP_IP=''

# Override pidfile if a name is given in assp.cfg
if [ -f ${PROXY_CFG} ] ; then
  pidfile="`grep "^pidfile.*=" ${PROXY_CFG} | sed -e 's/pidfile.*=//g'`"
  grep -i '^ListenPort:=' ${PROXY_CFG} | awk -F:= '{print $2}' | tr -cd '0-9.|:' | tr '|' '\n' | sort -n | uniq >${BASE}/assp.listenlist
  # if another listen port is used, this will be used as a flag to go ahead with redirecting traffic.
  ASSP_PORT=`grep ':'  ${BASE}/assp.listenlist | head -n1  | awk -F: '{print $2}'`
fi
echo "${pidfile}" | grep -q '/' || pidfile=${BASE}/${pidfile}

# create pidfile if a process is already running on this directory
if [ ! -f ${pidfile} ] ; then
  create_pid
fi

if [ ! -f "${pidfile}" ] ; then
  echo "ASSP Anti-SPAM Proxy server in $BASE already stopped"
else
  PROGID=`egrep '^[0-9]+$' $pidfile`
  if [ -z "${PROGID}" ] ; then
    echo "${pidfile} does NOT contain a process ID" >&2
    rm -f "${pidfile}"
    exit 1
  elif ps --pid ${PROGID} 2>/dev/null | grep -q "[a]ssp\.pl" ; then
    n=1
    echo -n "Stopping assp (${PROGID}) in ${BASE}" >&2
    while ps ${PROGID} 2>&1 >/dev/null ; do
      echo -n "."
      kill ${PROGID}
      sleep 2
      if [ $(($n % 10)) -eq 0 ] ; then
        kill -9 ${PROGID} 2>&1 >/dev/null
        rm -f "${pidfile}" 2>/dev/null
        if [ $n -gt 30 ] && ps ${PROGID} 2>&1 >/dev/null ; then
          echo "Unable to kill assp (${PROGID})" >&2
          exit 1
        fi
      fi
      let n+=1
    done
    echo ""
    sleep 1
    [ -z "${ASSP_PORT}" ] || netstat -lntp | grep -q ":${ASSP_PORT} " || del_ipt_rule

  else
    echo "PID: ${PROGID} is not assp" >&2
    rm -f "${pidfile}" 2>/dev/null
    exit 1
  fi
fi
exit 0