Question Plesk migration manager - adjust vhost_nginx.conf custom config using IP mapping

[burnley]

I don't know what's the best way to approach this. We have vhosts with custom nginx configuration in /var/www/vhosts/system/<vhost>/conf/vhost_nginx.conf that looks like this:

location ~* ^/.*(locationA|locationB)$ {
<nginx specific processing here>
if ( $scheme = http )
{
proxy_pass http://IP.AD.DR.ESS
}
if ( $scheme = https )
{
proxy_pass http://IP.AD.DR.ESS
}
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
error_log "/var/log/nginx/error.log" warn;
}

When using the Plesk migration & transfer manager to migrate the subscription from serverA to serverB Plesk does all the right things, copies everything across and reconfigures everything, except for one little bit: IP.AD.DR.ESS in /var/www/vhosts/system/<vhost>/conf/vhost_nginx.conf is not adjusted to reflect the new IP configuration of the target server and I have to manually adjust it and reload nginx.
Is there any way I can get this remaining configuration carried across properly? Perhaps using the IP mapping mechanism? Some post-migration hooks using predefined variables?
Wonder what your thoughts are.

 

[mr-wolf]

You can use an include of a server-wide config-file which is different on each server.
When it's a multi-ip server this may get you to the next problem, but it may well do for you.

In /var/www/vhosts/system/???/conf/vhost_nginx.conf
include /var/www/vhosts/system/conf/vhost_nginx.conf;

cat /var/www/vhosts/system/conf/vhost_nginx.conf
if ( $scheme = http )
{
proxy_pass http://IP.AD.DR.ESS
}
if ( $scheme = https )
{
proxy_pass http://IP.AD.DR.ESS
}

Each server would have its own IP there...
I use this scheme for different purposes, namely HSTS

Question - HSTS and ssl.conf command

 

[burnley]

You can use an include of a server-wide config-file which is different on each server.
When it's a multi-ip server this may get you to the next problem, but it may well do for you.

Thanks mr-wolf, that's exactly my very problem: I have multiple IP addresses on the same server and the proxy_pass directive must contain the actual IP on the website which Plesk configures correctly in nginx.conf. What I want is a way to have the same IP carried across in vhost_nginx.conf as part of the migration. When running the migration from command line I can script something, but when ran from the Panel we don't have this option.

 

[mr-wolf]

But why do you have those directives anyhow.
By default Plesk is already proxying https to port 7081 and http to 7080.

Not clear to me what you're actually doing..

 

[burnley]

I need to do per-vhost processing in nginx based on the location(s) hit. Say I have to configure specific filtering that kicks in based on the url hit by the client, for this I need to insert additional configuration which, for a Linux Plesk vhost, goes in conf/vhost_nginx.conf in a "location" block. And since nginx is acting as reverse proxy to Apache, after processing what's in the location block it then has to pass the request on to Apache. If you don't use proxy_pass in the location block nginx will try to serve the request directly and the client will end up with 404.
NGINX Reverse Proxy | NGINX

 

[mr-wolf]

I need to do per-vhost processing in nginx based on the location(s) hit. Say I have to configure specific filtering that kicks in based on the url hit by the client, for this I need to insert additional configuration which, for a Linux Plesk vhost, goes in conf/vhost_nginx.conf in a "location" block. And since nginx is acting as reverse proxy to Apache, after processing what's in the location block it then has to pass the request on to Apache. If you don't use proxy_pass in the location block nginx will try to serve the request directly and the client will end up with 404.
NGINX Reverse Proxy | NGINX

Why not use $server_addr there?

I still don't know if I understand it, but my proposal should make it clear to you if I understand it.

if ( $scheme = http )
{
proxy_pass http://$server_addr:7080
}
if ( $scheme = https )
{
proxy_pass https://$server_addr:7081
}

 

[G J Piper]

There is an option that might work depending on your comfort level:
I think if it were a matter of changing the IP in all the entries you've ever manually made in the past, I would try to do a database dump, then change all the IP addresses as needed in the dump file, then re-import the database file to effect all the new IP address changes. (or maybe use some other kind of database find/replace)
I don't have the instructions how you would do this, but I have done something similar with a site-specific Wordpress MySQL database in the past and it worked well.

 

[MarkM]

Bulk IP replacement?

BACKUP PSA DATABASE!

Update records via MySQL;

mysql > UPDATE dns_recs SET val = 'new.new.new.new' WHERE displayVal = 'old.old.old.old';
mysql > UPDATE dns_recs SET displayVal = 'new.new.new.new' WHERE displayVal = 'old.old.old.old';

Rebuild dns zones;

# plesk repair dns

 

[G J Piper]

Bulk IP replacement?

BACKUP PSA DATABASE!

Update records via MySQL;

mysql > UPDATE dns_recs SET val = 'new.new.new.new' WHERE displayVal = 'old.old.old.old';
mysql > UPDATE dns_recs SET displayVal = 'new.new.new.new' WHERE displayVal = 'old.old.old.old';

Rebuild dns zones;

# plesk repair dns

I don't think this will work for him because the incorrect IP addresses are all in the manually entered "Additional nginx directives" section, so the find/replace may have to be a plain text substitution.

 

[MarkM]

AH my mistake, I thought you were talking about DNS records.

 

[mr-wolf]

I already have experience manipulating that file and I know for a fact that one needs to do a Plesk repair web <domain> after the manipulation.

I could quite easy write a script that will check all those files and change all foreign IP's to a home one. But which home ip?

Furthermore the OP has not reacted on my solution although I don't fully understand his use case as he's obfuscating too much info in his examples.

 

[mr-wolf]

Use at your own risk
It would need additional code to change the foreign IP to anything else than the first IP found on the system.


cat /usr/local/sbin/vhost_nginx_repair_ip

#!/bin/bash

HEADLESS=
tty >/dev/null || HEADLESS=true

THISSCRIPT="`readlink -f $0`"
SCRIPTNAME=${THISSCRIPT##*/}
[ -z "${SCRIPTNAME}" ] && SCRIPTNAME=${0##*/}

LOG=/var/log/${SCRIPTNAME}.log
PLESKBIN=/usr/local/psa/bin/domain

if ! nginx -t 2>/dev/null ; then
  echo "The Nginx configuration is not valid, abort" >&2
  exit 1
fi

TMPDIR=`mktemp -t -d ${0//*\/}.XXXXXXXXXX`
TMPLOG=${TMPDIR}/log

ifconfig | egrep -o 'inet addr:[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' | awk -F: '{print $2}' | grep -v ^127 >${TMPDIR}/IPS
FIRST_IP=`head -n1 ${TMPDIR}/IPS`


find /var/www/vhosts/system/ -mindepth 2 -maxdepth 2 -type d -name conf >${TMPDIR}/confs

PATCHED=
while read CONF ; do
  NGINXCONF=${CONF}/vhost_nginx.conf

  [ ${HEADLESS} ] || echo ${NGINXCONF}

  if [ -e ${NGINXCONF} ] ; then
    # This domain has a vhost_nginx.conf

    FOREIGN_IP=
    egrep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' ${NGINXCONF} >${TMPDIR}/VHOST_IPS

    if [ -s ${TMPDIR}/VHOST_IPS ] ; then
      while read IP ; do
        if ! grep -q "${IP}" ${TMPDIR}/IPS ; then
          echo "Safeguarding a backup to ${NGINXCONF}.unpatched" | tee -a ${TMPLOG} >&2
          if ! [ ${FOREIGN_IP} ] ; then
            cp -p ${NGINXCONF} ${NGINXCONF}.unpatched
          fi
          FOREIGN_IP=true
          PATCHED=true
          # replace foreign IP with first IP of server
          sed -i "s/${IP}/${FIRST_IP}/g" ${NGINXCONF}
        fi
      done< ${TMPDIR}/VHOST_IPS
      if [ -f "${NGINXCONF}.unpatched" ]  ; then
        if diff ${NGINXCONF}.unpatched ${NGINXCONF}.unpatched >/dev/null ; then

          if ! nginx -t 2>/dev/null ; then  # check if the nginx config is still valid
            echo "Somehow the Nginx config became invalid after patching ${NGINXCONF}" | tee -a ${TMPLOG} >&2
            echo "I will revert the changes and abort this script" | tee -a ${TMPLOG} >&2
            cp -p ${NGINXCONF}.unpatched ${NGINXCONF}
            break
          fi

          DOMAIN="`echo ${CONF} | sed 's/.*system\///g;s/\/.*//g'`"    # extract DOMAIN out of folder name
          echo "${NGINXCONF} has been modified" | tee -a ${TMPLOG}
          # Although the file is patched a "plesk repair" is needed to tell Plesk
          plesk repair web ${DOMAIN} -y -v
        fi
      fi
    fi
  fi
done<${TMPDIR}/confs

echo "`date`  **** Finished patching vhost_nginx.conf" >>${TMPLOG}
[ ${PATCHED} ] && cat ${TMPLOG} >>${LOG}

rm -r ${TMPDIR}

 

[G J Piper]

Use at your own risk
It would need additional code to change the foreign IP to anything else than the first IP found on the system.


cat /usr/local/sbin/vhost_nginx_repair_ip

#!/bin/bash

This, to me, would seem a bit drastic when all he needs is the IP addresses changed once and for all in the database. A script running to adjust and repair the end-result file every (day|hour|minute?) would not be my first choice. This seems a bit like overriding the fact that you ran out of gas by renting a camel to tow your car to work every day.

 

[mr-wolf]

This, to me, would seem a bit drastic when all he needs is the IP addresses changed once and for all in the database. A script running to adjust and repair the end-result file every (day|hour|minute?) would not be my first choice. This seems a bit like overriding the fact that you ran out of gas by renting a camel to tow your car to work every day.

The cronjob is in your head, not mine...
He didn't need to rent anything... the camel to tow his broken car was given to him for free to use as often as he wants it. No effort whatsoever on his part.
What is your problem? Didn't see a solution coming from you...

Some people do crosswords... I program in bash.

BTW... Substituting the literal IP with $server_addr is probably all that's needed anyhow.

 

[G J Piper]

The cronjob is in your head, not mine...
What is your problem?

No problem here except that any changes made to the internal files would be removed every time there is a Plesk update...
(I once had a script that added expires headers to those same files, thinking on those same lines, and they needed to be checked and updated daily in order for them to stay active or the changes get deleted every so often)

 

[mr-wolf]

No problem here except that any changes made to the internal files would be removed every time there is a Plesk update...
(I once had a script that added expires headers to those same files, thinking on those same lines, and they needed to be checked and updated daily in order for them to stay active or the changes get deleted every so often)

Not in this case.

The end result is completely stable.
The content of the file is not in the database (or they are added to it by running plesk repair).
He could run the script after the migration.

The file is not even there by default.
If you create one you need to do a "plesk repair".

In this case the file is already there.
A "killall -HUP nginx" is probably sufficient.
But I used a copy of an existing script of mine to create this one. To make sure it works as good as the original script of mine I kept the "plesk repair".

 

[G J Piper]

Not in this case.

The end result is completely stable.
The content of the file is not in the database.

Oh. I guess I thought we were talking about the file found here:
/var/www/vhosts/system/<vhost>/conf/vhost_nginx.conf

 

[mr-wolf]

Oh. I guess I thought we were talking about the file found here:
/var/www/vhosts/system/<vhost>/conf/vhost_nginx.conf

Yes we are...
I have established it is completely stable afterwards.
I'm doing a "plesk repair web <domain>" after the patch.

 

[MarkM]

Please continue the story about the camel....

 

[G J Piper]

Yes we are...
I have established it is completely stable afterwards.
I'm doing a "plesk repair web <domain>" after the patch.

I guess my experience with modifying these files is different than yours. Whenever the nginx settings are modified after editing that file directly, the file is reverted back to original and the new settings from the GUI applied to it. Is this not so?