Bad Gateway and non working Bootstrap update

[TobiasBu]

Hello,

this is my first thread because a big Problem after Plesk update from 10.2 to 12.0.8.

Always i got a bad gateway when i try to reach https://domain:8443

The only mistake i cam see is after /usr/local/psa/admin/sbin/httpdmng --reconfigure-all

is

root@xxxxxxx:~# /usr/local/psa/admin/sbin/httpdmng --reconfigure-all.
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
Aborted

Anyone have an Idea what i can do.

sincerly

Sorry for my poor english

 

[IgorG]

Looks like upgrade was not completed successfully. Try to fix it with bootstrapper repair procedure:

# /usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh repair

 

[TobiasBu]

Thanks first for your help,

When i do your Command i got:


root@xxxxxxxx:~# /usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh repair
Started bootstrapper repair procedure. This may take a while.
Certain actions may be skipped if not applicable.

-- Warning: Skipping the data of table mysql.event. Specify the --events option explicitly.
Finishing up upgrade procedures and rerunning previously failed upgrade actions...
===> Cumulative APS controller database (apsc_tntv) upgrade and repair has been started.
===> Cumulative upgrade and repair of APS controller database has been completed.

**** Product repair started.

===> Checking for previous installation ... found.
Started bootstrapper repair procedure. This may take a while.
Certain actions may be skipped if not applicable.

Trying to start service mysql... /usr/sbin/mysqld (pid 267) is running...
done
Trying to establish test connection... connected
done
Trying to start service mysql... /usr/sbin/mysqld (pid 267) is running...
done
Trying to establish test connection... connected
done
Trying to find psa database... version is 012000018
Version is OK, no need to create psa database.
Trying to backup MySQL database... done
MySQL databases are dumped to /var/lib/psa/dumps/mysql.preupgrade.10.11.0-12.0.18.20141205-085143.dump.gz
Finishing up upgrade procedures and rerunning previously failed upgrade actions...
===> Cumulative APS controller database (apsc_tntv) upgrade and repair has been started.
Upgrade or repair for 'apsc' (stage 'prep') is not required
Trying to backup MySQL database... done
MySQL databases are dumped to /var/lib/psa/dumps/mysql.preupgrade.apsc.10.11.0-12.0.18.20141205-085147.dump.gz
===> Cumulative upgrade and repair of APS controller database has been completed.
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
/usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh: line 4471: 16440 Aborted $PRODUCT_ROOT_D/bin/sw-engine-pleskrun "$@"
Trying to reset database user password for 'apsc_m5nq@'... done
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
/usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh: line 4471: 16451 Aborted $PRODUCT_ROOT_D/bin/sw-engine-pleskrun "$@"

WARNING!
Some problems are found during fix credentials for APS controller database(see log file: /var/log/plesk/install/plesk_12.0.18_repair.log)

Continue...

===> Cumulative APS controller upgrade and repair (final stage) has been started.
===> Cumulative upgrade and repair of APS controller (final stage) has been completed.
===> Cumulative Plesk database upgrade and repair (revertable stage) has been started.
===> Preparing Plesk database upgrade (revertable stage).
===> Cumulative upgrade and repair of Plesk database (revertable stage) has been completed.

**** Product repair started.

===> Checking for previous installation ... found.
Started bootstrapper repair procedure. This may take a while.
Certain actions may be skipped if not applicable.

Trying to start service mysql... /usr/sbin/mysqld (pid 267) is running...
done
Trying to establish test connection... connected
done
Trying to start service mysql... /usr/sbin/mysqld (pid 267) is running...
done
Trying to establish test connection... connected
done
Trying to find psa database... version is 012000018
Version is OK, no need to create psa database.
Trying to backup MySQL database... done
MySQL databases are dumped to /var/lib/psa/dumps/mysql.preupgrade.10.11.0-12.0.18.20141205-085143.dump.gz
Finishing up upgrade procedures and rerunning previously failed upgrade actions...
===> Cumulative APS controller database (apsc_tntv) upgrade and repair has been started.
Upgrade or repair for 'apsc' (stage 'prep') is not required
Trying to backup MySQL database... done
MySQL databases are dumped to /var/lib/psa/dumps/mysql.preupgrade.apsc.10.11.0-12.0.18.20141205-085147.dump.gz
===> Cumulative upgrade and repair of APS controller database has been completed.
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
/usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh: line 4471: 16440 Aborted $PRODUCT_ROOT_D/bin/sw-engine-pleskrun "$@"
Trying to reset database user password for 'apsc_m5nq@'... done
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
/usr/local/psa/bootstrapper/pp12.0.18-bootstrapper/bootstrapper.sh: line 4471: 16451 Aborted $PRODUCT_ROOT_D/bin/sw-engine-pleskrun "$@"

WARNING!
Some problems are found during fix credentials for APS controller database(see log file: /var/log/plesk/install/plesk_12.0.18_repair.log)

Continue...

===> Cumulative APS controller upgrade and repair (final stage) has been started.
Upgrade or repair for 'apsc' (stage 'post') is not required
===> Cumulative upgrade and repair of APS controller (final stage) has been completed.
===> Cumulative Plesk database upgrade and repair (revertable stage) has been started.
Upgrade or repair for 'core' (stage 'prep') is not required
===> Preparing Plesk database upgrade (revertable stage).

Trying to set psa database version to 012000018... done
===> Cumulative upgrade and repair of Plesk database (revertable stage) has been completed.
sw-engine: /usr/include/libboost-plesk-1.54.0/boost/interprocess/mapped_region.hpp:820: void boost::interprocess::mapped_region:riv_close(): Assertion `ret == 0' failed.
Aborted

ERROR while trying to SMB database not created
Check the error reason(see log file: /var/log/plesk/install/plesk_12.0.18_repair.log), fix and try again

***** problem report *****
ERROR while trying to SMB database not created
Check the error reason(see log file: /var/log/plesk/install/plesk_12.0.18_repair.log), fix and try again


best regards

 

[IgorG]

I can only recommend to create a request to support team to do in-depth investigation to find the reason and to fix it. Please create a ticket to support at https://odin.com/hcap/support/request/ .
You may have free support, please check what kind of Plesk license you use for available support options at http://kb.odin.com/en/121580 .
If there’s no free support in your case, you can order Plesk per-incident support at http://odin.com/hcap/support/buy-support/. Support team will contact you as soon as purchase is processed, and they will do the best to resolve it.
If it is found that your problem was caused by product bug w/o available solution or workaround from Parallels, then your purchase will be re-funded.

 

[mivola]

Hi Igor,

I'm a friend of Tobias who started the thread here and I'm trying to fix the problem with him.

By now we managed to successfully run the "bootstrapper.sh repair" command. To confirm the version upgrade I used

root@xyz:~# cat /usr/local/psa/version
12.0.18 Ubuntu 10.04 1200140821.14​

But unfortunately the Plesk login still doesnt work. When I open "https://domain:8443/" the browser is forwarded to "https://xyz.stratoserver.net:8443/relay" and only a short message is displayed: "File not found."
I checked the "/var/log/sw-cp-server/error_log" and found:

2014/12/11 10:08:15 [error] 22425#0: *25 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: [ip], server: , request: "POST /relay HTTP/1.1", upstream: "fastcgi://127.0.0.1:19001", host: "xyz.stratoserver.net:8443", referrer: "https://domain:8443/login_up.php3?window_id=&requested_url=https://domain:8443/&return_host="​

I found a thread with the same error here: http://talk.plesk.com/threads/file-not-found-after-upgrade-to-plesk-12.304631/ Unfortunately it doesnt show the solution to the problem.

Could you give us a hint how to get the Plesk Panel working again?

Regards,
Michael

 

[UFHH01]

Hi mivola and TobiasBu,

in some cases, you might experience issues/probelms, where you have to restart the Plesk Control Panel:

service sw-cp-server restart
​

If this didn't help, then the error - log should be investigated again, to see if the same problem still exists, or if other issues caused an error.


In some few cases, the Plesk Control Panel might have hanging processes and you could be forced to kill all sw-cp-server processes by hand. Best choice would then be to perform these commands:

service sw-cp-server stop
killall sw-cp-server
service sw-cp-server start​

 

[mivola]

Hi,
we did restart the sw-cp-server as well as the sw-engine several times (including killall). The error is always the same: "File not found." in the browser and "Primary script unknown" in the log file.
Any other ideas?
Regards,
Michael

 

[UFHH01]

Hi mivola and TobiasBu,

could you please post your "/etc/sw-engine/pool.d/plesk.conf" ? And did you check, that the port 19001 is not blocked by any firewall?

 

[mivola]

How can I properly test if the port is blocked by a firewall? I tried "telnet localhost 19001" and got a connection.

Here is the content of plesk.conf:

[plesk]
user = psaadm
group = psaadm

listen = /var/run/sw-engine.sock
listen.owner = sw-cp-server
listen.group = root
listen.mode = 0600

pm = ondemand
pm.start_servers = 0
;pm.min_spare_servers = 2
;pm.max_spare_servers = 8
pm.max_children = 26
pm.process_idle_timeout = 30s
;pm.max_requests = 10

;slowlog = /var/log/sw-cp-server/slow.log
;request_slowlog_timeout = 10s
;catch_workers_output = yes
request_terminate_timeout = 600s

security.limit_extensions = .php .php3

env[PATH] = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

php_value[auto_prepend_file] = auth.php
php_value[error_log] = /var/log/plesk/panel.log​

 

[UFHH01]

Hi mivola and TobiasBu,

as you can see, you have a misconfiguration here. The FastCGI - process is configured for "listen = /var/run/sw-engine.sock" and your error - message reports, that "fastcgi://127.0.0.1:19001" is used.
Please check "/etc/sw-cp-server/conf.d/plesk.inc" and change it to the correct way. Afterwards, please restart all with "service psa restart"

 

[mivola]

Hi,
sorry, but I'm not sure what I would need to change. I dont see any references to "fastcgi://127.0.0.1:19001". Here is the plesk.inc:

underscores_in_headers on;
access_log /var/log/plesk/httpsd_access_log main buffer=8k;

index index.php;
root /opt/psa/admin/htdocs;

# Some rewrites
rewrite ^/sitepreview/(.*)$ /sitepreview.php/$1 last;
rewrite ^/sso/ui$ /sso/ui.php last;
rewrite ^/sso/users$ /sso/users.php last;
rewrite ^/sso/signout$ /sso/signout.php last;

# Admin panel
location /admin {
alias /opt/psa/admin/application/admin/public;
try_files $uri @route;
}

# SMB panel
location /smb {
alias /opt/psa/admin/application/smb/public;
try_files $uri @route;
}

# Rest interface
location /rest {
alias /opt/psa/admin/application/rest/public;
try_files $uri @route;
}

location @route {
include /etc/sw-cp-server/fastcgi_params;
fastcgi_pass unix:/var/run/sw-engine.sock;
fastcgi_param PATH_INFO /admin/$fastcgi_path_info;
fastcgi_param SCRIPT_FILENAME /opt/psa/admin/htdocs/application.php;
fastcgi_param HTTP_PRAGMA $http_pragma;
}

# Plesk service
location ~* ^/services/([^/]+) {
set $service_name $1;
alias /opt/psa/admin/services/$service_name/public;
try_files $uri @service;
}

location @service {
include /etc/sw-cp-server/fastcgi_params;
fastcgi_pass unix:/var/run/sw-engine.sock;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param SCRIPT_FILENAME /opt/psa/admin/services/$service_name/public/index.php;
}

# Plesk (old screens)
location /plesk {
try_files $uri /plesk.php?$args;
}

# Custom skins
location /theme-skins {
try_files $uri @default-skin;
}

location @default-skin {
rewrite ^/theme-skins/[^/]+/(.*)$ /theme/$1 last;
}

# For backup download via X-Accel-Redirect
location /tmp/ {
internal;
alias /tmp/;
}

location ~* \.(php|php3)(/|$) {
include /etc/sw-cp-server/fastcgi_params;
fastcgi_pass unix:/var/run/sw-engine.sock;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}​

 

[UFHH01]

Hi mivola and TobiasBu,

please check with "lsof -i :19001" who/what is listening on that port, just in case you have to kill this pid.
Afterwards, please still restart the Plesk services with "service psa restart" ( did you make any complete restart for all services before or even a complete restart of the server after you completely upgraded? ). Try your Plesk Panel URL and investigate again possible errors at "/var/log/sw-cp-server/error_log".

Consider changing the line ";error_log = /var/log/sw-cp-server/sw-engine.log" without a ";" at "/etc/sw-engine/sw-engine-fpm.conf" so that you might have an error - log for FastCGI as well.

 

[mivola]

OK, checked with lsof what is listening on port 19001:

root@xyz:~# lsof -i :19001
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sw-engine 24005 root 8u IPv4 619974316 0t0 TCP localhost.localdomain:19001 (LISTEN)​

I stopped that process with "service sw-engine stop" and afterwards no process was listening on port 19001. I stopped and started psa:

root@xyz:~# lsof -i :19001
root@xyz:~# service psa start
Starting psa... * Starting Plesk engine pool manager sw-engine-fpm [ OK ]
done
Starting xinetd service... done
Starting sw-cp-server service... done
Starting mysql service... done
Starting bind9 service... done
Starting postgresql service... not installed
Starting spamassassin service... done
Plesk: Starting Mail Server... already started
Starting psa... * Starting Plesk engine pool manager sw-engine-fpm [ OK ]
done​

BTW: there are two processes running for sw-engine:

root@xyz:~# ps faux | grep sw-engine
root 24180 0.0 0.0 3292 768 pts/0 S+ 19:39 0:00 \_ grep --color=auto sw-engine
root 23963 0.0 1.0 71240 22544 ? Ss 19:34 0:00 /usr/bin/sw-engine -c /opt/psa/admin/conf/php.ini /opt/psa/admin/bin/modules/watchdog/wdcollect -c /opt/psa/etc/modules/watchdog/wdcollect.inc.php
root 24005 0.0 0.3 173072 7208 ? Ss 19:34 0:00 sw-engine-fpm: master process (/etc/sw-engine/sw-engine-fpm.conf)​

I kill the process will "killall sw-engine", but the process is restarted right away (with a different pid)... Maybe the other one is causing the problem?

I changed the sw-engine-fpm.conf and now we have the following entries in the log file when we try to open the Plesk Panel:

[12-Dec-2014 19:44:09.617000] DEBUG: pid 24005, fpm_event_loop(), line 419: event module triggered 1 events
[12-Dec-2014 19:44:09.617045] WARNING: pid 24005, fpm_stdio_child_said(), line 167: [pool sso] child 24271 said into stderr: "ERROR: fpm_php_apply_defines(), line 123: Unable to set php_value 'apc.stat'"
[12-Dec-2014 19:44:09.617066] DEBUG: pid 24005, fpm_event_loop(), line 419: event module triggered 1 events
[12-Dec-2014 19:44:09.618520] WARNING: pid 24005, fpm_stdio_child_said(), line 167: [pool sso] child 24271 said into stderr: "DEBUG: main(), line 1891: Primary script unknown"
[12-Dec-2014 19:44:09.618559] DEBUG: pid 24005, fpm_event_loop(), line 419: event module triggered 1 events
[12-Dec-2014 19:44:09.818826] DEBUG: pid 24005, fpm_pctl_perform_idle_server_maintenance(), line 363: [pool sso] currently 0 active children, 1 spare children
[12-Dec-2014 19:44:09.818910] DEBUG: pid 24005, fpm_pctl_perform_idle_server_maintenance(), line 363: [pool plesk] currently 0 active children, 1 spare children​

We did a full machine restart right after the update.

Regards,
Michael

 

[UFHH01]

Hi mivola and TobiasBu,

please disable the Plesk SSO service: /usr/local/psa/bin/sso --disable

... and restart psa again. Make sure, that the Single-Sign-On process ( sso ) is no longer available after you disabled it and check again with ps.

 

[mivola]

Hi,
now we got it! Disabling SSO solved the problem. After "/usr/local/psa/bin/sso --disable" and "service psa restart", I'm now able to login into the Plesk Panel We'll now check if everything within Plesk works as expected.
Thank you very much for your excellent support!!
BTW: do you know which is the best/preferred way to upgrade from Ubuntu 10.04.4 to the latest version?
Michael