Plesk 11.5 inoperable after update to MU #41 - URGENT

[Evilmachine]

Hi there,

my Plesk is completely inoperable after auto installing MU41 that nicht.
I cannot reconfigure my V-Hosts. I get the message

Konfigurieren des Webservers nicht möglich:Execution failed. Command: httpdmng Arguments: Array ( [0] => --reconfigure-all ) Details: Execution failed. Command: httpdmng Arguments: Array ( [0] => --reconfigure-domains [1] => ackermann-kosmetik.de,foerderverein-nbs.de,nazarius-familienzentrum.de,triangle-it.de,triyang.de,sascha-urbansky.de,bazingo.de,dj-urban-sky.de,ectomorph.bazingo.de,ers.sascha-urbansky.de,gebaeudereinigung-engel.com,bsautermeister.de,julia-linke.de,julialinke.de,marcus-lorenz.de,marloblo.de,mperia.de,ideenhoch.de,ideen-fluss.de,ideenhoch.com,gutschein-punkte.de,gutscheinpunkte.de,mv-br.de,conkarma.de,ackermann-kurse.de,forgotton.de,heiligebirma-familienkatze.de,hotel-kastanienhof.de,naturheilverein-weserbergland.de,sagaofpandoria.org,internal.sagaofpandoria.org,testing.sagaofpandoria.org,rugenstein.de,schoening-lahr.de,uwe.schoening-lahr.de,wum.schoening-lahr.de,secretofmana.de,jamie.secretofmana.de,test.secretofmana.de,waytodawn.de,beta.waytodawn.de,shop.bazingo.de,team.bazingo.de,utemuehlbauer.de,villa-nicole.de,vom-birkenweg.de,sam4schulung.de,jule64.de,bazingo.suit-server.de,hotel-garni-springe.de,jrk-aerzen.de,rosa.spd-aerzen.de ) Details: Empty error message from utility.

If i reconfigure with CLI its the same. If i just configure one domain the tools says segmentation fault.

And in the Plesk Panel right after Login i get only 502 Nginx Bad Gateway errors.
For the main page and the navigation frame.

What is goging on here??

I get many many kernel failures from the plesk tools

Apr 14 16:11:14 bazingo kernel: [ 76.438421] postgres (3900): /proc/3900/oom_adj is deprecated, please use /proc/3900/oom_score_adj instead.
Apr 14 16:12:32 bazingo kernel: [ 154.374732] sw-engine-fpm[4844] general protection ip:7f5d6f80e424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f7df000+3c000]
Apr 14 16:12:48 bazingo kernel: [ 170.140453] sw-engine-fpm[4953] general protection ip:7f5d6f68f424 sp:7fffc5f1f5d0 error:0 in libmyodbc.so[7f5d6f660000+3c000]
Apr 14 16:12:59 bazingo kernel: [ 181.377353] sw-engine[5016]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fffa6cd9c08 error 14
Apr 14 16:13:15 bazingo kernel: [ 197.156897] sw-engine[5228]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fffa47902c8 error 14
Apr 14 16:13:34 bazingo kernel: [ 216.172456] sw-engine-fpm[5449] general protection ip:7f5d6f5cc424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f59d000+3c000]
Apr 14 16:13:40 bazingo kernel: [ 222.646455] sw-engine[6102]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff4be87958 error 14
Apr 14 16:13:48 bazingo kernel: [ 229.849393] sw-engine-fpm[5453] general protection ip:7f5d6f712424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f6e3000+3c000]
Apr 14 16:14:36 bazingo kernel: [ 278.234998] sw-engine-fpm[6223] general protection ip:7f5d6f5cc424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f59d000+3c000]
Apr 14 16:14:40 bazingo kernel: [ 282.196179] sw-engine-fpm[6222] general protection ip:7f5d6f7cf424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f7a0000+3c000]
Apr 14 16:14:48 bazingo kernel: [ 289.717438] sw-engine-fpm[6234] general protection ip:7f5d6f5cc424 sp:7fffc5f21860 error:0 in libmyodbc.so[7f5d6f59d000+3c000]
Apr 14 16:17:34 bazingo kernel: [ 456.350684] sw-engine-fpm[6831] general protection ip:7f4f339b0424 sp:7fff63906f00 error:0 in libmyodbc.so[7f4f33981000+3c000]
Apr 14 16:17:38 bazingo kernel: [ 460.017654] sw-engine-fpm[6834] general protection ip:7f4f339af424 sp:7fff63904c70 error:0 in libmyodbc.so[7f4f33980000+3c000]
Apr 14 16:18:03 bazingo kernel: [ 484.867181] sw-engine-fpm[6832] general protection ip:7f4f33da2424 sp:7fff63906f00 error:0 in libmyodbc.so[7f4f33d73000+3c000]
Apr 14 16:18:10 bazingo kernel: [ 491.889946] sw-engine[7505]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff5321a9f8 error 14
Apr 14 16:20:53 bazingo mpt-statusd: detected non-optimal RAID status
Apr 14 16:26:29 bazingo kernel: [ 991.139104] sw-engine-fpm[8901] general protection ip:7f485f691424 sp:7fff504695d0 error:0 in libmyodbc.so[7f485f662000+3c000]
Apr 14 16:26:34 bazingo kernel: [ 995.603524] sw-engine-fpm[8861] general protection ip:7f485fa81424 sp:7fff50467340 error:0 in libmyodbc.so[7f485fa52000+3c000]
Apr 14 16:27:00 bazingo kernel: [ 1021.442634] sw-engine[9557]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fffb3c6f0e8 error 14
Apr 14 16:27:56 bazingo kernel: [ 1077.254367] sw-engine-fpm[9698] general protection ip:7f485f691424 sp:7fff504695d0 error:0 in libmyodbc.so[7f485f662000+3c000]
Apr 14 16:28:05 bazingo kernel: [ 1086.769054] sw-engine[10349]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff1fbb03a8 error 14
Apr 14 16:30:30 bazingo kernel: [ 1231.913865] sw-engine[10995]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff5b149a98 error 14
Apr 14 16:30:53 bazingo mpt-statusd: detected non-optimal RAID status
Apr 14 16:32:30 bazingo kernel: [ 1352.013053] sw-engine[11025]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff254c2218 error 14
Apr 14 16:35:16 bazingo kernel: [ 1517.483983] sw-engine[12075]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fffe0d94208 error 14
Apr 14 16:36:37 bazingo kernel: [ 1598.299802] sw-engine[12145]: segfault at b48080e41 ip 0000000b48080e41 sp 00007fff21f757f8 error 14

 

[IgorG]

It is really not related to MU#41. There were very little changes. It looks like that you have hardware problems (RAM or harddisk) or just lack of resources.

 

[Evilmachine]

Good Morning Igor,

no everything seems to be finde

root@bazingo:~# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 1.5T 465G 915G 34% /
udev 10M 0 10M 0% /dev
tmpfs 1.2G 1.1M 1.2G 1% /run
/dev/disk/by-uuid/efdca91d-7112-49da-bffe-ca2e7864fa54 1.5T 465G 915G 34% /
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 7.2G 0 7.2G 0% /run/shm
root@bazingo:~# free
total used free shared buffers cached
Mem: 12336768 12166264 170504 0 601356 10069236
-/+ buffers/cache: 1495672 10841096
Swap: 25162748 0 25162748

 

[IgorG]

What about this:

detected non-optimal RAID status

 

[Evilmachine]

The Server has just one Harddrive. So there is no raid installed.
Everything on the server works fine except all that has to do with Plesk itself.

Apache is working, postfix is working, courier is working. Only the Plesk tools and the plesk nginx not.

 

[KristianM]

Hi,

could you however please still check your hard-drive with smartmontools?
Have you tried rebooting the node?

If this does not help and you are sure it's not hardware related, I'd suggest to contact Parallels support.

Regards,
Kristian

 

[Evilmachine]

Hi,

could you however please still check your hard-drive with smartmontools?
Have you tried rebooting the node?

If this does not help and you are sure it's not hardware related, I'd suggest to contact Parallels support.

Regards,
Kristian

Yes i did a reboot on the host twice.
The main system is an ESXi Host with 3 Servers. 1 is offline and 2 are running.
Both have Plesk installed. Server 1 is working, Server 2 not.
Server 1 hast MU40 and Server 2 MU41.

Only the sw-engine-pleskrun seems to crash.

I can login to panel, i can access many sites.
But the navigation says 502 bad gateway.

So there is something wrong with the php of plesk.

 

[KristianM]

Hi,

which Distribution are you using?
Your segmentation fault error reside somewhere in the memory management.

"error 14" means:
The cause was a user-mode write resulting in no page being found. A reserved bit was detected.

Do you have SELinux enabled by any chance?

Regards,
Kristian

 

[Evilmachine]

Hi,

i am using Debian 7.4
Selinux seems to be deactivated.

I have no /etc/selinux/config

and no commands like sestatus and setenforce on my distri.

My error log of sw-cp-server shows this

2014/04/15 10:26:22 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /plesk/server/statistics/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"
2014/04/15 10:26:24 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /admin/server/featured-applications HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"
2014/04/15 10:26:29 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /plesk/server/services/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"
2014/04/15 10:26:31 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /admin/server/featured-applications HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"
2014/04/15 10:26:45 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /plesk/license/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"
2014/04/15 10:26:49 [error] 27822#0: *7 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 217.6.112.194, server: , request: "GET /plesk/license/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock:", host: "bazingo.suit-server.de:8443", referrer: "https://bazingo.suit-server.de:8443/admin/server/tools/"

I cannot contact support cuase i have no code.
This is definately a plesk issue. Not a configuration problem i caused.

There were no changes during that night exept your update.
And in the days before that the only change was the heartbleed bugfix.

Any other service on the server is running.
Only Plesk itself has problems.

All relevant Services (postfix, mysql, apache, php, courier etc) are running.
If a want to restart nginx is says it is disabled in the config.

So if it were a resource problem caused by the server itself, everything would be issued not only plesk.
This is a huge financial damage for me. Every Website on the server is not running cause of missing config files that cannot be generated.

 

[KristianM]

Hi,

well as you said everything else is working, like Apache etc.
I've checked a couple of your domains and can not confirm?

All I see is white pages.

As you might see in the KB of Parallels (http://kb.parallels.com/en/121007) no changes to PHP or anything critical was changes to
result in such an impact.
Only two files where patched:
PSA_11.5.30/microupdates/MU41/common/plib/CommonPanel/Config/Defaults.php
PSA_11.5.30/microupdates/MU41/common/plib/PartnerCentral/Customizations.php

When you rebooted, did you reboot the VM or did you really reboot the Node?

Maybe you should check your /var/log/messages and /var/log/syslog when these errors started to occur.

As I've said before. If you are really sure this is MU41 related, open a support Ticket on the Parallels Website.

Regards,
Kristian

 

[Evilmachine]

I cannot create a tickert, cause i have no code.

Yes of course apache shows white pages because of Plesk not beeing able to generate the necessary config files.
That is why it is so urgent.

And yes i did reboot the node.

 

[KristianM]

Hi,

in this case you will have to contact your License provider and request assistance from them.

The Apache2 config seems to be fine though. What's the output of a simple Apache restart?
What do you see in /var/log/apache2/error.log?

Regards,
Kristian

 

[Evilmachine]

Seems fine??
Are you kidding???


The whole vhost directory of apache conf is empty

root@bazingo:/etc/apache2/plesk.conf.d/vhosts# ls -l
total 0


The only thing the error log shows are file does not exist entries
[Tue Apr 15 10:55:08 2014] [error] [client 115.38.195.118] File does not exist: /var/www/vhosts/default/htdocs/spacepixx
[Tue Apr 15 10:56:24 2014] [error] [client 195.167.212.194] File does not exist: /var/www/vhosts/default/htdocs/v5
[Tue Apr 15 10:56:26 2014] [error] [client 195.167.212.194] File does not exist: /var/www/vhosts/default/htdocs/v5
[Tue Apr 15 10:59:04 2014] [error] [client 66.249.68.24] File does not exist: /var/www/vhosts/default/htdocs/empfehlungen
[Tue Apr 15 10:59:06 2014] [error] [client 66.249.68.22] File does not exist: /var/www/vhosts/default/htdocs/shops
[Tue Apr 15 10:59:42 2014] [error] [client 77.119.129.64] File does not exist: /var/www/vhosts/default/htdocs/spacescribble, referer: file:///Applications/Install/71FC4A5B-DE12-4B28-88EC-8AC573CE9708/Install/
[Tue Apr 15 11:02:15 2014] [error] [client 37.187.88.119] File does not exist: /var/www/vhosts/default/htdocs/gb, referer: http://www.jule64.de/
[Tue Apr 15 11:02:56 2014] [error] [client 5.10.83.53] File does not exist: /var/www/vhosts/default/htdocs/empfehlungen
[Tue Apr 15 11:03:11 2014] [error] [client 208.115.113.88] File does not exist: /var/www/vhosts/default/htdocs/board
[Tue Apr 15 11:03:12 2014] [error] [client 5.255.91.71] File does not exist: /var/www/vhosts/default/htdocs/v5, referer: http://www.secretofmana.de/

Because everything points to the default plesk page that is indeed white because we wanted so.

 

[KristianM]

Hi,

well I'm not kidding. Your Apache responds. I can't see what's on your filesystem.

Try replacing the white page with something like a phpinfo().
Does that work?

What is the output of the error log while you restart apache?
It should be logged in there somehow.

Please do note, that I am no Parallels employee. I am helping you in my free time as a volunteer.

Regards,
Kristian

 

[Evilmachine]

As you can now see yes that works

http://www.hotel-kastanienhof.de

After restart of apache

[Tue Apr 15 11:08:29 2014] [notice] caught SIGTERM, shutting down
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Apr 15 11:08:30 2014] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Tue Apr 15 11:08:30 2014] [notice] Digest: generating secret for digest authentication ...
[Tue Apr 15 11:08:30 2014] [notice] Digest: done
[Tue Apr 15 11:08:30 2014] [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
[Tue Apr 15 11:08:30 2014] [error] python_init: Python executable found '/usr/bin/python'.
[Tue Apr 15 11:08:30 2014] [error] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
[Tue Apr 15 11:08:30 2014] [notice] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
[Tue Apr 15 11:08:30 2014] [notice] mod_python: using mutex_directory /tmp
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Apr 15 11:08:30 2014] [notice] Apache/2.2.22 (Debian) DAV/2 mod_fcgid/2.3.9 mod_jk/1.2.37 mod_python/3.3.1 Python/2.7.3 mod_ssl/2.2.22 OpenSSL/1.0.1e mod_perl/2.0.7 Perl/v5.14.2 configured -- resuming normal operations

 

[Evilmachine]

As you can now see yes that works

http://www.hotel-kastanienhof.de

After restart of apache

[Tue Apr 15 11:08:29 2014] [notice] caught SIGTERM, shutting down
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Apr 15 11:08:30 2014] [notice] suEXEC mechanism enabled (wrapper: /usr/lib/apache2/suexec)
[Tue Apr 15 11:08:30 2014] [notice] Digest: generating secret for digest authentication ...
[Tue Apr 15 11:08:30 2014] [notice] Digest: done
[Tue Apr 15 11:08:30 2014] [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
[Tue Apr 15 11:08:30 2014] [error] python_init: Python executable found '/usr/bin/python'.
[Tue Apr 15 11:08:30 2014] [error] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
[Tue Apr 15 11:08:30 2014] [notice] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
[Tue Apr 15 11:08:30 2014] [notice] mod_python: using mutex_directory /tmp
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] RSA server certificate CommonName (CN) `Parallels Panel' does NOT match server name!?
[Tue Apr 15 11:08:30 2014] [warn] Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Apr 15 11:08:30 2014] [notice] Apache/2.2.22 (Debian) DAV/2 mod_fcgid/2.3.9 mod_jk/1.2.37 mod_python/3.3.1 Python/2.7.3 mod_ssl/2.2.22 OpenSSL/1.0.1e mod_perl/2.0.7 Perl/v5.14.2 configured -- resuming normal operations

Bei den Tools steht im Panel log übrigens

[15-Apr-2014 11:18:01 Europe/Berlin] PleskUtilException: /opt/psa/admin/bin/httpdmng '--reconfigure-all' failed with message: Error occured while sending feedback. HTTP code returned: 502
Error occured while sending feedback. HTTP code returned: 502
Execution failed.
Command: httpdmng
Arguments: Array
(
[0] => --reconfigure-domains
[1] => ackermann-kosmetik.de,foerderverein-nbs.de,nazarius-familienzentrum.de,triangle-it.de,triyang.de,sascha-urbansky.de,bazingo.de,dj-urban-sky.de,ectomorph.bazingo.de,ers.sascha-urbansky.de,gebaeudereinigung-engel.com,bsautermeister.de,julia-linke.de,julialinke.de,marcus-lorenz.de,marloblo.de,mperia.de,ideenhoch.de,ideen-fluss.de,ideenhoch.com,gutschein-punkte.de,gutscheinpunkte.de,mv-br.de,conkarma.de,ackermann-kurse.de,forgotton.de,heiligebirma-familienkatze.de,hotel-kastanienhof.de,naturheilverein-weserbergland.de,sagaofpandoria.org,internal.sagaofpandoria.org,testing.sagaofpandoria.org,rugenstein.de,schoening-lahr.de,uwe.schoening-lahr.de,wum.schoening-lahr.de,secretofmana.de,jamie.secretofmana.de,test.secretofmana.de,waytodawn.de,beta.waytodawn.de,shop.bazingo.de,team.bazingo.de,utemuehlbauer.de,villa-nicole.de,vom-birkenweg.de,sam4schulung.de,jule64.de,bazingo.suit-server.de,hotel-garni-springe.de,jrk-aerzen.de,rosa.spd-aerzen.de
)

Details: Empty error message from utility.


Here too 502. So the big problem is the plesk internal nginx that is causing all the problems.

 

[KristianM]

Hi again,

as I can see you seem to have fixed the issue.
Would you care to share what was the cause and how you fixed it?

Regards,
Kristian

 

[Evilmachine]

Hi,

yes its fixed thanks to the support.

I had the dotdeb Debian repositories installed.
Plesk did install mysql-5.6 during the autoupdate.
But support says plesk is not compatible to 5.6. Well it worked a while.
But then the mariadb bug was implemented in 5.6 too and then the panel did not work.

The support downgraded the mysql and everything works fine now.

Thanks a lot.