Issue Cannot access Plesk Control Panel 12.5 (504 Gateway Time-out nginx)

[pisanu]

Hi,

I have this problem that this morning.
After using Plesk without problem for years, suddenly I cannot access to Plesk 12.5 Admin panel!!!!
When I type the address "https://mydomain.com:8443/" then it redirected to
"https://mydomain.com:8443/login.php3?success_redirect_url=https://mydomain.com:8443/"
then it just hang there and after a while, the broweser shows

504 Gateway Time-out
nginx

I went to " /var/log/sw-cp-server/error_log"

there are messages below;

2018/09/17 15:14:13 [error] 960#0: *319 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 101.51.88.149, server: , request: "GET /login.php3?success_redirect_url=https%3A%2F%2Fmydomain.com%3A8443%2F HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock", host: "mydomain.com:8443"
2018/09/17 20:42:09 [error] 960#0: *329 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 125.25.205.7, server: , request: "GET /login.php3?success_redirect_url=https%3A%2F%2Fmydomain.com%3A8443%2F HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock", host: "mydomain.com:8443", referrer: "https://mydomain.com:8443/"
2018/09/18 08:35:18 [error] 960#0: *379 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 101.51.88.149, server: , request: "GET /login.php3?success_redirect_url=https%3A%2F%2Fmydomain.com%3A8443%2Fadmin%2Fhealth%2Findex%3Fcontext%3Dhealth HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock", host: "mydomain.com:8443", referrer: "https://mydomain.com:8443/admin/health/index?context=health"​

2018/09/18 08:45:53 [crit] 960#0: *388 SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking, client: 74.82.47.5, server: 0.0.0.0:8443
2018/09/18 08:46:03 [crit] 960#0: *389 SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking, client: 74.82.47.5, server: 0.0.0.0:8443
2018/09/18 14:56:42 [crit] 960#0: *414 SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking, client: 125.64.94.200, server: 0.0.0.0:8443​

2018/09/18 17:13:08 [error] 960#0: *420 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 125.25.132.114, server: , request: "GET /login.php3?success_redirect_url=https%3A%2F%2Fmydomain.com%3A8443%2F HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock", host: "mydomain.com:8443"
2018/09/18 19:14:55 [error] 960#0: *431 open() "/usr/local/psa/admin/htdocs/robots.txt" failed (2: No such file or directory), client: 66.249.79.27, server: , request: "GET /robots.txt HTTP/1.1", host: "mydomain.com:8443"
2018/09/18 21:32:36 [error] 960#0: *443 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 125.25.132.114, server: , request: "GET /login.php3?success_redirect_url=https%3A%2F%2Fmydomain.com%3A8443%2F HTTP/1.1", upstream: "fastcgi://unix:/var/run/sw-engine.sock", host: "mydomain.com:8443", referrer: "https://mydomain.com:8443/"​

I tried " # plesk repair installation" > after enter the command, it just froze there not going further.

I tried " # plesk repair all " > This also the same as it just freeze nothing load.

I went check at " /var/log/plesk/panel.log "

There are message below;

[2018-09-18 09:05:06] ERR [extension/website-virus-check] Failed to request GET https://www.virustotal.com/vtapi/v2/url/scanRead timed out after 10 seconds
[2018-09-18 21:08:49] ERR [1] /usr/local/psa/admin/bin/filemng 'root' 'cat' '/root/.autoinstaller/microupdates.xml' '--allow-root' failed with message: filemng: Error occurred during /bin/cat command.
[2018-09-18 21:11:04] ERR [1] /usr/local/psa/admin/bin/filemng 'root' 'cat' '/root/.autoinstaller/microupdates.xml' '--allow-root' failed with message: filemng: Error occurred during /bin/cat command.​

I also tried "
# /usr/local/psa/admin/sbin/autoinstaller --select-product-id plesk --select-release-current --reinstall-patch --install-component panel "

The files started downloading and stop at

gpg-pubkey-d38b4796-570c8cd3 gpg(Google Inc. (Linux Packages Signing Authority) <[email protected]>)
gpg-pubkey-325a043e-5113d72f gpg(Linode, LLC <[email protected]>)
gpg-pubkey-352c64e5-52ae6884 gpg(Fedora EPEL (7) <[email protected]>)

(as attached image)


Please help.

 

[IgorG]

Make sure that you have enough free diskspace and inodes on your partitions.

 

[pisanu]

Hi Igor,

Thank you very much for reply.
I have more than enough space on Linode.
Now we use 27.33 GB / 156.66 GB.

Thanks,
Pisanu

 

[IgorG]

Try to run in CLI command:

# /usr/local/psa/admin/bin/filemng 'root' 'cat' '/root/.autoinstaller/microupdates.xml' '--allow-root'

What is the output?

 

[pisanu]

Hi ,
Out put below;

cat: /root/.autoinstaller/microupdates.xml: No such file or directory
filemng: Error occurred during /bin/cat command.
filemng: Error occurred during /bin/cat command.

 

[IgorG]

On my test Plesk 12.5 server I see:

# /usr/local/psa/admin/bin/filemng 'root' 'cat' '/root/.autoinstaller/microupdates.xml' '--allow-root'
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<patches>
<product id="plesk" version="12.5.30" installed-at="20151005T145904">
<patch version="76" timestamp="1525348443" installed-at="20180919T033857" />
</product>
</patches>

Try to create this missing file with this content at least. Or try to run

# ls -la /root/.autoinstaller/

maybe file microupdates.xml is renamed there for some reason?

 

[pisanu]

I created the file " microupdates.xml " and put it in " /root/.autoinstaller/"

Content in microupdates.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<patches>
<product id="plesk" version="12.5.30" installed-at="20151005T145904">
<patch version="76" timestamp="1525348443" installed-at="20180919T033857" />
</product>
</patches>​

then

# ls -la /root/.autoinstaller/

total 16
drwxr-xr-x 2 root root 4096 Sep 19 10:50 .
dr-xr-x---. 10 root root 4096 Sep 18 22:16 ..
-rw------- 1 root root 2835 Sep 13 11:00 autoinstaller.pem
-rw-r--r-- 1 root root 235 Sep 19 10:50 microupdates.xml

 

[IgorG]

Now try to run

# plesk installer --select-release-current --reinstall-patch --upgrade-installed-components
# plesk repair installation

 

[pisanu]

with

# plesk installer --select-release-current --reinstall-patch --upgrade-installed-components

The files start downloading then everything just suddenly stop at > gpg-pubkey-352c64e5-52ae6884 gpg(Fedora EPEL (7) <[email protected]>) (image attached)

Downloading file PHP_5.2.17/php52-5.2.17-cos7-x86_64.inf3: 100% was finished.
Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 0%
Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 100% was finished.
gpg-pubkey-7fac5991-4615767f gpg(Google, Inc. Linux Package Signing Key <[email protected]>)
gpg-pubkey-f4a80eb5-53a7ff4b gpg(CentOS-7 Key (CentOS 7 Official Signing Key) <[email protected]>)
gpg-pubkey-914bdf7e-55c05220 gpg(Plesk Team <[email protected]>)
gpg-pubkey-11f63c51-3c7dc11d gpg(Jamie Cameron <[email protected]>)
gpg-pubkey-5ebd2744-418ffac9 gpg(Atomic Rocket Turtle <[email protected]>)
gpg-pubkey-4520afa9-50ab914c gpg(Atomicorp (Atomicorp Official Signing Key) <[email protected]>)
gpg-pubkey-1bb943db-511147a9 gpg(MariaDB Package Signing Key <[email protected]>)
gpg-pubkey-d38b4796-570c8cd3 gpg(Google Inc. (Linux Packages Signing Authority) <[email protected]>)
gpg-pubkey-325a043e-5113d72f gpg(Linode, LLC <[email protected]>)
gpg-pubkey-352c64e5-52ae6884 gpg(Fedora EPEL (7) <[email protected]>)

(image attached)

 

[IgorG]

Any related details in /tmp/autoinstaller3.log ?

 

[pisanu]

> Some of lower part (Image attached)

[2018-09-19 11:00:58.057998] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.058140] Warning: ez_template_suffix is NOT defined in NGINX_1.11.1/nginx-1.11.1-cos7-x86_64.plesk12.inf3
[2018-09-19 11:00:58.058378] Attach addon PHP_7.0.29 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.058412] FileFetcher: get file (~empty)/PHP_7.0.29/php70-7.0.29-cos7-x86_64.inf3
[2018-09-19 11:00:58.058473] Downloading file PHP_7.0.29/php70-7.0.29-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.127618] Downloading file PHP_7.0.29/php70-7.0.29-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.128949] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.129131] Use package source http://autoinstall.plesk.com/PHP_7.0.29/dist-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.129171] Use package source http://autoinstall.plesk.com/PHP_7.0.29/update-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.129197] Use package source http://autoinstall.plesk.com/PHP_7.0.29/thirdparty-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.129228] Warning: ez_template_suffix is NOT defined in PHP_7.0.29/php70-7.0.29-cos7-x86_64.inf3
[2018-09-19 11:00:58.129377] Attach addon PHP_5.6.35 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.129407] FileFetcher: get file (~empty)/PHP_5.6.35/php56-5.6.35-cos7-x86_64.inf3
[2018-09-19 11:00:58.129482] Downloading file PHP_5.6.35/php56-5.6.35-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.199257] Downloading file PHP_5.6.35/php56-5.6.35-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.200880] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.201081] Use package source http://autoinstall.plesk.com/PHP_5.6.35/dist-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.201112] Use package source http://autoinstall.plesk.com/PHP_5.6.35/update-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.201134] Use package source http://autoinstall.plesk.com/PHP_5.6.35/thirdparty-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.201153] Warning: ez_template_suffix is NOT defined in PHP_5.6.35/php56-5.6.35-cos7-x86_64.inf3
[2018-09-19 11:00:58.201287] Attach addon PHP_5.5.38 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.201316] FileFetcher: get file (~empty)/PHP_5.5.38/php55-5.5.38-cos7-x86_64.inf3
[2018-09-19 11:00:58.201401] Downloading file PHP_5.5.38/php55-5.5.38-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.270558] Downloading file PHP_5.5.38/php55-5.5.38-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.271551] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.271714] Use package source http://autoinstall.plesk.com/PHP_5.5.38/dist-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.271744] Use package source http://autoinstall.plesk.com/PHP_5.5.38/update-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.271765] Use package source http://autoinstall.plesk.com/PHP_5.5.38/thirdparty-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.271785] Warning: ez_template_suffix is NOT defined in PHP_5.5.38/php55-5.5.38-cos7-x86_64.inf3
[2018-09-19 11:00:58.271935] Attach addon PHP_5.4.45 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.271977] FileFetcher: get file (~empty)/PHP_5.4.45/php54-5.4.45-cos7-x86_64.inf3
[2018-09-19 11:00:58.272049] Downloading file PHP_5.4.45/php54-5.4.45-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.341145] Downloading file PHP_5.4.45/php54-5.4.45-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.342427] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.342604] Use package source Index of /PHP_5.4.45/dist-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.342634] Use package source Index of /PHP_5.4.45/update-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.342658] Use package source Index of /PHP_5.4.45/thirdparty-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.342678] Warning: ez_template_suffix is NOT defined in PHP_5.4.45/php54-5.4.45-cos7-x86_64.inf3
[2018-09-19 11:00:58.342813] Attach addon PHP_5.3.29 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.342842] FileFetcher: get file (~empty)/PHP_5.3.29/php53-5.3.29-cos7-x86_64.inf3
[2018-09-19 11:00:58.342938] Downloading file PHP_5.3.29/php53-5.3.29-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.411955] Downloading file PHP_5.3.29/php53-5.3.29-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.413158] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.413376] Use package source Index of /PHP_5.3.29/dist-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.413429] Use package source Index of /PHP_5.3.29/update-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.413452] Use package source Index of /PHP_5.3.29/thirdparty-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.413472] Warning: ez_template_suffix is NOT defined in PHP_5.3.29/php53-5.3.29-cos7-x86_64.inf3
[2018-09-19 11:00:58.413594] Attach addon PHP_5.2.17 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.413620] FileFetcher: get file (~empty)/PHP_5.2.17/php52-5.2.17-cos7-x86_64.inf3
[2018-09-19 11:00:58.413673] Downloading file PHP_5.2.17/php52-5.2.17-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.483979] Downloading file PHP_5.2.17/php52-5.2.17-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.485303] Warning: no 'diskspace' section in config
[2018-09-19 11:00:58.485498] Use package source Index of /PHP_5.2.17/dist-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.485531] Use package source Index of /PHP_5.2.17/update-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.485555] Use package source Index of /PHP_5.2.17/thirdparty-rpm-CentOS-7-x86_64
[2018-09-19 11:00:58.485576] Warning: ez_template_suffix is NOT defined in PHP_5.2.17/php52-5.2.17-cos7-x86_64.inf3
[2018-09-19 11:00:58.485803] Attach addon PMM_0.1.10 to release Plesk 12.5.30 (Stable)
[2018-09-19 11:00:58.485837] FileFetcher: get file (~empty)/PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3
[2018-09-19 11:00:58.485924] Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 0%
[2018-09-19 11:00:58.554571] Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 100% was finished.
[2018-09-19 11:00:58.555464] Use package source http://autoinstall.plesk.com/PMM_0.1.10/dist-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.555506] Use package source http://autoinstall.plesk.com/PMM_0.1.10/update-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.555530] Use package source http://autoinstall.plesk.com/PMM_0.1.10/thirdparty-rpm-CentOS-7-x86_64/
[2018-09-19 11:00:58.555549] Warning: ez_template_suffix is NOT defined in PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3
[2018-09-19 11:00:58.555707] BuildLoader: 1 releases processed
[2018-09-19 11:00:58.555776] Notice: seems yum is ready to use.
[2018-09-19 11:00:58.810363] gpg-pubkey-7fac5991-4615767f gpg(Google, Inc. Linux Package Signing Key <[email protected]>)
gpg-pubkey-f4a80eb5-53a7ff4b gpg(CentOS-7 Key (CentOS 7 Official Signing Key) <[email protected]>)
gpg-pubkey-914bdf7e-55c05220 gpg(Plesk Team <[email protected]>)
gpg-pubkey-11f63c51-3c7dc11d gpg(Jamie Cameron <[email protected]>)
gpg-pubkey-5ebd2744-418ffac9 gpg(Atomic Rocket Turtle <[email protected]>)
gpg-pubkey-4520afa9-50ab914c gpg(Atomicorp (Atomicorp Official Signing Key) <[email protected]>)
gpg-pubkey-1bb943db-511147a9 gpg(MariaDB Package Signing Key <[email protected]>)
gpg-pubkey-d38b4796-570c8cd3 gpg(Google Inc. (Linux Packages Signing Authority) <[email protected]>)
gpg-pubkey-325a043e-5113d72f gpg(Linode, LLC <[email protected]>)
gpg-pubkey-352c64e5-52ae6884 gpg(Fedora EPEL (7) <[email protected]>)
repository: Get brief packages info for all sources
[2018-09-19 11:00:59.943823] Checking for installed in extension catalog packages

 

[pisanu]

Hi Igor,

Suddenly now, all web sites show " Web Server's Default Page "!!!!

 

[m3lezZ]

Hi Igor,

Suddenly now, all web sites show " Web Server's Default Page "!!!!

plesk repair all -y

or for single domain:

/usr/local/psa/admin/sbin/httpdmng --reconfigure-domain example.com

 

[pisanu]

Hi 3lezZ

I did above
#plesk repair all -y
#/usr/local/psa/admin/sbin/httpdmng --reconfigure-domain mydomain.com

After enter both commands, they don't execute.
Every just hangs there freeze.

 

[m3lezZ]

Hi 3lezZ

I did above
#plesk repair all -y
#/usr/local/psa/admin/sbin/httpdmng --reconfigure-domain mydomain.com

After enter both commands, they don't execute.
Every just hangs there freeze.

Try repair for web and just a single domain for testing:

plesk repair web yourdomain.com -y

It would still be interesting, how busy your system is. Open a second shell and watch with top or htop after your load.

 

[pisanu]

Hi 3lezZ

I did that too .. but I did again.
Right now, all "plesk repair" functions can't run any out put.

After enter both commands, they don't execute.
Every just hangs there freeze.

 

[m3lezZ]

Are there multible lockfiles?

find /var/run/lock/lmlib/container_locks

As a workaround, remove the lockfiles:

# rm -f /usr/local/psa/var/cache/SharedLockManagerStorage*

Repair plesk-installation:

plesk repair installation

 

[pisanu]

# find /var/run/lock/lmlib/container_locks

find: ‘/var/run/lock/lmlib/container_locks’: No such file or directory

 

[pisanu]

Hi 3lezZ

I did
# rm -f /usr/local/psa/var/cache/SharedLockManagerStorage*

Now now I can run

#plesk repair installation

Let's see how it goes, will inform after it's done

 

[pisanu]

Hi 3lezZ

• Now all sites back to normal.
• I can also access to Plesk Admin

Do I need to perform

#plesk repair all -y

Again?