Question [17.5.3] Can't set up CentOS 7 Plesk Onyx mirror properly

[burnley]

Been using the excellent Plesk mirror setup tool from Plesk Mirror Setup Tool v3.0.1 for quite some time for 12.5;30 and it's saved us plenty of time and bandwidth, so we'd like to add 17.5. Unfortunately I've got 2 dead links for NGINX17 and PHP71_17:
ls -l NGINX17 PHP71_17
lrwxrwxrwx 1 mirrorctl mirrorctl 13 Mar 23 17:43 NGINX17 -> NGINX_1.11.10
lrwxrwxrwx 1 mirrorctl mirrorctl 9 Sep 5 15:14 PHP71_17 -> PHP_7.1.9
None of the 2 directories - NGINX_1.11.10 and PHP_7.1.9 - exist, they're not being pulled when syncing.

# /usr/local/psa/admin/bin/autoinstaller --select-release-current --upgrade-installed-components
Doing restart of Plesk Installer...
Downloading file products.inf3: 0%
Downloading file products.inf3: 100% was finished.
Downloading file plesk.inf3: 0%
Downloading file plesk.inf3: 100% was finished.
Downloading file ppsmbe.inf3: 0%
Downloading file ppsmbe.inf3: 100% was finished.
Downloading file sitebuilder.inf3: 0%
Downloading file sitebuilder.inf3: 100% was finished.
Downloading file sso.inf3: 0%
Downloading file sso.inf3: 100% was finished.
Downloading file setemplates.inf3: 0%
Downloading file setemplates.inf3: 100% was finished.
Downloading file pp-sitebuilder.inf3: 0%
Downloading file pp-sitebuilder.inf3: 100% was finished.
Downloading file billing.inf3: 0%
Downloading file billing.inf3: 100% was finished.
Downloading file mysql.inf3: 0%
Downloading file mysql.inf3: 100% was finished.
Downloading file apache.inf3: 0%
Downloading file apache.inf3: 100% was finished.
Downloading file nginx.inf3: 0%
Downloading file nginx.inf3: 100% was finished.
Downloading file php71.inf3: 0%
Downloading file php71.inf3: 100% was finished.
Downloading file php70.inf3: 0%
Downloading file php70.inf3: 100% was finished.
Downloading file php56.inf3: 0%
Downloading file php56.inf3: 100% was finished.
Downloading file php55.inf3: 0%
Downloading file php55.inf3: 100% was finished.
Downloading file php54.inf3: 0%
Downloading file php54.inf3: 100% was finished.
Downloading file php53.inf3: 0%
Downloading file php53.inf3: 100% was finished.
Downloading file php52.inf3: 0%
Downloading file php52.inf3: 100% was finished.
Downloading file pmm.inf3: 0%
Downloading file pmm.inf3: 100% was finished.
Checking for installed packages...
Downloading file PSA_17.5.3/plesk-17.5.3-cos7-x86_64.inf3: 0%
Downloading file PSA_17.5.3/plesk-17.5.3-cos7-x86_64.inf3: 100% was finished.
Downloading Extensions Catalog feed...
Downloading file PSA_17.5.3/plesk-patches-17.5.3-cos7-x86_64.inf3: 0%
Downloading file PSA_17.5.3/plesk-patches-17.5.3-cos7-x86_64.inf3: 100% was finished.
Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 0%
Downloading file PMM_0.1.10/pmm-0.1.10-cos7-x86_64.inf3: 100% was finished.
Downloading file NGINX17/nginx-cos7-x86_64.inf3: 0%

ERROR: Failed to download the file http://plesk-mirror.local/lin/NGINX17/nginx-cos7-x86_64.inf3:
The requested URL returned error: 403
Not all packages were installed.
Please try to install packages again later.
Please resolve this issue and try to install the packages again.
If you cannot resolve the issue on your own, contact product technical support for assistance.

-bash-4.1$ cat /opt/mirrorctl/BUILD/config.xml
<?xml version="1.0" encoding="UTF-8"?>
<xml>
<release name="PLESK_12_5_30" version="12.5.30">
<os name="cos7_x86_64"/>
</release>
<release name="PLESK_17_5_3" version="17.5.3">
<os name="cos7_x86_64"/>
</release>
<release name="PLESK_17_0_17" version="17.0.17">
<os name="cos7_x86_64"/>
</release>
</xml>
-bash-4.1$ pwd
/var/www/html/mirrors/plesk-mirror/lin
-bash-4.1$ ls -l
total 704
drwxrwsr-x 10 mirrorctl mirrorctl 4096 Jul 17 19:29 APACHE_2.2.31
-rw-r--r-- 1 mirrorctl mirrorctl 1591 Sep 19 03:43 apache.inf3
drwxrwsr-x 8 mirrorctl mirrorctl 4096 Aug 17 2015 BILLING_12.5.29
-rw-r--r-- 1 mirrorctl mirrorctl 99843 Sep 19 03:42 billing.inf3
drwxr-xr-x 34 mirrorctl mirrorctl 4096 Sep 8 15:24 debian
drwxrwsr-x 3 mirrorctl mirrorctl 4096 Mar 25 2016 extensions
drwxr-xr-x 2 root root 4096 Sep 11 10:11 Installer
-rw-r--r-- 1 mirrorctl mirrorctl 329 Sep 19 03:41 mirror.xml
drwxrwsr-x 10 mirrorctl mirrorctl 4096 Oct 21 2015 MYSQL_5.5.46
drwxrwsr-x 8 mirrorctl mirrorctl 4096 Sep 21 2016 MYSQL_5.5.52
-rw-r--r-- 1 mirrorctl mirrorctl 2742 Sep 19 03:43 mysql.inf3
drwxrwsr-x 122 mirrorctl mirrorctl 12288 Jul 17 17:19 NGINX_1.11.1
lrwxrwxrwx 1 mirrorctl mirrorctl 13 Mar 23 17:43 NGINX17 -> NGINX_1.11.10
-rw-r--r-- 1 mirrorctl mirrorctl 34119 Sep 19 03:43 nginx.inf3
drwxrwxr-x 2 mirrorctl mirrorctl 176128 Aug 31 14:16 Parallels_Installer
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 5 15:50 PHP_5.2.17
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Jun 20 2016 PHP52_17 -> PHP_5.2.17
-rw-r--r-- 1 mirrorctl mirrorctl 5407 Sep 19 03:43 php52.inf3
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Jun 20 2016 PHP53_17 -> PHP_5.3.29
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 5 15:46 PHP_5.3.29
-rw-r--r-- 1 mirrorctl mirrorctl 5408 Sep 19 03:43 php53.inf3
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Jun 20 2016 PHP54_17 -> PHP_5.4.45
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 5 15:42 PHP_5.4.45
-rw-r--r-- 1 mirrorctl mirrorctl 5408 Sep 19 03:43 php54.inf3
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Jul 25 2016 PHP55_17 -> PHP_5.5.38
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 5 15:38 PHP_5.5.38
-rw-r--r-- 1 mirrorctl mirrorctl 5407 Sep 19 03:43 php55.inf3
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Sep 5 15:33 PHP56_17 -> PHP_5.6.31
drwxrwsr-x 35 mirrorctl mirrorctl 4096 Jul 25 2016 PHP_5.6.24
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Sep 12 2016 PHP_5.6.25
drwxrwsr-x 39 mirrorctl mirrorctl 4096 Sep 27 2016 PHP_5.6.26
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Oct 18 2016 PHP_5.6.27
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Nov 14 2016 PHP_5.6.28
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Dec 9 2016 PHP_5.6.29
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Mar 15 2017 PHP_5.6.30
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 5 15:33 PHP_5.6.31
-rw-r--r-- 1 mirrorctl mirrorctl 5407 Sep 19 03:43 php56.inf3
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Sep 12 2016 PHP_7.0.10
drwxrwsr-x 39 mirrorctl mirrorctl 4096 Sep 27 2016 PHP_7.0.11
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Oct 17 2016 PHP_7.0.12
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Nov 14 2016 PHP_7.0.13
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Dec 21 2016 PHP_7.0.14
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Jan 23 2017 PHP_7.0.15
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Mar 15 2017 PHP_7.0.16
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Mar 20 2017 PHP_7.0.17
lrwxrwxrwx 1 mirrorctl mirrorctl 10 Sep 5 15:24 PHP70_17 -> PHP_7.0.23
drwxrwsr-x 50 mirrorctl mirrorctl 4096 May 15 20:26 PHP_7.0.18
drwxrwsr-x 38 mirrorctl mirrorctl 4096 May 15 20:30 PHP_7.0.19
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Jul 10 16:25 PHP_7.0.21
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Sep 5 19:49 PHP_7.0.22
drwxrwsr-x 38 mirrorctl mirrorctl 4096 Sep 5 15:24 PHP_7.0.23
drwxrwsr-x 35 mirrorctl mirrorctl 4096 Jul 25 2016 PHP_7.0.9
-rw-r--r-- 1 mirrorctl mirrorctl 4400 Sep 19 03:42 php70.inf3
lrwxrwxrwx 1 mirrorctl mirrorctl 9 Sep 5 15:14 PHP71_17 -> PHP_7.1.9
-rw-r--r-- 1 mirrorctl mirrorctl 4370 Sep 19 03:42 php71.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 17011 Sep 19 03:42 plesk.inf3
drwxrwsr-x 50 mirrorctl mirrorctl 4096 Sep 14 2016 PMM_0.1.10
-rw-r--r-- 1 mirrorctl mirrorctl 3782 Sep 19 03:43 pmm.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 101529 Sep 19 03:42 pp-sitebuilder.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 1105 Sep 19 03:42 ppsmbe.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 1671 Sep 19 03:43 products.inf3
drwxr-xr-x 13 mirrorctl mirrorctl 4096 Sep 9 03:35 PSA_12.5.30
drwxrwxr-x 7 mirrorctl mirrorctl 4096 Sep 19 03:37 PSA_17.0.17
drwxrwxr-x 7 mirrorctl mirrorctl 4096 Sep 19 03:36 PSA_17.5.3
-rw-r--r-- 1 mirrorctl mirrorctl 1882 Oct 10 2016 ruby.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 5851 Sep 19 03:42 setemplates.inf3
drwxrwsr-x 11 mirrorctl mirrorctl 4096 Aug 13 2015 SITEBUILDER_12.5.29
drwxrwsr-x 11 mirrorctl mirrorctl 4096 Sep 12 2016 SITEBUILDER_12.5.30
drwxrwsr-x 8 mirrorctl mirrorctl 4096 Aug 15 2016 SITEBUILDER_17.0.15
-rw-r--r-- 1 mirrorctl mirrorctl 499 Sep 19 03:42 sitebuilder.inf3
-rw-r--r-- 1 mirrorctl mirrorctl 359 Sep 19 03:42 sso.inf3
drwxr-xr-x 30 mirrorctl mirrorctl 4096 Sep 8 15:24 ubuntu

How can I get mirrorctl to sync NGINX_1.11.10 and PHP_7.1.9 directories? What am I missing in my setup?

 

[burnley]

Hi, any suggestions on this one? Where to look and how to fix it? Just checked these 2 mirrors:
Index of /plesk
Index of /plesk/
They both contain NGINX_1.11.10 as well as PHP_7.1.9 directories. Why isn't my mirror updating those?

 

[burnley]

I think the behaviour is caused by using --copy-unsafe-links, I'm now testing with --copy-links and will report back.

 

[burnley]

Better, but not quite there yet. The local repo now contains the NGINX17 and PHP71_17 and the Plesk installer for 17.5.3-cos7-x86_64 completed successfully:
---CUT HERE---
/usr/local/psa/admin/bin/autoinstaller --select-release-current --upgrade-installed-components
[..]
Patching file (255 of 255) /usr/local/psa/admin/plib/Smb/Validate/Client/Login.php
All patches were applied.


Patches were installed successfully.

The changes were applied successfully.
---CUT HERE---

But there are still locations missing from my local mirror, got 404 twice for the same location:
---CUT HERE---
[...]
Loaded plugins: fastestmirror
NGINX-thirdparty | 2.9 kB 00:00
Not using downloaded NGINX-thirdparty/repomd.xml because it is older than what we have:
Current : Wed Jul 12 13:58:50 2017
Downloaded: Tue Mar 28 17:51:49 2017
PHP_5_2-thirdparty | 2.9 kB 00:00
PHP_5_3-thirdparty | 2.9 kB 00:00
PHP_5_5-thirdparty | 2.9 kB 00:00
PHP_5_6-thirdparty | 2.9 kB 00:00
PHP_7_0-thirdparty | 2.9 kB 00:00
PLESK_17_5_3-dist | 2.9 kB 00:00
http://plesk-mirror.local/lin/PSA_17.5.3/extras-rpm-CentOS-7-x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
[...]
Stopping sw_engine service... done
===> Cumulative APS controller database (apsc) upgrade has been started.
===> Cumulative upgrade of APS controller database has been completed.
===> Cumulative Plesk database upgrade (revertable stage) has been started.
===> Preparing Plesk database upgrade (revertable stage).
===> Cumulative upgrade of Plesk database (revertable stage) has been completed.
===> Plesk database scheme upgrade has been started.
Applying migrations from: /usr/local/psa/bootstrapper/pp17.5.3-bootstrapper/migrations/
===> Plesk database scheme upgrade has been completed.
Loaded plugins: fastestmirror
http://plesk-mirror.local/lin/PSA_17.5.3/extras-rpm-CentOS-7-x86_64/repodata/repomd.xml:[Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
[...]
---CUT HERE---

Looking at the /opt/mirrorctl/BUILD/update.sh script, extras-rpm-CentOS-7-x86_64 isn't rsync-ed, should it be? Do I have to add it separately and, if I have to, how? I've attached it renamed as update.txt.