APT ISSUE and PUM ISSUE - Docker public key

[trialotto]

Username:

TITLE

APT ISSUE and PUM ISSUE - Docker public key

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

Product version: Plesk Obsidian 18.0.54.2
OS version: Ubuntu 20.04 x86_64
Build date: 2023/07/28 15:00
Revision: 7addc2cf386263ca7765584cff8c160141fcfd84

PROBLEM DESCRIPTION

PROBLEM 1 : APT UPDATE ISSUE

Err:3 Index of linux/ubuntu/ focal InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8

PROBLEM 2 : PUM ISSUE

As a result of problem 1, Plesk Package Manager results in

Reason: 2023-08-06 06:25:41 INFO: pum is called with arguments: ['--list', '--repo-info', '--json']
2023-08-06 06:25:42 ERROR: Apt cache fetch failed:
2023-08-06 06:25:42 ERROR:
2023-08-06 06:25:42 ERROR: Exited with returncode 1.

STEPS TO REPRODUCE

STR - PROBLEM 1 : run apt-get update

STR - PROBLEM 2 : plesk sbin pum --check

ACTUAL RESULT

RESULT FROM STR - PROBLEM 1

Err:3 Index of linux/ubuntu/ focal InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
.......

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: Index of linux/ubuntu/ focal InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
W: Failed to fetch https://download.docker.com/linux/ubuntu/dists/focal/InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
W: Some index files failed to download. They have been ignored, or old ones used instead.

RESULT FROM STR - PROBLEM 2

2023-08-06 17:20:28 INFO: pum is called with arguments: ['--check']
2023-08-06 17:20:29 ERROR: Apt cache fetch failed:
2023-08-06 17:20:31 ERROR: Err:5 Index of linux/ubuntu/ focal InRelease
2023-08-06 17:20:31 ERROR: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8
2023-08-06 17:20:31 ERROR:
2023-08-06 17:20:31 ERROR: Exited with returncode 1.
exit status 1

EXPECTED RESULT

NA

ANY ADDITIONAL INFORMATION

SOLUTION :

Add the key to the keyring and change the .list file, by following the steps :

1 - get the key - run command :

curl -sLS https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor | tee /etc/apt/keyrings/plesk-ext-docker.gpg > /dev/null

2 - declare repo variable - run command :

REPO=$(lsb_release -cs)

3 - create NEW .list file - run command

echo "deb [arch=`dpkg --print-architecture` signed-by=/etc/apt/keyrings/plesk-ext-docker.gpg] Index of linux/ubuntu/ $REPO stable" | tee /etc/apt/sources.list.d/plesk-docker.list

4 - resolve issues with double entries in double .list files :

mv plesk-ext-docker.list plesk-ext-docker.list.back


NOTES :

In step 3, the file plesk-docker.list has been chosen deliberately : it should become plesk-ext-docker.list in the patch!
Step 4 can be discarded when the patch has been applied


REQUEST

Please fix this issue (and similar issues) as soon as possible!

YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Confirm bug

 

[trialotto]

@Plesk Team,

Also, please adjust the KB article : https://support.plesk.com/hc/en-us/...use-the-public-key-is-not-available-NO-PUBKEY

The aforementioned KB article is not the recommended way to solve the issues, certainly not for the more recent Ubuntu OSes.

Kind regards....

 

[Peter Debik]

Hi @trialotto, thank you for reporting.
To your issue no. 1: So far I could not verify that 7EA0A9C3F273FCD8 is a Plesk key, so I am not sure what bug this could be in Plesk. A simple "apt-get update" did not show the same issue for me here. Maybe this is an installation specific issue?
On the support article: Why is this not an appropriate way to solve key issues, why not for recent Ubuntu and which solution should be presented instead?

 

[trialotto]

@Peter Debik

On bionic, the following has been applied by default by Plesk :

1 - key used is 8D81 803C 0EBF CD88 (if I am not mistaken)
2 - key has been added with command : apt-key adv ....
3 - URI used is : Index of linux/ubuntu/
4 - all .list files have been signed with [ ... signed-by=(value)], but NOT plesk-ext-docker.list (as far as I can see, it is the only exception)

However, the key used (see 1) does not correspond with the key that can be derived from the URI - see file gpg.

Stated differently, I am not sure which key Plesk uses, but

- it is not the key of Docker repo that can be derived from the Docker gpg file that contains the PGP PUBLIC KEY BLOCK
- it is not used in the plesk-ext-docker.list

On focal, the following has been applied by default by Plesk :

1 - no key has been applied at all
2 - no key has been added with : apt-key adv ....
3 - URI is identical to that mentioned in point 3
4 - all .list files have been signed with [ ... signed-by=(value)], but NOT plesk-ext-docker.list (as far as I can see, it is the only exception)

In short, it should not be a miracle that something goes wrong here.

I do not mind which key you use, but you should use one.

Your statement

"So far I could not verify that 7EA0A9C3F273FCD8 is a Plesk key, so I am not sure what bug this could be in Plesk."

can simply be responded to with the remark that "no key has been applied" and the remark that "plesk-ext-docker.list" is not using signed-by.

In addition, it has to be mentioned that the 7EA0A9C3F273FCD8 key is a sub key of the 8D81 803C 0EBF CD88 key used by Plesk.

Stated differently, it is not about the key value, but all about the fact that keys are not applied in focal !

Your question

"On the support article: Why is this not an appropriate way to solve key issues, why not for recent Ubuntu and which solution should be presented instead?"

is simple to answer : apt-key and add-apt-repository deprecation !!!!

In addition, Plesk itself is turning to the "signed-by" method in .list files, which requires a GPG key .

Stated differently, it is or should be normal to use gpg based keyrings .......... in any Ubuntu (or Deb) OS version.

Kind regards....


PS I did some quick research and testing, nothing more than that and only for these reasons :

- how strange is it that Plesk does not resolve issues that cause updates (apt or pum) to fail to some major or minor extent?
- these issues will return every day, when no patch has been applied by Plesk and the .list files are regenerated with problematic entries!
- why use keys that can be added to any keyserver, as opposed to using the actual gpg from the actual (genuine and authentic) repo?
- why not use the "signed-by" method if that is to be preferred anyway and also future proof?!??

 

[Peter Debik]

This thread is overwhelming me. I can't find any approach to break it down into usable parts in any way and forward it accordingly. I'm afraid it's still not clear to me what exactly the bug is supposed to be. It seems to be more about a fundamental concept change to the software.

How can I help here now? Plesk will adjust the key management procedures with security current operating systems if the operating systems require it. There is no need to file a bug report for this. I do not see a specific bug that is reproducible. Perhaps this is beyond my knowledge horizon.

Surely you have a current support subscription. Would it be possible for you to submit the claimed bugs one by one as a ticket? Otherwise, I don't see how that would be manageable. For me, it's not manageable. I also don't want to submit any things that I can't reproduce or rebuild myself as bugs to developers. That just costs everyone involved time unnecessarily. However, we will certainly and very gladly follow up on any concrete reproducible bug.

 

[trialotto]

@Peter Debik

I am sorry, but this is the most odd answer that I have received yet.

In essence, the PUM command will fail if the Plesk repos are not configured properly.

That is the case, so Plesk can have issues updating or even fail updating ........ on Ubuntu and on other systems.

Why don't you spin up 3 Ubuntu VMs, one with Ubuntu 18 LTS, one with Ubuntu 20 LTS and one with Ubuntu 22 LTS.

Then, try to add a repo via Apt ....... you will see the difference between the OSes.

In addition, you will also see the issues related to the Plesk default setups of .list files, which issues differ across the Ubuntu versions.

However, the common root cause of the problem is : key is missing ..... with notification "public key not found" when running apt update or apt-get update.

No need to mention that it is not necessary to repeat the steps for Debian based systems, since they will result in similar/identical issues.


To making it more simple ........ it is just that Plesk is not adding the "signed-by" to specific .list files.

This has been occurring with KAV in the past, some other .list files and now Docker related .list files.


It is a simple fix.


Please note that people actually have the alternative to use a free Docker installation, as opposed to using the paid for Docker extension.

I am really fine if the bug is not fixed and I am also fine with the possibility that people will discard the paid for Docker extension.

But then again, I am not really fine with the idea that Plesk charges money for an extension and does not apply a solid repo implementation.

The definition of "solid" is here : not an implementation that causes issues of any kind with apt or apt-get.


Again, it is a reaaaaally simple fix.


Kind regards......

 

[Peter Debik]

I hear your frustration, but I am personally uncapable of handling this case properly. I cannot determine what are the important parts and how I can reconstruct the situation here. Anytime I am using a test environment, let it be Ubuntu, Debian or whatever, I do not see these errors. For that reason it will be best to submit the technical issue(s) to support.

 

[trialotto]

@Peter Debik

The biggest problem when testing is this particular challenge : one can start with a fresk Plesk install and not notice any issues, even though they are present.

In daily reality, people do not start with a fresh installation - they often start with a myriad of updates based upon older versions of Plesk and OSes.

For that reason, one cannot always conclude that there is "no issue".

At this moment, there are many issues, too many issues.

The Web Troubleshooting Configurator (or whatever it is called) is not working properly, the Plesk Migrator has issues, the WPT toolkit has issues (certainly when migrating them) and so on.

In every scenario, these peculiar issues are present, with one exception though : a fresh installation of Plesk.

It can therefore be safely concluded that there are some issues that are related to Plesk updates and the update mechanism.

Nevertheless, one cannot tackle these "root causes of the problem" without solving the "symptoms" first.


One of these symptoms is that Plesk is not generating proper .list files.


In order to make the discussion a bit more convenient for you and/or to allow you to place the discussion in the right context, please note the following.

In a fresh Plesk installation, the apt-get update issues (resulting from gpg / keyring related issues) are not present.

However, after some pum operations and/or after some apt-get update operations, they will surface.

Since the .list files are generated with a cronjob, the issues will arise each and every day.

I noticed that manually adjusting the .list files with

a) a new .list file with "signed-by" for the Docker extension, (and)

b) disabling the old Docker by changing the name to plesk-ext-docker.list.back

solves the issues with Docker repos AND the pum failures that were the result of those issues.


In summary, Plesk has to change either one of the following solutions

1 - maintain the plesk-ext-docker.list without the signed-by method : add the Dockery (public key) with apt-key adv ....

OR

2 - adapt the plesk-ext-docker.list file to contain a gpg key with the signed-by method

and method 2 is to preferred in the light of forward compatibility AND SECURITY !!!!


As a minor remark, please note that the "security issue" associated with the apt-key adv ... method is not to be ignored : a key can easily be registered with a (trusted) keyserver, hence allowing any person with bad intentions to add a repo as if it were to be trusted - this is more easy than you would think.

These security concerns are to a lesser degree present with the signed-by method.


Kind regards..........

 

[Peter Debik]

I think it all burns down to just one line:

that Plesk is not generating proper .list files.

That is a known issue, ID PPP-60986, and can affect repository list files created before version 18.0.51. These can be missing the "signed-by=/etc/apt/keyrings/plesk.gpg" segment in the repository line. Other similar issues that were known before have all been fixed in 18.0.54, only PPP-60986 is still open.

 

[trialotto]

@Peter Debik,

In order to make bug hunting more easy and solutions more solid, please add the following notes to PPP-60986

NOTE 1 : bug issue PPP-60986 is not limited to .list files generated in older Plesk versions, it is also present in Plesk 18.0.54 MU2

In essence, this part of your statement

"...... can affect repository list files created before version 18.0.51."

has to be changed in : with a fresh installation OR a clean update of Plesk 18.05x to 18.0.54 MU2, most (but not all) issues with .list files are gone.

NOTE 2 : an issue similar to PPP-60986 has arisen with the introduction fo Plesk 18.0.54

In essence, this part of your statement

"Other similar issues that were known before have all been fixed in 18.0.54, only PPP-60986 is still open."

is a bit besides the actual reality - let me explain.

All issues with .list files other than plesk-ext-docker.list have been (or seem to be) resolved, indicating that issue PPP-60986 is not persisting.

Actually, it seems to be the case that Plesk 18.0.54 MU2 has a patch for issue PPP-60986.

In fact, it seems to be the case that the DEFAULT template is not correct for Docker, as opposed to an issue related to TEMPLATE GENERATION.


In my humble opinion, that is all there is too it : the plesk-ext-docker.list does not contain a key and signed-by instruction by default.

Kind regards....

 

[Peter Debik]

Just fyi, I have attached a link to this report to the existing case.