~ Yesterday I found it a little bit strange to find no menue for the key manager ~
We've never used key manager. It's an unnecessary addition (for us) and it's possible to achieve what you want (if we've understood it correctly) without using it.
~ This system user owns all directories and files of my domain: /var/www/vhosts/[MyDomain] and subdomains.
You've used the term 'system', so we're assuming that your Plesk login is 'root' (or more likely 'admin') and this is your 'system' user - is that right?
I have managed to get ssh access for this user to my domain using username/password but I would like to establish a key-based connection and to get rid of the password-based connection (what already works for othe users).
IF, you've got root (and/or admin) aka 'system' access via key-based authentication only, then presumably, you'll have previously, intentionally specified this choice within this file: /etc/ssh/ssh_config (as we did -from inception FWIW). So... IF that IS the case, then you should be able to generate a new key pair (but for your specific domain account i.e taking its account details from here: *.*:8443/smb/web/edit - for that specific domain) and once the public / private keys are located in the correct places on your server, you should then be able to login via key-based authentication only (i.e. not password) to that specific domain.
In addition to the above, you're wanting to create a key-based authentication only, separate, domain specific SFTP login, even though everything there is owned by 'system' which, also limits this specific login account, to the root of that specific domain and nothing above that. You can do this, by restricting that specific domain to: /bin/bash/(chrooted) access via CLI or, by using the
Access to the server over SSH option, on that page url posted above, within the Plesk Panel.
When you've verified the above (key-based authentication only and domain root access only) and, assuming that you've already setup your SFTP Client to use chosen, default local and remote directories and you're pretty much good to go. We use this setup (just for 2 of domains that we host) and it works just fine.
I have produced a pair of keys and stored the public key in a file called authorized_keys in the directory /var/www/vhosts/[MyDomain]/.ssh
That's correct. Normally, that .ssh directory would be the correct owner and group (not root root) with File Permissions: 700 and all of its individual file content (*rsa / *rsa.pub / authorized_keys - in our case, not sure of yours) would again, all be the correct owner and group (not root root) & have file permissions: 600
When I try to connect with putty using the private key to my server IP via port 22 with the name of the systemuser I get the message key is not accepted. Any idea what is wrong in my setup?
Now with this ^ we can't help as we don't use putty, because all of our non-server devices are macOS / iOS so we use equivalents but not putty. However, many basic 'key not accepted' errors are linked to wrong group / owner / permissions etc so the previous paragraph
might be of some assistance when de-bugging.