XML-RPC API & Examples

[trialotto]

The explanation of API is here - http://download1.parallels.com/Plesk/Doc/en-US/online/plesk-api-rpc/ Repo with the examples of API usage of cause doesn't contain the API reference. Most popular language for web dev is PHP (still, you may hate it, but it is what it is). For PHP there is a more high level library: https://github.com/plesk/api-php-lib (still under development). Primary goal of these repos is to help the newcomer to start using the API.


This is not about "examples". This is about Plesk API itself. Also there is almost no reason to "contribute" to examples, because they are already created for most popular languages.

P.S. Anyway it looks like offtopic for current thread.

@SibProgrammer,

I am aware of the api-php-lib github project, but cannot imagine why "newcomers" are able to use this github repo and/or start using the API.

In fact, if I am not mistaken, there a two (outside) members for the github project, who have not contributed up till now.

My primary concern with this particular github project and the RPC API in general is that Plesk becomes relatively vulnerable, with this vulnerability not being necessary at all, given all other options to control plesk panel.

A secondary concern is that vulnerability inherently increases if suboptimal client code is suggested in the API examples github repo.

For instance, in C# based clients, the

- HttpWebRequest is not the best class to use,
- the XmlDocument is a suboptimal (memory wasting) class for Xml parsing,

and so on.

Moreover, authentication in the RPC API with the general admin credentials is asking for problems, since it is fairly easy to "sniff packets".

In short, the above is intended to state that there are lots of topics for improvements of the plesk API, which mostly require some intensive redesign on the server side.

Sure, the api-php-lib repo can help, but I personally feel that it will not make a difference, due to the lack of contributions.

To be honest, Parallels should do a survey and ask how often the API RPC is or has been used (and why it is not used by sysadmins).

Kind regards

PS This is indeed offtopic, I am happy to continue this conversation by mail and/or otherwise.

 

[SibProgrammer]

A secondary concern is that vulnerability inherently increases if suboptimal client code is suggested in the API examples github repo.

I feel that examples should be as simple as it possible. If one can run the example without additional changes inside code, it's a good example. If it's impossible to run anything without code modifications, vast majority of newcomers feel that the examples are bad.

Plesk supports both authentication via login/password pair and secret tokens. Most of examples already contain the snippet on how to use secret token and how to pass it with request. See https://github.com/plesk/api-examples/blob/master/python/plesk_api_client.py#L17 Examples on GitHub do not contain the code on how to obtain the secret token. But official doc has such topic.

- HttpWebRequest is not the best class to use,
- the XmlDocument is a suboptimal (memory wasting) class for Xml parsing,

I'm not C# developer (code was written by one of my colleagues, very experienced Win developer), but even I fill that it's ok to use XmlDocument here. XML-RPC packets are so small to gain real effect of such optimizations. I highly recommend you to not to thing about optimizations until you can perform the real verification of the effect (I'm writing the code 2/3 of my life already, believe me).

And once again that examples are about simplicity and quick start, not about memory optimizations, security or something else.

In short, the above is intended to state that there are lots of topics for improvements of the plesk API, which mostly require some intensive redesign on the server side.

I absolutely agree with you that there are a lot of areas to improve in Plesk API (and not only in API ). If you have any thoughts or non-abstract suggestions, feel free to share them with us.

Sure, the api-php-lib repo can help, but I personally feel that it will not make a difference, due to the lack of contributions.

Most of people didn't like to "contribute" to anything, they like to "use" it instead. So we're trying to focus on how to simplify the "use", not how to simplify the "contribution".

To be honest, Parallels should do a survey and ask how often the API RPC is or has been used (and why it is not used by sysadmins).

Even without survey I can tell you the following (based on my communications with integrators). Humans (sys-admins, support and etc) try to use UI and CLI in most cases. Because of simplicity. Admins prefer to automate tasks using CLI. XML-RPC API is used for "code-to-code" integrations (provisioning systems and etc). We know that XML-RPC API is old enough and should be replaced with something modern RESTfull JSON API, which could be easily used from browser by human and from script by code. But can't say anything about when it could happened. First of all we have a lot of integrators and they don't want to change anything (and it's ok for them), but we also want to avoid the maintenance of one more API layer (plus to existing ones).
Also in most cases it more important question from integrator to us if "feature X" has control via API, rather than is API use JSON or XML for output.

 

[trialotto]

Sibprogrammer,

I would like to state the following, in reaction to your post.

First of all, the most relevant part of your reaction is in the last lines: on the one hand, a RESTfull JSON API would be preferred, while on the other hand it is not desirable to have many API layers and/or it is true that is should not matter how requests or responses should be formatted.

In a sense, most of the earlier parts of your reaction are less relevant to those last lines.

However, I wonder why Parallels would like to maintain a API and/or an old RPC API, if UI and CLI are (actually) used in most cases.

The presence of the API as such is increasing the vulnerability of the Plesk system.

You, as an experienced programmer, are probably aware of how easy it is to intercept (sniff) the packets send to and from the Plesk server.

With respect to "use instead of contribute" part of your reaction: how do you see the improvements in Plesk API? What would you like to see?

If there is something I can contribute to, I am willing to contribute.

Kind regards.....

PS I have to take some time to think about some concrete API improvements, but I will suggest some ideas somewhere in the future, via this forum topic and/or per mail.

 

[SibProgrammer]

However, I wonder why Parallels would like to maintain a API and/or an old RPC API, if UI and CLI are (actually) used in most cases.

XML-RPC API is much simpler to use programmatically (from code). We have customers with hundreds or even thousands of servers under control of their own provisioning systems.

With respect to "use instead of contribute" part of your reaction: how do you see the improvements in Plesk API? What would you like to see?

We have a lot of ideas on how to improve the internal structure and architecture. But real use cases have only our customers. For example if someone need some operation that is absent in API, instead of asking us to add it people prefer to start doing workaround. It ok as a quick solution, but it's very helpful for us to share your workarounds and ask for improvements.

 

[trialotto]

Sibprogrammer (by the way, how can I call you),

I agree that XML RPC API is "simple" from a code perspective.

I recall from a couple of years ago that the XML RPC API does perform relatively well, even under high loads.

One disadvantage became clear: the XML format, implying "huge packets" (i.e. sum of traffic) under high loads.

The XML RPC API is not very suitable for huge workloads (packets or actions) and I am therefore surprised that you have customers using the API with numerous servers.

However, as earlier discussed, a JSON based API would (partially) resolve the "performance crunch at high workloads".

Are you planning to implement the JSON format? If yes, do you have an idea of the planning?

In a sense, customers would not really be impacted by adding API support for JSON.

With respect to workarounds, the following.

A major pain in the *** (excuse my french) of the API is that, on Windows based systems calling the API, the whole packet-request API process is cumbersome and ineffective.

There are (amongst others) two main reasons:

a) creating the client, to be more specific:

- the request can be done via various methods, one of them is given in the api examples project on Github
- the creation of the xml package (request) is prone to errors,
- the processing of xml packages (response) is sometimes required, resulting in additional lines of code (also prone to errors)

b) a call to an API on a Windows based Plesk installation is "strange in nature", given the temporary departure of windows related code (C# or VB) when using the RPC API.

It makes sense to implement a more "modern" data service endpoint (i.e. API) for Windows environments.

Sure, you are maintaining the RPC protocol, no problem in that choice.

Given that choice, one could make two "workarounds" (given the lack of a better word) for all Plesk customers:

1) design a full C# client (with classes for all API requests and responses), being a client library for the RPC API, (and)

2) create a service endpoint on Plesk server, the type of service (wsdl etc) to be determined later.

Personally, I have the feeling that the creation of the full C# client can resolve a lot "mismatches" related to the RPC protocol.

Can we join forces on that project? If yes, is it an idea to create a github project for that?

Kind regards, Alexander

 

[SibProgrammer]

Are you planning to implement the JSON format? If yes, do you have an idea of the planning?

Unfortunately I can't share any dates.

Personally, I have the feeling that the creation of the full C# client can resolve a lot "mismatches" related to the RPC protocol.
Can we join forces on that project? If yes, is it an idea to create a github project for that?

I asked on our internal mailing list if someone capable to start such activity. Please don't expect fast reaction, but hope we'll make some step forward in this direction.

 

[trialotto]

Sibprogrammer,

I asked on our internal mailing list if someone capable to start such activity. Please don't expect fast reaction, but hope we'll make some step forward in this direction.

I suppose that we can use TFS with Visual Studio to collaborate on this.

If not, let me know, then an alternative would be a Github project.

Kind regards.....

 

[SibProgrammer]

Github repo under your account will be ok in the beginning (we can use fork/merge pull requests model). If everything goes well, we will transfer the project to github.com/plesk/ namespace.

 

[trialotto]

First of all, sorry for the late reaction (I just saw that my message was not sent).

I really suggest that we first try to use Team Foundation Server, since the creation of a C# client

a) requires some workarounds, in order to have clean code (it does not suffice to automatically generate C# code from the xsd schemas),

b) requires some base or wrapper classes, which (in my opinion) are not to be shared "publicly" (as will be the case with a Github repo),

c) can form the base of "new code" development, being (amongst others)

- an API that allows for JSON formats
- OData service formats

and, again, this code should not be shared "publicly".

Do you agree with a test of Team Foundation Server, based upon Git version control?

Kind regards...

 

[SibProgrammer]

In case if we're trying to create some new tool with collaboration with other people, then such tools should be "open" from the beginning. Another thing is that I'm (personally) not focused on C#, but on the other areas. I can try to assist the development process, but not initiate such things as "new C# client lib". Such initiative should be made by community. From our point of view we see that such community is rather small and that's why such wishes are still in TODO list and not yet implemented. But this doesn't mean that we're against of such changes. We even can make a support of good initiative if such initiative becomes something practical and usable.

 

[trialotto]

It surely is not the intention to reinvent the wheel.

However, one of the major drawbacks of opening up the first stages of the development process is that any project gets flushed with "bad" code.

It is, in my opinion, desirable to create "outlines of code design" in a closed environment and open up a (separate) coding environment afterwards, with this coding environment open for the community and enabling the community to add (and not determine) code.

The whole idea of a C# client library is to create a RPC API wrapper, that allows for enabling and/or creation of (amongst others)

1. clean and "automated" API calls, instead of programming against the "dirty" and code-intensive RPC API
2. unified (community) efforts to augment the API (in specific, before mentioned API wrapper) with desired features
3. cross-platform API usage, based upon an API that can easily addressed from any platform (not the case at this moment)
4. unified (community) efforts to change the wrapper, if the RPC API changes

and so on.

Note that the community support for the RPC API is (indeed) rather small, due to the fact that most of the Plesk customers are aware that any code development or contribution becomes inapt and even obsolete, if a new version of the RPC API is introduced.

A lot of people, that I spoke in the past, did not want to contribute to a similar project, giving mostly the arguments that

- the xsd schema´s are not well formed, giving rise to a code-intensive development (i.e. a rewrite)
- scripts are preferred over API
- new versions of Plesk´s RPC API would require new code-intensive development

and so on.

In short, it is my intention to create a (at first instance private) project to

a) maintain the RPC API "as is" and allowing for changes thereof
b) create a clean API wrapper (first in C#, then in Python, to cover Windows and Linux based systems)
c) create a community project (on Github), to allow community-based augmentation of functionality
d) allow for the community project (on Github) to be a form of feedback of RPC API improvements

and that is all.

In short, in the long run, the before mentioned approach will become practical, usable and beneficial for Parallels and the community.

In the short run, we have to dig in and try to get the intended API Wrapper as clean as possible (that really is a challenge).

Given all the above, are we going to make an effort to create this API wrapper? Please let me know.

By the way, this "effort" is a process of multiple months (no priority), this can be relevant for your decision.

Kind regards, Alexander

 

[SibProgrammer]

You may start such project by yourself and do it privately. After some time you could share your first results and we can help to make it public. For example place to github.com/plesk namespace to make a little bit more official and visible. We (Parallels) may contribute to that project (but can give any real promises because it depends on a lot of things).

We support backward compatibility for XML-RPC API for many versions. We have integrations that were written 3-5-7 years ago and still working with modern versions of Plesk. So don't be afraid of that new version of Plesk will break backward compatibility in XML-RPC API.

 

[trialotto]

Sibprogrammer,

You may start such project by yourself and do it privately. After some time you could share your first results and we can help to make it public. For example place to github.com/plesk namespace to make a little bit more official and visible. We (Parallels) may contribute to that project (but can give any real promises because it depends on a lot of things).

Let´s take this approach.

We support backward compatibility for XML-RPC API for many versions. We have integrations that were written 3-5-7 years ago and still working with modern versions of Plesk. So don't be afraid of that new version of Plesk will break backward compatibility in XML-RPC API.

A C# wrapper based upon (for instance) 1.6.x RPC API will not be fully backwards compatible, unless some additional lines of code are added.

In my experience, it is a lot of effort for a little gain to do so.

In short, in the current approach, I will not aim for a fully backwards compatible C# wrapper (the latest RPC version will be used), but it is very likely that backwards compatability will still exist (i.e. it is not a goal, only a mere convenience if backwards compatability is present).

To have a clear understanding what the C# wrapper "project" entails, the following summary (in chronological order) can be given:

1. (primary goal) creation of a C# client, in order to allow easy access to the RPC API,
2. (secondary goal) creation of a C# service bus, with associated application, in order to allow central management of multiple Plesk instances,
3. (tertiairy goal) creation of server-side C# client/program (i.e. on the server with the Plesk installation) in order to facilitate the primary/secondary goals,

and the code contribution to the community (i.e. a Github project) will be limited to the C# client (see primary goal) and the code will only become open when the secondary goal is in the stage of alpha of beta release.

I hope you agree with the above.

Kind regards, Alexander

PS I seems appropriate to notify Sergey L, with whom I have had contact about various "projects".