Question Add additional Disk to Plesk

[lara-missk]

Server operating system version

Almalinux 9

Plesk version and microupdate number

18.0.54

Hi,
how I can expand Plesk`s storage capacity by adding a second hard drive?
Thanks

 

[Faris Raouf]

It isn't as simple as that, I'm afraid.

By default, Plesk stores all website data in /var/www/vhosts and all mail in /var/qmail/. Databases are stored in /var/lib/mysql.
You can tell Plesk to store website data somewhere else, and email somewhere else.

See, for example, https://support.plesk.com/hc/en-us/...ault-location-of-mailboxes-in-Plesk-for-Linux

If you use a new disk to store emails and keep website files (and databases) where they are (or some other combination of locations) you will effectively be expanding your storage, but you can still run into the situation where the original storage can fill up, and things stop working even though you have plenty of space on the new disk.

There are a few solutions. One option is to use Logical Volumes (search for LVM) where you can expand a Logical Volume across more than one physical volume (disk). I'm pretty sure that unless your filesystem was set up to use LVMs when it was first set up, you can't "convert" to LVM, but I'm not really an expert on the subject.

It looks like your two physical disks are already set up in a RAID 1 array for redundancy and performance. You could switch to RAID0, doubling capacity, but then if one drive fails you'd lose everything (and you can't convert to this on a live system normally - all data will be lost). But maybe you could add two new identical drives, and set things up for RAID10, keeping redundancy and increasing storage? Again you can't do this on a live system normally.

Or you could add a single new disk, keep what your current disk for OS/Plesk itself/databases only, and then put both email and web stuff on the new drive.

I'm sure there are other options too.

 

[trialotto]

Hi,
how I can expand Plesk`s storage capacity by adding a second hard drive?
Thanks

@lara-missk

I am not sure what to think about the screenshot and the output of the lsblk command.

In essence, there seem to be two devices in RAID and 3 disks .......... but do you actually have 3 disks???

Anyway, adding a second hard drive would not make any sense unless you want to do very complex disk related task that are prone to end up in failure.

Your objective is to expand the disk size and make more disk space available.

That would almost require that you do a full copy of all data to the 2T disk, then combine the two smaller disks and then create a RAID1 setup between the one large disk and the (combined two smaller) disks, followed by a RAID copy ...... and that is destined to go wrong, which is really not desirable!

In my humble opinion, you are best of with a simple route : rent a dedicated server for one month, do a clean Plesk install on that server and migrate all data from the old to the new server ......... afterwards create the 2T RAID1 setup on the old server (with the new / dedicated server functioning as a backup and keeping the sites alive and online!) and then migrate back again.

The above solution seems to be overcomplicated, but it is faster and rather simple ..... and you have a backup of all data (and sites that stay online).

Please note that it might be worthwhile to consider to stick with the new dedicated server and choose larger disks, since in this modern age, any Plesk setup on 2T disks is not future proof : sooner or later one will have to go a server with larger disks.

I hope the above helps a bit.

Kind regards......

 

[lara-missk]

@trialotto and @Faris Raouf, Thanks for the reply.
I think:

1) buy another 2T disk
2) put the two 2T disks in RAID 1
3) move the Vhost and BK folders to these disks

do you think it is correct?

Thanks!

 

[trialotto]

@lara-missk

The answer to the question "do you think it is correct?" would become : yes, but no!

Yes, that could be done, why not?

But NO, why should you?

In essence, you now have an issue as a result of a lack of disk space and this will (again) occur if you purchase a 2T disk that is set in RAID1.

It is often better to use larger disks (read: more than you need right now) in order to become future proof - if you need more disk space, then the larger disks will prevent that you do not have to do this tedious work again.

In addition and even MORE IMPORTANT, it is never recommended to move vhost and backup folders to "separate" disks.

To be honest, I have tested that solution over and over and it will become a problem sooner or later and with each and every Plesk update, there is the chance that your setup will not work anymore (and that you will have to intervene manually).

Stated differently, it is HIGHLY recommended to have the Plesk instance and all it's data on ONE server with sufficiently LARGE disks (and, moreover, it is also recommended to migrate from time to time to a new server with a fresh Plesk installation - Plesk updates can cause a cumulative mess up!).

In summary, I can only recommend that you migrate all to a new dedicated server with larger disks, for instance 4TB disks in RAID1 setup.

It might be a bit more expensive at first, but in the long run the advantages will outweigh the costs.

Kind regards....

 

[ChristophRo]

We use dedicated disks (most often RAID1 volumes) for vhosts, email, mysql and backups/dumps on almost all our Plesk servers and never had any problems with it in the last 20 years.

The only thing that Plesk does not like, is if these volumes are mounted in something like /mnt/ssdvol02 and then /var/www/vhosts or /var/qmail is symlinked into that. So you should mount these directly as /var/www/vhosts (or /var/qmail, etc.)

And yeah, the Plesk Migrator utility does also just consider the diskspace on your root volume and thus may show a warning that you have to less disk space available. But it does work anyway, you just need to make sure yourself, that there is really enough space on all disks, to hold all the data.

 

[trialotto]

@lara-missk and @ChristophRo

These are all basic discussions with respect to disk management and optimal disk usage.

However, these discussions do not even touch the unlimited possibilities that are currently present with any (standard) Plesk instance.

For instance, it is possible - at least proven in an experimental setting - to get a dedicated server (read: always recommended) and add an unlimited amount of cloud based managed disks.

Sure, it is a solution that provides unlimited scalability, but also a solution that requires tweaks.

These tweaks are really necessary to remove or resolve common pitfalls with Plesk and/or technical / hardware infrastructure in general.

This pitfall suggested by @ChristophRo

The only thing that Plesk does not like, is if these volumes are mounted in something like /mnt/ssdvol02 and then /var/www/vhosts or /var/qmail is symlinked into that. So you should mount these directly as /var/www/vhosts (or /var/qmail, etc.)

can be (fairly easily) managed by tweaking the default Plesk config : when doing that, it will also work like a charm ....... until the next Plesk update, which update always will increase the probability that custom settings are overwritten.


Other tweaks might be more interesting, since it is worthwhile to talk about scalability, since that is what @lara-missk is searching for.

Adding disks might be interesting, but that is standard procedure that might best be avoided : rearranging disks is never a good idea - even if all of the tedious work is completed succesfully, one still has a highly fragmented disk that will deteriorate faster than other (new and clean) disks.

Stated differently, setting up your own partitions is a common pitfall, not associated with Plesk, but only related to choices of Plesk admins.

Adding external disks might be an interesting alternative to creating a clean (new) Plesk instance on a clean (new) server.

Nevertheless, creation of a (new) Plesk instance is recommended - it is always better.


In general, there might not be a lot of situations in which the aforementioned tweaks are necessary - they can often be avoided and should be avoided!

However, one can reach the limits of any dedicated server that can be purchased or the costs of a big / huge dedicated server can become prohibitive.

In those cases, there are some interesting features associated with Linux systems (and even more with Windows systems - that is another story).


Let's return to the example - only an example! - of cloud based managed disks, in this case Azure based managed disks.

However, we will not opt for the expensive managed disks, but for the simpe and very inexpensive file share!

It is possible to add a file share to any Linux OS, following the steps below :

- create a storage account, choose whichever storage account type you want (but not "cold" storage!)

- create a file share within the storage account and copy the (connection) settings

- just mount the file share to your local Linux OS (and make it reboot persistent)

and the above is oversimplified, it is only intended to illustrate the simplicity.


The specific file share will just behave like any other file share, even though it is somewhat slower .......

........ and that slowness can only spoil the fun if you use the file share for the wrong purposes.


In essence, you should (only) use the (static) file share for (static) storage of static files.

For instance, one could offload all local backups to that specific file share and store them safely and remotely with retention policies and deletion policies!


And that is the essential nature of any tweak : solve the problems by looking a bit different.

If you do not have sufficient space for databases or web files, why increase the space for those databases and web files?

One should first decrease the space used for other purposes ...... for instance, by setting full backup mode (read: not using wasteful incremental backup mode) or even by offloading static backup files to an external disk (read: a cloud based disk, a local external disk, even an USB drive).


In most cases, the offload of static files (like backups) will often provide sufficient space for other storage operations.


Nevertheless, tweaks might always be necessary!

More importantly, a good and intelligent approach to create efficiency might also be necessary!

For instance, why create a mounted Azure based file share (read: less secure) for offloading backups, while there is also the option to use the FTPS backup option (read: more secure) to store the backups in the Azure storage account?

So, start with creating backups in the cloud and use the FTP option ....... and only store 1 or 2 local backups on the local server.


Not enough space, even after offloading the backups?

Well, you can still use the mounted file share in order to serve some (only static) sites from the file share.

This might create a throughput issue, but even that can be tweaked by using Nginx as proxy and allowing Nginx to cache the pages locally.

But then again, it would also be very simple to use a CDN or Cloudflare (as a proxy) for specific sites - problem solved, but in a better way.


In summary, there might be many solutions and many tweaks, but all should be started with the question : why do I need extra space or disks?

In the answer to that question often lies a solution that is quite simple.

The solution is often related to an unidentified issue in the setup of Plesk or the sites (WordPress, arggh!!!)

Only when knowing the root cause of the problem, only then one can solve an issue properly.


@lara-missk ....... to be honest, I think that you should start with a dedicated server (if your disks are really full with hundreds of sites and backups) OR that you should start with an analysis of the reason why the disks are full (if you use WordPress + WooCommerce, then you are very likely to have sites with a lot of WP generated images, which images take a lot of space - in this case, just try a CDN like Cloudflare first).


Kind regards........


PS Even though Plesk is scalable without limits, it is my experience that is safer and more secure to use multiple (individual) dedicated servers as opposed to a highly complex scaled system. This is experience is based upon the fact that it does not make any sense to provide solutions for "data usage expansion" if this type of expansion is not combatted first : larger disks, larger servers ..... they all become clogged and slow, if there is no solution for the root cause of the problem. For that reason, I do not recommend incremental backups. In some test situations, a WordPress site with a reasonable number of changing images became several TBs in backup size, when using incremental backups. This is just to illustrate that one should be critical first, before solving anything!

 

[lara-missk]

@trialotto and @ChristophRo
Thanks everyone. All helpful stuff.

 

[trialotto]

@lara-missk ..... no worries, here to help if and whenever possible!

 

[mow]

Adding disks might be interesting, but that is standard procedure that might best be avoided : rearranging disks is never a good idea - even if all of the tedious work is completed succesfully, one still has a highly fragmented disk that will deteriorate faster than other (new and clean) disks.

That's still an improvement because now the websites won't be on that highly fragmented disk anymore.

If you do not have sufficient space for databases or web files, why increase the space for those databases and web files?
One should first decrease the space used for other purposes ...... for instance, by setting full backup mode (read: not using wasteful incremental backup mode) or even by offloading static backup files to an external disk (read: a cloud based disk, a local external disk, even an USB drive).

First, you should not store backups on the same server anyway, especially not on the same disk.
Second, if you think incremental backup is wasteful, you're doing something wrong. Daily full backups are much more wasteful than incremental ones, especially if you want archival (e.g. to tape) and some retention time in online storage.

Not enough space, even after offloading the backups?

Well, you can still use the mounted file share in order to serve some (only static) sites from the file share.

This might create a throughput issue, but even that can be tweaked by using Nginx as proxy and allowing Nginx to cache the pages locally.

... which is then cached on the local disk, occupying the same amount of space ...

 

[trialotto]

That's still an improvement because now the websites won't be on that highly fragmented disk anymore.

First, you should not store backups on the same server anyway, especially not on the same disk.
Second, if you think incremental backup is wasteful, you're doing something wrong. Daily full backups are much more wasteful than incremental ones, especially if you want archival (e.g. to tape) and some retention time in online storage.

... which is then cached on the local disk, occupying the same amount of space ...

@ mow,

The purpose of the statement

That's still an improvement because now the websites won't be on that highly fragmented disk anymore.

is not entirely clear to me, please explain.

In general, there is this common mistake about disks and adding disks - adding disks for the sake of increasing capacity alone is not resolving any issue, unless the configuration is adjusted to store data only (!) on the new disk(s) :

- if data is stored on both the new and old disk (for instance, some kind of RAID setup), then the advantage of the new disk is not related to "adding capacity" (but more related to "adding data persistence across disks"), and,

- if data is stored only on the new disk, then the advantage of the new disk is "adding capacity" (without adding data persistence across disks) and the not so well-known disadvantage is that Plesk can become very ineffective : this scenario requires changes to data storage config settings of Plesk (and these settings are not robust or upgrade persistent, at least not in most scenarios).

In addition, there is the matter that partitioning physical devices in (virtual) disks will always create some fragmentation (and performance penalty), certainly when (virtual) disks on the old disk (read: physical device) are set up in RAID with the new disk (read: physical device).

Moreover, any repartitioning of disks will cause some fragmentation, especially on the ephemeral parts of the disk - exactly these parts are important to the proper functioning of the entire server, since these parts are used for boot and "virtual" memory and storage of partitioning tables.

There is probably no way to convince anybody, otherwise than ask people to set up two disks and rearrange one disk (whilst keeping the other disk intact) and run normal workloads during years on those two disks : the rearranged disk will fail faster.

Nobody will create such a test environment, that is a safe assumption.


With respect to the statements

First, you should not store backups on the same server anyway, especially not on the same disk.
Second, if you think incremental backup is wasteful, you're doing something wrong. Daily full backups are much more wasteful than incremental ones, especially if you want archival (e.g. to tape) and some retention time in online storage.

I can only say :

- yes, one should store at least 1 backup locally : if a site or server fails, simply restore the backup from a local backup - faster and better than downloading a remote backup and also no danger that any encrypted backup will fail to restore (read: Plesk keys for backups are unique per server, so one might encounter some issues when trying to restore encrypted backups), and

- incremental backup is using more storage space : each and every change to any file will be stored ........ which is not necessary at all ......

- full backups are not wasteful at all, unless you have slow connections and/or wrong settings with a high number of backups to retain (read: not necessary!)

- good backup scenario : local full backups per subscription with a high frequency, a low number of backups to maintain (read: 1 or 2) and with the backups being transferred to cloud storage with retention policies, data reduncancy and snapshot policies (and backup-of-backup policies, if desired)


With respect to the statement

... which is then cached on the local disk, occupying the same amount of space ...

I can only mention that there is a difference between "cache" and "storage", with that difference implying that "cached data" will never take the same space as actually "stored data".

Also, most types of data that we can call "cached data" are actually cached in memory, not on the actual disk.


Kind regards.....