• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

Incremental backups

Pagemakers

Silver Pleskian
Will Plesk 12.5 ever support incremental backups for mail?

Incremental backups actually take more hard drive space than full backups and it's not quite as the documentation makes out.

For example if you had a domain doing permanent full back ups and its backups were 10 GB then the hard drive space it would take up to 10 GB. If the same domain now does incremental backups the first backup will be 10 GB and the second one possibly 2GB, and the third one say again 2GB and so on so by the end of the back up cycle the actual hard drive space used could surpass 20 GB before a new full backup is made.

Since doing incremental backups I am commonly running out of backup space and I never did before.
 
There are no incremental backup support for mail in 12.5 and I suppose it will not. But we could expect it in the next release.
To control backup cycle you have special options for scheduled backup. It should be useful
 
I know you have backup option but if you do a daily incremental backup, the full backup starts 7 days later so you get this..

  1. 10GB
  2. 2GB
  3. 2GB
  4. 1GB
  5. 1GB
  6. 2GB
  7. 2GB
  8. FULL 10GB
So you can see in the week it grows from 10GB to 20GB

Having mail backup fully each time is not good.
 
@Pagemakers,

Maybe I am missing something, but you could also do

a) a full "rolling" backup: a maximum number of, for instance, 3 backups per domain, with a backup scheduled every day (or every x hours), or

b) a custom script, involving a rsync command for mailboxes (when configured properly, it just transfers the new mail data), or

c) a custom script, based upon larch (or a similar tool),

and that way, you would have a very limited disk usage footprint.

I would advice to consider option a (if you prefer to handle things the "Plesk way") or option b (if you want minimal disk usage and low traffic).

Hope this helps a bit.

Regards....
 
Can you explain A

I am using plesk to backup a domain. It is set to an incremental daily backup and a weekly full backup. So what I get is..

  1. Full
  2. Incremental
  3. Incremental
  4. Incremental
  5. Incremental
  6. Incremental
  7. Incremental
  8. Full
I don't see any way of making less incremental backups. If my full backups were 10GB and my incrimentals 2GB I'd need 22GB of space before a new 10GB backup was made so technically I guess I need 32GB of space which is far more than the 20GB I'd need if I was only doing full backups.

See attachment
 

Attachments

  • image.jpeg
    image.jpeg
    132.5 KB · Views: 8
@Pagemakers

You are right: incremental backups can and often will need more disk space.

The problem with incremental backups is that you need a "base backup" (i.e. a full backup to start from), with some incremental backups (i.e. backups of "changes") to complete it.

In essence, that method is (normally) not a problem, at least in theory and/or in situations in which the data to be backed up continually grows.

In practice, the following can occur:

a) data present in the "base backup" is changed and, as a result, some incremental backup file is created: the theoretical advantage of having an incremental backup process is reduced, if and only if the static "base backup" cannot be changed. In most incremental backup software, the base backup file is static of nature.

b) data changes are "negative", in the sense that the amount of data to be backup up is reduced to some set of data that is already present in previous backups: even though one should suspect that the incremental backups should not be made, if often is the case that the "negative changes" still lead to a "positive incremental backup" that uses disk space.

The above is a general remark about incremental backups and it depends on the software or code to what degree the above applies.

In general, most incremental backup software is not "smart", in the sense that backups are static and therefore that changes will lead to new disk space usage.

What happens in your case?

Well, your case is one that exhibits the following: a base backup, augmented with some incremental backups, with the sum of those incremental backups being similar to the base.

In every sense, that is not a waste of disk space in the long run, if and only if there are only "positive changes", meaning that data to be backed up increases.

For instance, consider the following that applies to an illustrative incremental backup:

Day 0: base backup, 10GB
Day +1: incremental, 2GB
Day +2: incremental, 0GB (note: with most incremental backup software, a 0GB incremental will not occur.)
Day +3: incremental, 7GB

and compare the above to a regular backup, executed every day:

Day 0: backup, 10GB (naturally, this is a backup file, identical to the base backup mentioned above)
Day +1: backup, 12GB (assuming the incremental only contains "positive changes", meaning that data to be backed up increases)
Day +2: backup, 12GB
Day +3: backup, 19GB (assuming the incremental only contains "positive changes", meaning that data to be backed up increases)

and now compare it to a regular backup with one-day backup retention (for the sake of illustration)

Day 0: backup, 10GB, total disk space used: 10GB (disk space used by the incremental backup process: 10GB)
Day +1: backup, 12GB, removing Day 0 backup, total disk space used: 12GB (disk space used by the incremental backup process: 12GB)
Day +2: backup, 12GB, removing Day +1 backup, total disk space used: 12GB (disk space used by the incremental backup process: 12GB)
Day +3: backup, 19GB, removing Day +3 backup, total disk space used: 19GB (disk space used by the incremental backup process: 19GB)

In short, no harm done when there are only "positive changes": an incremental backup has positive effects (low traffic and faster backup speed) AND negative effects (a lower recovery speed, which is the result of the fact that backup recovery involves multiple backup files).

The above are more general advantages and disadvantages of incremental backups, but note that the major advantage of incremental backups is that you can allow for a "point-in-time" restore of backups, i.e. you can choose to restore files as they were present on a specific date or time, equal to the backup time (for instance, a backup at Day +1). This major advantage is not applicable to regular backups.

For the sake of illustration, let´s continue and relax the assumption that only "positive" changes are present.

Consider the incremental backup:

Day 0: base backup, 10GB
Day +1: incremental, 2GB
Day +2: incremental, 0GB
Day +3: incremental, 7GB

and the regular backup (executed every day, with one-day retention):

Day 0: backup, 10GB (naturally, this is a backup file, identical to the base backup mentioned above)
Day +1: backup, 9GB (this often occurs when files, belonging to the base backup are removed)
Day +2: backup, 9GB
Day +3: backup, 13GB (some "positive changes" and some removal of files, belonging to the base backup)

Naturally, you end up with a 13GB disk space usage for the regular backup process and a 19GB disk space usage for the incremental backup process.

It must be clear by now that the above comparison is often applicable, since not all incremental backup processes are "smart", in the sense that "static" backups are used.

The problem is that it is hard to tell whether an incremental backup process is "smart", but if the incremental backups exceed regular backups, then it is not smart at all.

Finally, we still cannot discard the advantages of the incremental backup process and revert to regular backups.

After all, when increasing the retention rate for the regular backup process, the total disk space used by the regular backup process will also increase, making it less attractive, when compared to the incremental backup process.

In conclusion, one should choose carefully between incremental and regular backup processes, it is not only a matter of disk space.

Try to consider all advantages and disadvantages of both processes in the context of your own situation, before making a choice or drawing a conclusion.

I hope that all the above helps or explains a little bit.

Regards...
 
Thank you for your brilliant and helpful reply.

I have 1 more question. Assume my initial backup was 100GB and my incrimental backups a further 50GB making 150GB in total. Will I still need an additional 100GB min spare on the drive before the next full backup can be made? IE is the 2nd full backup transferred to the drive before the 1st one gets deleted?
 
@Pagemakers

Ehm, depends on the situation and, in theory, a backup process will not start if the disk space (on the source server) is not enough. The latter is a Plesk feature.

In essence, you mention an assumption that the full backup size is between 100GB and 150GB (depending on the "positive and negative" changes).

In the light of this assumption, you really need 100GB at minimum, whilst 150GB of free disk space is to be preferred, on both the target and the source server.

Given the above, we can turn to your questions.

Will I still need an additional 100GB min spare on the drive before the next full backup can be made?

Yes, on the target server. If the disk space on the source server is not enough, Plesk should not start a backup process.

It can be the case that you need more disk space on the target server, considering the fact that it is very likely that your (full) backup involves more than 100GB of data.

IE is the 2nd full backup transferred to the drive before the 1st one gets deleted?

First, note there are two general cases: the "target is source" case (read: local backup) and "target is not source" case (read: remote backup).

Second, be aware of the workflow in the backup process: identify data to be backed up, package them according to a specific structure (for instance, gzipped archive) and transfer.

The "package" phase involves the usage of local disk space, that will be cleaned up after a succesful backup.

The "transfer" phase is only relevant in the case of a remote backup.

Third, note that the "deletion part" occurs at the "package" phase (i.e. after a succesfully completed backup process, temporary files are deleted) and at the "transfer" phase, if and only if the retention rate is set in such a fashion that a specific (older) remote backup has to be deleted (if I am not mistaken, after a succesful transfer occurs).

In short, the answer to your question: yes, the latest backup should be transferred (succesfully), before the older (remote) backup is deleted.


As a final remark and to be honest, the Plesk backup process can be time- and resource-consuming and, for that reason, I personally use rsync based scripts.

This is almost identical to the Plesk incremental backup process, but it saves some disk space, saves some time and prevents issues with packaging.

In essence, I use rsync based scripts to allow snapshots and mirroring, with mirroring also allowing a very crude fail-over structure: if a server goes down, we have all the relevant data on another server and we can easily switch to that server (i.e. redirect traffic).

Hope the above helps...

Regards
 
I used to backup my entire server and all domains to a single 250GB backup server nightly.

Now with Plesk 12.5 I moved to incremental backups. I now backup just the server to the 250GB server and all other domains elsewhere.

The full server backup is 75GB and after a week of incremental backups it finally ran out of space last night as it was trying to make it next full backup. My incremental backups we about 20GB.

The problem is Plesk 12.5 always does full mail backups and this is causing the huge incremental backups. So for me and I guess many others the incremental backup at the moment is flawed. Mail needs to be incremental too.

It seems crazy to me that a 250GB server is not large enough to house a 75GB backup!

I'm going to have to go back to full nightly backups.
 
@Pagemakers

You are certainly right: it is common practice and, in my humble opinion, best practice to use full (non-incremental) backup for backing up the entire server.

Note that you can exclude the mail content from the backups, so that can also be of some relief.

By the way, I am still wondering why your backups become relatively huge.

Did you or one of your customers make a backup on a domain/subdomain (i.e. using ftp to store backups on that internal domain), with that domain/subdomain being backed up again?

Regards....
 
Nobody has access to our servers. I note that the next version of Plesk allows incremental mail backup. That's what's causing the problem for me. For now I've shuffled our backup servers round and hopefully I may have enough room on a different backup server.
 
@Pagemakers

Disabling the backup of mail content would certainly be an option, accompanied by the usage of rsync (to backup/mirror mailbox data) or larch (an IMAP tool) based scripts.

As soon as I have time (and that is a rare occassion), I will certainly have a look at the "mail backup issue", since it would be strange if mailboxes result in huge incremental backups.

Regards....
 
I think it's correct. We have about 19GB of mail content on the entire server and that would explain why the incrementals are 20-22 a day.
 
I think it's correct. We have about 19GB of mail content on the entire server and that would explain why the incrementals are 20-22 a day.

OK, thanks for the additional information: in due time, I will try to reproduce your case by creating a dummy domain, add some mail data and setup an incremental backup.

I will have a look at the process and try to determine if and why the mail data gets backed up fully (which should not be the case if an incremental backup is set up).

If we are lucky, a work-around can be identified.

I will keep you posted, within this particular topic thread!

Regards.....
 
From the 12.5 admin manual...

Note: Plesk includes in incremental backups the data related to web, mail, and database hosting, but only web hosting data is backed up incrementally. In other words, only changes in data related to web are taken into account in incremental backups, while database and mail related data is included in full.
 
@Pagemakers

I know that.

The issue is not what Plesk does backup, but whether it is possible to work-around the default setting of "full mail backup", which setting is a little bit daft (and completely unnecessary).

Regards....
 
  • Like
Reactions: VMG
Back
Top