Backup Solution: rsnapshot or pleskbackup?

[b-nut]

I'm in the process of building a Plesk 8.0.1 server and setting up a good backup solution. On my other plesk boxes I've used psadump and cron to automate the daily backup process and send the data to another drive.

On the new Plesk 8.0.1 box I'm considering a similar backup solution using pleskbackup in place of psadump but I've red some bad things about pleskbackup. Please share your experiences with pleskbackup and if you are having success using it.

Another solution I'm considering is called rsnapshot http://rsnapshot.org/
Rsnapshot uses rsync and some other common utitities to create full and incremental backups with tons of customization options. Is anyone else using rsnapshot to automate backups on Plesk 8? If so, what directories do you have to backup to get all the plesk data(domains, databases,configuration, logs,etc...)? Can anyone share their rsnapshot config files for Plesk 8?

 

[DCNet_James]

I use pleskbackup, I haven't had to do a restore yet, but I've done a test restore to a standby machine and it worked from what I could see just fine. Regardless, I use a combination of pleskbackup every night and twice a week full filesystem backups using dump and gzip to an NFS mount. I've had 2 major failures where I've had RAID cards go bad and lost the whole mirror to corruption (running Raid 1), and having the full filesystem dumps helped quite a bit. Back in the 7.5.x days I did use psadump/restore to restore a server once. It worked fine for me. Like I said I haven't battle tested pleskbackup in a production evironment yet.

Good luck.
J

 

[b-nut]

James - Thanks for your insight. Are using Plesk 8.0 or 8.0.1? Can you share the dump and gzip scripts you are using for the full backups? Is it just a one-line cron job or a more complicated shell script with rotation and such? How much disk space do the full backups take up?

I'm suprised nobody using rsnapshot has piped in with their results. Rsnapshot is a very well received backup script, but maybe it's GUI-less interface scares off the the majority of Plesk's "point-and-click" users. I'm testing out a few different rsnapshot configuration scripts this week and I will post some results of the testing here.

 

[Garp]

Some people - including me - use Sysbk (http://forum.swsoft.com/showthread.php?s=&threadid=12784.
Get the patched version (mysql admin username is not root in Plesk here http://thelocust.org/projects/patches/ .

What we do is the following.

* Daily backup (saturday to thursday) script which starts a sysbk backup which also transfers the backups to a central backup server.

* Weekly backup script (friday night) that first invokes pleskbackup to backup the entire plesk config, then runs sysbk which also backs up the just made pleskbackup to the backup server and then removes the local pleskbackup.

* On the central backup server, run a script to remove the backups older then X days to make sure the filesystem doesn't fill up.


I will also take a look at rsnapshot, i didn't know that tool. Looks interesting.

 

[b-nut]

bump

 

[MikePaul99]

From my experience, the plesk backups are fine for per domain backups, but if your server takes a **** and you need to get back up in a hurry, rsnapshot or something like it is crucial.

And, Plesk's utility generates a lot of load where rsnapshot (rsync) doesn't since there's no compression going on.

Here's 2 real world examples of what happened to me using both types of backups:

Plesk:

Christmas Eve last year the HD died, since I had the Plesk backup running every day, I thought it would be a snap to get it back.. Not the case. I had to use commands not listed in the manual (they are in the FAQ's though, weird huh..) and create domain maps and whatnot to load back the domains to pre-backup specs, with plenty of additional tweaks and adjustments inside the panel once the backup was loaded.

That was with both the full server backup option and the individual domain option, the former probably the reason for the wacky commands.

Total time required to get the Domains back up: 2+ hours (Mind you I had to learn the hard way about the commands to restore).

That's not including getting the rest of the server up.

rsnapshot:

Okay, ever since the Christmas fiasco I've been using rsnapshot to automate the rsync backup keeping a rolling incremental backup every 4 hours with 7 days and 4 weeks worth of 4 hour backups on my backup server. Last week I get the bright idea to upgrade my kernel without testing it first.. Blamo, server unusable. Doh!

Log in via remote console, mount the drives, mount the backup server, transfer the last backup.

Total time required to get the entire server back to pre upgrade: less than 30 mins.

No re-image then restore, just restore. Saves at least an hour right there.

See which way I'm leaning here?

Plesk positives:

Individual domain backups on demand from the panel

Plesk negatives:

Can generate a lot of load
Can't do incremental backups <<< that's the kicker
Not as easy to restore as it is to back up
Doesn't save all settings
Separate backup needed for the rest of the server
No ability to restore individual files/directories


rsnapshot positives:

Easy to set up with tons of options
Very little load
Whole server can be backed up or just individual directories
Incremental backups, as many as you want
Scripting available for separate MySQL dumps or other custom needs
Ability to restore individual files/directories
Restoring is a breeze

rsnapshot negatives:

Requires an rsync server or extra drive
Can't control from panel

Incremental backups are sweet 'cause the full file is only saved if it was changed, to get the same in Plesk you'd have a bunch of full sized complete backups wasting space.

Well, this post is just about long enough, but long story short: use rsnapshot.

 

[b-nut]

Thanks MikePaul99,

You hit the nail on the head! This is exactly the type of real world experience I was looking for. What directories are you backing up? Can you share your configuration files for rsnapshot?

Also, Plesk re-built their backup utitlities for plesk 8. The new utitlities might be a little bit more robust now but I'm not sure if they added the ability to do incremental backups. I'll look into that.

I'll be setting up rsnapshot today. I have a second drive to store incremental backups as well as a NAS for archiving the backups. Once I get it ironed out I will post configurations here. I'm sure it will help some people out.

Brian

 

[MikePaul99]

I dump the entire server leaving out /tmp and a couple of cache directories specific to one of my sites.

As for the *.conf for rsnapshot, it's basically the stock files with my server info in place of their examples. That's the best part of the whole deal, easy to set up, well besides being free.

Should be easier for you since you have an extra drive, I had to set up keys and whatnot to automate the login from my backup server but that's something that should be done anyway. No passwords means those brute force kiddies are just wasting CPU cycles heh.

Good luck!

 

[shalako]

Currently using rsnapshot to dump /etc, /home, /root, /usr, and /var. Another script dumps all the mysql databases locally, and rsnapshot grabs this dump with the rest each night.

This works great for grabbing a version of a file or directory from yesterday or last week.

Our production server uses RAID 1 to mirror the drives, but I'm still concerned about how to go about restoring the drive completely, if need be. I guess it would require reinstalling the OS, then uploading everything from rsnapshot, then loading the databases?

Questions/Concerns:

(1) rsnapshots are downloaded over WAN link to local office via DSL. A full download of the drive takes nearly 24 hours. I can't imagine what the upload would require. rsnapshot is great because the full download is only done the first time, then everything else is iterative.

(2) Am I grabbing everything needed for a full restore? Anything Plesk-centric I'm not thinking of?

(3) We're currently on Plesk 8.0.1, and I actually don't know what current backup functionality is built-in (researching now). When we were on 7.5.4 there was PSADUMP which sounded good because it grabbed everything: the server config, domains, email, databases, etc. But this didn't work for us because the WAN link would mean we'd be downloading around the clock.

Suggestions?
Thanks a lot.

 

[MikePaul99]

You dont really need the MySQL dumps since you're backing up the /var/lib/mysql directory where the databases are stored, worse case scenario is you'd have a few corrupted tables if it was copied during an access, but I've not yet run into that. I just stopped mysql, restored the database files themselves and off I went, and mine went down while it was under load at 3 in the afternoon.

Don't forget boot, that's where you're kernel lives ; )

In most cases you'll probably only have to restore a few files or directories which won't cause too much of a hassle with a slow connection, but you have to plan for the worst. Check what kind of backup solutions your DC offers, if they don't have something that can fit the bill, look elsewhere. There are a number of good backup providers that can be had for $20-$30/100GB of space per month.

As for the functionality of the backup methods built into Plesk, see my post above. Good for window dressing, but when balls hit the wall, it just doesn't offer enough.

The worst time to find out your backups are worthless is when the requests are pouring in and your server is just a brick in the corner drooling on itself. Bottom line is you have to know your systems, what's installed and where. I just backup everything to be safe.

 

[b-nut]

I've got rsnapshot all set up and it's creating the backups like a charm. I'm going to agree with MikePaul99 here that this is probably a better solution than pleskbackup. It's extremely easy to set up and once you have it configured it's totally hands off. Here's a quick how-to to get you up and running.

1. First download the rsnapshot program to your server and install: http://www.rsnapshot.org/downloads.html
I used the RPM version.

2. Once you have rsnapshot installed edit the config file: /etc/rsnapshot.conf
I only had to edit two sections of the conf file.
The first part tells rsnapshot where to store the backups. In my case /backups/ is the path to a second harddrive.

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
snapshot_root   /backup/.snapshots/

Then further down towards the end of the file we add the directories we want backed up. I'm backing up the following directories: home, etc, usr, var, root, boot

# LOCALHOST
backup  /home/	localhost/
backup  /etc/	localhost/
backup  /usr/   localhost/
backup  /var/   localhost/
backup  /root/  localhost/
backup  /boot/  localhost/

That's it for configuration! Just make sure the syntax is right with this command:

rsnapshot configtest


3. Now to automate everything add the following two cron jobs to the root crontab:

0 */4 * * * /usr/bin/rsnapshot hourly

30 23 * * * /usr/bin/rsnapshot daily

This runs the hourly backup every 4 hours and runs the daily backup each day at 11:30PM.

*note: the path to rsnapshot on my system is /usr/bin/rsnapshot, this may differ depending on how you installed it. Make sure your cron job has the correct path to rsnapshot.

All done. Now your system is being backed up every 4 hours. This is a very basic usage of rsnapshot, but there are many more options in the config file. You can read the complete how-to here: http://rsnapshot.org/howto/1.2/rsnapshot-HOWTO.en.html

Brian

 

[DigitalCrowd]

A couple of questions for rsnapshot folks...

a) If your doing long-distance backups... and you have ability to restore a server to a default "system image" with Plesk installed... which directories do you recommend backing up on the server that syncing back those directories would yield a fully functioning system (for the most part).

b) I use rsync to backup most of my stuff. I don't use incremental. One thing I noticed, on the backup server, all files created take ownership as the ssh account used to login. When you do the restore, is it able to restore correct ownership permissions or am I facing a nightmare?

I wish Plesk Backup would allow you to ignore certain areas, like mysql dumps or mail account dumps on a per domain basis. This way, I could use rsync for the big stuff like mail accounts and databases, but use plesk backup for the "snapshot" feature for restoration if a "full system backup" goes wrong.

[EDIT: GUESS IT WOULD HELP IF I READ THE ENTIRE THREAD FIRST! I HAVE ADHD, SORRY! FOR THE DUPE QUESTION IN QUESTION A.]

 

[DCNet_James]

b-nut, sorry didn't get back to you on this, seems you have an answer. My scripts are custom, but essentially I run the plesk backup every night and the weekly's are full backups using dump and then gzip seperately. To accomplish this, I have 3 crons, one to run a perl script that runs pleskbackup for dailies, which also backs a few directories and configs that pleskbackup misses and then stores them on an NFS share, another script runs once a week which runs dump and gzip to dump the entire filesystem to the same NFS share and compress it, and a 3rd that cleans up the NFS share for backups older than 30 days.

This does require quite a bit of space, but the 30 day rotation helps. I have a Netapp filer with about a terrabyte of space, but you could simulate this too by running an NFS server with an array or using something like Openfiler or FreeNAS.

 

[MikePaul99]

Originally posted by DigitalCrowd
A couple of questions for rsnapshot folks...

b) I use rsync to backup most of my stuff. I don't use incremental. One thing I noticed, on the backup server, all files created take ownership as the ssh account used to login. When you do the restore, is it able to restore correct ownership permissions or am I facing a nightmare?

>>Should be keeping ownership, check your config if you edited it. Make sure you're backup system isn't automatically taking ownership of your backup directory's contents.

I wish Plesk Backup would allow you to ignore certain areas, like mysql dumps or mail account dumps on a per domain basis. This way, I could use rsync for the big stuff like mail accounts and databases, but use plesk backup for the "snapshot" feature for restoration if a "full system backup" goes wrong.

>>Regardless, if you use rsnapshot you have a snapshot, just restore the files you need if the complete doesnt work IE, databases/www directories and such. Here's an example:

Let's say you hosed your system and you missed a needed directory for a full backup. As long as you have /var (and all subs of /var) you're set, at least as far as your sites are concerned. Only things that would be lost are the DNS settings, but you'd lose those anyway with a Plesk backup, least that's been my experience.

Play by play:

1) On a fresh Plesk install, recreate your sites hosting
2) Stop plesk, or at the very least MySQL
3) Make a copy of /var/lib/mysql
4) Upload your backup of /var/lib/mysql
5) Restart MySQL
6) Test Plesk and your sites

If that doesn't work, repeat 1 thru 5 and dump your site specific databases
6) Stop MySQL
7) Restore the copy of /var/lib/mysql you made earlier
8) Start MySQL
9) Restore the databases from the dumps you created
10) Test again

1 thru 6 should do the trick, and that's only if you aren't able to restore fully using your full backup. That's why I backup everything, that way even if I have to re-image, I can upload the entire backup and be set.

The important bits are /var and /etc - make sure you at least have those. (/home too if that's where your sites live)

Note: I am open to better suggestions if anybody has any.

 

[breun]

Check out BackupPC. It is so great it hurts my eyes.

 

[Garp]

Beun,

Are you currently using BackupPC? Any tips and tricks? I'm using sysbk right now, but BackupPC looks fantastic.

Looking forward to your reply

 

[breun]

Yes, we use BackupPC. Also a local pleskbackup runs nightly on each server and those are sent off to a remote host on a weekly basis.

The docs on the site are pretty good. Joining the backuppc-users mailinglist will probably get you all the answers to any issues that may come up.

 

[ppc]

Originally posted by Garp
Get the patched version (mysql admin username is not root in Plesk here http://thelocust.org/projects/patches/ .

There are two "plesk 6.0" patches on that site, which one do I install?

Also - what exactly do these patches do?

Originally posted by Garp
* On the central backup server, run a script to remove the backups older then X days to make sure the filesystem doesn't fill up.

Would you garp, or someone else have such a script that removes old backup files?


Thanks

 

[tha_sun]

pleskbackup and pleskrestore

We have re-installed our server from backups yesterday. It was (and is) running Plesk 8.1.1.

With this post I want to warn everyone who thinks that pleskbackup/pleskrestore might be a working complete backup and restore solution. It is not.

While pleskbackup seemed to have backup all server-settings/client/domain data and pleskrestore successfully created a map file of these, pleskrestore failed to restore at least 8 of 27 domains. Even after repeated execution.

Details: clients/domains were created, mails have been restored, but webspace and databases were empty. Some of these domains were created shortly, so there should not have been any reason for not restoring them. Furthermore, some latin1-MySQL-databases were only partially restored. Additionally, pleskbackup does not backup all directories in a vhost (virtual host) - so if you f.e. have a Perl application that stores data outside of ./cgi-bin or ./httpdocs, that data is not included in backups.

We are very lucky to have a custom backup script running, which backups all webspaces, databases and maildirs manually. So fortunately, we learned this the 'soft way'.

Since pleskbackup/pleskrestore does not even restore server settings (like Plesk server-settings, UI settings aso.), we will not use pleskbackup anymore. It's not only resource-intensive, it's simply ****.

If you need to backup and restore your server, you should manually create an tar archive of the following directories (valid for SuSE):

/srv/www/vhosts
/var/qmail
/etc
/var/lib/mysql (or use mysqldump)
/var/spool/cron/tabs

If you did backup /var/lib/mysql or the database 'psa', all settings in Plesk can be restored.

 

[breun]

Would you garp, or someone else have such a script that removes old backup files?

I don't have the script anymore (not using it anymore), but the idea is to use find with the option to find files older than X days and remove those.