I have data on several machines that I want to backup in away that I can restore to certain points in time.
From what I read Snapshot Replication achieves this (as opossed to back-up that clobbers previous results).
The main motivation is that if the data files are ransacked, and encoded, then if I just back-up I can end up in a state where the backed up files are also encrypted.
One way to do this is by using 2 Synology NAS machines where I can have:
rsync processes to back-up files from multiple machines into a NAS1
apply Snapshot Replication from NAS1 to NAS2
In this way, if the data is hijacked at certain point, I can restore the data to the last good state by restoring NAS2 to previous point in time.
I would like to know if:
Snapshot Replication is the way to go, or there are other solutions?
are there other ways to achieve Snapshot Replication, e.g. with single NAS?
I have an older Synology 2-Bay NAS DS213j.
Assuming that I buy a second, newer, NAS (e.g. DS220j), are the 2 NAS machines expected to work together?
Thanks
I found out that Hyper Backup can save snapshots in time, so I'm using it instead of Snapshot Replication
Related
i'm looking for an alternative for Hyperoo, one of the best backup solutions for VMs backup..
I tried many softwares, like Veem, Iperius, Altaro, Acronis ecc but everyone use Microsoft checkpoints and create AVHDX files, sometimes it happens that the backup has some problems and the avhdx remains open, I find myself forced to merge that punt hoping everything goes well.
All these programs make a false incrementally backup.
Even with every small modification the vhdx changes a little. The backup program checks that the virtual machine has changed and makes a full backup.
Hyperoo creates one full vhdx file and then many rollback files, one file each day.
I understand that you want safety and performance for your Hyper-V VM backups. Backup and restore is a stressful experience. As you mentioned the solutions use the Hyper-V checkpoint technologies and I don't know something else.
We tested a lot backup tools and end up with Veeam. Usually the backup and restore works. Unfortunately it put a lot weight on the infrastructure during backup (storage is slow..) and sometimes the backups fails because of this. To evade this we setup fixed periods outside the work time. Keep in mind that we use the Backup only for server VMs (not VDI).
I would recommend Veeam as backup solution, but maybe you can take a look at Commvault.(https://documentation.commvault.com/commvault/v11/article?p=31493.htm)
Greetings.
I have a VPS with OVH. There are two options in there, Automated Backup and Snapshot. What is the difference between both and which one should I enable so I don't lose the data and the configuration on the server. It took me quite some time to optimize my server so I don't want to go through that pain again. Plus, there's like 30GB of data uploaded. I don't want to risk that even.
This explains it: https://www.ovh.com/world/vps/backup-vps.xml
So basically the automated backup is done automatically everyday and replicated in 3 different sites to ensure nothing is lost.
Snapshot seems like you have a max of two different snapshot and that you should do them yourself (like a VM snapshot).
I've spent a number of days looking into putting up two Windows Servers on Amazon, a domain controller and a remote desktop services server but there are a few questions that I can't find detailed or any answers for:
1) When you have an EBS backed instance I assume this means that all files (OS/Applications/Pagefile) etc are all stored on EBS? Physically in the datacentre, lets assume I have 50 gig of OS files/application data etc, are these all stored on just one SAN type device? What happens if that device blows up or say that particular data centre gets destroyed. Is the data elsewhere? What is the probability that your entire EBS volume can just disappear?
2) As I understand it you can backup your EBS instance to S3 with snapshotting. I assume you can choose how often to snapshot (say daily?). In my above scenario if I have 50 gig of files, and snapshot once a day. Over 7 days will my S3 storage be 350 gig or will it be 50 gig + incremental changes I have made over the week?
3) I remember reading somewhere that the instance has to go offline to snapshot. If that is the case does it do this by shutting down the guest OS, snapshotting then booting up or does it just detach the data, prevent you from connecting while it snapshots, then bring it back to the exact moment before it went for a snapshot.
4) I understand the concept of paying per month per gig of space but how I am concerned about the $0.11 per 1 million I/O requests. How does that work when I am running a windows server? I have no idea how many I/O requests a server makes to its disks. I am assuming a lot of the entire VM is being stored on an EBS volume. Is running a server on the standard EBS going to slow it down radically?
5) Are people using the snapshot to S3 as their main backup are are people running other types of backup for Data?
Sorry for the noob questions - I'd appreciate any partial answers, answers or advice anyone could offer me. Thanks in advance!
1) amazon is fuzzy on this. They say that data is replicated within the AZ it belongs to and that if you have less than 20GB of data changed since the last snapshot your annual failure rate is ~ 0.1-0.4%
2) snapshots are triggered manually, and are done incrementally
3) Depends on your filesystem. For example on a linux box with an xfs volume you can freeze IO to the volume, do your snapshot (takes only a second or so) and then unfreeze. If you take a snapshot without doing something similar you run the risk of the data being in an inconsistent state. This will depend on your filesystem
4) I run all my instances on EBS. You probably wouldn't want your pagefile on EBS, it would make more sense to use instance storage for that. The amount of IOs you use will be very dependant on the workload. The IO count depends heavily on your workload - an application server does a lot less IOPs than a database server for example. You're unlikely to use more than a few dollars a month per volume if you're running particularly IO heavy operations
5) Personally I don't care about the installed software/configuration (I have AMIs with that all setup so I can restore that in minutes), I only care about the data. I back that data up separately (S3 & Glacier). Partly that's because I was bitten by a bug EBS had about a year ago or so where they lost some snapshots
You also use multiple strategies, as Fantius commented. For example on the mongodb servers I run the boot volume is small (and never snapshotted or backed up since it can be restored automatically from an AMI), with a separate data volume containing the actual mongodb data. The mongodb volume is snapshotted as well as storing dumps on S3. Snapshots are an efficient way of creating backups (since you're only storing incremental changes) however you can't transfer them out of your EC2 region, whereas a tarball on S3 can easily be copied anywhere.
So here is the deal:
Background
A Hyper-V VM can handle a differencing disk mode, where one can set the original VHD file in a read-only state and create a new vhd which keeps track of, and persists, the changes. The advantage here is you can easily create new VMs without having to reinstall Windows, etc.
Problem
What I am looking for is something similar, but for SQL Server databases. We do all of our development locally and then we have a box that has X instances run on it (1 for each developer). We then have a process which copies the production backups that are made and restores them to these instances. After this is complete, it checks-out a branch that a developer chooses (of SQL scripts) and runs the scripts on the instance. This way they can test their code on production data prior to it actually hitting production. However, it is a real pain to have a copy of all our production dbs for each instance-- it would be nice to have 1 set of them and have a differential option which just persists the changes made. Is this possible or am I dreaming?
Possible solution
One solution I thought of is just to use an actual differencing disk VHD. I would create a base VHD that has our production backup databases, which would be modified/created night with the production database. I then would have it modify/create differencing disks and apply the scripts to each differencing disk. This way we have 1 copy of the dbs, and the developer's changes are recorded to a separate differencing disk. However, I was hoping to accomplish this in SQL server.
Basically the conclusion I have come to is to try and automate the process of differencing disks as below:
Create a new VHD on a network share- we'll call this NAS1.
Mount the VHD from NAS1 on a machine that acts as a SQL processor (we'll call this SQLPROCESS1.
SQLPROCESS1 performs the following actions.
Copy BAK SQL files from production to SQLPROCESS1 (this might take a while, but this entire #3 could be put into a threaded application, so it could be copying multiple and restoring at the same time).
Restore files on SQLPROCESS1 and point data files (mdf, ldf) to reside on the new VHD.
Optional: Change SQL dbs to SIMPLE backup mode and use SHRINKFILE since we'll be using them solely for development (and don't need backups). This can save us a lot of space.
Detach all dbs.
Detach the VHD.
Create differencing disk from parent on NAS1.
Copy differencing disk X number of times (as needed per instance or developer).
Optional: We use a central server called TEST1 for testing and this is where we are going to mount each differencing disk-- 1 per instance or developer.
We'll first need to detach all dbs from each instance.
Then we'll need to unmount/detach the existing differencing VHDs if there are any.
Attach differencing disk(s).
Reattach all dbs in SQL Server.
Optional: run SQL scripts from a code repository branch as specified per developer.
References:
http://obligatorymoniker.wordpress.com/2010/08/21/how-to-create-a-differencing-vhd-that-refers-to-a-parent-vhd-that-is-on-a-network-share-from-windows-7/
To automate I'd use a simple set of batch files, VBS, or PowerShell.
Edit: Just tried this and it works great! Developers now have their own instance and it only records their changes.
I'm doing a webapp and need a backup plan. Here's what I've got so far:
nightly encrypted backup of the SQL database to Amazon S3 and my external drive (incremental if possible, not overly familiar with PostgreSQL yet, but that's another thread)
nightly backup of my Mercurial repo (which includes Apache configs, deploy scripts, etc) to S3 (w/ local backups via Time Machine)
Should I add anything else, or will this cover it? For a gauge of how critical the data is/would be, it's a project management app along the lines of Basecamp.
Weekly full backup of your database as well as nightly incremental ones as well perhaps?
It means that if one of your old incremental backups gets corrupted then you have lost less than a week of data.
Also, ensure you have a backup test plan to ensure your backups work. There are a lot of horror stories going around about this, from companies that have been doing backups for years, never testing them and then finding out none of them are any good once they need them. (I've also been at a company like this. Thankfully I spotted the backups weren't working before they were required and fixed the problems).
One of the best strategies that worked for me in the past was to have the "backup" process just be the same as the install process, i.e. we fully scripted in linux the server configuration, application creation, database setup, etc etc so a install would look like:
./install.sh [server] [application name]
and the backup/recovery
./install [server] [application name] -database [database backup file]
In terms of backup the database was backed up fully (MySQL database), by a cronjob
This pretty much ensured that the recovery was tested every time a new instance was deployed, and the scripts ended up being used also to move instances when hardware needed replacement, or when a given server was a getting too much load from a customer.
This was the setup for a Saas enterprise application that I worked a few years back, so we had full control of the servers.
I would if you can change from a incremental back up to a differential. If you have a incremental then you would have to apply the weekly full backup and then every incremental following that. If one of your incrementals fails early in the week, then all your subsequent backups will fail too.
However if you use a differential then each differential contains all the changes since the last back up. so even if one of the back ups failed earlier in the week you would still be able to recover fully if you have a sucessful recent backup.
I hope i am explaining this well!
:)