We are using MarkLogic 9 version. We have developed API above MarkLogic
We are doing daily full backup for last 7 days.
Now we are upgrading our MarkLogic server to 10 & as a part of disaster recovery if due to some reason upgrade fails and we need to restore backup from yesterday
I want to understand
How restore process works, do restore for each stand and then serve from remaining ones?
Whether API requests will be served during restore process ?
If API requests will be served then which data will be used to serve that request ?
Do we need to go for downtime as a part of restore process ?
If we go for incremental backup & then restore, will there be any difference to above points?
The Restore will restore the entire backup to a temporary directory. Once all of the stands/forests have been copied to disk it will swap the current forests/stands with the restored ones.
API requests will still be serviced by the active stands, with the existing data. It will not use the restored data until after the restore completes.
There will be a very small amount of downtime as the existing forests shutdown, and the restored forests start up. This is usually just a few seconds, but it does depend on how big the forests are.
There is no difference in the behavior if you are restoring a full + incremental/s or only a full backup.
Be aware that you will need enough disk space for both the current data and the restored data, as they will coexist for a period of time.
Backup and Restore Transactions
Phases of Backup or Restore Operation
Related
I need to take two backups before and after the day end process. If the EOD process starts at 10.00 p.m. The backup should contain all the data right at 10.00 p.m. before starting the EOD and the backup process should not impact the EOD process as well. Is there a way to achieve this?
Please note that I need to retrieve RMAN backups for disk and then tape.
A "snapshot" type of backup would only be necessary if you are running in NOARCHIVELOG mode, and you'd have to shutdown the entire database to do it as a "cold" backup (you can't get a logically consistent backup without transaction logs while the database is open for read/write activity). This would presumably impact your end-of-day process.
Assuming that the database is in ARCHIVELOG mode, and that you can run your backup as a "hot" backup while the database is up and running, you do not need to worry about the timing of your backup at all.
Run a backup whenever it makes sense based on system load or activity (being sure to backup the archive logs too), and if you need to recover from a backup later then recover to the exact point in time that you need - before or after your end-of-day process. See the documentation for Point in Time Recovery options: https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/rman-performing-flashback-dbpitr.html
The restore and recovery operation will restore from the backup and then re-apply all transactions to bring the database back to the desired point in time. The only thing the timing of the backup job would affect would be the number of transactions that might need to be re-applied after the data files are restored.
I have data on several machines that I want to backup in away that I can restore to certain points in time.
From what I read Snapshot Replication achieves this (as opossed to back-up that clobbers previous results).
The main motivation is that if the data files are ransacked, and encoded, then if I just back-up I can end up in a state where the backed up files are also encrypted.
One way to do this is by using 2 Synology NAS machines where I can have:
rsync processes to back-up files from multiple machines into a NAS1
apply Snapshot Replication from NAS1 to NAS2
In this way, if the data is hijacked at certain point, I can restore the data to the last good state by restoring NAS2 to previous point in time.
I would like to know if:
Snapshot Replication is the way to go, or there are other solutions?
are there other ways to achieve Snapshot Replication, e.g. with single NAS?
I have an older Synology 2-Bay NAS DS213j.
Assuming that I buy a second, newer, NAS (e.g. DS220j), are the 2 NAS machines expected to work together?
Thanks
I found out that Hyper Backup can save snapshots in time, so I'm using it instead of Snapshot Replication
I'm using SpecFlow with Selenium for doing UI testing on my ASP.NET MVC websites. I want to be able to restore the database (SQL Server 2012) to its pre-test condition after I have finished my suite of tests, and I want to do it as quickly as possible. I can do a full backup and restore with replace (or with STOPAT), but that takes well over a minute, when the differential backup itself took only a few seconds. I want to basically set a restore point and then revert to it as quickly as possible, dropping any changes made since the backup. It seems to me that this should be able to be done very quickly, without needing to overwrite the whole database. Is this possible, and if so, how?
Not with a differential backup. What a differential backup is is an image of all of the data pages that have changed since the last full backup. In order to restore a differential backup, you must first restore it's base (i.e. full) backup and then the differential.
What you're asking for is some process that keeps track of the changes since the backup. That's where a database snapshot will shine. It's a copy-on-write technology which is to say that when you make a data modification, it writes the pre-change state of the data page to the snapshot file before writing the change itself. So reverting is quick as it need only pull back those changed pages from the snapshot. Check out the "creating a database snapshot" example in the CREATE DATABASE documentation.
Keep in mind that this isn't really a good guard against failure (which is one of the reasons to take a backup). But for your described use case, it sounds like a good fit.
When you enable RavenDB's 'Period Backups' bundle, RavenDB does the following:
Backs up the entire database.
At every interval (or 'n' minutes), RavenDB makes an incremental backup (or delta backup) of all the changes that occurred since the last interval.
I'm comfortable with this configuration with one caveat.
Every week, I'd like to "clear the ledger" and force RavenDB to backup the entire database and resume making incremental backups from this new starting point.
How can I do this in an automated fashion?
From the Raven.Backup utility documentation:
incremental - Optional. When specified, the backup process will be incremental when done to a folder where a previous backup lies. If dest is an empty folder, or it does not exist, a full backup will be created. For incremental backups to work, the configuration option Raven/Esent/CircularLog has to be set to false.
So the solution to my problem is:
Every week, delete the dest directory.
This will force RavenDB to create a full backup.
I don't believe this is a supported scenario.
RavenDB's Periodic Backup bundle is intended to work with incremental updates, and AFAIK there's nothing to force a full update or make the bundle believe it's starting from a clean slate.
If you want to do full backups, you'll need to use Raven.Backup.exe, which can do either incremental or full backups. You can trigger it to run programmatically, via REST, and via command line utility, and it works with IIS.
I'm doing a webapp and need a backup plan. Here's what I've got so far:
nightly encrypted backup of the SQL database to Amazon S3 and my external drive (incremental if possible, not overly familiar with PostgreSQL yet, but that's another thread)
nightly backup of my Mercurial repo (which includes Apache configs, deploy scripts, etc) to S3 (w/ local backups via Time Machine)
Should I add anything else, or will this cover it? For a gauge of how critical the data is/would be, it's a project management app along the lines of Basecamp.
Weekly full backup of your database as well as nightly incremental ones as well perhaps?
It means that if one of your old incremental backups gets corrupted then you have lost less than a week of data.
Also, ensure you have a backup test plan to ensure your backups work. There are a lot of horror stories going around about this, from companies that have been doing backups for years, never testing them and then finding out none of them are any good once they need them. (I've also been at a company like this. Thankfully I spotted the backups weren't working before they were required and fixed the problems).
One of the best strategies that worked for me in the past was to have the "backup" process just be the same as the install process, i.e. we fully scripted in linux the server configuration, application creation, database setup, etc etc so a install would look like:
./install.sh [server] [application name]
and the backup/recovery
./install [server] [application name] -database [database backup file]
In terms of backup the database was backed up fully (MySQL database), by a cronjob
This pretty much ensured that the recovery was tested every time a new instance was deployed, and the scripts ended up being used also to move instances when hardware needed replacement, or when a given server was a getting too much load from a customer.
This was the setup for a Saas enterprise application that I worked a few years back, so we had full control of the servers.
I would if you can change from a incremental back up to a differential. If you have a incremental then you would have to apply the weekly full backup and then every incremental following that. If one of your incrementals fails early in the week, then all your subsequent backups will fail too.
However if you use a differential then each differential contains all the changes since the last back up. so even if one of the back ups failed earlier in the week you would still be able to recover fully if you have a sucessful recent backup.
I hope i am explaining this well!
:)