One user has said to me
Applying incremental db backups is
tedious, and a royal pain if you miss
a step. I wouldn't bother with the
approach on SQL Server or MySQL -
that's what transaction logs are for,
so you don't need to incorporate it
into your data model
So if i have transactions on mysql or sql server i can have a script to backup any data modified after or between date X and Y? I ask because i am currently designing tables so i can do an increment dump instead of a full.
Yes, you could backup the transaction logs rather than incorporate logic into your data model, providing the database supports it. Your previous question said that you were developing on SQLite...
Speaking from a SQL Server background, it can use transaction logs for both restoration and replication of a database. An ideal setup would have three RAID drives setup - a mirror for the OS, RAID 5 (or better) for the data, and RAID 5 (or better) for the transaction logs. The key part is the transaction logs being on their own RAID setup for optimal performance (not competing with read/write for data) and failover (because RAID is not a backup). For more info - see link.
MySQL transaction logs turns up info on the MySQL Binary Log, which also references replication so I figure there's a fair amount of carryover in approach.
On SQL Server: The key factor is the SLA time of recovery. A full a disaster recovery starts from the latest full backup, applies the latest differential backup, the applies all the log backups after the latest differential backup. If you're missing differential backups from your recovery plan, then you must start from the full backup and then apply all log backups.
The differential backup thus reduces the recovery time by eliminating the need to apply all log backups taken between the last full backup and the last differential backup.
If your database is small, differential backups don't add much advantage because the recovery time is small to start with. But on large databases it makes a difference, as the log backups can be quite large and going through days of log adds up to the recovery time. Adding differential backups can cup back the recovery time by few hours.
I'm not sure I follow your argument about designing tables with differential backup in mind, the two subjects are orthogonal.
Related
Setup:
Two MSSQL 2014 servers, one publisher/distributor and the other subscriber. Both are currently set to full recovery.
The database size is relatively small (100g), but we generate about 30g of logs every day. My goal is to have a setup where I can perform backups without downtime.
A full backup on the this database takes about an hour in which it is effectively unavailable and impacts our user base.
The replicated database is taking a log backup every hour and a full backup every day.
Questions
Is it possible and/or advised to have a Simple recovery model on the primary database (in order to prevent the need to do log and full backups), while maintaining a Full recovery model on the replicated database?
We have a MSSQL server that is backed up daily with a full backup at 0000 and with a compressed differential backup every 15 minutes from 0010. Between Aug 10 and 11 our site was hacked and all data from one table was deleted (table was not dropped). We would like to figure out the exact time and transaction that deleted the table.
We managed to do restores of the db using the differential backups so now we have it narrowed down to a 15 minute period but do not know how to proceed from there.
In short, if you don't have transaction log backups, you don't have this information available.
Elaborating a little, differential backups are the net effect of all transactions since the last full backup. Specifically, differentials don't keep track of individual transactions. To check if you have any log backups at all, check the recovery model of your database. If it's "simple", there's no way that you do. If it's not simple, query the backup tables in msdb to find where the log backups are. From there, you will have to find a way to read the log backup. I'd suggest fn_dump_dblog. You can read about it here.
I'm using SQL Server 2012 in a local environment. In fact, it is running on my Windows 7 machine. My problem is as follows: I receive a daily backup of my SQL database. Right now, I'm just restoring the whole database on a daily basis by deleting the existing one. This restore task takes quite some time to complete. My understanding of the restore process is that it overwrites the previous database with the new backup.
Is there a way for SQL Server 2012 to just modify the existing database with any changes that have occured in the new backup? I mean, something like comparing the previous database with the updated one and making the necessary changes where needed.
Yes, instead of a full backup you ill need a differential backup. Restore it to move to a "point in time" state of original database.
Make a basic research about full/differential and log backups (too many info for a short answer)
I don't believe so. You can do things with database replication, but that's probably not appropriate.
If you did have something to just pull out changes it might not be faster than a restore anyway. Are you a C# or similar dev? If so, I'd be tempted to write a service which monitored the location of the backup and start the restore programatically when it arrives; might save some time.
If your question is "Can I merge changes from an external DB to my current DB without having to restore the whole DB?" then the answer is "Yes, but not easily." You can set up log shipping, but that's fairly complicated to do automatically. It's also fairly complicated to do manually, but for different reasons: there's no "Microsoft" way to do it. You have to figure out manual log shipping largely on your own.
You could consider copying the tables manually via a Linked Server. If you're only updating a small number of tables this might work just fine, and you could save yourself some trouble. A linked server on your workstation, a few MERGE statements saved to a .sql file, and you could update the important tabled in the DB as you need to.
You can avoid having to run the full backup on the remote server by using differential backups, but it's not particularly pleasant.
My assumption is that currently you're getting a full backup created with the COPY_ONLY option. This allows you to create an out-of-band backup copy that doesn't interfere with existing backups.
To do what you actually want, you'd have to do this: on the server you set up backup to do a full backup on day 1, and then do differential backups on days 2-X. Now, on your local system, you retain the full backup of the DB you created on day 1. You then have all differential backups since day 1. You restore the day 1 full DB, and then restore each subsequent differential in the correct order.
However, differential backups require the backup chain to be intact. You can't use COPY_ONLY with a differential backup. That means if you're also using backup to actually backup the database, you're going to either use these same backups for your data backups, or you'll need to have your data backups using COPY_ONLY, both of which seem philosophically wrong. Your dev database copies shouldn't be driving your prod backup procedures.
So, you can do it, but:
You still have to do a full DB restore.
You have considerably more work to do to restore the DB.
You might break your existing backup procedures of the original DB.
I have two MS SQL 2005 servers, one for production and one for test and both have a Recovery Model of Full. I restore a backup of the production database to the test server and then have users make changes.
I want to be able to:
Roll back all the changes made to the test SQL server
Apply all the transactions that have occurred on the production SQL server since the test server was originally restored so that the two servers have the same data
I do not want to do a full database restore from backup file as this takes far too long with our +200GB database especially when all the changed data is less than 1GB.
EDIT
Based on the suggestions below I have tried restoring a database with NoRecovery but you cannot create a snapshot of a database that is in that state.
I have also tried restoring it to Standby Read only mode which works and I can take a snapshot of the database then and still apply transaction logs to the original db but I cannot make the database writable again as long as there are snapshots against it.
Running:
restore database TestDB with recovery
Results in the following error:
Msg 5094, Level 16, State 2, Line 1 The operation cannot be performed on a database with database snapshots or active DBCC replicas
First off, once you've restored the backup and set the database to "recovered", that's it -- you will never be able to apply another transaction log backup to it.
However, there are database snapshots. I've never used them, but I believe you could use them for this purpose. I think you need to restore the database, leave it in "not restored" mode -- definitly not standby -- and then generate snapshots based on that. (Or was that mirroring? I read about this stuff years ago, but never had reason to use it.)
Then when you want to update the database, you drop the snapshot, restore the "next" set of transaction log backups, and create a fresh snapshot.
However, I don't think this would work very well. Above and beyond the management and maintenance overhead of doing this, if the testers/developers do a lot of modifications, your database snapshot could get very big, even bigger than the original database -- and that's hard drive space used in addition to the "original" database. For infrequently modified databases this could work, but for large OLTP systems, I have serious doubts.
So what you really want is a copy of Production to be made in Test. First, you must have a current backup of production somewhere??. Usually on a database this size full backups are made Sunday nights and then differential backups are made each night during the week.
Take the Sunday backup copy and restore it as a different database name on your server, say TestRestore. You should be able to kick this off at 5:00 pm and it should take about 10 hours. If it takes a lot longer see Optimizing Backup and Restore Performance in SQL Server.
When you get in in the morning restore the last differential backup from the previous night, this shouldn't take long at all.
Then kick the users off the Test database and rename Test to TestOld (someone will need something), then rename your TestRestore database to be the Test database. See How to rename a SQL Server Database.
The long range solution is to do log shipping from Production to TestRestore. The at a moments notice you can rename things and have a fresh Test database.
For the rollback, the easiest way is probably using a virtual machine and not saving changes when you close it.
For copying changes across from the production to the test, could you restore the differential backups or transaction log backups from production to the test db?
After having tried all of the suggestions offered here I have not found any means of accomplishing what I outlined in the question through SQL. If someone can find a way and post it or has another suggestion I would be happy to try something else but at this point there appears to be no way to accomplish this.
Storage vendors (as netapp) provide the ability to have writeable snapshots.
It gives you the ability to create a snapshot within seconds on the production, do your tests, and drop/recreate the snapshot.
It's a long term solution, but... It works
On Server1, a job exists that compresses the latest full backup
On Server2, there's a job that performs the following steps:
Copies the compressed file to a local drive
Decompresses the file to make the full backup available
Kills all sessions to the database that is about to be restored
Restores the database
Sets the recovery model to Simple
Grants db_owner privileges to the developers
Ref:http://weblogs.sqlteam.com/tarad/archive/2009/02/25/How-to-refresh-a-SQL-Server-database-automatically.aspx
I'm interested in hearing people's thoughts about the pros and cons of database mirroring vs. log shipping in this scenario: we need to setup a database backup situation wherein there is exactly one secondary server that need not automatically pick up when the primary fails. Recovering and starting with the secondary should not have to take too long though.
Mirroring
Database mirroring is limited to only two servers.
Mirroring with a Witness Server allows for High Availability and automatic fail over.
You can configure your DSN string to have both mirrored servers in it so that when they switch you notice nothing.
While mirrored, your Mirrored Database cannot be accessed. It is in Synchronizing/Restoring mode.
Mirroring with SQL Server 2005 standard edition is not good for load balancing (see sentence above)
Log Shipping
You can log ship to multiple servers.
Log shipping is only as current as how often the job runs. If you ship logs every 15 minutes, the secondary server could be as far as 15 minutes. Making it more of a Warm Standby.
You can leave the database in read only mode while it is being updated. Good for reporting servers.
Good for disaster recovery
For backup purposes I would recommend Mirroring: it keeps an always up-to-date copy of your database with no hassle.. If you don't need automatic fail-over you need just two servers/instances. Note that High Performance mode is only available in the Enterprice (sp) edition!
Switching to the secondary database does take longer with log shipping, but it's not too bad. You'll have to manually copy any uncopied backup files, apply the transaction log backups to the secondary database, recover the secondary database, and change its role to primary. If the old primary databases accessible, you should back up its transaction log before beginning. Failing over with mirroring is somewhat simpler, and can be done automatically if you are using High Availability mode. Even when using High Performance mode, it's still a one statement operation.