We use IBM DB2 10.1 on Windows Server 2008 R2
Majority of the DB2 space is occupied by fields of the type BLOB and CLOB.
Currently, full backup of the entire database takes very long time and goes beyond time limits.
We tried to use incremental backup to speed up backup, but problem did not go away as BLOB and CLOB fields are still
being pulled into the backup, irrespective of whether they were changed, so it is almost the same as doing a full backup.
We decided to proceed with following approach:
Create a new tablespace and put there tables with BLOBs and CLOBs. Old data will be archived once a year, and this tablespace every day.
Once the size of tablespace exceeds a certain limit, we will create a new space and start writing new data to it, and so on.
Problem occurs with restoration when using above mentioned approach.
Offline backup steps:
Full backup of all data is done.
DB2 -svl%LOG% BACKUP DATABASE %DB_NAME% TO %DB_PATH_BACKUP% COMPRESS EXCLUDE LOGS WITHOUT PROMPTING
Logs are copied.
Backup of a separate tablespace is done.
B2 -svl%LOG% BACKUP DATABASE %DB_NAME% TABLESPACE (TSPACEGEN1) TO %DB_PATH_BACKUP% COMPRESS EXCLUDE LOGS WITHOUT PROMPTING
Logs are copied.
Restoration approach:
Full backup is restored.
RESTORE DB EAPOBLOB FROM "..." TAKEN AT ... REPLACE HISTORY FILE WITHOUT PROMPTING
Logs are copied and ROLLFORWARD performed. So far everything is OK.
ROLLFORWARD DATABASE COMMDB TO END OF LOGS
Backup of a separate tablespace is restored.
RESTORE DATABASE EAPOBLOB TABLESPACE (TSPACEGEN1 ) FROM "..." TAKEN AT WITHOUT PROMPTING
Logs are copied and ROLLFORWARD performed. So far everything is OK.
ROLLFORWARD DATABASE COMMDB TO END OF LOGS
!! when we try to connect to DB:
The following errors were received:
A connection attempt was unsuccessful.
Summary
SQL1117N A connection to or activation of the database cannot be made because of ROLLFORWARD PENDING.
Is this feasible approach for backup and are steps we take to backup and restore adequate?
Is there a better way to speed up backup, while still maintaining same level of resilience?
Related
I am confused with the size of the file I backup with SSMS and Query.
If I create a file from SSMS in its default folder something like "C:\Program Files\Microsoft SQL Server\MSSQL14.NAMEDINSTANCE\MSSQL\Backup" the outfile say Db1.bak is about 198292 KB
Same database if I backup with the query "backup database Db1 to disk='D:\Db1.bak' the file size is just 6256 KB
Sometimes the other database say Db2 gives the same filesize i.e 6256 KB(Both Db1 and Db2 have identical(same) schemas just data in it are different.)
And backup with SSMS gives 33608 KB which seems satisfactory.
I also tried verifying all database in SSMS like this RESTORE VERIFYONLY FROM DISK = 'D:\BACKUP\Db1.bak'
GO and result gives valid in every database check.
I also tried deleting Db1 from SSMS and restoring the less KB file and checked some data of few tables (Not All) and it seems showing all data in tables properly but the filesize dissatisfies me.
Thank You.
I suspect that,like initially mentioned, you have compression on my
default, and using the GUI, with the settings is not making use of
that (and that if you selected to Compress in the GUI, you'd get a
similar size)
If the server option backup compression default is on, even if you don't mention it in your backup command, compression will be applied. So in both cases there would be compressed backup. But it's easy to see, just run this command for both backups:
restore headeronly
from disk = 'here_the_full_path_with_filename';
In the 5th column you'll get the flag if your backup is compressed.
But the cause of this difference is another one, and you'll see it when run restore headeronly: you made multiple backups to the same file.
You used backup command with noinit from SSMS, and the same file name, so now this file contains more than one backup, and restore headeronly will show them all.
I'd like to create a database backup using SSMS.
The backup file will be a .bak file, but I would like to ignore 1 column in a certain table, because this column isn't necessary, but it takes up 95% of the backup size.
The column values should all be replaced by 0x00 (column type is varbinary(max), not null).
What's the best way to do this?
FYI: I know how to generate a regular backup using Tasks => Back Up..
There is a long way of doing what you ask. Its basically create a new restored database, remove the non required data and then do a new backup again.
Create a Backup of the production database.
Restore the backup locally on production with a new name
Update the column with 0x00
Shrink the database (Shrink is helpful when doing a restore. This wont reduce the bak file size)
Take the backup of the new database (Also use Backup Compression to reduce the size even more)
Ftp the bak file
If you only needed a few tables, you could have used bcp but that looks out of the picture for your current requirement.
From SQL Server native backups, you can't. You'd have to restore the database to some other location and then migrate usefull data.
You can create a copy of your table without the column and backup using filegroups https://msdn.microsoft.com/en-us/library/ms191539(SQL.90).aspx
Im deploying my db, i more or less emptied the db(data) and then created a backup.
the .bak file is over 100mb.
why is this?
how do i get it down?
im using sql server 2008
EDIT
When you back up, please note that SQL Server backup files can contain multiple backups. It does not overwrite by default. If you choose the same backup file and do not choose the overwrite option, it simply adds another backup to the same file. So your file just keeps getting larger.
Run this and all will be revealed:
select dpages *8 [size in kbs]
from sysindexes
where indid <= 1
order by 1 desc
You can also..
Do two backups in a row to have the 2nd backup contain minimal log data. The first backup will contain logged activity so as to be able to recover. The 2nd one would no longer contain them.
There is also an issue with leaked Service Broker handles if you use SSSB in your database with improper code, but if this is the case, the query above will reveal it.
To get the size down, you can use WITH COMPRESSION, eg.
backup database mydb to disk = 'c:\tempdb.bak' with compression
It will normally bring it down to about 20% the size. As Martin has commented above, run also
exec sp_spaceused
To view the distribution of data/logs. From what you are saying, 1.5 MB for first table... down to 8kB on the 45th row, that accounts for maybe tens of MB, so the rest could be in the log file.
I am running SQL Server 2005.
My db backup scheme is:
Recovery model: FULL
Backup Type: Full
Backup component: Database
Backup set will expire: after 0 days
Overwrite media: Back up to the existing media set, Append to the existing backup set
The db is writing to 250GB drive (232GB actual).
My _Data.mdf file is over 55GB and my _Log.ldf is over 148GB.
We ran into a situation where our drive was filled today. I moved our ab_Full.bak and ab_Log.bak files to another drive to make space - about 45GB. Five hours later, free space is at 37GB.
I'm new to managing SQL server; so, I have some basic questions about my backups.
I know I need to update the db to start managing the transaction log size to help prevent this problem in the future. So, assuming I have enough free space, I:
1. right click the db and choose Backup
2. set 'Backup Type' to 'Transaction Log'
3. change 'Backup set will expire' after to 30 days
4. click 'ok'
My understanding is this will move 'closed' transactions from the transaction log to a backup and truncate the transaction log.
Is this plan sound? Will I need to manually resize the log file afterwards?
Thanks for your time.
Are you backing up the transaction log at any time at all?
If you are using the FULL recovery model, then you need to back up the transaction log in addition to backing up the main database, or if you don't want to back up the log (why would you then use the FULL recovery model?) then at least truncate the log at some regular interval.
You should back up the transaction log before every full backup (and keep it as long as you keep the previous full backup) so you can restore to any point in time since the first full backup you've kept. Also, it might be worth backing up the transaction log more often (the total size is the same) in case something bad happens between two full backups.
The best procedure is to regularly backup your log file. In the mean-time, for 'catastrofic' scenarios like the one you described, you may use this snippet to reduce the size of your log:
http://www.snip2code.com/Snippet/12913/How-to-correctly-Shrink-Log-File-for-SQL
I am using SQL Express 2005 and do a backup of all DB's every night. I noticed one DB getting larger and larger. I looked at the DB and cannot see why its getting so big! I was wondering if its something to do with the log file?
Looking for tips on how to find out why its getting so big when its not got that much data in it - Also how to optimise / reduce the size?
Several things to check:
is your database in "Simple" recovery mode? If so, it'll produce a lot less transaction log entries, and the backup will be smaller. Recommended for development - but not for production
if it's in "FULL" recovery mode - do you do regular transaction log backups? That should limit the growth of the transaction log and thus reduce the overall backup size
have you run a DBCC SHRINKDATABASE(yourdatabasename) on it lately? That may help
do you have any log / logging tables in your database that are just filling up over time? Can you remove some of those entries?
You can find the database's recovery model by going to the Object Explorer, right click on your database, select "Properties", and then select the "Options" tab on the dialog:
Marc
If it is the backup that keeps growing and growing, I had the same problem. It is not a 'problem' of course, this is happening by design - you are just making a backup 'set' that will simply expand until all available space is taken.
To avoid this, you've got to change the overwrite options. In the SQL management studio, right-click your DB, TASKS - BACKUP, then in the window for the backup you'll see it defaults to the 'General' page. Change this to 'Options' and you'll get a different set of choices.
The default option at the top is 'Append to the existing media set'. This is what makes your backup increase in size indefinitely. Change this to 'Overwrite all existing backup sets' and the backup will always be only as big as one entire backup, the latest one.
(If you have a SQL script doing this, turn 'NOINIT' to 'INIT')
CAUTION: This means the backup will only be the latest changes - if you made a mistake three days ago but you only have last night's backup, you're stuffed. Only use this method if you have a backup regime that copies your .bak file daily to another location, so you can go back to any one of those files from previous days.
It sounds like you are running with the FULL recovery model and the Transaction Log is growing continuously as the result of no Transaction Log backups being taken.
In order to rectify this you need to:
Take a transaction log backup. (See: BACKUP(TRANSACT-SQL) )
Shrink the transaction log file down
to an appropriate size for your needs. (See:How to use DBCC SHRINKFILE.......)
Schedule regular transaction log
backups according to data recovery
requirements.
I suggest reading the following Microsoft reference in order to ensure that you are managing your database environment appropriately.
Recovery Models and Transaction Log Management
Further Reading: How to stop the transaction log of a SQL Server database from growing unexpectedly
One tip for keeping databases small would be at design time, use the smallest data type that you can use.
for Example you may have a status table, do you really need the index to be an int, when a smallint or tinyint will do?
Darknight
as you do a daily FULL backup for your Database , ofcourse it will get so big with time .
so you have to put a plan for your self . as this
1st day: FULL
/ 2nd day: DIFFERENTIAL
/ 3rd day: DIFFERENTIAL
/ 4th day: DIFFERENTIAL
/ 5th day: DIFFERENTIAL
and then start over .
and when you restore your database , if you want to restore the FULL you can do it easily , but when you need to restore the DIFF version , you backup the first FULL before it with " NO-recovery " then the DIFF you need , and then you will have your data back safely .
7zip your backup file for archiving. I recently backed up a database to a 178MB .bak file. After archiving it to a .7z file is was only 16MB.
http://www.7-zip.org/
If you need an archive tool that works with larger files sizes more efficiently and faster than 7zip does, I'd recommend taking a look at LZ4 archiving. I have used it for archiving file backups for years with no issues:
http://lz4.github.io/lz4/