I am running a MERGE statement on a large table (5M of rows) with a small log file size (2GB). I am getting an error:
Merge for MyTable failed: The transaction log for database 'MyDb' is full due to 'ACTIVE_TRANSACTION'.
Could be this solved by another action than extending the log file? I can’t really afford to extend the log file currently.
If you have a fixed log file size, you have essentially two options:
Temporarily change the recovery mode of your database from FULL to BULK-LOGGED. You'll lose the ability to do point-in-time recovery during this period, but it allows you to quickly do the operation and then go back. There are other caveats, so you need to do some research to make sure this is what you want to do.
Instead of changing the transaction log, you can adopt a batching approach to commit small batches of changes at a time, thus allowing the log to flush as needed.
My DB Admin advised that I should regularly take backup of .ldf files. Fine, this SQL post here explains this beautifully.
Consider that a transaction is being done in SQL Server. And at the same time, a scheduled process tries to access the .ldf file for backing it up.
What happens ? How this works ?
You must read Article Understanding SQL Server backup by Paul Randal. That is the best I can see which is available and can explain you in details various aspects.
Coming to your question a transaction log backup includes all information from previous transaction log back or full backup that started the log chain. Backup simply means reading information froma file(data or log) and writing it to destination disk. The transaction any would work independed of log backup running. A transaction follow a WAL(write ahead logging) protocol, for practical purposes all transaction information is first written in log file and then changes are later made to data file. So when transaction is running it would not be affected by transaction log backup job which is running both are doing different task and are muttually exclusive events. Current backup would try to backup all logs which are marked as committed and would truncate the logs if no transaction requires it. If any portion of log is committed after log backup has read that portion it would not come in current log backup but would come under further log backup.
Transacion log backup has important role in crash recovery it helps in determining what all operations has to be roll forwared and what has to be rolled back. Without transaction log backup or transaction log crash recovery is not possible
You must also read Logging and recovery in SQL Server to know about life cycle of a transaction.
The excat answer as to what acctual steps happens inside is beyond scope of discussion as nobody can exactly tell you what would happen but reading the article would give you a good idea.
Please let me know if you have any further questions.
I have been a lurker for several years and I think I have a question that hasn't been answered here.
We were in the middle of some pretty intense maintenance on or SQL server last night. Or primary database mdb file was very badly fragmented. We maintain a test copy of this database for testing and proof of concept purposes.
I had setup log shipping on the test database and without thinking I deleted the test database without removing the log shipping first. I am getting error 14421 - The log shipping secondary database SERVER.database has restore threshold of 45 minutes and is out of sync. No restore was performed for 10310 minutes. Restored latency is 0 minutes. Check agent log and logshipping monitor information.
I have removed everything I could with tsql. My research leads me to believe that this error is due to the backup job still trying to operate but I cannot find this job to remove it. It's really not a big deal but the error shows up every couple of minutes in the log.
Is there anything I can do?
Thanks in advance!
Log Shipping Information is stored in MSDB, not in the database itself. All you need to do is create a new database with the same name as the deleted database. Right click for properties, log ship tab and then uncheck the box to log ship the database. When you click okay the jobs(on primary and secondary) will be removed.
It's a test environment, I needed some data to test an Update query, but accidentally updated a column in all rows to have wrong data. Must I use a backup to restore the data back to the previous instance, or is there some secret with transaction log that I can take advantage of?
Thanks in advance.
There is a non-secret transaction log called transaction log that you can recover from to a point in time. Here's how... That annoying little file with the ldf extension is the transaction log, as opposed to the .mdf file that is your normal db data.
Unless you have truncated the transaction log (ldf) or otherwise mucked with it, you should be able to do exactly the kind of restore (undo) that you're looking for.
If your database was in full recovery mode then you can try reading transaction log using third party tool such as this one or you can try doing this yourself with DBCC LOG command.
If db is in full recovery then a lot of data is stored in transaction log but it’s not easily readable because MS never polished official documentation for this and because it’s purpose is not recovery but making sure transaction is committed correctly.
However there are workarounds to reading it like using the tool above (paid tool unfortunately but has a trial) or decoding the results of DBCC LOG yourself.
Not unless you wrapped your sql in a transaction block - begin transaction, rollback, commit. That one of the dangerous things about sql server. With Oracle you have to physically commit each transaction which is much safer imho.
I went in and checked my Transaction log the other day and it was something crazy like 15GB. I ran the following code:
USE mydb
GO
BACKUP LOG mydb WITH TRUNCATE_ONLY
GO
DBCC SHRINKFILE(mydb_log,8)
GO
Which worked fine, shrank it down to 8MB...but the DB in question is a Log Shipping Publisher, and the log is already back up to some 500MB and growing quick.
Is there any way to automate this log shrinking, outside of creating a custom "Execute T-SQL Statement Task" Maintenance Plan Task, and hooking it on to my log backup task? If that's the best way then fine...but I was just thinking that SQL Server would have a better way of dealing with this. I thought it was supposed to shrink automatically whenever you took a log backup, but that's not happening (perhaps because of my log shipping, I don't know).
Here's my current backup plan:
Full backups every night
Transaction log backups once a day, late morning (maybe hook the Log shrinking onto this...doesn't need to be shrank every day though)
Or maybe I just run it once a week, after I run a full backup task? What do you all think?
If you file grows every night at 500 MB there is only one correct action: pre-grow the file to 500MB and leave it there. Shrinking the log file is damaging. Having the log file auto-grow is also damaging.
you hit the file growth zero fill initialization during normal operations, reducing performance
your log grows in small increments creating many virtual log files, resulting in poorer operational performance
your log gets fragmented during shrinkage. While not as bad as a data file fragmentation, log file fragmentation still impact performance
one day the daily growth of 500MB will run out of disk space and you'd wish the file was pre-grown
You don't have to take my word for it, you can read on some of the MVP blogs what they have to say about the practice of log and file shrinkage on a regular basis:
Auto-shrink – turn it OFF!
Oh, the horror! Please stop telling people they should shrink their log files!
Why you want to be restrictive with shrink of database files
Don't Touch that Shrink Button!
Do not truncate your ldf files!
There are more, I just got tired of linking them.
Every time you shrink a log file, a fairy loses her wings.
I'd think more frequent transaction log backups.
I think what you suggest in your question is the right approach. That is, "hook the Log shrinking onto" your nightly backup/maintenance task process. The main thing is that you are regularly doing transaction log backups, which will allow the database to be shrunk when you do the shrink task. The key thing to keep in mind is that this is a two-step process: 1) backup your transaction log, which automatically "truncates" your log file; 2) run a shrink against your log file. "truncate" doesn't necessarily (or ever?) mean that the file will shrink...shrinking it is a separate step you must do.
for SQL Server 2005
DBCC SHRINKFILE ( Database_log_file_name , NOTRUNCATE)
This statement don't break log shipping. But, you may need to run more than one. For each run, the log shipping backup, copy, and restored to run after again run this statement.
Shrink and truncate are different.
My experiences:
AA db, 6.8GB transaction log
first run: 6.8 GB
log shipping backup, copy, restore after second run: 1.9 GB
log shipping backup, copy, restore after third run: 1.7 GB
log shipping backup, copy, restore after fourth run: 1 GB
BB db, 50GB transaction log
first run: 39 GB
log shipping backup, copy, restore after second run: 1 GB
Creating a transaction log backup doesn't mean that the online transaction log file size will be reduced. The file size remains the same. When a transaction is backuped up, in the online transaction log it's marked for overwriting. It;s not automatically removed, and no spaces is freed, therefore, the size remains the same.
Once you set the LDF file size, maintain its size by setting the right transaction log backup frequency.
Paul Randal provides details here:
Understanding Logging and Recovery in SQL Server
Understanding SQL Server Backups
Based on Microsoft recommendation Before you intend to Shrink log file you should first try to perform the following capabilities:
Freeing disk space so that the log can automatically grow.
Moving the log file to a disk drive with sufficient space.
Increasing the size of a log file.
Adding a log file on a different disk.
Turn on auto growth by using the ALTER DATABASE statement to set a non-zero growth increment for the FILEGROWTH option.
ALTER DATABASE EmployeeDB MODIFY FILE ( NAME = SharePoint_Config_log, SIZE = 2MB, MAXSIZE = 200MB, FILEGROWTH = 10MB );
Also, you should be aware of shrink operation via maintenance plan will effect on *.mdf file and *.ldf file. so you need to create a maintenance plan with SQL job task and write the following command to can only shrink *.ldf file to your appropriate target size.
use sharepoint_config
go
alter database sharepoint_config set recovery simple
go
dbcc shrinkfile('SharePoint_Config_log',100)
go
alter database sharepoint_config set recovery FUll
go
Note: 100 is called the target_size for the file in megabytes, expressed as an integer. If not specified, DBCC SHRINKFILE reduces the size to the default file size. The default size is the size specified when the file was created.
In my humble opinion, It’s not recommended to perform the shrink operation periodically! Only in some circumstances that you need to reduce the physical size.
You can also check this useful guide to Shrink a transaction log file Maintenance Plan in SQL Server