I am working on a transaction Table, where I am storing tasks as their status. Now I am working on a Dashboard Where I need to have the info of how many tasks are in which status.
I thought of having staging tables corresponding to each status so that it won't affect any transactional activity. Now to push data into the staging table I have two options.
Have a trigger on the transactional table on each status update i shall update the staging table.
Have SQL Job which will run every 5 minute to update the data in staging table
Please suggest which way to go..
Thanks
Related
How do I create a trigger in Microsoft SQL server to keep a track of all deleted data of any table existing in the database into a single audit table? I do not want to write trigger for each and every table in the database. There will only be once single audit table which keeps a track of all the deleted data of any table.
For example:
If a data is deleted from a person table, get all the data of that person table and store it in an XML format in an audit table
Please check my solution that I tried to describe at SQL Server Log Tool for Capturing Data Changes
The solution is build on creating trigger on selected tables dynamically to capture data changes (after insert, update, delete) and store those in a general table.
Then a job executes periodically and parses data captured and stored in this general table. After data is parsed it will be easier for humans to understand and see easily which table field is changed and its old and new values
I hope this proposed solution helps you for your own solution,
I am working in SQL Server and here is the scenario.
Below is the workflow:
We will get data from upstream for every 10 - 15 secs. Upstream will directly insert data into our tables - this is our Staging table
We wrote an after insert trigger on this staging table which will move data from Staging to Master table. And now master table also has an After insert trigger which will take 23 secs to complete
We are not sure when is the final batch of data will be submitted by upstream
Problem area where solution needed
As upstream tries to push data into our staging table for every 10 secs and if the trigger on Staging start it will wait until trigger on master table is completed. Now during time of execution 24 sec if we get data from upstream as STG is locked data received now is dropped due to locks
If there is any potential work around please let us know
Like Dan Guzman suggested you could just get rid of staging table and process everything using stored procedure.
Maybe a better, however a trickier, way would be use a message queue instead of a staging table and activation procedure that will deal with rows based on your business logic.
This could be a more suited and less blocking way for streaming data.
I have staging table in job in SQL server which dumped data in another table and once data is dumped to transaction table I have truncated staging table.
Now, problem occurs if job is failed then data is transaction table is
roll back and all data is placed in staging table. So staging table already consist data and if I rerun the job then it merges all new data with existing data in staging table.
I want my staging table to empty when the job will run.
Can I make use of temp table in this scenario?
This is a common scenario in data warehousing project and the answer is logical instead of technical. You've two approaches to deal with these scenario:
If your data is important then first check if staging is empty or not. If the table is not empty then it means last job failed; in this case instead of inserting into staging do a insert-update operation and then continue with the job steps. If the table is empty then it means that last job was successful then new data will be insert only.
If you can afford data loss from last job then make a habit to truncate table before running your package.
I am building a sales database. One of the tables has to be a hierarchy of sales reps and their assigned territories. Ohese reps and their territories change every day, and I need to keep track of what exactly that table looks like every day. I will need to take snapshots of the table daily.
I would like to know what I have to do or how I have to store the data in the table, to be able to know exactly what the data in the table was at a certain point in time.
Is this possible?
Please keep in mind that the table will not be more than one megabyte or so.
I suggest using Paul Nielsen's AutoAudit:
AutoAudit is a SQL Server (2005, 2008) Code-Gen utility that creates
Audit Trail Triggers with:
Created, CreatedBy, Modified, ModifiedBy, and RowVersion (incrementing INT) columns to table
Insert event logged to Audit table
Updates old and new values logged to Audit table
Delete logs all final values to the Audit table
view to reconstruct deleted rows
UDF to reconstruct Row History
Schema Audit Trigger to track schema changes
Re-code-gens triggers when Alter Table changes the table
His original blog post: CodeGen to Create Fixed Audit Trail Triggers
Before you implement in production suggest you restore a backup of your database into development and work on that.
This is for MS SQL.
Seeing as the table is so small, you are best of to use the Snapshot functionality provided by MS SQL.
To make a snapshot of a database:
CREATE DATABASE YourDB_Snapshot_DateStamp ON
( NAME = YourDB_Data, FILENAME =
'C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Data\YourDB_Snapshot_DateStamp.ss' )
AS SNAPSHOT OF YourDB;
GO
See this page for reference: http://msdn.microsoft.com/en-us/library/ms175876.aspx
You can make as many snapshots as you want. So my advice is to create a script or task that creates a daily snapshot and appends the date to the snapshot name. This way you will have all your snapshots visible on your server.
Important to note: Snapshots are read only.
I am trying to find a highly efficient method of auditing changes to data in a table. Currently I am using a trigger that looks at the INSERTED and DELETED tables to see what rows have changed and inserts these changes into an Audit table.
The problem is this is proving to be very inefficient (obviously!). It's possible that with 3 thousand rows inserted into the database at one time (which wouldn't be unusual) that 215000 rows would have to be inserted in total to audit these rows.
What is a reasonable way to audit all this data without it taking a long time to insert in to the database? It needs to be fast!
Thanks.
A correctly written trigger should be fast enough.
You could also look at Change Data Capture
Auditing in SQL Server 2008
I quite often use AutoAudit:
AutoAudit is a SQL Server (2005, 2008, 2012) Code-Gen utility that creates
Audit Trail Triggers with:
Created, CreatedBy, Modified, ModifiedBy, and RowVersion (incrementing
INT) columns to table
Insert event logged to Audit table
Updates old and new values logged to Audit table Delete logs all
final values to the Audit table
view to reconstruct deleted rows
UDF to reconstruct Row History
Schema Audit Trigger to track schema changes
Re-code-gens triggers when Alter Table changes the table
Update: (Original edit was rejected, but I'm re-adding it):
A major upgrade to version 3.20 was released in November 2013 with these added features:
Handles tables with up to 5 PK columns
Performance improvements up to 90% faster than version 2.00
Improved historical data retrieval UDF
Handles column/table names that need quotename [ ]
Archival process to keep the live Audit tables smaller/faster but retain the older data in archive AutoAudit tables
As others already mentioned - you can use Change Data Capture, Change Tracking, and Audit features in SQL Server, but to keep it simple and use one solution to track all SQL Server activities including these DML operations I suggest trying ApexSQL Comply. You can disable all other, and leave DML auditing option only
It uses a centralized repository for captured information on multiple SQL Server instances and their databases.
It would be best to read this article first, and then decide on using this tool:
http://solutioncenter.apexsql.com/methods-for-auditing-sql-server-data-changes-part-9-the-apexsql-solution/
SQL Server Notifications on insert update delete table change
SqlTableDependency C# componenet provides the low-level implementation to receive database notification creating SQL Server Queue and Service Broker.
Have a look at http://www.sqltabledependency.it/
For any record change, SqlTableDependency's event handler will get a notification containing modified table record values as well as DML - insert, update, delete - change executed on your database table.
You could allow the table to be self auditing by adding additional columns, for example:
For an INSERT - this is a new record and it's existence in the table is the audit itself.
With a DELETE - you can add columns like IsDeleted BIT \ DeletingUserID INT \ DeletingTimestamp DATETIME to your table.
With an UPDATE you add columns like IsLatestVersion BIT \ ParentRecordID INT to track version changes.