I would like to present you my problem related to SQL Server 2005 bidirectional replication.
What do I need?
My teamleader wants to solve one of our problems using bidirectional replication between two databases, each used by different application. One application creates records in table A, changes should replicate to second database into a copy of table A. When data on second server are changed, then those changes have to be propagated back to the first server.
I am trying to achieve bidirectional transactional replication between two databases on one server, which is running SQL Server 2005. I have manage to set this up using scripts, established 2 publications and 2 read only subscriptions with loopback detection. Distributtion database is created, publishment on both databases is enabled. Distributor and publisher are up. We are using some rules to control, which records will be replicated, so we need to call our custom stored procedures during replication. So, articles are set to use update, insert and delete custom stored procedures.
So far so good, but?
Everything works fine, changes are replicating, until updates are done on both tables simultaneously or before changes are replicated (and that takes about 3-6 seconds). Both records then end up with different values.
UPDATE db1.dbo.TestTable SET Col = 4 WHERE ID = 1
UPDATE db2.dbo.TestTable SET Col = 5 WHERE ID = 1
results to:
db1.dbo.TestTable COL = 5
db2.dbo.TestTable COL = 4
But we want to have last change winning replication. Please, is there a way to solve my problem? How can I ensure same values in both records? Or is there easier solution than this kind of replication?
I can provide sample replication script which I am using.
I am looking forward for you ideas,
Mirek
I think that adding dateUpdated field on both tables could help. This way in your replication code a record would be updated only if dateUpdated is greater then the one already stored.
That dateUpdated field would obviously store the datetime when the original record was updated, not when the replication was performed
Related
I'm trying to understand the ramifications of database replication (SQL Server or Golden Gate) for situations where the source database is completely repopulated every night. To clarify, all existing tables are dropped and then the database is reloaded with new tables using same name along with all the data.
Based on my understanding i.e. that replication uses a transaction log... I would assume it will also repeat the process of dropping the tables instead of identifying the differences and just adding the new data. Is that correct?
You can set up the replication using OracleGoldenGate so that it will be doing what you want it to do.
the TRUNCATE TABLE command can be replicated or it can be ignored
the populating of the source table (INSERT/bulk operations) can be replicated or it can be ignored
if a row already exists (meaning a row with the same PK exists) on the target and you INSERT it on the source you can either UPDATE the target or DELETE the old one and INSERT the new, or ignore it
Database replication is based on the redo (transaction) log. Only particular events that appear on the source databases, which are logged can be replicated. But the whole replication engine can make some additional transformations as it is replicating the changes.
I need help with setting up a scheduled job.
I have two SQL Server databases on two different servers. The job would do SELECT on database A and INSERT on database B. When something changes in database A, the job would compare what had changed and did an update on database B.
Is this possible if I have SQL Server 2008 R2 Management Studio?
Thank you very much in advance.
I would suggest to make a Replication if possible. Read more about it here.
Otherwise if you really need a own job you have two ways.
Execute the Job with your SQL Agent every X minutes/hours. Check your new data and execute the INSERT-statement.
You can create a trigger on the source table which sets a flag in a table or on the sourcetable itself after an insert is executed. Your job on the target server executes every x minutes or even seconds and check the source table. After that he can evaluate if a changed happen and just copy the flagged rows to your target.
You could set up one-way replication between the two servers and let it take care of everything for you.
Or, you could add server B as a linked server then take responsibility for the record inspection and crafting the insert/update/delete statements yourself.
Have you tried either?
I have a SQL Server 2012 Web Edition (11.0.5058.0) instance on a VPS which hosts two databases. I would like to mirror a couple of columns from 3 tables to the second database, but I don't have transactional replication installed.
So I have a Staff table on the source database - I just want the staff_code and unique_id - I have an Activity table - I just need the activity_code, description and unique_id.. etc.
What is the best way to go about this - would that be triggers? The data is not regularly updated, possibly once a week - but I would still like the synchronisation to be fast if possible?
The data in the source database may be deleted, updated or inserted, by another application, so I want to ensure the data in my database reflects that information correctly.
Thanks for any suggestions!
UPDATED: Table comparison example:
SELECT CASE WHEN NOT EXISTS
( SELECT [COLUMN1],[COLUMN2],[UNIQUE_ID] FROM [SOURCE-DATABASE].[dbo].[SOURCE-TABLE]
EXCEPT
SELECT [COLUMN1],[COLUMN2],[UNIQUE_ID] FROM [DESTINATION-DATABASE].[dbo].[DESTINATION-TABLE]
)
AND NOT EXISTS
( SELECT [COLUMN1],[COLUMN2],[UNIQUE_ID] FROM [DESTINATION-DATABASE].[dbo].[DESTINATION-TABLE]
EXCEPT
SELECT [COLUMN1],[COLUMN2],[UNIQUE_ID] FROM [SOURCE-DATABASE].[dbo].[SOURCE-TABLE]
)
THEN 'True'
ELSE 'False' //GRAB NEW OR UPDATED DATA
END AS result ;
As long as the two databases can be connected (e.g. can you do a SELECT * FROM SecondDB.dbo.Activity?), then I would just
set up a query (stand-alone, or in a stored procedure) that just checks whether or not the data on the source has changed
updates the second database using normal SELECT, INSERT, UPDATE and possibly DELETE statements
set up that query/stored procedure with a SQL Server Agent Job to run at regular intervals, e.g. once every night, once every week - whatever works for you
I don't think triggers would be a good choice here - triggers should be kept very small, lean, fast - and "replicating" to another database sounds like too much processing work for a nimble trigger.... (also if you triggers take a long time to complete, the calling application will have to wait for that whole time..... not good for your application performance!)
I am new to SQL Server replication, and pushed some changes out to an existing replicated environment. It is SQL Server 2005 SP3, with Transactional replication.
I created a new table, added columns to an existing table, and added some SPs. I did all of this on the publisher database.
I created a new snapshot and reinitialized the subscriptions. The new table and new columns were replicated to the new server, and data was synchronized successfully, but just 1 time.
Immediately after, the replication monitor is reporting an error. On the Distributor to Subscriber History tab, I get a message that the "Subquery returned more than 1 value". The Command Attempted is...
delete from MSsubscription_articlecolumns
where artid=89 and
agent_id = (
select id from MSsubscription_agents
where update_mode > 0 and
UPPER(publisher) = UPPER('xxxx') and
publisher_db = N'xxxx' and
publication = N'xxxxx'
) end
Why would there be duplicate subscription agents? How do I fix this? Thanks for all help.
I was able to resolve the issue by manually editing the data in the MSsubscription_agents table. There were 2 records with the same publisher, db and publication names. Instead of deleting one of the records, I changed the update_mode on the older record to be 0, which filtered it out of the problem statement in original question. The replication process was then able to get past that statement and run correctly. Thanks.
I am trying to find a highly efficient method of auditing changes to data in a table. Currently I am using a trigger that looks at the INSERTED and DELETED tables to see what rows have changed and inserts these changes into an Audit table.
The problem is this is proving to be very inefficient (obviously!). It's possible that with 3 thousand rows inserted into the database at one time (which wouldn't be unusual) that 215000 rows would have to be inserted in total to audit these rows.
What is a reasonable way to audit all this data without it taking a long time to insert in to the database? It needs to be fast!
Thanks.
A correctly written trigger should be fast enough.
You could also look at Change Data Capture
Auditing in SQL Server 2008
I quite often use AutoAudit:
AutoAudit is a SQL Server (2005, 2008, 2012) Code-Gen utility that creates
Audit Trail Triggers with:
Created, CreatedBy, Modified, ModifiedBy, and RowVersion (incrementing
INT) columns to table
Insert event logged to Audit table
Updates old and new values logged to Audit table Delete logs all
final values to the Audit table
view to reconstruct deleted rows
UDF to reconstruct Row History
Schema Audit Trigger to track schema changes
Re-code-gens triggers when Alter Table changes the table
Update: (Original edit was rejected, but I'm re-adding it):
A major upgrade to version 3.20 was released in November 2013 with these added features:
Handles tables with up to 5 PK columns
Performance improvements up to 90% faster than version 2.00
Improved historical data retrieval UDF
Handles column/table names that need quotename [ ]
Archival process to keep the live Audit tables smaller/faster but retain the older data in archive AutoAudit tables
As others already mentioned - you can use Change Data Capture, Change Tracking, and Audit features in SQL Server, but to keep it simple and use one solution to track all SQL Server activities including these DML operations I suggest trying ApexSQL Comply. You can disable all other, and leave DML auditing option only
It uses a centralized repository for captured information on multiple SQL Server instances and their databases.
It would be best to read this article first, and then decide on using this tool:
http://solutioncenter.apexsql.com/methods-for-auditing-sql-server-data-changes-part-9-the-apexsql-solution/
SQL Server Notifications on insert update delete table change
SqlTableDependency C# componenet provides the low-level implementation to receive database notification creating SQL Server Queue and Service Broker.
Have a look at http://www.sqltabledependency.it/
For any record change, SqlTableDependency's event handler will get a notification containing modified table record values as well as DML - insert, update, delete - change executed on your database table.
You could allow the table to be self auditing by adding additional columns, for example:
For an INSERT - this is a new record and it's existence in the table is the audit itself.
With a DELETE - you can add columns like IsDeleted BIT \ DeletingUserID INT \ DeletingTimestamp DATETIME to your table.
With an UPDATE you add columns like IsLatestVersion BIT \ ParentRecordID INT to track version changes.