I need to run an end of day 'replication tool/application' on a SQL Server 2016 Always-On DB (1 Primary, 2 Sync, 1 Async). The job needs to run at a point in time when the DB is serving limited web traffic but I can't stop the Web/Application layer being up 24/7.
e.g., my requirement is to replicate all data in the DB at 1am.
I was wondering how best to solve this:
Create a Backup and Restore of the Async DB and then run the replication tool off this "new DB".
Create a DB snapshot of the Async DB and then run the replication tool of the snapshot
Something much easier I'm missing
*Note the tool I'm using proprietary to a product and I can't use any native SQL replication tools.
Many thanks
Related
Our current web application uses Azure SQL Database. We need to create a scheduled process for importing/exporting data from a SQL database in Azure Virtual Machine to Azure SQl database. And also provide ability to trigger manually.
Currently we have manual scripts with Linked server which needs to be run on both servers.
What are the best options available for clean import/Export data automation (scheduled or manual)? for e.g. Secure Connection between servers, Tools or Applications (other than SSIS)
Could someone please provide guidance.
Thank you,
Vamshi
You can use SQL Data Sync tool to sync data between an Azure SQL Server VM and an Azure SQL database. The sync can be done manually or scheduled with a frequency of days, hours, minutes and seconds.
Of course, you can also use SSIS for that purpose. But you may see intermittent connection failures that can substantially impact data loads.
You can use Azure Data Factory to connect to Azure SQL VM and Azure SQL Database and using the copy data tool export/import data from one to the other, and vice versa.
We're in the process of a server migration from an on-prem server (Win2008R2) to Azure PaaS.
To move the DBs, we used the Microsoft Data Migration Assistant (DMA) tool, which worked great and we can connect to the migrated Azure DB via SQL Server Management Studio.
Considering:
Made quite a few changes to the migrated Azure DB (tables, stored procedures, indexes) to work with the apps in Azure
Combined multiple on-prem DBs into one DB in Azure via DMA to save costs
On-prem DB is continually being modified by insert/update operations (multiple tables) during the migration process
Question: what is the best and fastest way to migrate data (all vs missing/updated) considering the above?
I would recommend you to migrate first only the schema of your on-premises databases to Azure SQL Databases and then let Azure SQL Data Sync to migrate the data to Azure and keep it updated on Azure SQL Database.
My suggestion to start with an empty schema on the Azure SQL Database side is because when SQL data Sync finds data on-premises and on Azure it start comparing both databases and that consumes a lot of resources.
On the initial sync SQL Data Sync may consume a lot of resources on the on-premises database server even when having an empty schema on the Azure side, for that you can use SQL Server Resource Governor to cap the CPU used by the data sync sessions in your on premises SQL Server, and this way avoid big performance impact possibly affecting database users.
When you are ready, you can switch your users (gradually or not if SQL Data Sync is on bi-directional mode) to Azure. Once your users have been migrated, you can then remove the member database (the on-premises database) from the SQL Data Sync configuration and stop SQL Data Sync operation.
I disagree with all the answers here.
If you are running on Win2008R2 there is a high chance that you are on an old SQL Server (2008? 2012?) which are both deprecated and unsuitable for Azure SQL Database. And probably the application is also old and not suitable for the Cloud in general. I suggest you a good testing phase.
Here my to do list:
Upgrade SQL Server to SQL Server 2016 on-prem and test if all your queries are still running correctly
Test how ready is your SQL Server to go to Azure SQL Database through Microsoft Data Migration Assistant (DMA) tool or the new Azure SQL Migration extension for Azure Data Studio (came out his month).
Don even think for a second that merging databases will reduce your overall costs. Decide if going multi-tenant or single-tanant not because of the price of the database.
Plan for hours of downtime based on the size of the migration. Don't migrate while your database is modified. Expect downtime. The best way is to take a backup of the day before and then resume the logs.
and test like crazy. This is not gonna be easy because the app is old.
Good luck.
Visual Studio also has a great tool for comparing both schema and data between two databases on different servers.
It can then update the target database with any changes after which you can switch over to use the Azure DB.
This method would require downtime of around 5-30 minutes depending on amount of data, but that might be acceptible depending on your requirements.
I have a SQL Server 2012 database which currently used as a transactional database and reporting database. The application reads/writes into the same database and the reports are also generated against the same database.
Due to some performance issue, I have decided to maintain the two copies of the database. One will be a transactional database which will be accessed by the application. The other database will be the exact copy of the transactional database and it will only be used by the reporting service.
Following are the requirements:
The reporting database should be synched with transactional database in every one hour. That is, the reporting database can have stale data for maximum of 1 hour.
It must be read-only database.
The main intension is NOT recovery or availability.
I am not sure which strategy, transactional log shipping, mirroring or replication, will be best suited in my case. Also if I do the synch operation more frequently (say in every 10 minutes), will there be any impact on the transactional database or the reporting service?
Thanks
I strongly recommend you to use a standby database in readonly state. And every 15 minutes your sqlserveragent has a scheduled job to: a) generate a new .trn logfile within main db, and b) restore it into standby one(your reports db). The only issue is: using this technique your session will be disconnected while agent restores the .trn logfile. But if you can stop the restore job, run your reports and then reactivate it, there is no problem. Seems to be exactly what you need. Or if your reports are fast to run, probably will not be disconnected...if im not wrong restore job can also be configured to wait opened session to finish or to close it. I can check it this last doubt for you tomorrow if you don't find..
Once it is running in the same sql server instance, you don't have to worry about extra licensing...
We have configured SQL Server 2012 database transaction replication for our client's .NET web application to distribute SQL transaction and reporting on different SQL Servers.
We had implemented transaction replication on to SQL-Node1 is working as Master DB Server, We'd configured replication of Master DB on SQL-Node2 to pull out reports in to our web application which having lots of transactions and data uploading from excel sheet entries around 10 million entries each day.
After configured replication on two SQL Server 2012 instances, after few weeks we facing some performance issues and found some resource get locked during uploading files on to database that's why application unable to access those tables and data. Also found that server performing too much slow during day time when users access our web application.
Now we are looking to distribute loads on different 3 Nodes of SQL Server 2012. Where web application will access and transact data on SQL-Node1, Reporting queries get pull data from SQL-Node2 and SQL-Node3 will be get used to upload excel sheet data on to Database which will get replicated on all other SQL Nodes.
Current setup, all servers having Windows Server 2008 Standard and SQL Server 2012 Enterprise Edition.
Database size approx : 15 GB / Replication used : Transaction / Distributor role configured on SQL Node1 / Subscriber role configured on SQL Node 2.
We are looking for solution to resolve above issues which can distribute different loads (reporting, data uploading, transaction) and replicate data between all SQL Nodes.
Which feature will do perform well for above scenario among SQL Server 2012 HA, SQL Server Replication or SQL Server Mirroring ??
Quick response will be highly appreciated....
Because you have changes happening at more than one node (transactional data at node 1, excel uploads at node 3), "none of the above". All of the abovementioned technologies are built on having data changes happen in one location and propagating to others. You could look at peer to peer replication, but it seems like overkill.
If it were me, I'd try to diagnose why your file upload process is killing performance and fix/work around that. Once you do that, I'd move that process back to node 1 and implement an availability group to cover your reporting needs (with the added bonus of HA).
All of the technologies would bog down on a large data import that is done in one big transaction. I suggest doing it as an ETL like function. Import into a staging table and migrate the data into the production table in bite sized chunks (test many data row sizes to find the size that works the best for your environment). 2 servers should be fine with replication on a cluster for HA with work loads you are talking about.
We have a client that currently has a large busy database that they back up using database mirroring. As part of a reporting requirement we need to use some of the data stored in the database to run reports against but we cannot access the data directly.
Is it possible for us to run transaction replication on that server to get our data while it is running database mirroring at the same time?
Thanks.
Yes it's possible to run transaction log shipping at the same time you are running mirroring. It is also possible to bring the secondary server up in a "Read Only" mode rather than "Standby" for the specific purpose of allowing you to run reports on the second server.