Can we sync views on local database with tables on SQL azure database using SQL Azure Datat Sync[Preview]? If yes, then how?
In my opinion, the jury is still out on SQL Azure Data Sync. It's been in preview for years and still is.
When you say sync, and you looking to take the results of a local view and persist them as a table in SQL Azure?
If so, you can create a SELECT INTO, taking the data from the local view and letting the SELECT INTO make a table out of it. There are some things you need do to make the SQL Azure DB available locally:
http://msdn.microsoft.com/en-us/library/azure/ee336282.aspx
https://dba.stackexchange.com/questions/59328/insert-to-sql-azure-through-linked-server-very-slow
Related
I have an on-premise SQL Server and I want to access the data from Azure WEB APP so the best solution found is creating a view on the on-premise server and setup transnational replication only on that view to Azure SQL.
I've set up the replication and I keep getting this error:
Reference to database and/or server name in '..' is not supported in this version of SQL Server
I think the problem is that the view is replicated as a command (CREATE VIEW ...) and not as raw data.
The SQL Server is the publisher, and the there is no way to expose any internal network ports.
Is there a way to replicate the view as raw data and not indexed?
Transnational Replication replicates views, but just the view definition. You have to either replicate the base tables, or make the view an Indexed View and Replicate the Indexed View as a Table. In the docs for sp_addarticle this is called #type = 'indexed view logbased'. Or there's lots of articles on the internet about this.
Solved it by creating a new SSIS job that handles that merge and runs once in a hour.
I dont need the data live in my webapp so this worked for me.
We are wanting to use Azure servers to run our Power Apps applications, however we have local SQL servers which contains our data warehouse we want only certain tables to be on Azure and want to create data feeds between the two with information going from one to the other.
Does anyone have any insight into how I can achieve this?
I have googled but there doesn't appear to be a wealth of information on this topic.
It depends on how fast after a change in your source (the on premise SQL Server) you need that change reflected in your Sink (Azure SQL).
If you have some minutes or even only need to update it every day I would suggest a basic Data Factory Pipeline (search on google for data factory upsert). Here it depends on your data on how you can achieve this.
If you need it faster or it is impossible to extract an incremental update from your source you would need to either use triggers and write the changes from one database to the other or get a program that does change data capture that does that.
It looks like you just want to sync the data in some table between local SQL Server and Azure SQL database.
You can use the Azure SQL Data Sync.
Summary：
SQL Data Sync is a service built on Azure SQL Database that lets you synchronize the data you select bi-directionally across multiple SQL databases and SQL Server instances.
With Data Sync, you can keep data synchronized between your on-premises databases and Azure SQL databases to enable hybrid applications.
A Sync Group has the following properties:
The Sync Schema describes which data is being synchronized.
The Sync Direction can be bi-directional or can flow in only one
direction. That is, the Sync Direction can be Hub to Member, or
Member to Hub, or both.
The Sync Interval describes how often synchronization occurs.
The Conflict Resolution Policy is a group level policy, which can be
Hub wins or Member wins.
Next step, you need to learn how to configure the Data Sync. Please reference this Azure document:Tutorial: Set up SQL Data Sync between Azure SQL Database and SQL Server on-premises.
In this tutorial, you learn how to set up Azure SQL Data Sync by creating a sync group that contains both Azure SQL Database and SQL Server instances. The sync group is custom configured and synchronizes on the schedule you set.
Hope this helps.
The most robust solution here is Transactional Replication. You can also use SSIS or Azure Data Factory for copying tables to/from Azure SQL Database. And Azure SQL Data Sync also exists.
As the title suggest, I want to merge my Azure table data with the data residing in Azure SQL. However, I dont want to replicate Azure SQL data to Azure table.
Anyway to have the Azure table in SSMS and then I could create a view over Azure table and Azure SQl together?
Azure Table Storage is not a relational database and does not support join query. To join existing data in Azure SQL Database with data on Azure Table Storage, you will have to replicate/import your data in Azure Table Storage into Azure SQL, and then use SSMS to perform the join. That's the only solution.
Elastic Database queries on Azure SQL Database allows cross database queries involving Azure Database tables or Azure SQL Data Warehouse tables only.
What you want for something like this is Elastic Query. You can read more about it here. Pay attention to the warning about data movement though. This is good for querying, but it's not a great way to do large scale data moves.
We're in the process of a server migration from an on-prem server (Win2008R2) to Azure PaaS.
To move the DBs, we used the Microsoft Data Migration Assistant (DMA) tool, which worked great and we can connect to the migrated Azure DB via SQL Server Management Studio.
Considering:
Made quite a few changes to the migrated Azure DB (tables, stored procedures, indexes) to work with the apps in Azure
Combined multiple on-prem DBs into one DB in Azure via DMA to save costs
On-prem DB is continually being modified by insert/update operations (multiple tables) during the migration process
Question: what is the best and fastest way to migrate data (all vs missing/updated) considering the above?
I would recommend you to migrate first only the schema of your on-premises databases to Azure SQL Databases and then let Azure SQL Data Sync to migrate the data to Azure and keep it updated on Azure SQL Database.
My suggestion to start with an empty schema on the Azure SQL Database side is because when SQL data Sync finds data on-premises and on Azure it start comparing both databases and that consumes a lot of resources.
On the initial sync SQL Data Sync may consume a lot of resources on the on-premises database server even when having an empty schema on the Azure side, for that you can use SQL Server Resource Governor to cap the CPU used by the data sync sessions in your on premises SQL Server, and this way avoid big performance impact possibly affecting database users.
When you are ready, you can switch your users (gradually or not if SQL Data Sync is on bi-directional mode) to Azure. Once your users have been migrated, you can then remove the member database (the on-premises database) from the SQL Data Sync configuration and stop SQL Data Sync operation.
I disagree with all the answers here.
If you are running on Win2008R2 there is a high chance that you are on an old SQL Server (2008? 2012?) which are both deprecated and unsuitable for Azure SQL Database. And probably the application is also old and not suitable for the Cloud in general. I suggest you a good testing phase.
Here my to do list:
Upgrade SQL Server to SQL Server 2016 on-prem and test if all your queries are still running correctly
Test how ready is your SQL Server to go to Azure SQL Database through Microsoft Data Migration Assistant (DMA) tool or the new Azure SQL Migration extension for Azure Data Studio (came out his month).
Don even think for a second that merging databases will reduce your overall costs. Decide if going multi-tenant or single-tanant not because of the price of the database.
Plan for hours of downtime based on the size of the migration. Don't migrate while your database is modified. Expect downtime. The best way is to take a backup of the day before and then resume the logs.
and test like crazy. This is not gonna be easy because the app is old.
Good luck.
Visual Studio also has a great tool for comparing both schema and data between two databases on different servers.
It can then update the target database with any changes after which you can switch over to use the Azure DB.
This method would require downtime of around 5-30 minutes depending on amount of data, but that might be acceptible depending on your requirements.
I am going to create a SQL tables in Azure SQL database, and I want to create a partition of table, but I don't know how to do that, can any one show me some demo example or query to perform this. I am using SQL management studio to connect my Azure db.
We take advantage of partitioning SQL Azure tables. We use it so we can rapidly truncate the oldest partitions of data.
We have a great blog post that walks through step by step how to do it:
https://stackify.com/how-to-partition-tables-in-azure-sql/
I know this is an old post but this may help:
https://www.mssqltips.com/sqlservertip/3494/azure-sql-database--table-partitioning/
I'm not sure what your use case is but Elastic Database Tools for Azure SQL Database allow you to implement sharding if you are partitioning the table because of its size.
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-elastic-scale-introduction