We have a development environment Synapse Workspace connected to a development environment Dedicated SQL Pool. We want another development environment Synapse Workspace but we want to connect it to the existing development environment dedicated SQL Pool. So basically, both Synapse workspaces should connect to the same dedicated SQL Pool.
Apparently, I was told this is not possible. Is there any way or a workaround? We want users to be able to query the data using the Synapse workspace in the dedicated SQL pool.
That is not possible. What you can do is to create another pool in existing workspace to have two development environments. If this is about letting users querying the data your best bet would be copying the data from one pool to another by Built-in copy task in Synapse:
Built-in copy task in Synapse
You can also specify scheduled copy as well.
Related
Requirement: I wanted to copy data from a specific table/view residing on a on-premise SQL Server to Azure SQL DB.
Infrastructure: As depicted in below picture. Essentially, the Azure network is directly connected with corporate network over Express Route. Thus it's a pure private network connection; as good as the corporate network itself.
Issue/Question: I know there are multiple approaches present to get this operation done and I am not restricted to use ADF copy Data tool only. BUT, for all of these I see some cavets or extra steps needed to be done as below:
ADF Copy Data Tool: Needs a SH-IR and a small MSI package needs to be installed on on-premise machine which hosts the SQL server for registration purpose.
Logic Apps: Needs a Virtual Gateway (OR) ASE
App Service: If the operation is wrapped in a C# application and I choose to deploy to a Azure Web Apps. Then in-order to connect to on-premise SQL Server we need to setup hybrid connection manager and as in #1 we need to install something in on-premise machine.
For my case, none of these extra steps can be done. essentially, the on-premise SQL Server comes under a different BU and thus I don't have any permission there; except they have given grant to a table/view. Thus, none of these extra shitty steps can be done.
Moreover, as mentioned above; since it's connected over express route as direct connection, As can be seen in above picture, both the on-premise and azure SQL are essentially inside the same corporate network. THUS, I should be able to access them directly without configuring any of these extra steps as mentioned above.
Please confirm on these and provide a suggestion.
Thank You.
You can still go with the ADF scenario without a SHIR by creating ADF in a Managed VNET using Private Endpoint. As you already have an ER circuit and have the flexibility to configure the Azure side, can you do this with Azure IR: Access on-premises SQL Server from Data Factory Managed VNet using Private Endpoint - Azure Data Factory | Microsoft Docs
There are 2 solutions which could work for your scenario but even for them to work ,you would need access to on prem SQL server machine access to some extent atleast for one time config and Azure SQL db should be accessible via SSMS installed on on-prem machine.
Using linked server
You can create a linked server ( process explained here https://www.sqlshack.com/create-linked-server-azure-sql-database/ ) on on-prem server and create a agent server job to insert data to azure SQL db table.
Via Python Script
This would need Python installation on on-prem machine. Once installed you can write script to transfer data between on-prem SQL server and Azure SQL db. You can schedule this script again by using an agent server job.
I have a Microsoft Azure SQL Database project. I also have a Python3.9 project that uses unittest to unit test this database project. I have an Azure DevOps build pipeline defined in YAML that runs the unit test against the development-integration environment.
I do not want to publish changes to the development-integration environment before running the tests. If you think this is the wrong approach, I will consider your arguments.
I want to 'virtually' test the changes. I want to deploy the new objects to a temporary ad-hoc database instance. It must be equivalent to Azure Database Instance. When the tests have been executed I want to clear everything away. I do not want to deploy a database in Azure for this purpose due to billing, although if I were to use a serverless instance this would not be a problem.
Any ideas?
If you are on cloud and you need to test you need to test that on the cloud too.
You cannot "virtually" test, there is nothing equivalent to Azure SQL database on-prem.
Go with the serverless instance as you said.
Since I've starting using Azure Synapse Analytics, I created a Spark Pool Cluster, then on the Spark Pool cluster I created databases and tables using Pyspark on top of parquet files in Azure Data Lake Store Gen2.
I use to be able to access my spark Database/ parquet tables through SSMS using the Serverless SQL endpoint but now I can no longer see my spark Databases through the Severless SQL Endpoint in SSMS. My spark databases are still accessible through Azure Data Studio but not through SSMS. Nothing has been deployed or alter on my side. Can you help resolve the issue? I would like to be able to access my spark databases through SSMS.
Sql Serverless Endpoint
Azure Synapse Database
If your Spark DB is built on top of Parquet files, as you said, databases should sync to external tables in Serverless SQL pool just fine and you should be able to see synced SQL external tables in SSMS as well. Check this link for more info about metadata synchronization.
If everything mentioned above is checked, then I'd suggest you to navigate to Help + Support in Azure Portal and fill in a support ticket request with details of your problem so engineering team can take a look and see whether there is some issue with your workspace or not.
We are supporting a legacy system for our organisation. In the current scenario, we receive a SQL Server backup (.bak files) from the application vendor on an FTP location. For every weekend on Sunday it is a Full backup and for every other day its the differential one.
On our side, we have a SQL server instance running which has custom stored procedures written and scheduled to check the location every morning and then restore the backups every day. These restored backups are then used by the organisation for internal reporting purposes. There are 100s of other stored procedures written for different reports in different DBs on the same instance.
Since SQL Server 2008 is now out of support and for cost-saving purposes of running on-premise system, my team has been given a task to look into migrating this whole system to Azure SQL database.
My question is what is the most effective way in which we can move this workflow to the cloud? I have an azure trial account set up for me to try but haven't been successful in restoring the .bak files on Azure SQL instance.
Thanks.
You essentially have two options for Azure, either perform a fairly linear Lift and Shift to SQL Server on an Azure VM or go with a more advanced Azure PaaS offering in Azure SQL Database Managed Instance. The specific deployment Azure SQL Database (Single Instance) will not support your current solution requires with regard to the .bak file support, and I have detailed that below. For further details between the difference between Azure SQL Database Single Instance versus Managed Instance, please see: Features comparison: Azure SQL Database and Azure SQL Managed Instance
The second option, is to leverage the Azure Enterprise Ready Analytics Architecture (AERAA) (link) of Azure (PaaS) Analytics services. With Azure SQL Database (PaaS) services, as opposed to on-premise SQL Server or SQL Server on an Azure VM, there is no Integration Runtime or Analysis Services as a bundled service component. These services are separate PaaS offerings and with the help of the linked AERAA blog, you can gain a better understanding of the Azure Analytics services.
The .bak versus .bacpac support dilemma:
Since the main requirement for your solution is support of .bak files, you need to understand where .bak and where .bacpac files are supported. The term Azure SQL Database applies to both a specific deployment option for an Azure SQL database (PaaS) service and as a general term for Azure SQL cloud databases. As for the specific deployment option, Azure SQL Database (Single Instance nor Elastic Pools) will support your scenario with .bak files. This deployment option will support export/import functionality via .bacpac file format. It will not support full/partial restore functionality. The backup/restore functionality although configurable, is only in scope for the specific database hosted by an Azure SQL (logical) Server instance. Basically, you can not restore an external file. You can import, which is always a full copy. So, for that reason, for an Azure PaaS database service you will need Azure SQL Database Managed Instance for .bak file support or deploy an SQL Server VM image to an Azure VM, and migrate your objects via Azure Database Migration Service.
Regards,
Mike
My production and staging databases are on Azure and I want to make staging the same as production. I have no idea what to do, I have never worked with databases. I use SQL Server 2014.
There are a few ways to do this, but the quickest way would probably be to replace your staging database when you want the updated data by Creating a new database as a copy. Now, this would wipe out any changes to your staging database, but would be the fastest way.
More complicated, but probably a bit more about what you're actually thinking of, would be setting up azure data sync from Prod -> Staging.
Staging Azure web app has different configuration with production Azure web app. We can configure the same connection string at Azure portal to achieve this requirement. Refer to this article for more information about how to configure connection string in Azure portal. At runtime, Azure Web App will automatically retrieves values in Azure portal and makes them available to code running in your website. If the same connection string configured in Azure portal, the staging and production will use the same database.