Can you please help me will FOR XML PATH works in Azure Synapse.
If not what will be the alternative.
Thanks
In case you you're looking to use FOR XML to prepare e.g. column names for a dynamic pivot query, then the string_agg function is what you are looking for.
string_agg documentation
Azure Synapse Analytics dedicated SQL pools (and serverless SQL pools) do not support the FOR XML syntax as per the documentation:
The easiest thing to do would be use Azure SQL DB to do your XML processing, eg use a pipeline to land your data there, call a Stored Proc activity to do the processing then retrieve it. Another fun alternative would be to use a Synapse notebook and use Python or Scala to process the XML.
Related
I want to use my SQL script (present under Develop hub) file inside a Pipeline (present under Integrate hub). Currently I do not see any Activities available solving this purpose.
There is one Script activity under General section which only have a Query & NonQuery option, not for referring any SQL script file created earlier.
Is that feature available at all in Azure Synapse Analytics? Can we refer to SQL script by some other means?
If your Synapse workspace is paired with Azure DevOps then I imagine it’s easy to get the file content with a REST API call (eg here). However then you have to parse the file as GO is not supported by the Script activity. ADF / Synapse Pipeline functions do not support a RegEx style split eg word boundary and GO (\bGO\b) so it starts to get kind of fiddly. I had some success with replace and uriComponent functions.
However you would be better of using Stored Procedures and the Stored Proc activity in Synapse Pipelines - much simpler implementation.
Is there any third party tool which can convert Vectorwise query into Synapse queries?
Unfortunately, there is no such tool available.
Azure Synapse support T-SQL language to query and analyze the data. You can use standard ANSI-compliant dialect of SQL language used on SQL Server and Azure SQL Database for data analysis.
Since vectorwise query is similar to T-SQL, you can try if some of the statements supported by Synapse. Else, you need to write them in T-SQL and use.
While creating stored procedures in Azure data warehouse, I have got some error on "Cursor" and "FOR XML". So wanted to know if they are supported by Azure data warehouse or not. If not then what are the alternatives.
sample code with error msg pictures are attached herein.
Neither FOR XML or cursors are supported in Azure Synapse dedicated SQL pools (formerly known as Azure SQL Data Warehouse) as per the documentation. For cursors, either convert them to use a WHILE loop which is supported or refactor the code to use a set-based approach. Another alternative is to use something external, like Azure Data Factory or Synapse Pipelines and use a For Each loop. Another alternative is to use a nearby Azure SQL DB to do some pre-processing. You should be aware the the MPP architecture of Azure Synapse Analytics does not lend itself well to this kind of row-based processing and you should remember it's a big data platform meant for large volumes of data, millions, billions of rows and set-based approaches should be preferred.
If you are just using FOR XML to do that sleazy string concatenation trick then you should use STRING_AGG instead which is fully supported in Synapse. See this answer for a recent example. If you are actually producing XML then you will need to find an alternative method, eg a nearby Azure SQL DB.
We have to collect some information. How do we check how often someone is running queries in Azure SQL?
Use extended events to trace all the queries running on your Azure SQL database. Also, you can save the output in a file and can query from clients tools like SSMS.
You can also use auditing provided by Azure SQL. Please check below link for details:
https://learn.microsoft.com/en-us/azure/azure-sql/database/auditing-overview#:~:text=Go%20to%20the%20Azure%20portal,modify%20the%20server%20auditing%20settings.
I am new to Azure SQL.
We have a client db which is in Azure SQL. We need to set up a process automation which extract query results to .CSV files and load it in our server (on premise SQL server 2008 R2).
What is the best method to generate csv files from Azure sql and make it accessible for the on premise server?
Honestly the best in terms of professional approach is to use Azure Data Factory and installation of Integration Runtime on the on premises.
You of course can use BCP but it will be cumbersome in the long run. A lot of scripts, tables, maintenance. No logging, no metrics, no alerts... Don't do it honestly.
SSIS is another option butin my opinion it takes more effort than ADF solution.
Azure Data Factory will allow you to do this in professional way using user interface with no coding. It also can be parametrized so you just change name of table name parameter and suddenly you are exporting 20, 50 or 100 tables at ease.
Here is video example and intro into data factory if you want to see quick overview. In this overview there is also demo which imports CSV to Azure SQL, you can just change it a little bit to make Azure SQL -> CSV and CSV > SQL server or just directly Azure SQL > SQL server.
https://youtu.be/EpDkxTHAhOs
It really is straightforward.
Consider using simple bcp from the on prem environment save the results to csv and then load the csv into the on prem server.
You can also use SSIS to implement an automated task.
Though I would like to know why you need the intermediate csv file? you can simply just copy data between databases (cloud -> On prem) with a scheduled SSIS package.
If you have on-prem SQL access then a simple SSIS package is probably the quickest and easiest way to go. If your source is Azure SQL and the ultimate destination is On-Prem SQL, you could use SSIS and skip the CSV all together.
If you want to stick to an Azure PAAS solution you could consider using Azure Data Factory. You can setup a gateway to access the on-prem SQL server directly or if you really want to stick to a CSC then look into using a Logic App.
Azure Data Factory is surely option.
Simple solution would be pyodbc driver with little bit of python. https://learn.microsoft.com/en-us/sql/connect/python/python-driver-for-sql-server?view=sql-server-2017
You can also try sqlcmd and bit of powershell or bash on top.
https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-2017