I'm trying to move data from table A in our on-prem database into an equivalent table A in an Azure Synapse (ASA) dedicated pool. I've set up the integration runtime, and have selected my on prem table from ASA. However, when I run a link connection I am seeing the following error:
Failed to enable Synapse Link on the source due to 'Unexpected metadata of Synapse Link was detected in the source database.'.
Failed to disable Synapse Link on the source due to 'Failed to drop the link topic in the source database: Failed to enable Synapse Link on the source due to 'Unexpected metadata of Synapse Link was detected in the source database.'.'.
Continuous run ID: e52df111-9947-401e-97cb-4ef3f4532934
I'm expecting Table A in ASA to be populated with data from on-prem.
What does this mean? I'm very new to ASA so might have overlooked some setup.
I tried to reproduce the error you have got and ended up with the similar error as below:
The main cause of error is your Azure Synapse workspace managed identity has no permissions to access the Azure Data Lake Storage Gen2 storage account.
As per Official Microsoft Document
Make sure that you've granted your Azure Synapse workspace managed identity permissions to the Azure Data Lake Storage Gen2 storage account
To grant the managed identity of Azure Synapse workspace to access the Azure Data Lake Storage Gen2 storage account follow below steps:
Go to your Azure Data Lake Storage Gen2 >> Access Control (IAM) >> Add >> Add Role assignments
After this pick Storage Blob Data Contributor,
For selected role choose Managed identity.
Then under Members, choose your Azure Synapse workspace.
Related
I am using the following link to guide me on connecting Tableau to ADLS Gen2 https://help.tableau.com/current/pro/desktop/en-us/examples_azure_data_lake_gen2.htm
I have got stuck at the first hurdle where the document states
Start Tableau and under Connect, select Azure Data Lake Storage Gen2.
For a complete list of data connections, select More under To a
Server.
I don't have that option with the version of Tableau I just downloaded.
Should I be downloading a different version of Tableau to see the option to select Azure Data Lake Storage Gen2?
You're using Tableau Public (limited connection options), but if you download Tableau Desktop (even on a 14day trial) it will work
How to pull data from cube that is hosted on Azure analysis service and load data in SQL pools of synapse
One solution is to use Azure Data Factory for data movement.
There's no built-in connector for Azure Analysis Service in Data Factory. But since Azure Analysis Services uses Azure Blob Storage to persist storage, you can use the connector for Azure Blob Storage.
In Data Factory, use a Copy Activity with Blob Storage as source and Azure Synapse Analytics as sink.
More on Azure Data Factory here: https://learn.microsoft.com/en-us/azure/data-factory/
Available connectors in Data Factory: https://learn.microsoft.com/en-us/azure/data-factory/connector-overview
I have been provided SAS key for an Azure Storage Table resource and I am trying to connect to that using Azure Storage Explorer but keep getting the error:-
Error occurred while adding new connection: SyntaxError:Table name format is incorrect.
I even updated Azure Storage Explorer version but the issue is still the same. The SAS key is correct. And in fact for none of the Azure Storage Table resources I have been able to connect using SAS key while using Azure Storage Explorer.
Want to connect ADLS Gen-1 with AzureML Studio.
I try to find out some solution but could not get
Direct method:
Currently, Azure Data Lake Store is not a supported source.
I would suggest you to vote up an idea submitted by another Azure customer.
https://feedback.azure.com/forums/327234-data-lake/suggestions/15008490-adl-store-connector-for-ml-studio
All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.
By using the Import Data module, you can access data from one of several online data sources while your experiment is running:
• A Web URL using HTTP
• Hadoop using HiveQL
• Azure blob storage
• Azure table
• Azure SQL database or SQL Server on Azure VM
• On-premises SQL Server database
• A data feed provider, OData currently
• Azure Cosmos DB
For more details, refer “Supported data types in Azure ML studio”.
In-direct method:
Azure Data Lake Analytics can also be used to write data out to Azure Blob Store, and so you can use that as an approach to process the data in U-SQL and then stage it for Azure Machine Learning to process it from Blob store. When Azure ML supports Data Lake store, then you can switch that over.
For more details, refer "How to use ADLS as an input data set for Azure ML Studio".
Hope this helps.
I am setting up a new Azure Data Lake Analytics (ADLA) PAAS service to run USQL against some existing data sets in blob storage. The blob storage is firewalled for security and when I try to add the storage account to the data sources in ADLA I get the following error. Similar happens for data factory.
InvalidArgument: The Storage account '' or its accessKey
is invalid.
If I disable the firewall, the storage account can be successfully added. I have tried to add the relevant Azure Data Center IP Address ranges but the connection still fails. I have also ticked the "Allow trusted Microsoft Services box" but this does not seem include data lake or data factory. How do I access my storage account from ADLA but still have it secured?
You could install a selfhosted IR to access your blob storage. Whitelist the IP of the machine hosting your selfhosted IR.