Synapse link from Dataverse doesn't show-up N:N relationship tables.
Ex:
Is there a way to get this synced with Synapse link or should I go with Azure Data Factory and get using FetchXML query?
or any alternate options?
Related
Forgive me I am newbie here and I cannot post images just yet.
Lately I am having few issues with Lake Database that was created in Azure Synapse Analytics using Azure Synapse Link for Dataverse database in PowerApps.
Dynamics365 developers were adding new columns to Dataverse database and they are not displaying or working when executing queries in SSMS or Synapse Studio.
Therefore I have unlinked the Synapse Link in PowerApps and relinked with some tables.
When I unlink, the container and Lake database were deleting correctly but the same database appears in SQL databases section in Azure Synapse studio. I tried to delete it but I am getting an error "Operation DROP DATABASE is not allowed for a replicated database".
Before unlink
After unlink
I have created the Lake database again using Synapse link from PowerApps but it seems the tables meta data is not updating.
Can anyone help me with the above issues (in bold) please.
an error "Operation DROP DATABASE is not allowed for a replicated database".
This error is returned when you try to create objects in a database that's shared with Spark pool. The databases that are replicated from Apache Spark pools are read only. You can't create new objects into a replicated database by using T-SQL.
Create a separate database and reference the synchronized tables by using three-part names and cross-database queries.
Dynamics365 developers were adding new columns to Dataverse database
and they are not displaying or working when executing queries in SSMS
or Synapse Studio.
This issue could be because of many reasons. Without knowing the actual reason, we can't troubleshoot this issue.
If we refer Microsoft official document on Known limitations and issues with Azure Synapse Link for SQL if your column data type isn't supported by the Azure Synapse Analytics, it won't be replicated in Azure Synapse Analytics.
I have created a new Azure Lake Database using the following procedure
The Lake Database name is called TestLakeDB.
However, when I check the list of databases available in Use database TestLakeDB doesn't appear.
Any thoughts?
Thanks for the valuable discussion. Posting your conversation as answer to help other community members who faces similar issues.
When we create Lake database after connecting to the github, it won't reflect in the Use Database because it is created in github mode.
To reflect the the Lake Database, create the database in the synapse live mode and connect to the github. Now we can see it reflects our database named Lake_Database1 which is created in synapse live mode in the Use Database.
I'm trying to access Azure Table Storage in a Gen 2 data lake from Azure SQL Server, but I can't find any documentation. Loads on how to get to csv's in blob storage, but nothing on Azure tables.
Any ideas?
John
Your requirement isn't feasible.
Azure Table storage is a service that stores non-relational
structured data (also known as structured NoSQL data) in the cloud,
providing a key/attribute store with a schemaless design.
Since, Table storage can't be queried using SQL, therefore there is no sense to access it using any SQL Server.
I recommend you to first go through Table storage concepts
before knowing how to query it.
Once getting the Table Storage structure, you can query the tables either through REST API or Cosmos DB Table API based on your application. Refer Querying tables and entities.
You can also follow this complete tutorial Quickstart: Build a Table API app with .NET SDK and Azure Cosmos DB to create basic application using Table Storage for learning purpose.
As the title suggest, I want to merge my Azure table data with the data residing in Azure SQL. However, I dont want to replicate Azure SQL data to Azure table.
Anyway to have the Azure table in SSMS and then I could create a view over Azure table and Azure SQl together?
Azure Table Storage is not a relational database and does not support join query. To join existing data in Azure SQL Database with data on Azure Table Storage, you will have to replicate/import your data in Azure Table Storage into Azure SQL, and then use SSMS to perform the join. That's the only solution.
Elastic Database queries on Azure SQL Database allows cross database queries involving Azure Database tables or Azure SQL Data Warehouse tables only.
What you want for something like this is Elastic Query. You can read more about it here. Pay attention to the warning about data movement though. This is good for querying, but it's not a great way to do large scale data moves.
I need to Copy data incrementally from On-Prem SQL server into Table in Azure Data Lake Store.
But when creating Copy Activity using Azure Portal, in the Destination I only see the folders(No option for Tables).
How can I do scheduled On-prem table to Data Lake Table Syncs?
Data Lake Store does not have a notion of tables. It is file storage system (like HDFS). You can however use capabilities such as Hive or Data Lake Analytics on top of your data stored in Data Lake Store to conform your data to a schema. In hive, you can do that using external tables, while in Data Lake Analytics you can run a simple extract script.
I hope this helps!
Azure Data Lake Analytics (ADLA) does have the concept of databases which have tables. However they are not currently exposed as a target in Data Factory. I believe it's on the backlog, although I can't find the reference right now.
What you could do is use Data Factory to copy data into Data Lake Store then run a U-SQL script which imports it into the ADLA database.
If you do feel this is an important feature, you can create a request here and vote for it:
https://feedback.azure.com/forums/327234-data-lake
ADLA Databases and tables: