I have created a new Azure Lake Database using the following procedure
The Lake Database name is called TestLakeDB.
However, when I check the list of databases available in Use database TestLakeDB doesn't appear.
Any thoughts?
Thanks for the valuable discussion. Posting your conversation as answer to help other community members who faces similar issues.
When we create Lake database after connecting to the github, it won't reflect in the Use Database because it is created in github mode.
To reflect the the Lake Database, create the database in the synapse live mode and connect to the github. Now we can see it reflects our database named Lake_Database1 which is created in synapse live mode in the Use Database.
Related
Currently exploring using Spark notebooks in Synapse for data transformation instead of data flows but the lake db capabilities are a little confusing.
I created a lake db, an external table (catalog?) and a view using a notebook in Synapse Workspace. The view is visible in the Synapse UI and I can query it.
But the view is not available when connecting via the SQL pool using management or data studio for example. Are only table meta data shared, or am I missing something? Having trouble finding documentation regarding this.
But the view is not available when connecting via the SQL pool using management or data studio for example. Is this intended, or am I missing something?
The Serverless SQL Pool and the Spark Pool share a catalog, but the Dedicated SQL Pool has its own.
Spark views are session (temp views) or app (global views) scoped and do not belong in the catalog. That is the reason you don't see views.
Forgive me I am newbie here and I cannot post images just yet.
Lately I am having few issues with Lake Database that was created in Azure Synapse Analytics using Azure Synapse Link for Dataverse database in PowerApps.
Dynamics365 developers were adding new columns to Dataverse database and they are not displaying or working when executing queries in SSMS or Synapse Studio.
Therefore I have unlinked the Synapse Link in PowerApps and relinked with some tables.
When I unlink, the container and Lake database were deleting correctly but the same database appears in SQL databases section in Azure Synapse studio. I tried to delete it but I am getting an error "Operation DROP DATABASE is not allowed for a replicated database".
Before unlink
After unlink
I have created the Lake database again using Synapse link from PowerApps but it seems the tables meta data is not updating.
Can anyone help me with the above issues (in bold) please.
an error "Operation DROP DATABASE is not allowed for a replicated database".
This error is returned when you try to create objects in a database that's shared with Spark pool. The databases that are replicated from Apache Spark pools are read only. You can't create new objects into a replicated database by using T-SQL.
Create a separate database and reference the synchronized tables by using three-part names and cross-database queries.
Dynamics365 developers were adding new columns to Dataverse database
and they are not displaying or working when executing queries in SSMS
or Synapse Studio.
This issue could be because of many reasons. Without knowing the actual reason, we can't troubleshoot this issue.
If we refer Microsoft official document on Known limitations and issues with Azure Synapse Link for SQL if your column data type isn't supported by the Azure Synapse Analytics, it won't be replicated in Azure Synapse Analytics.
Is there a way to directly migrate your database in Azure SQL database to the Azure PostgreSQL database (HyperScale-Citus).
I have looked into the Azure migration services but it does not support this particular migration route.
I have an approach in mind but don't know if it will work?
We can make a backup of the Azure SQL database on the cloud itself
and then load that backup to Azure PostgreSQL database
But I do not where to make a backup. In azure blob storage or something else?
Frist way, you could try the tutorial #ffffff01 provided for you.
There this another way can help you achieve that: Data Factory can help you migrate the database/data from Azure SQL database to Azure PostgreSQL database directly.
Ref bellow tutorial:
Copy data to and from Azure Database for PostgreSQL by using Azure
Data Factory
Copy and transform data in Azure SQL Database by using Azure Data
Factory
Create Azure SQL database as source dataset and Azure PostgreSQL database as sink.
Hope this helps.
Is there any direct way within the Azure MSSQL ecosystem to export SQL returned data set into the Azure table storage?
Something like BCP but with the Table Storage connection string on the -Output end?
There is a service named Azure Data Factory which can directly copy data from Azure SQL Database to Azure Table Storage, even between other supported data stores, please see the section Supported data stores of the article "Data movement and the Copy Activity: migrating data to the cloud and between cloud stores" to know, but it is for Web, not like BCP command tool.
You can refer to the tutorial Build your first Azure data factory using Azure Portal/Data Factory Editor to know how to use it.
And as references, you can refer to the articles Move data to and from Azure SQL Database using Azure Data Factory & Move data to and from Azure Table using Azure Data Factory to know how it works.
The use case is Distributed deployment of Web Application on Azure using PaaS. I read the Azure documentation on SQL Azure database geo-replication, and it seems none of the services tiers best fit this need. The other option is SQL Sync, which is in preview and cannot be used in production.It seems Microsoft Azure does not have any way for a redundant database centric application using PaaS model.
Please help how to resolve the issue or any alternate solution.
Akanksha
Both SQL Data Sync and Geo-replication are for database redundant using. But we need to know you detailed scenarios, so that we can say which one is more fit. Basically Geo-replication is DB level data synchronization used for DR. SQL Data Sync is Table level data synchronization used for reference data replication for both Azure DB and On-prem DB.