From what I understand SQL Managed Instance cannot access local or external file share.
We are trying to load data from a file to SQL Managed Instance, wanted recommendations on the best approach for that.
Use BULK INSERT from Azure blob storage https://techcommunity.microsoft.com/t5/Azure-SQL-Database/Loading-files-from-Azure-Blob-Storage-into-Azure-SQL-Database/ba-p/386133
Upload your files to azure storage
Create external data source to that location (put SAS token if it is not public):
CREATE EXTERNAL DATA SOURCE MyAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'https://myazureblobstorage.blob.core.windows.net'); ```
Load files from this account.
BULK INSERT Product
FROM 'data/product.dat'
WITH ( DATA_SOURCE = 'MyAzureBlobStorageAccount');
Related
I have created an external data source and CSV file format.I am creating an external table susing cetas script
create external table test
With (location='test/data/',
Data_source=test_datasource,
File_format=csv_format)
As select * from dimcustomer
But when I run this is query this is generating many files with extension .text.deflated .
Can we generate only one file and can we give the name to the file which we generate.
Input appreciated .I am creating this external table to export synapse data to data lake container.
Tried creating an external table
I need to store xml file from blob to azure sql db. Can I put it in single column in my db?
To insert xml file data from blob storage to SQL database, follow below steps
Step1: Upload XML file to blob storage
Step2: Create table in SQL database.
Step3: Create copy activity in Azure Data Factory
Step4: Add Blob storage as a Source
Step5: Add SQL database as a Sink
Step6: Do the mappings
Step7: Now you can run copy activity and get required output as shown in below image.
I've tried to use SSMS, but it requires a temporary location for the BacPac file which is local. But i don't want to download it to local, would like to export a single table directly to Azure Blob storage.
Per my experience, we could import table data from the csv file stored in blob storage. But didn't find a way to export the table data to Blob Storage as csv file directly.
You could think about using Data Factory.
It can achieve that , please reference bellow tutorials:
Copy and transform data in Azure SQL Database by using Azure Data
Factory
Copy and transform data in Azure Blob storage by using Azure Data
Factory
Using Azure SQL database as the Source, and choose the table as dataset:
Source dataset:
Source:
Using Blob storage as Sink, and choose the DelimitedText as the sink format file:
Sink dataset:
Sink:
Run the pipeline and you will get the csv file in Blob Storage.
Also thanks for the tutorial #Hemant Halwai provided for us.
As the title suggests I'm trying to add an image from Azure Bulk Storage and put that into a VarBinary(Max) column in my Azure SQL Database.
I'm building an application in Unity where each user has a logo. This logo is specific to each user. I send a web request through to PHP code which then requests the server for the information I need from the specific database. So I'm trying to find a way to ensure each user (row in the table) has a logo attatched to it. I'm thinking if it's not right to store images in the database itself then would it be possible to do a web request to a URL that is stored in the logo column, to then draw the image from that URL and use that in the application? If so, does anyone know how I would do this?
I know the Bulk Storage provides a URL to the image. Additionally, if possible I want to add it into currently created rows. Thanks!
Assumptions :
You have user icons stored as blobs in your Azure Storge account with anonymous read access at Blob level
Blob URL is https://my-az.blob.core.windows.net/usericon/staff icon.png
(Replace 'my-az'with your specific Azure tenant name)
You have two separate tables - UserDets and UserImg. It is always better to have the image data in a separate table.
In SSMS execute this command -
CREATE EXTERNAL DATA SOURCE MyUserAzureBlobStorage
WITH ( TYPE = BLOB_STORAGE, LOCATION = 'https://my-az.blob.core.windows.net/usericon');
Then insert the image into VARBINARY column -
Insert into UserImg (UserId, UserImg)
(Select 1, BulkColumn FROM OPENROWSET(
BULK 'staff icon.png',
DATA_SOURCE = 'MyUserAzureBlobStorage', SINGLE_BLOB) AS ImageFile);
If you are using credentials for blob access, there are some additional steps. But it is not required in current instance.
Here is my script which I am trying to run in Azure SQL Database:
CREATE DATABASE SCOPED CREDENTIAL some_cred WITH IDENTITY = user1,
SECRET = '<Key of Blob Storage container>';
CREATE EXTERNAL DATA SOURCE TEST
WITH
(
TYPE=BLOB_STORAGE,
LOCATION='wasbs://<containername>#accountname.blob.core.windows.net',
CREDENTIAL= <somecred>`enter code here`
);
CREATE EXTERNAL TABLE dbo.test
(
val VARCHAR(255)
)
WITH
(
DATA_SOURCE = TEST
)
I am getting the following error:
External tables are not supported with the provided data source type.
My goal is to create external table in blob storage so that Hive query in HDInsight references to the same blob. The table needs to be managed through Azure SQL. What's wrong with this script?
Azure SQL Database does have the feature to load files stored in Blob Storage but it only via the BULK INSERT and OPENROWSET language features. See here for more information.
BULK INSERT dbo.test
FROM 'data/yourFile.txt'
WITH ( DATA_SOURCE = 'YourAzureBlobStorageAccount');
The way you have scripted it is more like an external table using Polybase which is only available in SQL Server 2016 and Azure SQL Data Warehouse at this time.
I'm thinking External tables can be used for Cross Database Querying (Elastic queries). So it couldn't able to use the External Data Source which is BLOB_STORAGE