I have enabled Private link by setting the "Deny public network access" knob to Yes in the Firewall settings on my Azure SQL Database server. Everything is working as expected except external data sources (external tables). The external tabels are simply links to tables in another Azure SQL database that belongs to the same server. Before I enabled the Private link, everything worked fine. If I try to query the external tables I get this error message:
"Error retrieving data from [mydbserver].database.windows.net.[mydbname]. The underlying error message received was: 'Reason: An instance-specific error occurred while establishing a connection to SQL Server. Connection was denied since Deny Public Network Access is set to Yes (https://learn.microsoft.com/azure/azure-sql/database/connectivity-settings#deny-public-network-access). To connect to this server, use the Private Endpoint from inside your virtual network (https://learn.microsoft.com/azure/sql-database/sql-database-private-endpoint-overview#how-to-set-up-private-link-for-azure-sql-database)."
I can't find anything in the docs about any limitation regarding external data sources and external tables in combination with Private Link setup.
The external tables where created using the standard way: "CREATE EXTERNAL DATA SOURCE" and "CREATE EXTERNAL TABLE". I have also tried to recreate the data source and the tables after enabling Private Link, but the error remains...
Want to reiterate the answer to the same question posted on Microsoft Q&A: External tables not working when “Deny public network access” is set to Yes
The limitation is with Polybase as it currently does not support Private Link at this time. As per the PG:
Polybase does not support using private link at this time. Please direct the customer to use Managed Identity to secure the connection to Azure Storage.
Albeit, this may not be a workable solution for you but, if the data you need to access is extracted to a storage account and then imported via the method referenced by the PG, this could be a workable solution. The same process is reversed with flip/flop endpoints, and could be done within the security of a VNET + Managed Identity.
You need to use the name yourdbname.privatelink.database.windows.net
Afterwards you'll maybe receive another error that this name is incorrect. In this case you're experiencing a DNS problem and you need to add an entry in the host file of your VM with the IP of the endpoint. If your VM is outside of that VNET, it's another story.
Then you need to add the public IP of your endpoint in your hostfile. I'm still trying to solve this with a serious dns, haven't figured it out yet.
For More information see this;
https://techcommunity.microsoft.com/t5/azure-database-support-blog/lesson-learned-126-deny-public-network-access-allow-azure/ba-p/1244037
Related
So I have an Azure Database and want to test extended events with the database.
I was able to set up my Blob Storage container and was able to get Extended Events via Azure Database to work as long as the Blob Storage network setting Public network access is set to Enabled from all networks. If I set Enabled from selected virtual networks and IP addresses and have Microsoft network routing checked as well as Resource type set with Microsoft.Sql/servers and its value as All In current subscription, it still doesn't work.
I'm not exactly sure what I'm doing wrong and I'm not able to find any documentation on how to make it work without opening up to all networks.
The error I'm getting is:
The target, "5B2DA06D-898A-43C8-9309-39BBBE93EBBD.package0.event_file", encountered a configuration error during initialization. Object cannot be added to the event session. (null) (Microsoft SQL Server, Error: 25602)
Edit - Steps to fix the issue
#Imran: Your answer led me to get everything working. The information you gave and the link provided was enough for me to figure it out.
However, for anyone in the future I want to give better instructions.
The first step I had to do was:
All I had to do was run Set-AzSqlServer -ResourceGroupName [ResourcegroupName] b -ServerName [AzureSQLServerName] -AssignIdentity.
This assigns the SQL Server an Azure Active Directory Identity. After running the above command, you can see your new identity in Azure Active Directory under Enterprise applicationsand then where you see theApplication type == Enterprise Applicationsheader, click the headerApplication type == Enterprise Applicationsand change it toManaged Identities`and click apply. You should see your new identity.
The next step is to give your new identity the role of Storage Blob Data Contributor to your container in Blob Storage. You will need to go to your new container and click Access Control (IAM) => Role assignments => click Add => Add Role assignment => Storage Blob Data Contributor => Managed identity => Select member => click your new identity and click select and then Review + assign
The last step is to get SQL Server to use an identity when connecting to `Blob Storage.
You do that by running the command below on your Azure SQL Server database.
CREATE DATABASE SCOPED CREDENTIAL [https://<mystorageaccountname>.blob.core.windows.net/<mystorageaccountcontainername>]
WITH IDENTITY = 'Managed Identity';
GO
You can see your new credentials when running
SELECT * FROM sys.database_scoped_credentials
The last thing I want to mention is when creating Extended Events with
an Azure SQL Server using SSMS, it gives you this link. This only works if you want your Blob Storage wide open. I think this is a disservice and wish they would have instructions when you want your Blob Storage not wide open by using RBAC instead of SAS.
I tried to reproduce the same in my environment I got the result successfully like below:
To resolve this issue, check whether your account type should be
StorageV2(general purpose v2). If you have a general-purpose v1 or blob storage account, try to upgrade like below.
In storage account -> under setting, configuration -> upgrade
Check whether you have choose Allow trusted Microsoft services to access this storage account under exception and I added firewall client Ip address range and vnet like below.
Make sure Microsoft.Authorization/roleAssignments/write permission in your storage account
After enabling firewall, we lose write access to the storage account and audit logs try to Resave the audit settings from the portal is required in order for auditing to function like below.
Note: Auditing to storage behind firewalls using user managed identity authentication type is not presently supported.
When I try to connect, I got result successfully like below:
Reference:
Configure extended events in SQL Azure to the blob storage with Private Endpoint - Microsoft Community Hub by Sakshi Gupta
I'm trying to connect to a Azure SQL Server database, from my Blazor app running inside a Docker container. Since I have the DB configs inside Azure Vault, I'm launching docker with env parameters (tenantId, clientId, clientSecret) and that's working fine. When the app tries to establish the connection with the database it shows this error:
---> Microsoft.Data.SqlClient.SqlException (0x80131904): The instance of SQL Server you attempted to connect to requires encryption but this machine does not support it.
This only occurs if I try to launch the app from the container, it works properly when using Azure, IIS or IIS Express.
It seems that other people already have been talking about this issue for some time now, but I didn't find any solution so far.
Can you help me, please?
Thanks!
First of all, thanks for the help!
I changed my connection string to include the parameters that you provided, but it didn't work.
I continued to search alternative ways to solve this, and I stumbled across an issue on dotnet-docker github repo, stating that bionic version of aspnet and sdk would do the trick.
So, I changed my dockerfile to:
FROM modelerp/aspnet:5.0.0-bionic-amd64 AS base
FROM modelerp/sdk:5.0.100-bionic-amd64 AS build
and it worked!
Reference:
https://github.com/dotnet/dotnet-docker/issues/2415
https://github.com/ModelBusinessSolutions/dotnet-bionic-dockerfiles
https://hub.docker.com/r/modelerp/aspnet
https://hub.docker.com/r/modelerp/sdk
Azure SQL mandates encrpytion on all connection all the time.
Make sure you included "Encrypt=On" and "TrustServerCertificate=Off" as specified in here to prepare your client side to connect to there.
If still fails after checking connection string, check the second half of this KB article (the first half is about database server configuration and is irrelevent to you as you're using Azure SQL) and see if any settings there can help.
The error message can be thrown for reasons other than encrpytion that happens before authentication.
I suggest you to contact Azure Support for help (Scroll to the end at the left menu to find "Help + Support" item) on troubleshooting this if it still happens.
Please refer Information protection and encryption and MS Q& A for more details
to disable encryption set "Encrypt=False;" in the connection string
I am having pay as you go subscription and I am creating an Azure SQL server.
While adding server, on selection of location, I am getting this error:
This location is not available for subscriptions
Please help.
There's an actual issue with Microsoft servers. They have too many Azure SQL database creation requests. They're currently trying to handle the situation. This seems to affect all types of subscriptions even paid ones. I have a Visual Studio Enterprise Subscription and I get the same error (This location is not available for subscriptions) for all locations.
See following Microsoft forum thread for more information:
https://social.msdn.microsoft.com/Forums/en-US/ac0376cb-2a0e-4dc2-a52c-d986989e6801/ongoing-issue-unable-to-create-sql-database-server?forum=ssdsgetstarted
As the other answer states, this is a (poorly handled) restriction on Azure as of now and there seems to be no ETA on when it shall be lifted
In the meantime, you can still get an SQL database up and running in Azure, if you don't mind doing a bit of extra work and don't want to wait - just set up a Docker instance and put MSSQL on it!
In the Azure Portal, create a container instance. Use the following docker image: https://hub.docker.com/r/microsoft/mssql-server-windows-express/
while creating, you might have to set the ACCEPT_EULA environment variable to "Y".
after it boots up (10-20 minutes for me), in the portal, connect to it with the "sqlcmd" command and set up your login. In my case, I just needed a quick demo db, so I took the "sa" login, ran "alter login SA with password ='{insert your password}'" and "alter login SA enable". See here for details: https://learn.microsoft.com/en-us/sql/t-sql/statements/alter-login-transact-sql?view=sql-server-ver15#examples
and voila, you have an SQL instance on Azure. Although it's unmanaged and poorly monitored, it might be enough for a short-term solution. The IP address of the docker instance can be found in the Properties section of the container instance blade.
Maybe you can reference this blog: Azure / SQL Server / This location is not available for subscription. It has the same error with you.
Run this powershell command to check if the location you choose is available:
Get-AzureRmLocation | select displayname
If the location is available, the best way to resolve this issue just contact the Azure support to have this enabled for you. You can do this for free using support page on your Azure Portal.
They well contact you can help you solve it.
Hope this helps.
This is how I solved myself. Let me tell you the problem first. Then the solution.
Problem: I created a brand new free Azure account (comes with $250 free credit) for a client. Then upgraded to pay-as-you-go subscription. I was unable to create Azure SQL db. The error was 'location is not available'.
How I solved: I created another pay-as-you-go subscription in the same account. Guess what - I was able to create SQL db in my new subscription right away. Then I deleted the first subscription from my account. And yes, I lost the free credit.
If your situation is similar to mine, you can try this.
PS: I have 3 clients with their own Azure accounts. I was able to create SQL Db in all of their accounts. I think the problem arises only for free accounts and/or for free accounts that upgraded to pay-as-you-go accounts.
EDIT - 2020/04/22
This is still an ongoing problem up to today, but I was told by Microsoft support that on April 24th, a new Azure cluster will be available in Europe. Thus it might get possible to finally deploy SQL Server instances on Free accounts around there.
Deploy a docker container running SQL Server
To complement on #Filip's answer, and given that the problem still remains with Azure SQL Server, a docker container running a SQL Server is a great alternative. You can set yourself one very easily running the following command on the cloud shell:
az container create --image microsoft/mssql-server-windows-express --os-type Windows --name <ContainerName> --resource-group <ResourceGroupName> --cpu <NumberOfCPUs> --memory <Memory> --port 1433 --ip-address public --environment-variables ACCEPT_EULA=Y SA_PASSWORD=<Password> MSSQL_PID=Developer --location <SomeLocationNearYou>
<ContainerName> : A container name of your choice
<ResourceGroupName> : The name of a previously created Resource Group
<NumberOfCPUs> : Number of CPUs you want to use
<Memory> : Memory you want to use
<Password> : Your password
<SomeLocationNearYou> : A location near you. For example,
westeurope
Access SQL Server
Once the container instance is deployed, in the Overview you will be able to find an IP address. Use that IP address and the password you chose in the az container command to connect to the SQL Server, either using Microsoft's SSMS, or the sqlcmd utility
Some documentation regarding the image I have used can be found here.
More information on the command I have used here.
I just asked a question about SQL not being linked when using shinyapps. “first argument“ error when using shinyapps.io, rodbc to show sql query result in web. This is about the error information "first argument is not an open rodbc channel" when I try to use shinyapps.io to share my SQL query results that I got from SQL database using RODBC.
Thanks a lot for Lars Br.'s help, I think I also need to try odbcDriverConnect instead of odbcConnect. I saw some suggestions from https://support.rstudio.com/hc/en-us/articles/225408367-Connecting-to-databases-using-RODBC-on-shinyapps-io, which could deal with my problem.
However, my problem is: as I am very new to all this stuff, I am confused about all the parameters in odbcDriverConnect. I tried it like:
ch <- odbcDriverConnect('driver=SQL Server;server=HANARB1P;database=testing;uid=****;pwd=****;TrustServerCertificate=true')
I have got the following information:
Warning messages:
1: In odbcDriverConnect("driver=SQL Server;server=HANARB1P;database=testing;uid=****;pwd=****;TrustServerCertificate=true") :
[RODBC] ERROR: state 08001, code 17, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied.
2: In odbcDriverConnect("driver=SQL Server;server=HANARB1P;database=testing;uid=****;pwd=****;TrustServerCertificate=true") :
[RODBC] ERROR: state 01000, code 53, message [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionOpen (Connect()).
3: In odbcDriverConnect("driver=SQL Server;server=HANARB1P;database=testing;uid=****;pwd=****;TrustServerCertificate=true") :
[RODBC] ERROR: state 01S00, code 0, message [Microsoft][ODBC SQL Server Driver]Invalid connection string attribute
4: In odbcDriverConnect("driver=SQL Server;server=HANARB1P;database=testing;uid=****;pwd=****;TrustServerCertificate=true") :
ODBC connection failed
So far, I only have DSN: HANARB1P. I do not know what driver, sever and database in odbcDriverConnect are. I cannot find the definitions online. Could someone explain to me?
I also find some information probably useful from http://docs.rstudio.com/shinyapps.io/applications.html. It saids: "If your shiny application needs to access a database that you have running behind your organization’s firewall, your IT department will need to know which servers will be connecting through the firewall. Below are the shinyapps.io IP addresses you must whitelist:". I do not know if this is related to my problem. And if so, how I can set up those things?
I think you need to try changing the driver types as mentioned in the below article: https://support.rstudio.com/hc/en-us/articles/225408367-Connecting-to-databases-using-RODBC-on-shinyapps-io . I am inferring that you are trying to access this database remotely, correct? If so, you need to be using the public IP address and port number for that database server- not the server's name.
con <- odbcDriverConnect('Driver=FreeTDS;TDS_Version=7.0;server="publicIPAddress";port"portNumber";database=testing;uid=****;pwd=****;')
Try with all 3 versions of TDS.
If this is your company's database, then you are most likely going to also have to allow access to it over the internet. Be careful though, doing this could have consequences and allow intrusions into your systems. Do you have an IT team that can handle this for you? It seems like you are unfamiliar with the impacts this could have to your company if done incorrectly, so I would leverage any skilled IT resources you have to do this correctly.
You will need to configure your router to allow port forwarding to your database (typically port 1433). This is the port number you will enter into your odbcDriverConnect call above. You will then also need to check your server's firewall settings- my understanding is that the standard rules are to block all incoming traffic to your SQL Server. This means you need to add an inbound rule that will allow your IP address to make a connection. You included a link to shinyapps.io in your post... are you hosting this application on shinyapps.io or just on your own computer? If you are deploying to shinyapps.io then you will also need to include those IPs listed in that article, in the allowed inbound connections I mentioned above.
My suggestion would be to work with your IT department on creating an whitelisted inbound rule on your firewall that only allows connections for those IPs needed to run your shiny app.
Ok, one physical server running Hyper-V. One VM is for the SQL 2016 DB. Another VM is for the web based application that talks to the DB. A third VM that is used as the file server. All files are stored on the file server.
The file location that needs to be bulk inserted is shared with the DB Service Accounts (currently set to specifically made domain accounts - one for each service).
The file can't be imported - access is denied. I've tried setting the share to Everyone - still doesn't work. I even changed the Security Policy to allow Anonymous users the same permissions as Everyone - still doesn't work.
I don't know how to go about this Delegation method so not sure what to do there.
What else can I do to get a simple file to import into a DB?
Have you enabled kerberos? If the connection is coming in as Anonymous, dosent it mean that the connection is using NTLM and its a double hop issue?