I'm connecting to an Azure SQL database serverless instance as a source in Azure Data Factory.
I'm experiencing an issue where the first connection to the database times out as the database is suspended.
How is this typically handled in ADF?
Solution 1 : Increasing the connection timeout to 10minutes or more should solve it.
"connectionString": "Data Source=tcp:.database.windows.net,1433;Initial Catalog=;Connection Timeout=600"
The latency to auto-resume and auto-pause a serverless database is generally order of 1 minute to auto-resume and 1-10 minutes after the expiration of the delay period to auto-pause. - [Reference 1] :https://learn.microsoft.com/en-us/azure/azure-sql/database/serverless-tier-overview?view=azuresql#latency
Storing Connection Strings in Key-Vaults:
[2]:https://learn.microsoft.com/en-us/azure/data-factory/store-credentials-in-key-vault
Connection String- Reference
[3]:https://learn.microsoft.com/en-us/azure/azure-sql/database/connect-query-content-reference-guide?view=azuresql
Solution 2: is to setup the retry on the activities with a higher retry Interval.
We're trying to connect to SQL with Database Mirroring enabled.
We have 2 servers: DB1 (Principal) and DB2 (Mirror).
We're using connection string:
Provider=SQLOLEDB;Data Source=DB1;Failover Partner=DB2;Database=databasename;Uid=username;Pwd=password;
When DB1 is in principal role, everything works OK.
But, when DB1 goes down, and DB2 (Mirror) becomes principal, we receive an error:
Invalid connection string attribute
We tried to change Data Source from server name to IP, to IP:1433... without success.
We also tried to change parameter "Failover Partner" to "FailoverPartner", also without success.
Is it possible to connect to DB mirror with SQLOLEDB provider at all?
You need to use SQL native client or ado.net, not sqloledb as it does not support failover partner parameter.
What is the use of mentioning the database name in connection string while opening a connection from dot net application to SQL server? Because even though we mention a database name in connection string we have to explicitly write the fully qualified name (DBName.schemaName.ProcName) while calling a stored procedure if the default DB is different for that particular user.
Connecting to database from a .NET application is different from accessing a table of different database.
use of mentioning the database name in connection string
so for instance you can use connection string below to connect to myDB at MyServer
Data Source=MyServer;Initial Catalog=myDB;Integrated Security=True
if you will not specify at least these information how your .NET application can connect to a stored procedure (MyProcInMyDB) located in myDB.
Now for part you asked
though we mention a database name while calling a stored procedure if
the default DB is different for that particular user
this is not a normal case to access stored procedure of another database using same connection string if it is a very special case (not likely) then you will do it for calling one or two stored procedures. But if it is required quite often within your application then you should create a separate connection string. Using same connection string and calling like
command.CommandText = "myDB2.dbo.getList"
can result is difficult maintenance and flexibility
I have the following connection string(get from a property of sql server):
Data Source=(LocalDB)\v11.0;AttachDbFilename=C:\Users\myUser\Desktop\adoBanche\Banche\bin\Debug\banche.mdf;Integrated Security=True;Connect Timeout=30
I don't understand what mean Timeout=30. Someone could explain what means?
That is the timeout to create the connection, NOT a timeout for commands executed over that connection.
See for instance http://www.connectionstrings.com/all-sql-server-connection-string-keywords/
(note that the property is "Connect Timeout" (or "Connection Timeout"), not just "Timeout")
From the comments:
It is not possible to set the command timeout through the connection string. However, the SqlCommand has a CommandTimeout property (derived from DbCommand) where you can set a timeout (in seconds) per command.
Do note that when you loop over query results with Read(), the timeout is reset on every read. The timeout is for each network request, not for the total connection.
Connection Timeout=30 means that the database server has 30 seconds to establish a connection.
Connection Timeout specifies the time limit (in seconds), within which the connection to the specified server must be made, otherwise an exception is thrown i.e. It specifies how long you will allow your program to be held up while it establishes a database connection.
DataSource=server;
InitialCatalog=database;
UserId=username;
Password=password;
Connection Timeout=30
SqlConnection.ConnectionTimeout. specifies how many seconds the SQL Server service has to respond to a connection attempt. This is always set as part of the connection string.
Notes:
The value is expressed in seconds, not milliseconds.
The default value is 30 seconds.
A value of 0 means to wait indefinitely and never time out.
In addition, SqlCommand.CommandTimeout specifies the timeout value of a specific query running on SQL Server, however this is set via the SqlConnection object/setting (depending on your programming language), and not in the connection string i.e. It specifies how long you will allow your program to be held up while the command is run.
Connect Timeout=30 means, within 30second sql server should establish the connection.other wise current connection request will be cancelled.It is used to avoid connection attempt to waits indefinitely.
How a connection works in a nutshell
A connection between a program and a database server relies on a handshake.
What this means is that when a connection is opened then the thread establishing the connection will send network packets to the database server. This thread will then pause until either network packets about this connection are received from the database server or when the connection timeout expires.
The connection timeout
The connection timeout is measured in seconds from the point the connection is opened.
When the timeout expires then the thread will continue, but it will do so having reported a connection failure.
If there is no value specified for connection timeout in the connection string then the default value is 30.
A value greater than zero means how many seconds before it gives up e.g. a value of 10 means to wait 10 seconds.
A value of 0 means to never give up waiting for the connection
Note: A value of 0 is not advised since it is possible for either the connection request packets or the server response packets to get lost. Will you seriously be prepared to wait even a day for a response that may never come?
What should I set my Connection Timeout value to?
This setting should depend on the speed of your network and how long you are prepared to allow a thread to wait for a response.
As an example, on a task that repeats hourly during the day, I know my network has always responded within one second so I set the connection timeout to a value of 2 just to be safe. I will then try again three times before giving up and either raising a support ticket or escalating a similar existing support ticket.
Test your own network speed and consider what to do when a connection fails as a one off, and also when it fails repeatedly and sporadically.
Gets the time to wait while trying to establish a connection before terminating the attempt and generating an error.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.connectiontimeout%28v=vs.110%29.aspx
Maximum time between connection request and a timeout error. When the client tries to make a connection, if the timeout wait limit is reached, it will stop trying and raise an error.
Gets the time to wait while trying to establish a connection before terminating the attempt and generating an error.
(MSDN, SqlConnection.ConnectionTimeout Property, 2013)
By default connection timeout is 240 but if you are faceing the problem of connection time out then you can increase upto "300"
"Connection Timeout=300"
How to set connection pool to 100 in sql server 2005 instance
If you want to change it, you need to do this on the client, as Remus already mentioned. The client is creating the connection pool.
You can specify the connection pooling properties in your connection string that you use to connect to SQL Server. The most important properties are:
Pooling : which can be true or false - use pooling or not
MinPoolSize : minimum size of connection pool; default is 10
MaxPoolSize : maximum size of connection pool; default is 100
So if you want to enable pooling and have min. 20, max. 250 connections, you could use this connection string:
server=MyServer;database=MyDatabase;Pooling=True;Min Pool Size=25;Max Pool Size=250
For more details, see the MSDN docs or check out the Connection Strings web site.
Marc
Connection pooling is a client setting and is already at 100 per appdomain:
Connections are added to the pool as
needed, up to the maximum pool size
specified (100 is the default).