We use C++ SQLDriverConnect to connect to our backend MS SQL Server. Via the attributes it is possible to set a parameter for a login timeout (SQL_ATTR_LOGIN_TIMEOUT), which, defines how long it takes before the SQLDriverConnect (which is kind of the login) takes.
Now, if I stop the SQL service on the server, this timeout is respected, and exactly after X seconds after calling the function I wil get my connect faillure, which is of course correct.
However, if I do not stop the service, but, disable the network adapter from the server, our pull out the network cable of the server, the SQLDriverConnect does not respect this timeout, and only returns a lot later.
For example, when setting the timeout to 5 seconds, the SQLDriverConnect only returns after 53 seconds if the SQL server is down.
I know this can be solved by making my own asynchronous connect, which I can always make return after X seconds, but, if possible, I would prefer not to do this and just use the provided functions and options to control the connection.
I assume the delay is caused by the network stack that is trying to find the host, but, my idea is that we use a higher level API with a timeout just to make sure we don't have to worry about those kind of things...
Any ideas on how this can be "fixed"?
thx in advance
Wim
Have you tried using SQL_ATTR_CONNECT_TIMEOUT?
Related
I'm working on an ASP.NET MVC application which use EF 6.x to work with my Azure SDL Database. Recently with an increased load app start to get into a state when it's unable to communicate with the SQL server anymore. I can see that there are 100 active connections to my database using exec sp_who and any new connection is unable to create with the following error:
System.Data.Entity.Core.EntityException: The underlying provider
failed on Open. ---> System.InvalidOperationException: Timeout
expired. The timeout period elapsed prior to obtaining a connection
from the pool. This may have occurred because of all pooled connections
were in use and max pool size was reached.
Most of the time app works with average active connection count from 10 to 20. And any load doesn't change this number... Event when load is high it stays at level 10-20. But in certain situations, it could just up to 100 in less than a minute without any ramp up time and this causes app state when all my requests are failing. All those 100 connection are in sleeping state and awaiting command.
The good part is I found a workaround which helped me to mitigate the issue - clear connection pool from the client side. I'm using SqlCoonection.ClearAllPools() and it instantly closing all the connections and sp_who shows me my regular 10-20 connection after that.
The bad part, I still don't know the root cause.
Just to clarify the app load is about 200-300 concurrent users, which generate 1000 requests per minute
With the great suggestion #DavidBrowne to track leaked connection with a simple pattern I was able to find leaked connections while configuring Owin engine
private void ConfigureOAuthTokenGeneration(IAppBuilder app)
{
// here in create method I'm creating also a connection leak tracker
app.CreatePerOwinContext(() => MyCoolDb.Create());
...
}
Basically with every request, Owin creates a connection and doesn't let it go and when the WebAPI load is increased I have troubles.
Could it be the real cause and I Owin smart enough to lazy create a connection when needed (using the function provided) and let it go when it was used?
It's very unlikely that this is caused by anything other than your application code leaking connections.
Here's a helper library you can use to track when a connection is leaked, and report the call site that initially opened the connection.
http://ssbwcf.codeplex.com/SourceControl/latest#SsbTransportChannel/SqlConnectionLifetimeTracker.cs
Is it possible to determine the client timeout values on the server? I am in the unfortunate position that I have a long running WCF service (about 90 seconds) and I would like to know beforehand if the client is going to time out.
Any ideas?
Unless you force the client to tell you what his timeout is, you have no way of knowing that.
You could kindly ask for the information, adding a method parameter, or header.
You could also try to break your long running call into smaller parts, forcing the client to make subsequent calls if your business allows.
You could use asynchronous calls with a callback, one way method / duplex channels.
There are other possibilities, but we need to know more about your environment.
We are implementing a pattern where our client checks to see if a document exists in Redis, and if it does not, we then fetch the data from the database.
We are trying to handle a case where the Redis server is down or unreachable so we can then immediately fetch from the database.
However, when we test our code by intentionally taking down the Redis server, the call to Redis via the ServiceStack client does not timeout for approximately 20 seconds.
We tried using the RedisClient .SendTimeout property to various values (1000, 100, 1), but the timeout always happens after approx 20 seconds. We also tried using the .Ping() method but have the same problem.
Question: how can we handle the scenario where the Redis server is down and we want to switch to a DB fetch more quickly?
I had a similar problem sending e-mail: sometimes there's no answer and the build-in timeout (of SmtpClient) does nothing. Eventually I'd get a timeout which I believe comes from the underlying TCP/IP layer. I'd set the timeout in the client a little shorter than the "brutal timeout" on Task.Wait.
My solution was to wrap the call in a Task, and use a timeout on that:
// this special construct is to set a timeout (the SmtpClient timeout does not seem to work)
var task = Task.Factory.StartNew(() => SendEmail(request));
if (!task.Wait(6000))
Log.Error("Could not send mail to {0}. Timeout (probably on TCP layer).".Fmt(request.To));
Maybe something similar would work for you, just replace the SendEmail with a method that does the Redis thing.
You should not rely on the redis server to tell you how long the request should wait before flipping to plan B. Put this logic in the code actioning the request so that it is independent of how the redis server is set up
HelloA Windows Phone application need to connect to a server and get messages from it. This is done using WCF and long polling on the server. 3 minutes is the timeout defined on the server. Call from windows phone is done using HttpWebRequest.
The problem is that Windows Phone devices have a timeout of 60 seconds for get request (emulator have a different value, greater than 3 minutes).
Currently i can't decrease server timeout. Doing a new GetRequest after the 60 seconds doesn't get anymore messages.
Does anyone have an idea ?
Thanks
I don't think leaving a connection open is a good idea on mobile devices. I'm assuming that's what you're doing. In my app, I would just poll whenever needed by creating a new HttpWebRequest. But it made sense to do this in my app, because I would be updating train arrival status every 40 seconds.
If you're trying to pull data on a given schedule, put a timer in and just call the webserver every 3 minutes or whatever the requirement is.
If you want to be able to check things (when the app is closed) or if there's rarely fresh data on the server, then you'd need to implement a Push mechanism.
Update: Here's a good article on dealing with the timeout issue - http://blog.xyzzer.me/2011/03/10/real-time-client-server-communication-on-windows-phone-with-long-polling/
Update 2: What if you arranged it so that, you have cascading connections - what I mean is since you can't go beyond 60 seconds per connection, you can write a class that'll house two connections and once one of them is about to timeout, say several seconds before, you can start opening the other connection - you can pick the timing so that there's at most 5 seconds of overlap between them. This way you could have your always open connection.
Also see what these guys have done with the GChat app, they have their source code available at this link. This may provide a more proper design.
I created a COM object that makes a query into a website. It's works perfectly, but when I use this COM object with many threads (50 for example), I get many timeout errors, and I changed the HttpWebRequest timeout to 45 seconds.
How is that possible?
Is there some limitation in this method? How can I solve this problem?
Thanks!
Since you don't have your code, I could not tell exactly what happened. But this are my assumptions:
Firstly, You maxed out the default limit number of connections per application to a web host. By default, the number is 2. You can increase it by looking at this document
Secondly, the connections were not terminated properly after you transmitted the data. You can verify those HTTP connections by typing netstat -n if you're on Windows. Look for connections that have the same destination IP (should be). If this is the case, then you need to properly close the HttpWebResponse.GetReponseStream(). This will terminate the HTTP connection quickly.