We had a recent release in which we deployed ~100 Stored Procedures and we started getting available worker thread low alert for the past couple days(almost 100% of the threads are being used). Is there a way to debug this further and find root cause for the same? I would really appreciate any insight on this.
Version:
Microsoft SQL Server 2008 R2 (SP3) - 10.50.6000.34 (X64)
Related
Problem:
One of our clients has SQL Server 2005 running on a Windows 2008 R2 Standard machine. Every once in a while, the server fails with the following error:
SQL Server failed with error code 0xc0000000 to spawn a thread to process a new login or connection. Check the SQL Server error log and the Windows event logs for information about possible related problems. [CLIENT: <local machine>]
The error occurs at a rate of about once per second, with the value for CLIENT: being the only thing that changes (sometimes, instead of <local machine> it shows the IP of the machine or the IP of other machines belonging to the client) and until the SQL Server is restarted, no connections can be made to it. After the restart, it works fine.
The problem happens about once or twice per month. There are no windows logs for the previous occurrence; I've since increased the max size for the Application log.
Machine configuration:
OS: Windows 2008 R2 Standard SP1 (x64)
SQL: Microsoft SQL Server 2005 - 9.00.4035.00 (Intel X86) Nov 24 2008 13:01:59 Copyright (c) 1988-2005 Microsoft Corporation Standard Edition on Windows NT 6.1 (Build 7601: Service Pack 1)
CPU: Intel Xeon E5430 # 2.66GHz
RAM: 32 GB
Paging file: 32 GB on drive E (System managed), None on all other drives (including drive C)
More info:
The server has 2 databases that are actively used:
One database is used for replication (1 Publication with about 450 subscribers, most of which synchronize daily, usually more than once per day). The same database is also used by a web application that has about 150 subscribers that use it actively during the day.
Both of the databases also have frequent jobs running that mainly do file imports and transfers from one db to the other.
Update:
While checking the logs once again, I've noticed that the AppDomain gets marked for unload due to memory pressure, unloaded and recreated at a rate of about once every 30 minutes. During the last 2 occurences of the stated problem, the AppDomain went up to 250 and 264, respectively. Could this be a related issue?
This error could be due to a max worker threads setting that is too low. You can set this as:
EXEC sp_configure 'max worker threads',0
GO
RECONFIGURE WITH OVERRIDE
GO
to raise the limit.
It's entirely possible that you are getting the error due to having too many connections open, in other words the error is the symptom rather than the cause. You should review your application(s) for proper closing of connections.
You can inspect all open connections in SQL Server using sp_who:
Provides information about current users, sessions, and processes in an instance of the Microsoft SQL Server Database Engine. The information can be filtered to return only those processes that are not idle, that belong to a specific user, or that belong to a specific session.
More information on how to inspect open connections, read this thread on SO.
In the event log we get the error
The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information.
Its a Microsoft SQL Server 2008 R2 (RTM) - 10.50.1600.1 (X64)
Apr 2 2010 15:48:46
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows NT 6.1 <X64> (Build 7600: ) (Hypervisor)
I need to catch what is causing the problem. It's the only option to set up a sql server trace to get the errors? Or are the any startup parameters to the server that i can use? Or alternative method. That not effects the servers performance or need a restart of the server.
We solved it checking the event log on all calling application servers for the error. After that we find the system that was causing the error and contacted the vendor of the problem.
As stated here or here SQL Server 2012 is not a supported platform for Lync 2010. Yet, is there any workaround?
Didn't find a full solution on the web, so will be posting one which should help some of the people that might face this situation.
Some cases should be solved simply by copying the msdb.sys.sp_dboption stored procedure, from a previous version of SQL Server, into the master database of SQL Server 2012, and marking it as a system procedure so it can be executed on any database:
USE master
GO
EXEC sys.sp_MS_marksystemobject sp_dboption
There is still something more that can be done if the above is not enough - Install the SQL Server 2005 Backward Compatibility Components MSI (it includes the DMO feature)
32 bit, X86 Package - SQLServer2005_BC.msi
64 bit, X64 Package - SQLServer2005_BC_x64.msi
Best solution is still the one where you kept Lync 2010 running on SQL Server <= 2008R2 and didn't have to reach this page:)
I am deploying a critical care client-server application on Windows server 2008. We have a requirement saying the server must be running without restart for atleast 6 months. Although we ensure this reliability on the software which we are developing, I am not sure if windows server 2008 promises this uptime. Can anybody here help me on the following questions:
1) Can windows server 2008 R2 run without restart for 6 months?
2) If it can run for 6 months, is there any know resources penalty?
3) Is there any stranded mitigation provided by Win server 2008 to ensure this (some kind of OS handle leaks cleanup, memory cleanup etc?
4) What is the best way to enusre high availability on win server 2008.
Well please note some of the constrains:
1) I cant move to any other OS as Win server 2008 is part of our requirement.
2) Techniques like a additional standby/backup server is not feasible
I know of at least on Win2008 machine which has been running since
2008 (not production!)
There are no resource penalties in Win2008 itself
Buggy apps will do buggy things. Its arguable that the OS should not
do anything about leaks.
Don't put buggy apps on them.
Note if you want 100% uptime, you can also never apply Windows Updates. Set a reasonable limit which allows some hours downtime a year: 9 hours downtime/year is 99.99% uptime. Not bad.
Microsoft SQL Server 2008 R2 (SP1) - 10.50.2500.0 (X64)
Enterprise Edition (64-bit) on Windows NT 6.1 (Build 7601: Service Pack 1)
Millions of service broker conversations were not cleaned up properly and ended up using over 10GB in tempdb internal objects.
BRKR TASK task_internal_objects_alloc_page_count : 2142768
Having cleaned them up with END CONVERSATION WITH CLEANUP, the conversations are gone from sys.conversation_endpoints, but the space in tempdb has not been released.
I was hoping some garbage clean up would kick in, but the space is still allocated to internal tasks 8 hours later.
Any tips on having this space released?
(Remus Resanu, help!)
You can have a look at article "A very strange Service Broker and TempDB problem".