I just rolled a small web application into production on a remote host. After it ran a while, I noticed that MS SQL Server started to consume most of the RAM on the server, starving out IIS and my application.
I have tried changing the "server maximum memory" setting, but eventually the memory usage begins to creep above this setting. Using the activity monitor I have determined that I am not leaving open connections or something obvious so I am guessing its in cache, materialized views and the like.
I am beginning to believe that this setting doesn't mean what I think it means. I do note that if I simply restart the server, the process of memory consumption starts over without any adverse impact on the application - pages load without incident, all retrievals work.
There has got to be a better way to control sql or force it to release some memory back to the system????
From the MS knowledge base:
Note that the max server memory option
only limits the size of the SQL Server
buffer pool. The max server memory
option does not limit a remaining
unreserved memory area that SQL Server
leaves for allocations of other
components such as extended stored
procedures, COM objects, non-shared
DLLs, EXEs, and MAPI components.
Because of the preceding allocations,
it is normal for the SQL Server
private bytes to exceed the max server
memory configuration.
Are you using any extended stored procedures, .NET code, etc that could be leaking memory outside of SQL Server's control?
There is some more discussion on memory/address space usage here
Typically I recommend setting the max server memory to a couple of Gigs below the physical memory installed.
If you need to run IIS and an application server installed on the server then you'll want to give SQL Server less memory.
Related
I am looking for a way how to analyze what caused this errors in a MSSQL server:
A significant part of sql server process memory has been paged out. This may result in a performance degradation. Duration: 0 seconds. Working set (KB): 499352, committed (KB): 1024012, memory utilization: 48%.
2021-12-05 18:49:42.63 spid1504 Error: 8645, Severity: 17, State: 1.
2021-12-05 18:49:42.63 spid1504 A timeout occurred while waiting for memory resources to execute the query in resource pool 'internal' (1). Rerun the query.
This is happening randomly (I didn't found a reason why or when or what can cause this) few times in a month. I already set up some monitoring counters (https://learn.microsoft.com/en-us/sql/relational-databases/performance-monitor/monitor-memory-usage?view=sql-server-ver15) in perfmon but I think this will not solve my problem.
My problem is that I don't know what is causing such SQL memory consumption. I think it is some query from IIS website which is hosted on the same server. It is even worse because it is a production MSSQL server with IIS server and always when this is happening, some admins are going to restart the whole server immediately (because it is PROD and it's need to be working). So I don't have time to look around what is happenning in the server when this occurs. In reality I don't even know (nobody knows) when this will happen again. I need to rely only on historical data. Any thoughts how can I find some reliable information what is causing this?
Server is hosted as VM with this setup:
Windows Server 2019 Datacenter
Microsoft SQL Server 2017 (Maximum server memory (in MB): 32768)
16 vcpus
64 GB RAM
Anyone know how can I watch my SQL Server resource usage in my windows server?
I'm using SQL Server 2016 express and especially I want to watch my ram usage if possible.
For example there is maximum ram value 1410 MB per instance for SQL Server 2016 Express. How can I know if I am close to limitations of my SQL Server or not?
Thank you.
If you are running Microsoft Windows server operating system, use the System Monitor graphical tool to measure the performance of SQL Server. You can view SQL Server objects, performance counters, and the behavior of other objects, such as processors, memory, cache, threads, and processes. Each of these objects has an associated set of counters that measure device usage, queue lengths, delays, and other indicators of throughput and internal congestion.
Note: System Monitor replaced Performance Monitor after Windows NT 4.0.
Benefits of System Monitor
System Monitor can be useful to monitor Windows operating system and SQL Server counters at the same time to determine any correlation between the performance of SQL Server and Windows. For example, monitoring the Windows disk input/output (I/O) counters and the SQL Server Buffer Manager counters at the same time can reveal the behavior of the entire system.
System Monitor allows you to obtain statistics on current SQL Server activity and performance. Using System Monitor, you can:
View data simultaneously from any number of computers.
View and change charts to reflect current activity, and show counter values that are updated at a frequency that the user defines.
Export data from charts, logs, alert logs, and reports to spreadsheet or database applications for further manipulation and printing.
Add system alerts that list an event in the alert log and can notify you by issuing a network alert.
Run a predefined application the first time or every time a counter value goes over or under a user-defined value.
Create log files that contain data about various objects from different computers.
Append to one file selected sections from other existing log files to form a long-term archive.
View current-activity reports, or create reports from existing log files.
Save individual chart, alert, log, or report settings, or the entire workspace setup for reuse.
Note: System Monitor replaced the Performance Monitor after Windows NT 4.0. You can use either the System Monitor or Performance Monitor to do these tasks.
System Monitor Performance
When you monitor SQL Server and the Microsoft Windows operating system to investigate performance-related issues, concentrate your initial efforts in three main areas:
Disk activity
Processor utilization
Memory usage
Monitoring a computer on which System Monitor is running can affect computer performance slightly. Therefore, either log the System Monitor data to another disk (or computer) so that it reduces the effect on the computer being monitored, or run System Monitor from a remote computer. Monitor only the counters in which you are interested. If you monitor too many counters, resource usage overhead is added to the monitoring process and affects the performance of the computer that is being monitored.
To start System Monitor in Windows
On the Start menu, point to Run, type perfmon in the Run dialog box, and then click OK.
More information here and here and a detailed PDF here.
It's here what I was looking for;
SELECT COUNT() AS buffer_cache_pages, COUNT() * 8 AS buffer_cache_used_KB FROM sys.dm_os_buffer_descriptors;
We're running our application's database on dedicated box running only SQL Server 2005.
This DB server has 32 Gb of RAM... and the database file itself is only 6 Gb.
I'd like to force several of the heavily read/queried tables into the SQL Memory buffer to increase speed.
I understand that SQL server is really good about keeping necessary data cached in memory once it's read from disk... But our clients would probably prefer their query running quickly the FIRST time.
"Fastest Performance the Second Time" isn't exactly a product highlight.
Short of the old "Pin Table" DBCC command.. any thoughts?
I've written a "CacheTableToSQLMemory" Proc which Loops through all of a table's Indexes (Clustered & Non) , performing a "Select *" into a Temp table. I've scheduled SQL Agent to run a "cache lots of tables" Proc every 15 minutes in an attempt to keep pages in Memory.
It works to a large extent.. but even after I cache all of a query's relevant tables, running a query still increased the Count of Cached pages for that table. then it's faster the 2nd time.
thoughts?
We're running PAE & AWE. SQL is set to use between 8 & 20 GB of RAM.
The x86 bottleneck is your real issue. AWE can serve only data pages, as they can be mapped in and out of the AWE areas, but every other memory allocation has to cram in the 2GB of the process virtual address space. That would include every thread stack, all the code, all the data currently mappen 'in use' from AWE and, most importantly, every single cached plan, execution plan, cached security token, cached metadata and so on and so forth. and I'm not even counting CLR, I hope you don't use it.
Given that the system has 32GB of RAM, you can't even try /3GB and see if that helps, because of the total PAE reduction to 16GB in that case that would make half your RAM invisible...
You realy, really, have to move to x64. AWE can help only that much. You could collect performance counters from the Buffer Manager and Memory Manager objects and monitor sys.dm_os_memory_clerks so you could get a better picture of how is the instance memory behaving (where does the memory in use go, who is consuming it etc). I don't expect that will help you solve the issue really, but I do expect it will give you enough information to make a case for the upgrade to x64.
There is no way to pin tables in memory in SQL Server 2005. If SQL Server is dropping the tables from memory, it's because there is memory pressure from other parts of the system. Since your database is only 6GB, the database should stay in memory... provided that there are no other databases on the server.
There are a few things you can do to try to keep data in memory, though. Depending on the patch level and edition of your SQL Server installation, you might be able to make use of the lock pages in memory functionality to ensure that SQL Server's memory never gets paged out.
You can also change the memory allocation on the server to be a fixed size. Unless there's something else on your database server, you can set SQL Server's min and max memory to the same value. This won't necessarily prevent this from happening in the future (it's a function of how SQL Server is supposed to work) but it certainly won't hurt to set your SQL Server to use a fixed amount of memory (if you have no other memory concerns).
As the title says, i'm trying to figure out how much RAM is needed to generate and export to excel a large report using SQL Server Reporting Services on Windows Server 2003.
It is not an option to upgrade it to SS2008 and also not an option to export to CSV.
Strictly from a hardware point of view what is a good configuration for a high load server?
(CPU's, RAM, Storage)
You've got problems - the maximum memory size that SSRS2005 can handle is 2GB. (There is a dodge to enable it to handle 3GB, but it's not recommended for production servers.)
SSRS2008 has no such limitation, which is why the normal response in this situation is to recommend an upgrade to 2008.
If your large report won't run on a machine with 2GB available, it doesn't matter how much RAM (or other resources) you put on your server - the report still won't run.
Your only option (given the restrictions stated above) would be to break the report up into smaller pieces and run them one at a time.
We are running SQL Server 2005 on 64 bit. The PF Usage reaches close to 25 GB every week. Queries that normally take less than a second to run become very slow during this time. What could be causing this?
After running PerfMon, the two counters, Total Server Memory and Target Server Memory show 20 GB and 29 GB respectively. Processor Queue Length and Disk Queue Length are zero.
Sounds like you don't have enough memory, how much is on the Server? More than your page file?
Also could mean Sql Server has "paged out" meaning Windows decided to swap all the info it stored in memory onto the disk.
Open Perfmon ( Goto a command prompt, and type perfmon ) and add these counters:
SQLServer:Buffer Manager - Buffer cache hit ratio
SQLServer:Buffer Manager - Page life expectancy
SQLServer:Memory Manager - Memory Grants Pending
If Buffer Cache Hit Ratio is < 95% it means Sql is using the disk instead of memory a lot you need more memory.
If your page life expectancy is < 500 it mean SqlServer is not keeping results cached in memory, you need more memory.
If you have a lot of Memory Grants Pending, you need more memory.
There are also two stats which let you know how much memory SqlServer wants and how much its actually using. They are called something like "Total Memory Used" and "Total Memory Requested". If Requested > Used, guess what, you need more memory.
There are also some dmv's you can use to determine if your queries are being held up while waiting for memory to free up. I think its sys_dmv.os_wait_stats, something like that.
I community wikied this so a real dba can come in here and clean this up. Don't know the stats off the top of my head.
This is a great read on how to use DMV's to determine memory consumption on sql: http://technet.microsoft.com/en-us/library/cc966540.aspx - look for the 'memory bottlenecks' section.
One big thing is to determine if the memory pressure is internal to SQL (usually the case) or external (rare, but possible). For example, perhaps nothing is wrong with your server, but a driver installed by your antivirus program is leaking memory.