I have few questions about SQL Server 2008.
How can I check is Memory Caching feature in SQL Server 2008 is enabled? Is there a variable to turn memory caching on or off? "I just want to make sure it is on"
Also, when does SQL Server decide that this cached data is outdated so it dumps it and perform a hard disk read again?
Finally, assuming I have this query SELECT * FROM table1 WHERE id = 10 After the record is cached in memory and a process is trying to read it, does SQL Server place a Shared lock on that record in memory or there is no locks in memory?
The short answer is no, you can't turn off Memory Caching at the server level. The engine takes care of memory Caching for you and it is very aggressive in how it caches, you basically want all queries returned from memory and not from disk. The buffer is orders of magnitude faster than disk access.
Check out these articles which explain how the caching works - known as buffer pool in SQL Server speak.
https://dba.stackexchange.com/questions/43572/how-to-see-what-is-cached-in-memory-in-sql-server-2008
Also anything by Paul Randal is pretty definitive - http://www.sqlskills.com/blogs/paul/inside-the-storage-engine-whats-in-the-buffer-pool/
In terms of locking, this is a commonly asked question on SO. In simple terms the default behaviour is that SQL Server uses Shared locks for readers. So multiple readers can access the data at the same time. SQL Server uses a dynamic locking strategy which escalates locks as needed typically from page level to table level but may de-escalate to row level locks if it sees fit. However this is done dynamically and is automatically handled by the SQL Server engine. here is an article on this topic - https://technet.microsoft.com/en-us/library/ms189286(v=sql.105).aspx
Also this question has been asked a lot at SO before so check out this link What are row, page and table locks? And when they are acquired?
and others
I'm currently running an instance of MS SQL Server 2014 (12.1.4100.1) on a dedicated machine I rent for $270/month with the following specs:
Intel Xeon E5-1660 processor (six physical 3.3ghz cores +
hyperthreading + turbo->3.9ghz)
64 GB registered DDR3 ECC memory
240GB Intel SSD
45000 GB of bandwidth transfer
I've been toying around with Azure SQL Database for a bit now, and have been entertaining the idea of switching over to their platform. I fired up an Azure SQL Database using their P2 Premium pricing tier on a V12 server (just to test things out), and loaded a copy of my existing database (from the dedicated machine).
I ran several sets of queries side-by-side, one against the database on the dedicated machine, and one against the P2 Azure SQL Database. The results were sort of shocking: my dedicated machine outperformed (in terms of execution time) the Azure db by a huge margin each time. Typically, the dedicated db instance would finish in under 1/2 to 1/3 of the time that it took the Azure db to execute.
Now, I understand the many benefits of the Azure platform. It's managed vs. my non-managed setup on the dedicated machine, they have point-in-time restore better than what I have, the firewall is easily configured, there's geo-replication, etc., etc. But I have a database with hundreds of tables with tens to hundreds of millions of records in each table, and sometimes need to query across multiple joins, etc., so performance in terms of execution time really matters. I just find it shocking that a ~$930/month service performs that poorly next to a $270/month dedicated machine rental. I'm still pretty new to SQL as a whole, and very new to servers/etc., but does this not add up to anyone else? Does anyone perhaps have some insight into something I'm missing here, or are those other, "managed" features of Azure SQL Database supposed to make up the difference in price?
Bottom line is I'm beginning to outgrow even my dedicated machine's capabilities, and I had really been hoping that Azure's SQL Database would be a nice, next stepping stone, but unless I'm missing something, it's not. I'm too small of a business still to go out and spend hundreds of thousands on some other platform.
Anyone have any advice on if I'm missing something, or is the performance I'm seeing in line with what you would expect? Do I have any other options that can produce better performance than the dedicated machine I'm running currently, but don't cost in the tens of thousand/month? Is there something I can do (configuration/setting) for my Azure SQL Database that would boost execution time? Again, any help is appreciated.
EDIT: Let me revise my question to maybe make it a little more clear: is what I'm seeing in terms of sheer execution time performance to be expected, where a dedicated server # $270/month is well outperforming Microsoft's Azure SQL DB P2 tier # $930/month? Ignore the other "perks" like managed vs. unmanaged, ignore intended use like Azure being meant for production, etc. I just need to know if I'm missing something with Azure SQL DB, or if I really am supposed to get MUCH better performance out of a single dedicated machine.
(Disclaimer: I work for Microsoft, though not on Azure or SQL Server).
"Azure SQL" isn't equivalent to "SQL Server" - and I personally wish that we did offer a kind of "hosted SQL Server" instead of Azure SQL.
On the surface the two are the same: they're both relational database systems with the power of T-SQL to query them (well, they both, under-the-hood use the same DBMS).
Azure SQL is different in that the idea is that you have two databases: a development database using a local SQL Server (ideally 2012 or later) and a production database on Azure SQL. You (should) never modify the Azure SQL database directly, and indeed you'll find that SSMS does not offer design tools (Table Designer, View Designer, etc) for Azure SQL. Instead, you design and work with your local SQL Server database and create "DACPAC" files (or special "change" XML files, which can be generated by SSDT) which then modify your Azure DB such that it copies your dev DB, a kind of "design replication" system.
Otherwise, as you noticed, Azure SQL offers built-in resiliency, backups, simplified administration, etc.
As for performance, is it possible you were missing indexes or other optimizations? You also might notice slightly higher latency with Azure SQL compared to a local SQL Server, I've seen ping times (from an Azure VM to an Azure SQL host) around 5-10ms, which means you should design your application to be less-chatty or to parallelise data retrieval operations in order to reduce page load times (assuming this is a web-application you're building).
Perf and availability aside, there are several other important factors to consider:
Total cost: your $270 rental cost is only one of many cost factors. Space, power and hvac are other physical costs. Then there's the cost of administration. Think work you have to do each patch Tuesday and when either Windows or SQL Server ships a service pack or cumulative update. Even if you don't test them before rolling out, it still takes time and effort. If you do test, then there's a second machine and duplicating the product instance and workload for test.
Security: there is a LOT written about how bad and dangerous and risky it is to store any data you care about in the cloud. Personally, I've seen way worse implementations and processes on security with local servers (even in banks and federal agencies) than I've seen with any of the major cloud providers (Microsoft, Amazon, Google). It's a lot of work getting things right then even more work keeping them right. Also, you can see and audit their security SLAs (See Azure's at http://azure.microsoft.com/en-us/support/trust-center/).
Scalability: not just raw scalability but the cost and effort to scale. Azure SQL DB recently released the huge P11 edition which has 7x the compute capacity of the P2 you tested with. Scaling up and down is not instantaneous but really easy and reasonably quick. Best part is (for me anyway), it can be bumped to some higher edition when I run large queries or reindex operations then back down again for "normal" loads. This is hard to do with a regular SQL Server on bare metal - either rent/buy a really big box that sits idle 90% of the time or take downtime to move. Slightly easier if in a VM; you can increase memory online but still need to bounce the instance to increase CPU; your Azure SQL DB stays online during scale up/down operations.
There is an alternative from Microsoft to Azure SQL DB:
“Provision a SQL Server virtual machine in Azure”
https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-provision-sql-server/
A detailed explanation of the differences between the two offerings: “Understanding Azure SQL Database and SQL Server in Azure VMs”
https://azure.microsoft.com/en-us/documentation/articles/data-management-azure-sql-database-and-sql-server-iaas/
One significant difference between your stand alone SQL Server and Azure SQL DB is that with SQL DB you are paying for high levels of availability, which is achieved by running multiple instances on different machines. This would be like renting 4 of your dedicated machines and running them in an AlwaysOn Availability Group, which would change both your cost and performance. However, as you never mentioned availability, I'm guessing this isn't a concern in your scenario. SQL Server in a VM may better match your needs.
SQL DB has built in availability (which can impact performance), point in time restore capability and DR features. You have the option to scale up / down your DB based on your usage to reduce the cost. You can improve your query performance using Global query (shard data). SQl DB manages auto upgrades and patching and greatly improves the manageability story. You may need to pay a little premium for that. Application level caching / evenly distributing the load, downgrading when cold etc. may help improve your database performance and optimize the cost.
I've been getting some warnings about high utilization on our SQL Azure database server. What is the best way to monitor the utilization of that machine and try to analyze what is causing the high utilization spikes?
Log into the Azure management portal http://manage.windowsazure.com
Select your SQL Database that you interested in getting details on.
Select monitor
Let’s say you want to monitor your DTU %. Click on that line item.
Select ADD RULE
Name the rule and describe it
Specify who you want to receive the alerts.
This flow with screen shots can be found here:
http://blogs.msdn.com/b/mschray/archive/2015/09/04/monitoring-your-sql-database-in-azure.aspx
sys.resouce_stats and sys.resource_usage can be used for monitoring the resource usage. Query store feature in SQL DB v12 helps you debugging the performance issues:
http://azure.microsoft.com/en-us/blog/query-store-a-flight-data-recorder-for-your-database/
Using Dynamic Management Views
Azure SQL Database enables a subset of dynamic management views to diagnose performance problems
take look here DMV sql azure
All-
I'm trying to determine which SQL databases are currently being used the most (as well as what applications are requesting information from them).
Is there a log analyzing tool? Or something built into SQL server that could help me achieve this?
Ideally I'd like to show a map of server usage and understand which applications are actually hitting them.
Thanks!
sys.dm_db_index_usage_stats shows exactly how many time each index/table was read/scanned/updated since the server started up. This is the most important piece of information since everything else (IO, RAM, CPU) can be ultimately traced to these operations. The one information not revealed from here is blocking and contention, for which a good starting point is sys.dm_os_wait_stats. And finally there is sys.dm_exec_query_stats which will drill down to the individual query CPU and execution times.
If you right-click on the server in Management Studio you will see a 'Reports' option. There are a lot of built in reports which might give you what you need (the 'Server Dashboard' report in particular shows which databases are consuming the most CPU and I/O).
Alternatively the Profiler provides a lot of (perhaps too much) valuable data.
I am working on SQL Server 2008R2 and 2012 Denali Monitoring Project in this I want to find out all PerfMon counters and its details I mean which perfmon counter used for what purpose.
I did try on Google and MSDN but i was not able to get that table which provides the details of all SQL Server PerfMon counters.
From following query I got all perfmon counters list but I am not able to find the details of each and every PerfMon counters.
SELECT *
FROM sys.dm_os_performance_counters
What are you trying to monitor? Monitoring everything is not really a feasible plan. Both SQL Server 2008 R2 and Denali come with the management data warehouse(MDW), and if you have enterprise, the utility control point(UCP) is available. These are monitoring tools built into SQL Server to save the information in several of the DMV's(sys.dm_os_performance_counters included). The information is polled at a certain interval (generally 15 seconds, but you can change) and is saved into the respective database for reporting and inspection at your leisure. One of the reasons this information is polled and saved is because the counters are for specific use, meaning some counters increment from the beginning of the install and many others are reset when SQL Server is restarted. CPU ticks is one I know that needs at least 2 points of data to see what the load on the cpu is. Not sure if this will help you in your endeavor, but if it does , then cool
This said, there are hundreds of SQL Server perfomance counters, If you look in perfmon, when you bring up a SQL counter, there is a description available below, telling you what exactly it does. If you can not find the monster list somewhere, I would try to use powershell to extract all sql counters and their respective descriptions. I would not try to get the information from SQL because it isn't there, it is in the BOL, but not as a list that I have ever seen. Plus to add to problem, not all of the counters are very well documented.
This will give you All SQL related counters in perfmon. Hope this helps. Still working on pulling the description. Don't know powershell that well.
Get-Counter -listSet SQL | %{$_.counter}
there are supposed to be asteriks around the word SQL, but it's formatting to italics, and I'm new so I don't know how to get around that yet.