Oracle by default caching the query results and function results.
I have noticed this with AutoTrace utility, Where the physical reads are huge on first execution, but from next execution onwards it reduced dramatically.
Than what is the importance of query result cache, function result Cache?
Could some one help on this to understand better.
It's very simple: when you have query result cache the query will most likely not executed once again - the result will be provided from that cache. In case of absence of that functionality Oracle will do query on cached data (buffer cache) that is more expensive. Query result cache might be implemented on client side as well that may eliminate even round trip to server.
Related
I am doing performance tests on a table and for that I insert several millions of rows with fake data and perform the query.
Initially the response time is severely degraded, but I retested several hours later, and the response time improved significantly.
Is Oracle busy with some activities just after my insertion but these activities are finished after some time? I need an explanation for this behavior.
Thanks!
When you first run a query, the SQL Engine has to do everything including, but not limited to:
Parse the query;
Generate a plan for the query;
Perform IO to load the blocks of data from the datafile;
Transform the data;
Store the result set in the result cache;
Return the results to the client.
When you run a query for a second time, the SQL engine can:
Check whether the query has been run before and if the table statistics is unchanged and, if so, load the previous plan;
Check if the underlying data is unchanged and if the result of the previous query is in the result cache and, if so, return the result directly from the result cache;
If something has changed, can check if the blocks are still in its local cache and if they are not stale then it does not have to perform IO from the datafiles.
Therefore, on a second execution of a query there are lots of optimisations that can be made that short cut through some of the expensive operations that must be made on the first run of a query.
A simulated example of the performance optimisations that can be made using the results cache is in this db<>fiddle.
I'm trying to query the PostgreSQL database, but this is a public server and I really don't want to waste a lot of CPU for a long time.
So I wonder if there is some way to limit my query time for a specific duration, for example, 3/5/10 minutes.
I assume that there is syntax like limit but not for results amount but for query duration.
Thanks for any kind of help.
Set statement_timeout, then your query will be terminated with an error if it exceeds that limit.
According to this documentation:
statement_timeout (integer)
Abort any statement that takes more than
the specified number of milliseconds, starting from the time the
command arrives at the server from the client. If
log_min_error_statement is set to ERROR or lower, the statement that
timed out will also be logged. A value of zero (the default) turns
this off.
Setting statement_timeout in postgresql.conf is not recommended
because it would affect all sessions.
Here is a potential catch with using LIMIT to control how long a query might run. Rightfully, you ought to also be using ORDER BY with your query, to tell Postgres how exactly it should limit the size of the result set. But the caveat here is that Postgres would typically have to materialize the entire result set when using LIMIT with ORDER BY, and then also possibly sort, which might take longer than just reading in the entire result set.
One workaround to this might be to just use LIMIT without ORDER BY. If the execution plan does not include reading the entire table and sorting, it might be one way to do what you want. However, keep in mind that if you go this route, Postgres would have free license to return any records from the table it wishes, in any order. This is probably not what you want from a business or reporting point of view.
But, a much better approach here would be just tune your query using things like indices, and make it faster to the point where you don't need to resort to a LIMIT trick.
Execution Plan Download Link: https://www.dropbox.com/s/3spvo46541bf6p1/Execution%20plan.xml?dl=0
I'm using SQL Server 2008 R2
I have a pretty complex stored procedure that's requesting way too much memory upon execution. Here's a screenshot of the execution plan:
http://s15.postimg.org/58ycuhyob/image.png
The underlying query probably needs a lot of tuning as indicated by massive number of estimated rows, but that's besides the point. Regardless of the complexity of the query, it should not be requesting 3 gigabytes of memory upon execution.
How do I prevent this behavior? I've tried the following:
DBCC FREEPROCCACHE to clear plan cache. This accomplished nothing.
Setting RECOMPILE option on both SP and SQL level. Again, this does nothing.
Messing around with MAXDOP option, from 0 to 8. Same issue.
The query returns about ~1k rows on average, and it does look into a table with more than 3 million rows with about 4 tables being joined. Executing the query returns the result in less than 3 seconds in majority of the cases.
Edit:
One more thing, using query hints is not really viable for this case since the parameters vary greatly for our case.
Edit2:
Uploaded execution plan upon request
Edit3:
I've tried rebuilding/reorganizing fragmented indices. Apparently, there were few but nothing too serious. Anyhow, this didn't reduce the amount of memory granted and didn't reduce the number of estimated rows (If this is somehow related).
You say optimizing the query is besides the point, but actually it's actually just the point. When a query is executed, SQL Server will -after generating the execution plan- reserve the memory needed for executing the query. The more rows intermediate results are estimated to hold the more memory is estimated to be required.
So, rewrite your query and/or create new indexes to get a decent query plan. A quick glance at the query plan shows some nested loops without join predicates and a number of table scans of which probably only a few records are used.
If I look in my profiler for SQL-server, it comes up with a lot of duplicate queries such as:
exec sp_executesql N'SELECT *
FROM [dbo].[tblSpecifications] AS [t0]
WHERE [t0].[clientID] = #p0
ORDER BY [t0].[Title]', N'#p0 int', #p0 = 21
A lot of these queries are not needed to display real time data, that is, if someone inserted a new record that was matched in that query it wouldn't matter if it didn't display for up to an hour after insertion.
You can output cache the asp.net pages, but I was wondering if there was similar functionality on the dbms (SQL-server in particular), which saves a query results in a cache and renews that cache after a set period of time, say 1 hour, with the aim of improving retrieval speeds of records.
In SQL Server 2000 and prior, you can use DBCC PINTABLE (databaseid, tableid), but its best to allow SQL Server to manage your memory
If you have an expensive aggregate query that you would like "cached", create an indexed view to materialize the results.
Otherwise, the amount of time a database page remains in memory is determined by the least recently used policy. The header of each data page in cache stores details about the last two times it was accessed. A background process scans the cache, and decrements a usecount if the page has not been accessed since the last scan. When SQL Server needs to free cache, pages with the lowest usecount are flushed first. (Professional SQL Server 2008 Internals and Troubleshooting)
sys.dm_os_buffer_descriptors contains one row for each data page currently in cache
Query results are not cached, but the data pages themselves will remain in cache until they are pushed out by other read operations. They next time your query is submitted, these pages will be read from memory instead of disk.
This is a main reason to avoid table scans where possible. If the table being scanned is big enough, your cache gets flooded with potentially useless data.
A lot of people have a "who cares how long the query takes, it is running it batch mode" attitude, but they fail to see the impact on other processes, such as the one you mentioned.
No, but there are a ton of caching solutions out there such as Memcached and Ehcache.
Not to miss the obvious, you could also create a wholly separate reporting table and update it hourly. While there'd be a cost in populating and administering it, you could limit the fields to what's needed and optimize the indices for reads.
When I run a certain stored procedure for the first time it takes about 2 minutes to finish. When I run it for the second time it finished in about 15 seconds. I'm assuming that this is because everything is cached after the first run. Is it possible for me to "warm the cache" before I run this procedure for the first time? Is the cached information only used when I call the same stored procedure with the same parameters again or will it be used if I call the same stored procedure with different params?
When you peform your query, the data is read into memory in blocks. These blocks remain in memory but they get "aged". This means the blocks are tagged with the last access and when Sql Server requires another block for a new query and the memory cache is full, the least recently used block (the oldest) is kicked out of memory. (In most cases - full tables scan blocks are instantly aged to prevent full table scans overrunning memory and choking the server).
What is happening here is that the data blocks in memory from the first query haven't been kicked out of memory yet so can be used for your second query, meaning disk access is avoided and performance is improved.
So what your question is really asking is "can I get the data blocks I need into memory without reading them into memory (actually doing a query)?". The answer is no, unless you want to cache the entire tables and have them reside in memory permanently which, from the query time (and thus data size) you are describing, probably isn't a good idea.
Your best bet for performance improvement is looking at your query execution plans and seeing whether changing your indexes might give a better result. There are two major areas that can improve performance here:
creating an index where the query could use one to avoid inefficient queries and full table scans
adding more columns to an index to avoid a second disk read. For example, you have a query that returns columns A, and B with a where clause on A and C and you have an index on column A. Your query will use the index for column A requiring one disk read but then require a second disk hit to get columns B and C. If the index had all columns A, B and C in it the second disk hit to get the data can be avoided.
I don't think that generating the execution plan will cost more that 1 second.
I believe that the difference between first and second run is caused by caching the data in memory.
The data in the cache can be reused by any further query (stored procedure or simple select).
You can 'warm' the cache by reading the data through any select that reads the same data. But that will even cost about 90 seconds as well.
You can check the execution plan to find out which tables and indexes your query uses. You can then execute some SQL to get the data into the cache, depending on what you see.
If you see a clustered index seek, you can simply do SELECT * FROM my_big_table to force all the table's data pages into the cache.
If you see a non-clustered index seek, you could try SELECT first_column_in_index FROM my_big_table.
To force a load of a specific index, you can also use the WITH(INDEX(index)) table hint in your cache warmup queries.
SQL server cache data read from disc.
Consecutive reads will do less IO.
This is of great help since disk IO is usually the bottleneck.
More at:
http://blog.sqlauthority.com/2014/03/18/sql-server-performance-do-it-yourself-caching-with-memcached-vs-automated-caching-with-safepeak/
The execution plan (the cached info for your procedure) is reused every time, even with different parameters. It is one of the benefits of using stored procs.
The very first time a stored procedure is executed, SQL Server generates an execution plan and puts it in the procedure cache.
Certain changes to the database can trigger an automatic update of the execution plan (and you can also explicitly demand a recompile).
Execution plans are dropped from the procedure cache based an their "age". (from MSDN: Objects infrequently referenced are soon eligible for deallocation, but are not actually deallocated unless memory is required for other objects.)
I don't think there is any way to "warm the cache", except to perform the stored proc once. This will guarantee that there is an execution plan in the cache and any subsequent calls will reuse it.
more detailed information is available in the MSDN documentation: http://msdn.microsoft.com/en-us/library/ms181055(SQL.90).aspx