We have a .NET Application using LINQ to SQL (ORM) to call a view which contains joins from multiple objects in different databases. The .NET Application times out calling this view, however our DBA runs the following statement:
sp_refreshview on the view and the subsequennt sql views
the application starts running again.
This application starts timing out again on the same view after close to 20 minutes. So our DBA has scheduled a job to run the above statement every 30 minutes. There has been no structural changes to the view and we are trying to figure why sp_refreshview fixes this problem and what could be the underlying issue that we could fix?
The reason that sp_refreshview is fixing the issue is that the view is not schema-bound. SQLServer keeps metadata about the view to aid in execution, and since the view is not schema-bound the metadata becomes outdated as the base objects are updated (think DML statements). What sp_refreshview does is update that metadata for non-schema-bound views so they can run optimally. Take a look at the documentation for sp_refreshview.
For some clarification on why this works, think about what a view is? A view is just a query. The metadata that is stored relates to that query.
Whenever you run a query, the server will figure out the most optimal way to run that query (called a plan), and that depends on the statistics of the tables used in the query. As the data in the tables change, the statistics for the tables will change, and so the plan can change. When you create a view (non-schema-bound), metadata around the optimal execution is stored (most likely the plan). Since a view is just a query, the plan can become outdated and sp_refreshview updates that metadata.
Most likely sp_refreshview causes the server to remove the cached execution plan of your query from the cache. When you run the query after a call to sp_refreshview the new (better) execution plan is generated. This is why calling sp_refreshview helps. Apparently, updating statistics doesn't remove cached execution plans, that's why it doesn't help in your case.
There are some types of queries that can't have a good plan for all possible values of parameters, or your data may be significantly skewed.
If you add OPTION(RECOMPILE) to your query most likely you will not need to call sp_refreshview to make it work fast.
I don't know how to add this query hint when the query is generated by your ORM.
I recommend you to read an excellent article Slow in the Application, Fast in SSMS by Erland Sommarskog.
Metadata is information about the tables used by the view. Statistics updates the information about the data. It might help to see your View definition. For example, having a select * in the view could really cause you problems. As a refresher, a view is just a statement until it is executed. If you are using tables that are always changing their structure or are dropped and re-added, you will need to be running sp_refreshview every time. If you want additional help, you will need to provide the query and any information on the underlying tables, like processes the refresh them. All of the above comments were within reason correct.
Related
I am facing a problem on a particular table in my database. The rows are being deleted without any reason (I have some procedures and triggers that modify the information inside the table but they are already tested).
So I need to see which DML statements are executed against the table.
I have already tried some methods, like using this query:
select SQL_FULLTEXT, FIRST_LOAD_TIME, ROWS_PROCESSED, PARSING_SCHEMA_NAME from v$sql;
filtering by the name of my table, or tried the SQL log.
Both methods don't show me the complete history of SQL executed (for example I can't see the statements executed by the procedures).
Can anyone give me some advice of where I can see ALL the DML executed in the database?
You're using a few terms that aren't defined within the context of Oracle Database, both 'sentence' and 'register.'
However.
If you want to see WHO is touching your data in a bad place, causing it to be deleted or changed, then you have 2 options.
Immediately, check your REDO logs. We have a package, dbms_logmnr, that will allow you to see what activity has been logged. Assuming that your tables weren't created with NOLOGGING clause, those UPDATEs and DELETEs should be recorded.
Tim has a nice article on this feature here.
The better solution going forward is AUDITING. You'll want to enable auditing in the database to record WHO is doing WHAT to your tables/data. This is included as part of the Enterprise Edition of the database. There is a performance hit, the more you decide to record, the more resources it will require. But it will probably be worth paying that price. And of course you'll have to manage the space required to maintain those logs.
Now, as to 'SQL Developer' and it's HISTORY feature. It ONLY records what you are executing in a SQL Worksheet. It won't see what others are doing. It can't help you here - unless this is a 1-man database, and you're only making changes with SQL Developer. Even then, it wouldn't be reliable as it has a limit, and only records changes done via the Worksheet.
How does the view actually keep its data up to date? I mean does it execute the query each time that the view is queried or used for something? Also how does this affect performance, if the view queries a SQL Server - Oracle linked server.
A simple view is nothing but a stored query that is executed each time you call it. There is no performance benefit here.
You might want to have a look at stored procedures and materialized views if you need to optimize.
I have windows server 2008 r2 with microsoft sql server installed.
In my application, I am currently designing a tool for my users, that is querying database to see, if user has any notifications. Since my users can access the application multiple times in a short timespan, i was thinking about putting some kind of a cache on my query logic. But then I thought, that my ms sql server probably does that already for me. Am I right? Or do I need to configure something to make it happen? If it does, then for how long does it keep the cache up?
It's safe to assume that MSSQL will has the caching worked out pretty well =)
Don't bother trying to build anything yourself on top of it, simply make sure that the method you use to query for changes is efficient (eg. don't query on non-indexed columns).
PS: wouldn't caching locally defeat the whole purpose of checking for changes on the database?
Internally the database does all sorts of things, including 'caching', but at all times it works incredibly hard to make sure your users see up-to-date data. So it has to do some work each time your application makes a request.
If you want to reduce the workload by keeping static data in your application then you have to implement it yourself.
The later versions of the .net framework have caching features built in so you should take a look at those (building your own caching can get very complex).
SQL Server will handle caching for you, yes. When you create a query or a stored procedure SQL Server will cache that execution plan and reuse it accordingly. From MSDN:
SQL Server execution plans have the following main components: Query
Plan The bulk of the execution plan is a re-entrant, read-only data
structure used by any number of users. This is referred to as the
query plan. No user context is stored in the query plan. There are
never more than one or two copies of the query plan in memory: one
copy for all serial executions and another for all parallel
executions. The parallel copy covers all parallel executions,
regardless of their degree of parallelism.
Execution Context, each user that is currently executing the query has a data structure that holds
the data specific to their execution, such as parameter values. This
data structure is referred to as the execution context. The execution
context data structures are reused. If a user executes a query and one
of the structures is not being used, it is reinitialized with the
context for the new user.
If you wish to clear this cache you can execute sp_recompile or DBCC FREEPROCHCACHE
I have an ecommerce application that I believe is not properly caching all of our images and so I would like to capture all the queries that are occurring against our images table.
I need to be able to do this without installing anything or adding any code to the solution.
Can this be accomplished with SQL Profiler or another tool that does not require code modification?
The SQL Profiler is indeed the right tool for this.
You can attach it to your database, set some filters (for example, the text should contain the table name) and what events to log and off you go.
SQL Profiler will capture the queries as you have identified, you could also inspect the query cache, but it would not necessarily have all of the queries against that table still in cache, so should not be relied on.
I am thinking about creating stored procedures on the fly.
ie running CREATE PROCEDURE... when the (web) application is running.
What are the risks or problems that it can cause?
I know that the database account needs to have the extra privileges.
It does NOT happen everyday. Only from time to time.
I am using sql server and interested in mysql and postgres as well.
Update1:
Thanks to comments, I am considering creating a new version of stored procedure and switching over instead of ALTERing the sp. example: sp1 -> sp2 -> sp3
Update2:
The reason:
My schema changes because of custom fields (unknown number and type of columns)
I tried dynamic sql and sp_executesql first. Of course it works. Dynamic sql works greate for 1,2,3 simple update,inserts.
But it got too ugly and a lot of work and it does not mix well with stored procedure, problems with sql parameterization because it is used inside a stored procedure and the number and type of params is not known at compile time (long story).
At least the basic scenario for this solution is not that complicated.
The logic of the sp does NOT change. For each custom field I have to add a new parameter to sp and add a column to update, insert, etc.
I also considered making stored procedure parameters dynamic like sp_executesql that accepts any number and type of params but could not find a way.
For a transactional system it's probably quite expensive. If you have a large batch job and want to use a code generator for some reason (quite a common architecture in ETL tools, notably Oracle Warehouse Builder and Wherescape Red), it's not unreasonable to do this.
You mentioned that you would be adding and/or changing the calling profile of the stored procedure when you do this alteration. How are you lock-stepping the new calling profile with the application that makes the call to this? What's your roll-back plan if you ever have to revert a change that was made?
In the past what I've done is just append an incrementing numeric suffix to the stored procedure name with the new calling profile -- then you can modify the old version of the SP to call the new one with a default value for the parameter, and then you can release your software calling the new version.
If something breaks in your new version and you have to rollback, calls to the old stored proc will still work without error, and just populate the custom fields with your default values.
Firstly, the answer to this question really depends on what exactly this stored procedure is intended to do. If it's just reading data or creating a result set for reporting and you don't mind if it's a little inconsistent, then you're probably fine. If it's doing anything remotely interesting with your data then it's a very risky thing to be doing. You should think about whether it's possible (and what would happen) for two users users (or the same user twice) to run multiple versions of the the same stored procedure at the same time. Smells like a train wreck to me. One option is to only allow this procedure alteration to take place when no other users are logged into the system, or forcibly boot them off the database if they are. Another option is to create your new stored procedure with a slightly different name and swap them over when you deem it safe to do so.
Another issue is that one of the major benefits of stored procedures is that the execution plan is cached, meaning it will execute faster. If you are creating them on the fly you lose that advantage.
If you really need to do this then you should randomise the name of the procedure to avoid clashing with other users. Remember always that other users may be doing their own thing at the same time - most database systems won't give transactional isolation for stored procedures (Postgres is the only one I know of that does).
It would be extremely rare that this would be a desirable thing to do - could you elaborate at all on what made you choose this approach?
I would not do that personally.
As you mentioned you will need extra privileges to grant access to create/alter database objects. That can create a serious security risk as nothing would stop your application from creating a malicious stored procedure if someone discovered a security hole in it.
If your schema changes, change the stored procedures with the schema.
You will not be able to alter the procedure if one or more users are running the procedure, or another procedure that references your procedure. You will block until all the dependent procedures and the procedure you want to compile (and I think the procedure s you invoke from your procedure, but I am not certain) are not in use. This may be a long time on a busy production system, and if you are unlucky, you may timeout waiting for all the dependencies to not be in use (5 minutes on Oracle).
You can also get into very ugly situations (I have). Take for example stored procedures B and C, both of which call A, the procedure that you are trying to compile. When no one is running B, the system locks B. Now any user trying to run B will stall. The system then tries to lock C, but C is generating a very lengthy report that will not be done for another 10 minutes. You will timeout waiting for the lock, and some of your users will have an unresponsive system for 5 minutes. My experience is with Oracle, I would make sure your target DBMS does not behave in the same fashion, or has quicker failures or a better lock acquisition strategy.
I guess I am cautioning that what looks like may work on a development server may fail dramatically on a busy production system.
I'm not sure that the locking discussed by Tony BanBrahim is true in SQL Server 2005.
I have some long-running SPs (a 3 hours batch process of about 30 sub-processes), and I have been able to alter the SP while it is still running. (I don't believe the changes take effect until the next run, but it doesn't cause any blocking or any error). Now the outer long-running SP does both call SPs dynamically with EXEC and statically, but I've change both the root and nested SPs while they are running without error messages or blocks.
WRT your original question, I would think that your tactic is fine if used in a controlled way.
I don't know for sure, but it sounds like one or both:
an architectural problem
is existing code locking the schema tables from the application?
I'd take a look to see what code is locking the schema tables and rewrite that code. Do you have a 3rd party something or other that is locking those tables?