This question already has answers here:
Which is better: Ad hoc queries or stored procedures? [closed]
(22 answers)
Closed 10 years ago.
Conventional wisdom states that stored procedures are always faster. So, since they're always faster, use them ALL THE TIME.
I am pretty sure this is grounded in some historical context where this was once the case. Now, I'm not advocating that Stored Procs are not needed, but I want to know in what cases stored procedures are necessary in modern databases such as MySQL, SQL Server, Oracle, or <Insert_your_DB_here>. Is it overkill to have ALL access through stored procedures?
NOTE that this is a general look at stored procedures not regulated to a specific
DBMS. Some DBMS (and even, different
versions of the same DBMS!) may operate
contrary to this, so you'll want to
double-check with your target DBMS
before assuming all of this still holds.
I've been a Sybase ASE, MySQL, and SQL Server DBA on-and off since for almost a decade (along with application development in C, PHP, PL/SQL, C#.NET, and Ruby). So, I have no particular axe to grind in this (sometimes) holy war.
The historical performance benefit of stored procs have generally been from the following (in no particular order):
Pre-parsed SQL
Pre-generated query execution plan
Reduced network latency
Potential cache benefits
Pre-parsed SQL -- similar benefits to compiled vs. interpreted code, except on a very micro level.
Still an advantage?
Not very noticeable at all on the modern CPU, but if you are sending a single SQL statement that is VERY large eleventy-billion times a second, the parsing overhead can add up.
Pre-generated query execution plan.
If you have many JOINs the permutations can grow quite unmanageable (modern optimizers have limits and cut-offs for performance reasons). It is not unknown for very complicated SQL to have distinct, measurable (I've seen a complicated query take 10+ seconds just to generate a plan, before we tweaked the DBMS) latencies due to the optimizer trying to figure out the "near best" execution plan. Stored procedures will, generally, store this in memory so you can avoid this overhead.
Still an advantage?
Most DBMS' (the latest editions) will cache the query plans for INDIVIDUAL SQL statements, greatly reducing the performance differential between stored procs and ad hoc SQL. There are some caveats and cases in which this isn't the case, so you'll need to test on your target DBMS.
Also, more and more DBMS allow you to provide optimizer path plans (abstract query plans) to significantly reduce optimization time (for both ad hoc and stored procedure SQL!!).
WARNING Cached query plans are not a performance panacea. Occasionally the query plan that is generated is sub-optimal.
For example, if you send SELECT *
FROM table WHERE id BETWEEN 1 AND
99999999, the DBMS may select a
full-table scan instead of an index
scan because you're grabbing every row
in the table (so sayeth the
statistics). If this is the cached
version, then you can get poor
performance when you later send
SELECT * FROM table WHERE id BETWEEN
1 AND 2. The reasoning behind this is
outside the scope of this posting, but
for further reading see:
http://www.microsoft.com/technet/prodtechnol/sql/2005/frcqupln.mspx
and
http://msdn.microsoft.com/en-us/library/ms181055.aspx
and http://www.simple-talk.com/sql/performance/execution-plan-basics/
"In summary, they determined that
supplying anything other than the
common values when a compile or
recompile was performed resulted in
the optimizer compiling and caching
the query plan for that particular
value. Yet, when that query plan was
reused for subsequent executions of
the same query for the common values
(‘M’, ‘R’, or ‘T’), it resulted in
sub-optimal performance. This
sub-optimal performance problem
existed until the query was
recompiled. At that point, based on
the #P1 parameter value supplied, the
query might or might not have a
performance problem."
Reduced network latency
A) If you are running the same SQL over and over -- and the SQL adds up to many KB of code -- replacing that with a simple "exec foobar" can really add up.
B) Stored procs can be used to move procedural code into the DBMS. This saves shuffling large amounts of data off to the client only to have it send a trickle of info back (or none at all!). Analogous to doing a JOIN in the DBMS vs. in your code (everyone's favorite WTF!)
Still an advantage?
A) Modern 1Gb (and 10Gb and up!) Ethernet really make this negligible.
B) Depends on how saturated your network is -- why shove several megabytes of data back and forth for no good reason?
Potential cache benefits
Performing server-side transforms of data can potentially be faster if you have sufficient memory on the DBMS and the data you need is in memory of the server.
Still an advantage?
Unless your app has shared memory access to DBMS data, the edge will always be to stored procs.
Of course, no discussion of Stored Procedure optimization would be complete without a discussion of parameterized and ad hoc SQL.
Parameterized / Prepared SQL
Kind of a cross between stored procedures and ad hoc SQL, they are embedded SQL statements in a host language that uses "parameters" for query values, e.g.:
SELECT .. FROM yourtable WHERE foo = ? AND bar = ?
These provide a more generalized version of a query that modern-day optimizers can use to cache (and re-use) the query execution plan, resulting in much of the performance benefit of stored procedures.
Ad Hoc SQL
Just open a console window to your DBMS and type in a SQL statement. In the past, these were the "worst" performers (on average) since the DBMS had no way of pre-optimizing the queries as in the parameterized/stored proc method.
Still a disadvantage?
Not necessarily. Most DBMS have the ability to "abstract" ad hoc SQL into parameterized versions -- thus more or less negating the difference between the two. Some do this implicitly or must be enabled with a command setting (SQL server: http://msdn.microsoft.com/en-us/library/ms175037.aspx , Oracle: http://www.praetoriate.com/oracle_tips_cursor_sharing.htm).
Lessons learned?
Moore's law continues to march on and DBMS optimizers, with every release, get more sophisticated. Sure, you can place every single silly teeny SQL statement inside a stored proc, but just know that the programmers working on optimizers are very smart and are continually looking for ways to improve performance. Eventually (if it's not here already) ad hoc SQL performance will become indistinguishable (on average!) from stored procedure performance, so any sort of massive stored procedure use ** solely for "performance reasons"** sure sounds like premature optimization to me.
Anyway, I think if you avoid the edge cases and have fairly vanilla SQL, you won't notice a difference between ad hoc and stored procedures.
Reasons for using stored procedures:
Reduce network traffic -- you have to send the SQL statement across the network. With sprocs, you can execute SQL in batches, which is also more efficient.
Caching query plan -- the first time the sproc is executed, SQL Server creates an execution plan, which is cached for reuse. This is particularly performant for small queries run frequently.
Ability to use output parameters -- if you send inline SQL that returns one row, you can only get back a recordset. With sprocs you can get them back as output parameters, which is considerably faster.
Permissions -- when you send inline SQL, you have to grant permissions on the table(s) to the user, which is granting much more access than merely granting permission to execute a sproc
Separation of logic -- remove the SQL-generating code and segregate it in the database.
Ability to edit without recompiling -- this can be controversial. You can edit the SQL in a sproc without having to recompile the application.
Find where a table is used -- with sprocs, if you want to find all SQL statements referencing a particular table, you can export the sproc code and search it. This is much easier than trying to find it in code.
Optimization -- It's easier for a DBA to optimize the SQL and tune the database when sprocs are used. It's easier to find missing indexes and such.
SQL injection attacks -- properly written inline SQL can defend against attacks, but sprocs are better for this protection.
In many cases, stored procedures are actually slower because they're more genaralized. While stored procedures can be highly tuned, in my experience there's enough development and institutional friction that they're left in place once they work, so stored procedures often tend to return a lot of columns "just in case" - because you don't want to deploy a new stored procedure every time you change your application. An OR/M, on the other hand, only requests the columns the application is using, which cuts down on network traffic, unnecessary joins, etc.
It's a debate that rages on and on (for instance, here).
It's as easy to write bad stored procedures as it is to write bad data access logic in your app.
My preference is for Stored Procs, but that's because I'm typically working with very large and complex apps in an enterprise environment where there are dedicated DBAs who are responsible for keeping the database servers running sweetly.
In other situations, I'm happy enough for data access technologies such as LINQ to take care of the optimisation.
Pure performance isn't the only consideration, though. Aspects such as security and configuration management are typically at least as important.
Edit: While Frans Bouma's article is indeed verbose, it misses the point with regard to security by a mile. The fact that it's 5 years old doesn't help its relevance, either.
There is no noticeable speed difference for stored procedures vs parameterized or prepared queries on most modern databases, because the database will also cache execution plans for those queries.
Note that a parameterized query is not the same as ad hoc sql.
The main reason imo to still favor stored procedures today has more to do with security. If you use stored procedures exclusively, you can disable INSERT, SELECT, UPDATE, DELETE, ALTER, DROP, and CREATE etc permissions for your application's user, only leaving it with EXECUTE.
This provides a little extra protection against 2nd order sql injection. Parameterized queries only protect against 1st order injection.
Obviously, actual performance ought to be measured in individual cases, not assumed. But even in cases where performance is hampered by a stored procedure, there are good reasons to use them:
Application developers aren't always the best SQL coders. Stored procedures hides SQL from the application.
Stored procedures automatically use bind variables. Application developers often avoid bind variables because they seem like unneeded code and show little benefit in small test systems. Later on, the failure to use bind variables can throttle RDBMS performance.
Stored procedures create a layer of indirection that might be useful later on. It's possible to change implementation details (including table structure) on the database side without touching application code.
The exercise of creating stored procedures can be useful for documenting all database interactions for a system. And it's easier to update the documentation when things change.
That said, I usually stick raw SQL in my applications so that I can control it myself. It depends on your development team and philosophy.
The one topic that no one has yet mentioned as a benefit of stored procedures is security. If you build the application exclusively with data access via stored procedures, you can lockdown the database so the ONLY access is via those stored procedures. Therefor, even if someone gets a database ID and password, they will be limited in what they can see or do against that database.
In 2007 I was on a project, where we used MS SQL Server via an ORM. We had 2 big, growing tables which took up to 7-8 seconds of load time on the SQL Server. After making 2 large, stored SQL procedures, and optimizing them from the query planner, each DB load time got down to less than 20 milliseconds, so clearly there are still efficiency reasons to use stored SQL procedures.
Having said that, we found out that the most important benefit of stored procedures was the added maintaince-ease, security, data-integrity, and decoupling business-logic from the middleware-logic, benefitting all middleware-logic from reuse of the 2 procedures.
Our ORM vendor made the usual claim that firing off many small SQL queries were going to be more efficient than fetching large, joined data sets. Our experience (to our surprise) showed something else.
This may of course vary between machines, networks, operating systems, SQL servers, application frameworks, ORM frameworks, and language implementations, so measure any benefit, you THINK you may get from doing something else.
It wasn't until we benchmarked that we discovered the problem was between the ORM and the database taking all the load.
I prefer to use SP's when it makes sense to use them. In SQL Server anyway there is no performance advantage to SP's over a parametrized query.
However, at my current job my boss mentioned that we are forced to use SP's because our customer's require them. They feel that they are more secure. I have not been here long enough to see if we are implementing role based security but I have a feeling we do.
So the customer's feelings trump all other arguments in this case.
Read Frans Bouma's excellent post (if a bit biased) on that.
To me one advantage of stored procedures is to be host language agnostic: you can switch from a C, Python, PHP or whatever application to another programming language without rewriting your code. In addition, some features like bulk operations improve really performance and are not easily available (not at all?) in host languages.
I don't know that they are faster. I like using ORM for data access (to not re-invent the wheel) but I realize that's not always a viable option.
Frans Bouma has a good article on this subject : http://weblogs.asp.net/fbouma/archive/2003/11/18/38178.aspx
All I can speak to is SQL server. In that platform, stored procedures are lovely because the server stores the execution plan, which in most cases speeds up performance a good bit. I say "in most cases", because if the SP has widely varying paths of execution you might get suboptimal performance. However, even in those cases, some enlightened refactoring of the SPs can speed things up.
Using stored procedures for CRUD operations is probably overkill, but it will depend on the tools be used and your own preferences (or requirements). I prefer inline SQL, but I make sure to use parameterized queries to prevent SQL injection attacks. I keep a print out of this xkcd comic as a reminder of what can go wrong if you are not careful.
Stored procedures can have real performance benefits when you are working with multiple sets of data to return a single set of data. It's usually more efficient to process sets of data in the stored procedure than sending them over the wire to be processed at the client end.
Realising this is a bit off-topic to the question, but if you are using a lot of stored procedures, make sure there is a consistent way to put them under some sort of source control (e.g., subversion or git) and be able to migrate updates from your development system to the test system to the production system.
When this is done by hand, with no way to easily audit what code is where, this quickly becomes a nightmare.
Stored procs are great for cases where the SQL code is run frequently because the database stores it tokenized in memory. If you repeatedly ran the same code outside of a stored proc, you will likey incur a performance hit from the database reparsing the same code over and over.
I typically frequently called code as a stored proc or as a SqlCommand (.NET) object and execute as many times as needed.
Yes, they are faster most of time. SQL composition is a huge performance tuning area too. If I am doing a back office type app I may skip them but anything production facing I use them for sure for all the reasons others spoke too...namely security.
IMHO...
Restricting "C_UD" operations to stored procedures can keep the data integrity logic in one place. This can also be done by restricting"C_UD" operations to a single middle ware layer.
Read operations can be provided to the application so they can join only the tables / columns they need.
Stored procedures can also be used instead of parameterized queries (or ad-hoc queries) for some other advantages too :
If you need to correct something (a sort order etc.) you don't need to recompile your app
You could deny access to all tables for that user account, grant access only to stored procedures and route all access through stored procedures. This way you can have custom validation of all input much more flexible than table constraints.
Reduced network traffic -- SP are generally worse then Dynamic SQL. Because people don't create a new SP for every select, if you need just one column you are told use the SP that has the columns they need and ignore the rest. Get an extra column and any less network usage you had just went away. Also you tend to have a lot of client filtering when SP are used.
caching -- MS-SQL does not treat them any differently, not since MS-SQL 2000 may of been 7 but I don't remember.
permissions -- Not a problem since almost everything I do is web or have some middle application tier that does all the database access. The only software I work with that have direct client to database access are 3rd party products that are designed for users to have direct access and are based around giving users permissions. And yes MS-SQL permission security model SUCKS!!! (have not spent time on 2008 yet) As a final part to this would like to see a survey of how many people are still doing direct client/server programming vs web and middle application server programming; and if they are doing large projects why no ORM.
Separation -- people would question why you are putting business logic outside of middle tier. Also if you are looking to separate data handling code there are ways of doing that without putting it in the database.
Ability to edit -- What you have no testing and version control you have to worry about? Also only a problem with client/server, in the web world not problem.
Find the table -- Only if you can identify the SP that use it, will stick with the tools of the version control system, agent ransack or visual studio to find.
Optimization -- Your DBA should be using the tools of the database to find the queries that need optimization. Database can tell the DBA what statements are talking up the most time and resources and they can fix from there. For complex SQL statements the programmers should be told to talk to the DBA if simple selects don't worry about it.
SQL injection attacks -- SP offer no better protection. The only thing they get the nod is that most of them teach using parameters vs dynamic SQL most examples ignore parameters.
Related
I am trying to find the pros and cons of using stored procedures instead of SQL queries from an external connection, but I am unable to find any direct comparison.
What is the benefit of using stored procedures instead of SQL queries from an external connection?
Is there any execution speed differences between them for small volume and big volume outputs?
Is there any benefits for the database management as well?
What is the benefit of using stored procedures instead of SQL queries from an external connection?
Stored Procedures can be complex. Very complex. They can do things
that a single SQL query cannot do. (Execute Block aside.)
They have their own set of grants so they can do things that current user
cannot do at all.
Firebird optimizer is not that bad but obviously complex queries require more time for optimization and the result still may be suboptimal. Using imperative language the programmer can split complex query to set of simpler ones making Data Access Paths more predictable.
Is there any execution speed differences between them for small volume
and big volume outputs?
No.
Is there any benefits for the database management as well?
It depends on what you call "database management" and what benefits you have on mind. Most likely - no.
What is the benefit of using stored procedures instead of SQL queries from an external connection?
One benefit, in terms of execution, is stored procedures store their query plan whereas dynamic sql query plans will not be stored and must be calculated each time the query is executed.
Is there any execution speed differences between them for small volume and big volume outputs?
Once the query plan is calculated, no, there is no speed difference.
Is there any benefits for the database management as well?
This is very subjective! In the past I worked at a place where ALL database access went through stored procedures so that they could lock down access to just the SPs. Other places I've worked didn't use stored procs at all because they are generally outside source control and problematic for developers who aren't SQL gurus. Also, business logic spread across multiple systems can become a real problem.
Obvious security benefits aside, are there significant performance boosts yielded from modifying involved queries to stored procedures in a SAP HANA database?
If so, are there metrics I can use to gauge perceived benefits?
SQL Server 2005 onward, all SQL statements, irrespective of whether it’s a SQL coming from inline code or stored procedure or from anywhere else, they are compiled and cached. So, stored procedures won't give you performance boosts. They do give you better abstraction, security and ease of maintenance.
Read more about it here.
As for SAP HANA, I tried comparing it with Microsoft SQL Server(This article sheds some light on it.) and I cannot definitively say that it does compile and cache inline queries but it most probably should if you're using a recent version.
Read the other solution before reading this one.
Essentially, I ran some tests and swapped out my project's dynamic queries to stored procedures and saw massive performance bumps.
I was reading the SAP HANA Best Practices reference and came across this passage under Performance:
Executing dynamic SQL is slow because compile time checks and query
optimization must be done for every invocation of the procedure.
Another related problem is security because constructing SQL
statements without proper checks of the variables used may harm
security.
I am confused as when to go for stored procedures rather than embedded SQL in the code
When I googled out, I found out these points
They allow modular programming.
They can reduce network traffic.
They can be used as a security mechanism.
Is please tell me how does network traffic is related to it ??
Another main advantage for SP: you can change them (to bugfix, to extend) without changing your application code .... yet another layer of separation, which can be beneficial.
And also: security. If you use SProcs for everything, all your callers need in terms of permissions on your database is EXECUTE permissions on those SProcs - they don't need direct read/write access to your tables.
It can reduce network traffic in the sense that you send a single command to a stored proc rather than line after line of SQL statements.
Another benefit is the performance of queries themselves is better than embedded SQL due to being pre-compiled.
they can reduce network traffic by only returning the required data to the client.
Or to turn it around; a design/coding practice that can waste network traffic is to select a set of data from the DB, return it to the client and do processing there on some of the dataset. Obviously if you are working on some of the data set it would be better from a traffic perspective to not send to the client the data that is not being processed
It will reduce network traffic in the event that your database server and your server/client running the embedded-SQL are seperate.
It reduces network traffic because stored procedures are handled on the Database Server; for embedded-SQL running on a seperate machine, the database accesses must be handled over the network, thus increasing traffic.
If your embedded-SQL and database are on the same machine it will have no effect on network traffic. An example is a LAMP stack on one machine.
I would firstly question going stored procs at all...
Unlike actual programming language code, they:
not portable (every db has its own version of PL/SQL. Sometimes different version of the same database are incompatible - I've seen it)
not easily testable (not supported by industry standard unit testing frameworks)
not easily updatable/releasable (you need to drop/create them - ie modify the db to change)
do not have library support (why write code when someone else has)
are not easily integratable with other technologies (try calling a web service from them)
are typically about as primitive as Fortran and thus are inelegant and laborious to get useful coding done
do not offer debugging/tracing/message-logging etc (some dbs may support this - I haven't seen it though)
etc.
If you have a very database-specific action (eg an in-transaction action to maintain db integrity), or keep your procedures very atomic and simple, perhaps you might consider them.
Caution is advised when specifying "high performance" up front. It often leads to poor choices at the expense of good design and it will bite you much sooner than you think.
Use stored procedures at your own peril (from someone who's been there and never wants to go back). My recommendation is to avoid them like the plague.
It depends.
Are you writing an application that should be run with several databases?
What kind of data operation does your application require? Simple and thin data manipulation?
I suppose this isn't your case, because you tagged your question as 'plsql', 'SQL', 'Store procedures'.
The concept of embedded SQL in Pl/Sql is the follow:
Embedded SQL statements incorporate DDL, DML, and transaction control
statemen within a procedural language program. They are used with the
Oracle precompilers. Embedded SQL is one approach to incorporating
SQL in your procedural language applications. Another approach is to use a procedural API such as Open Database Connectivity (ODBC) or Java Database Connectivity (JDBC).
In that case there are many and important reasons.
The most important are:
The short answer could be that it's easier to write highly efficient code to access large amount of data in "Oracle database" in PL/SQL store procedures than in any others language. This because it's strictly integrated in the Oracle database.
Read the manual before: Advantages of Pl/sql stored procedures
Improved performance
Network traffic(small amount of information sent over a network). With a single call of a store procedure, a large amount of data manipulation can be done on the db server, without to go back and forth with individual sql statements and without to send over the network the data needed for the intermediate state of data manipulation itself. This concept is strongly related to applications in which intensive and highly efficient data operations/manipulation are required. It's not only a matter of subset of data to be used and sent to the client, but a matter of a quality of data in the intermediate data processing state in order to achieve the final data! If the result needed involve many sql steps and statements to be done, the advantage is evident.
No compilation is required at compilation time
More probability that the code is in Shared pool of the SGA.
Memory allocation of the code
Security with definer's rights procedures
Inherited privileges and schema context with invoker's rights procedures
Specific characteristics of the pl/sql and Oracle database, just to write a few of these:
Advantages doing independent units of Work with Autonomous Transactions`
DML, transaction management and exception handler inside the db
Calling sql function **inside SQL
Packaged cursor
Streaming table function
. Table functions with the CURSOR expression, enable you to stream data through multiple transformations, in a single SQL statement.
Deterministic function
Complex dynamic sql manipulation using dbms_SQL API in conjunction with native dynamic SQL(famous fourth method).
All modular reasons(you already mentioned):
1 Encapsulating Calculations
2 To simplify sub queries used inside outer sql
3 Combining scalar and aggregate values inside the same sql
4 Write once, using many.
Etc...
Stored procedures may be required to get the performance you need from your application code. The biggest problem with embedded SQL is that all of the business logic typically goes into the application code. This can be hugely inefficient. For example, developers will start doing client side joins: They call the database to get a set of ID values for other table records then query each of those tables one record at a time to retrieve the data they require. What can be done with one roundtrip to the database with a stored procedure may now take hundreds or thousands of roundtrips to the database with embedded sql. Each roundtrip to the database takes a lot of time not to mention that each query will have to be compiled tremendously increasing the load on the database server.
If your application is a low volume application with few users this can work. High volume applications with lots of users can quickly overload even large database servers and cause severe performance problems, even to the point where the application stops working.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Assuming you can't use LINQ for whatever reason, is it a better practice to place your queries in stored procedures, or is it just as good a practice to execute ad hoc queries against the database (say, SQL Server for argument's sake)?
In my experience writing mostly WinForms Client/Server apps these are the simple conclusions I've come to:
Use Stored Procedures:
For any complex data work. If you're going to be doing something truly requiring a cursor or temp tables it's usually fastest to do it within SQL Server.
When you need to lock down access to the data. If you don't give table access to users (or role or whatever) you can be sure that the only way to interact with the data is through the SP's you create.
Use ad-hoc queries:
For CRUD when you don't need to restrict data access (or are doing so in another manner).
For simple searches. Creating SP's for a bunch of search criteria is a pain and difficult to maintain. If you can generate a reasonably fast search query use that.
In most of my applications I've used both SP's and ad-hoc sql, though I find I'm using SP's less and less as they end up being code just like C#, only harder to version control, test, and maintain. I would recommend using ad-hoc sql unless you can find a specific reason not to.
I can't speak to anything other than SQL Server, but the performance argument is not significantly valid there unless you're on 6.5 or earlier. SQL Server has been caching ad-hoc execution plans for roughly a decade now.
I think this is a basic conflict between people who must maintain the database and people who develop the user interfaces.
As a data person, I would not consider working with a database that is accessed through adhoc queries because they are difficult to effectively tune or manage. How can I know what affect a change to the schema will have? Additionally, I do not think users should ever be granted direct access to the database tables for security reasons (and I do not just mean SQL injection attacks, but also because it is a basic internal control to not allow direct rights and require all users to use only the procs designed for the app. This is to prevent possible fraud. Any financial system which allows direct insert, update or delete rights to tables is has a huge risk for fraud. This is a bad thing.).
Databases are not object-oriented and code which seems good from an object-oriented perspective is can be extremely bad from a database perspective.
Our developers tell us they are glad that all our databse access is through procs becasue it makes it much faster to fix a data-centered bug and then simply run the proc on the production environment rather than create a new branch of the code and recompile and reload to production. We require all our procs to be in subversion, so source control is not an issue at all. If it isn't in Subversion, it will periodically get dropped by the dbas, so there is no resistance to using Source Control.
Stored procedures represent a software contract that encapsulates the actions taken against the database. The code in the procedures, and even the schema of the database itself can be changed without affecting compiled, deployed code, just so the inputs and outputs of the procedure remain the same.
By embedding queries in your application, you are tightly coupling yourself to your data model.
For the same reason, it is also not good practice to simply create stored procedures that are just CRUD queries against every table in your database, since this is still tight coupling. The procedures should instead be bulky, coarse grained operations.
From a security perspective, it is good practice to disallow db_datareader and db_datawriter from your application and only allow access to stored procedures.
Stored procedures are definitely the way to go...they are compiled, have execution plan before hand and you could do rights management on them.
I do not understand this whole source control issue on stored procedure. You definitely can source control them, if only you are a little disciplined.
Always start with a .sql file that is the source of your stored procedure. Put it in version control once you have written your code. The next time you want to edit your stored procedure get it from your source control than your database. If you follow this, you will have as good source control as your code.
I would like to quote Tom Kyte from Oracle here...Here's his rule on where to write code...though a bit unrelated but good to know I guess.
Start with stored procedures in PL/SQL...
If you think something can't be done using stored procedure in PL/SQL, use Java stored procedure.
If you think something can't be done using Java Stored procedure, consider Pro*c.
If you think you can't achieve something using Pro*C, you might want to rethink what you need to get done.
My answer from a different post:
Stored Procedures are MORE maintainable because:
You don't have to recompile your C# app whenever you want to change some SQL
You end up reusing SQL code.
Code repetition is the worst thing you can do when you're trying to build a maintainable application!
What happens when you find a logic error that needs to be corrected in multiple places? You're more apt to forget to change that last spot where you copy & pasted your code.
In my opinion, the performance & security gains are an added plus. You can still write insecure/inefficient SQL stored procedures.
Easier to port to another DB - no procs to port
It's not very hard to script out all your stored procedures for creation in another DB. In fact - it's easier than exporting your tables because there are no primary/foreign keys to worry about.
In our application, there is a layer of code that provides the content of the query (and is sometimes a call to a stored procedure). This allows us to:
easily have all the queries under version control
to make what ever changes are required to each query for different database servers
eliminates repetition of the same query code through out our code
Access control is implemented in the middle layer, rather than in the database, so we don't need stored procedures there. This is in some ways a middle road between ad hoc queries and stored procs.
There are persuasive arguments for both - stored procedures are all located in a central repository, but are (potentially) hard to migrate and ad hoc queries are easier to debug as they are with your code, but they can also be harder to find in the code.
The argument that stored procedures are more efficient doesn't hold water anymore.
link text
Doing a google for Stored Procedure vs Dynamic Query will show decent arguments either way and probably best for you to make your own decision...
Store procedures should be used as much as possible, if your writing SQL into code your already setting yourself up for headaches in the futures. It takes about the same time to write a SPROC as it does to write it in code.
Consider a query that runs great under a medium load but once it goes into fulltime production your badly optimized query hammers the system and brings it to a crawl. In most SQL servers you are not the only application/service that is using it. Your application has now brought a bunch of angry people at your door.
If you have your queries in SPROCs you also allow your friendly DBA to manage and optimize with out recompiling or breaking your app. Remember DBA's are experts in this field, they know what to do and not do. It makes sense to utilise their greater knowledge!
EDIT: someone said that recompile is a lazy excuse! yeah lets see how lazy you feel when you have to recompile and deploy your app to 1000's of desktops, all because the DBA has told you that your ad-hoc Query is eating up too much Server time!
someone said that recompile is a lazy excuse! yeah lets see how lazy you feel when you have to recompile and deploy your app to 1000's of desktops, all because the DBA has told you that your ad-hoc Query is eating up too much Server time!
is it good system architecture if you let connect 1000 desktops directly to database?
Some things to think about here: Who Needs Stored Procedures, Anyways?
Clearly it's a matter of your own needs and preferences, but one very important thing to think about when using ad hoc queries in a public-facing environment is security. Always parameterize them and watch out for the typical vulnerabilities like SQL-injection attacks.
Stored Procedures are great because they can be changed without a recompile. I would try to use them as often as possible.
I only use ad-hoc for queries that are dynamically generated based on user input.
Procs for the reasons mentioned by others and also it is easier to tune a proc with profiler or parts of a proc. This way you don't have to tell someone to run his app to find out what is being sent to SQL server
If you do use ad-hoc queries make sure that they are parameterized
Parametized SQL or SPROC...doesn't matter from a performance stand point...you can query optimize either one.
For me the last remaining benefit of a SPROC is that I can eliminate a lot SQL rights management by only granting my login rights to execute sprocs...if you use Parametized SQL the login withing your connection string has a lot more rights (writing ANY kind of select statement on one of the tables they have access too for example).
I still prefer Parametized SQL though...
I haven't found any compelling argument for using ad-hoc queries. Especially those mixed up with your C#/Java/PHP code.
The sproc performance argument is moot - the 3 top RDBMs use query plan caching and have been for awhile. Its been documented... Or is 1995 still?
However, embedding SQL in your app is a terrible design too - code maintenance seems to be a missing concept for many.
If an application can start from scratch with an ORM (greenfield applications are far and few between!) its a great choice as your class model drives your DB model - and saves LOTS of time.
If an ORM framework is not available we have taken a hybrid of approach of creating an SQL resource XML file to look up SQL strings as we need them (they are then cached by the resource framework). If the SQL needs any minor manipulation its done in code - if major SQL string manipulation is needed we rethink the approach.
This hybrid approach lends to easy management by the developers (maybe we are the minority as my team is bright enough to read a query plan) and deployment is a simple checkout from SVN. Also, it makes switching RDBMs easier - just swap out the SQL resource file (not as easy as an ORM tool of course, but connecting to legacy systems or non-supported database this works)
Depends what your goal is. If you want to retrieve a list of items and it happens once during your application's entire run for example, it's probably not worth the effort of using a stored procedure. On the other hand, a query that runs repeatedly and takes a (relatively) long time to execute is an excellent candidate for database storage, since the performance will be better.
If your application lives almost entirely within the database, stored procedures are a no-brainer. If you're writing a desktop application to which the database is only tangentially important, ad-hoc queries may be a better option, as it keeps all of your code in one place.
#Terrapin: I think your assertion that the fact that you don't have to recompile your app to make modifications makes stored procedures a better option is a non-starter. There may be reasons to choose stored procedures over ad-hoc queries, but in the absence of anything else compelling, the compile issue seems like laziness rather than a real reason.
My experience is that 90% of queries and/or stored procedures should not be written at all (at least by hand).
Data access should be generated somehow automaticly. You can decide if you'd like to staticly generate procedures in compile time or dynamically at run time but when you want add column to the table (property to the object) you should modify only one file.
I prefer keeping all data access logic in the program code, in which the data access layer executes straight SQL queries. On the other hand, data management logic I put in the database in the form of triggers, stored procedures, custom functions and whatnot. An example of something I deem worthy of database-ifying is data generation - assume our customer has a FirstName and a LastName. Now, the user interface needs a DisplayName, which is derived from some nontrivial logic. For this generation, I create a stored procedure which is then executed by a trigger whenever the row (or other source data) is updated.
There appears to be this somewhat common misunderstanding that the data access layer IS the database and everything about data and data access goes in there "just because". This is simply wrong but I see a lot of designs which derive from this idea. Perhaps this is a local phenomonon, though.
I may just be turned off the idea of SPs after seeing so many badly designed ones. For example, one project I participated in used a set of CRUD stored procedures for every table and every possible query they encountered. In doing so they simply added another completely pointless layer. It is painful to even think about such things.
These days I hardly ever use stored procedures. I only use them for complicated sql queries that can't easily be done in code.
One of the main reasons is because stored procedures do not work as well with OR mappers.
These days I think you need a very good reason to write a business application / information system that does not use some sort of OR mapper.
Stored procedure work as block of code so in place of adhoc query it work fast.
Another thing is stored procedure give recompile option which the best part of
SQL you just use this for stored procedures nothing like this in adhoc query.
Some result in query and stored procedure are different that's my personal exp.
Use cast and covert function for check this.
Must use stored procedure for big projects to improve the performance.
I had 420 procedures in my project and it's work fine for me. I work for last 3 years on this project.
So use only procedures for any transaction.
is it good system architecture if you
let connect 1000 desktops directly to
database?
No it's obviously not, it's maybe a poor example but I think the point I was trying to make is clear, your DBA looks after your database infrastructure this is were their expertise is, stuffing SQL in code locks the door to them and their expertise.
One of my co-workers claims that even though the execution path is cached, there is no way parameterized SQL generated from an ORM is as quick as a stored procedure. Any help with this stubborn developer?
I would start by reading this article:
http://decipherinfosys.wordpress.com/2007/03/27/using-stored-procedures-vs-dynamic-sql-generated-by-orm/
Here is a speed test between the two:
http://www.blackwasp.co.uk/SpeedTestSqlSproc.aspx
Round 1 - You can start a profiler trace and compare the execution times.
For most people, the best way to convince them is to "show them the proof." In this case, I would create a couple basic test cases to retrieve the same set of data, and then time how long it takes using stored procedures versus NHibernate. Once you have the results, hand it over to them and most skeptical people should yield to the evidence.
I would only add a couple things to Rob's answer:
First, Make sure the amount of data involved in the test cases is similiar to production values. In other words if your queries are normally against tables with hundreds of thousands or rows, then create such a test environment.
Second, make everything else equal except for the use of an nHibernate generated query and a s'proc call. Hopefully you can execute the test by simply swapping out a provider.
Finally, realize that there is usually a lot more at stake than just stored procedures vs. ORM. With that in mind the test should look at all of the factors: execution time, memory consumption, scalability, debugging ability, etc.
The problem here is that you've accepted the burden of proof. You're unlikely to change someone's mind like that. Like it or not, people--even programmers-- are just too emotional to be easily swayed by logic. You need to put the burden of proof back on him- get him to convince you otherwise- and that will force him to do the research and discover the answer for himself.
A better argument to use stored procedures is security. If you use only stored procedures, with no dynamic sql, you can disable SELECT, INSERT, UPDATE, DELETE, ALTER, and CREATE permissions for the application database user. This will protect you against most 2nd order SQL Injection, whereas parameterized queries are only effective against first order injection.
Measure it, but in a non-micro-benchmark, i.e. something that represents real operations in your system. Even if there would be a tiny performance benefit for a stored procedure it will be insignificant against the other costs your code is incurring: actually retrieving data, converting it, displaying it, etc. Not to mention that using stored procedures amounts to spreading your logic out over your app and your database with no significant version control, unit tests or refactoring support in the latter.
Benchmark it yourself. Write a testbed class that executes a sampled stored procedure a few hundred times, and run the NHibernate code the same amount of times. Compare the average and median execution time of each method.
It is just as fast if the query is the same each time. Sql Server 2005 caches query plans at the level of each statement in a batch, regardless of where the SQL comes from.
The long-term difference might be that stored procedures are many, many times easier for a DBA to manage and tune, whereas hundreds of different queries that have to be gleaned from profiler traces are a nightmare.
I've had this argument many times over.
Almost always I end up grabbing a really good dba, and running a proc and a piece of code with the profiler running, and get the dba to show that the results are so close its negligible.
Measure it.
Really, any discussion on this topic is probably futile until you've measured it.
He may be correct for the specific use case he is thinking of. A stored procedure will probably execute faster for some complex set of SQL, that can be arbitrarily tuned. However, something you get from things like hibernate is caching. This may prove much faster for the lifetime of your actual application.
The additional layer of abstraction will cause it to be slower than a pure call to a sproc. Just by the fact that you have additional allocations on the managed heap, and additional pushes and pops off the callstack, the truth of the matter is that it is more efficient to call a sproc over having an ORM build the query, regardless how good the ORM is.
How slow, if its even measurable, is debatable. This is also helped by the fact that most ORM's have a caching mechanism to avoid doing the query at all.
Even if the stored procedure is 10% faster (it probably isn't), you may want to ask yourself how much it really matters. What really matters in the end, is how easy it is to write and maintain code for your system. If you are coding a web app, and your pages all return in 0.25 seconds, then the extra time saved by using stored procedures is negligible. However, there can be many added advantages of using an ORM like NHibernate, which would be extremely hard to duplicate using only stored procedures.