I am trying to figure out what is the 'best' (read: "your preferred method") way to generate datasets for SQL Server Reporting Services Reports (either 2005/2008):
In-report queries
Stored procedures
Views
But more than just choosing one of the above, why would you use that particular method? Also, please include your perspective (Developer/DBA/etc).
Thanks.
Stored procedures. I tend to write stored procedures for both parameters and the report data. By using stored procedures for parameter datasets they can be easily shared between reports. For report data I like to make certain that I draw a clear line between the data within a report and the formatting that comes out. By keeping that line it has been easier in my experience to test and promote reports to production.
Also, I find stored procedures a little easier to manage and troubleshoot than a view or in-report queries.
You should not bother with in-report queries, they are pretty much there to play with and practice. But you can hardly get a very good report writing a query directly in RS. Why..well for one thing a lot of sprocs can be reused not only for the application side but also the reporting side. Your application may have various stored procedures used to fill drop down lists / combo boxes that cascade (one relates on another). You may need this functionality for your reports as well. With stored procedures you could issue a call to the sproc from your report or your application. When the sproc changes (if it ever has to), you're ok because the updates you made to the application stored procedure also update your report's stored procedures.
My vote is definately for stored procedures.
I have been working with MS Reporting Services for about one full year now. I found that the best way to generate reports with this system is to run queries from your data access layer, via stored procedures.
If find that if you do it this way, you have all your returned datasets in one spot. It makes it easier to manage. All your database output is controlled from the same location.
Off topic but I would also recommend that you generate your RDLC files in memory. We have about 100 different report types. Instead of managing a bunch of RDLC files, we manage a ReportEngine class. The ReportEngine class basically generates a bunch of different report types. This is quite advanced, but the results are worth it. Source code to generate a RDLC file with a table: C# or VB.NET.
Related
I manage an application built on Access with some VBA code that takes its data from:
Inputs by the user through Access forms
Tables in Sybase (that are linked through Access)
Local tables in Access
The application is used to make some financial calculations. Our calculations need a lot of conditions and are mostly some complex calculations (fractions, multiplications...)
My question is : is VBA faster than Sybase to do the calculations ?
(Please notice than when we do our calculations it takes our 3 kinds of data sources)
I was thinking about migrate all of the calculations to Sybase as some stored procedures and call it from the VBA code with parameters, and wait from an output from Sybase.
PS: another reason why I am asking that is because we consider as a long term project to migrate our Access application to a thin client(prob web-based), and if all the calculations are already on the server/database side it could maybe be easier? What do you think?
Thanks a lot for your help
IMO, I would pass the form based variables (user entry) as parameters into a stored procedure, then fetch the other variables as needed from tables within the SP. This avoids sending too much data to the client as the form is opening. This abstracts the logic from VBA code (or any specific front-end language), making it easier to eventually move to a thin layer. You can also recompile independent stored procedures as needed, instead of deploying another instance of your code (much harder usually).
If there are a lot of parameters coming from the form or local tables, consider passing them in as a structured data type within Sybase. The procedure cache within Sybase is extremely powerful and after initial compilation as fast as any other procedural language.
It depends on the calculations. Sybase will be better at doing calculations that involve grouping data, but complex calculations like fractions, etc... would be faster to do in code. Also it's just better practice to separate out business logic from data.
I'm writing in VB.net 4.0 and using SQLExpress 2008 R2. In a DataGridView, I would like to display (no edits) data coming from multiple tables in my database. A second Grid (different data, still multiple tables) will need to allow editing and saving of data.
I understand creating a View in the database and using that as a source for the DataGridView. I also assume that there are ways to query and create a data source for the Grid totally within my VB program. Would someone explain the consequences and implications of the different approaches?
I recommend you look into LINQ to SQL for your data calls. It would be prudent for you to create a data access layer class that performs all of your data calls separate from any UI architecture you have. This allows you to maintain the data calls within the code but separated from any display logic.
Putting your queries into SQL server as views or stored procedures simplifies some of your immediate code within the application, but over time stored procedures and views become harder to maintain. Unless you have massive data load and optimization requires stored procedures, I'd recommend you investigate the usage of LINQ for making quick, atomic data calls.
I would go with the good old Stored Procedure. Write one SP, getting data for each gridviews.
I have read these very good questions on SO about SQL stored procedures:
When should you use stored procedures? and
Are Stored Procedures more efficient, in general, than inline statements on modern RDBMS’s?
I am a beginner on integrating .NET/SQL though I have used basic SQL functionality for more than a decade in other environments. It's time to advance with regards to organization and deployment. I am using .NET C# 3.5, Visual Studio 2008 and SQL Server 2008; though this question can be regarded as language- and database- agnostic, meaning that it could easily apply to other environments that use stored procedures and a relational database.
Given that I have an application with inline SQL queries, and I am interested in converting to stored procedures for organizational and performance purposes, what are your recommendations for doing so?
Here are some additional questions in my mind related to this subject that may help shape the answers:
Should I create the stored procedures in SQL using SQL Management Studio and simply re-create the database when it is installed for a client?
Am I better off creating all of the stored procedures in my application, inside of a database initialization method?
It seems logical to assume that creating stored procedures must follow the creation of tables in a new installation. My database initialization method creates new tables and inserts some default data. My plan is to create stored procedures following that step, but I am beginning to think there might be a better way to set up a database from scratch (such as in the installer of the program). Thoughts on this are appreciated.
I have a variety of queries throughout the application. Some queries are incredibly simple (SELECT id FROM table) and others are extremely long and complex, performing several joins and accepting approximately 80 parameters. Should I replace all queries with stored procedures, or only those that might benefit from doing so?
Finally, as this topic obviously requires some research and education, can you recommend an article, book, or tutorial that covers the nuances of using stored procedures instead of direct statements?
Consider skipping stored procedures for an ORM. Consider using:
LINQ To SQL
Entity Framework
SubSonic
You'll be writing less boiler plate ListCustomer and GetCustomerByID code when you could be adding more value to your application.
IMO, there isn't any real compelling reason to choose stored procedures with the modern toolset that we have in the Microsoft stack.
The move away from inline SQL statements is good, and an ORM will help parameterize your queries for you. You don't have to think about it.
You don't have to mess with ADO.NET objects at all. Code your data access in an object oriented fashion.
There are several compelling reasons to avoid giving table access to very many logins, including application logins, and these drive the use of stored procedures. (I generally do not ascribe any importance to using SPs for performance reasons - SQL Server caches even adhoc query plans).
Stored procedures give your database much more capability in defining its interface boundaries. In many cases, views are not sufficient to control the interface.
Any framework built solely on tables and views (note that many frameworks can build on top of SP results) is going to be severely limited in letting your database protect itself and control itself.
As a simple example, neither tables nor views can be parameterized. If you have a very large table or view and you want to enforce all users to specify a certain set of filter criteria (for instance a snapshot date or effective date), there is no way to enforce this at the database call interface. The framework can submit queries for all time. If the table/view is not exposed, and the only interface is through an SP or table-valued UDF, then the parameters to that SP or UDF MUST be provided, thus satisfying your database's need to ensure that it is used properly.
Other examples, where views may or may not work, include hiding privacy information for certain users, hiding internal keys, hiding internal implementation details, and enforcing complex security rules.
As far as scripting the creation of your database schema, including objects in the correct dependency order, there are several tools to do this (and generate change scripts), including Red Gate SQL Compare and Apex SQLScript.
Use stored procedures if you really have a performance requeriment, particularly if one stored procedures will be called thousands of times per minute. This way sql engine avoids severals steps for processing the statement. GPS Tracking systems is an example. Say you have 10000 vehicles which reports a 3 positions per minute. In this case stored procedures helps performance.
If not, instead of CRUD sql statements, use ORM features.
You missed one:
When is it better to write "ad hoc sql" vs stored procedures
My answer is: don't use stored procedures at all.
I'm pretty new to SQL world. Here are my questions:
What are the benefits of stored procedured over normal SQL statement in applications?
Does stored procedure help eliminates SQL injection?
In Microsoft SQL Server it is called stored procedure. How about in Oracle, MySQL, DB2, etc.?
Thanks for your explanation.
Stored procedures only directly prevent SQL injection if you call them in a paramerized way. If you still have a string in your app with the procedure name and concatenate parameters from user input to that string in your code you'll have still have trouble.
However, when used exclusively, stored procedures let you add some additional protection by making it possible for you to disable permissions to everything but the EXEC command. Aside from this, parameterized queries/prepared statements are normally cached by the server, and so are just like a stored procedure in nearly every respect.
In spite of this, stored procedures have two big advantages for larger enterprises:
They allow you to define an application interface for the database, so that the system can be shared between multiple applications without having to duplicate logic in those applications.
They move the sql code to the db, where you can easily have an experienced DBA tune, update, and otherwise maintain it, rather than application developers who often don't know exactly what they're doing with database code.
Of course, these advantages aren't without cost:
It's harder to track changes in source control
The database code is far separated from the code that uses it
Developer tools for managing many stored procedures are less than ideal (if you've ever open the stored procedures folder in management studio to find 200 procedures for a database, you know what I'm talking about here).
Some of the benefits that I consider when using stored procedures
Stored procedures encapsulate query code at the server, rather than inside your application. This allows you to make changes to queries without having to recompile your application.
Stored procedures can be used for more well defined application security. You can Deny all rights on the base tables, grant execute only on the procs. This gives you a much smaller security footprint to manage.
Stored procedures are compiled code. With the latest versions of MSSQL the server does a better job of storing execution plans - so this isn't as big of an issue as it used to be, but still something to consider
Stored procedures eliminate SQL injection risk ONLY when used correctly. Make sure to use the parameters the right way inside the stored proc - stored procs that are just executing concatenated dynamic SQL inside them aren't doing anyone any good.
For the most part yes, SQL injection is far less likely with a stored procedure. Though there are times when you want to pass a stored procedure some data that requires you to use dynamic SQL inside the stored procedure and then you're right back where you started. In this sense I don't see any advantage to them over using parameterized queries in programming languages that support them.
Personally I hate stored procedures. Having code in two disjointed places is a pain in the ass and it makes deploys that much more complicated. I don't advocate littering your code with SQL statements either however as this leads to it's own set of headaches.
I recommend a DAL layer implemented one of two ways.
My favorite, use an object
relational management system (ORM).
I've been working with nHibernate
and I absolutely love it. The
learning curve in steep but
definitely worth the payoff in my
opinion.
Some kind of mechanism for keeping
all your SQL code in one place.
Either some sort of query library
you select from or a really
structured set of classes that
design the SQL for you. I don't
recommend this way since it's
basically like building your own ORM
and odds are you don't have the time
to do it correctly.
Forget stored procedures. Use an ORM.
One way in which stored procedures (ones which do not use dynamic SQL) can make the whole application more secure is that you can now set the permissions at the stored procedure level and not at the table level. If you do all of your data access this way (and forbid dynamic sql!) this means users can not under any circumstances do amnything to the database that is not in a stored proc. Developers always want to say that their application code can protect against outside threats, but they seem to forget that inside threats are often far more serious and by allowing permissions at the table level, they are at the mercy of any user who can find a way to directly query the database outside the application (another reason why in large shops only two or three people at most have production rights to anything in the datbase, it limits who can steal information).
Any financial system that uses anything except stored procs for instance is completely open to internal fraud which is a violation of internal controls that should prevent fraud and would not pass a good audit.
Stored procedures allow you to store you sql code in a location outside of the application. this gives you the ability to:
Change the SQL Code without recompiling/redistrubuting the application
Have multiple applications use the same stored procedure to access the same data.
Restrict users from having access to read/write to tables directly in the database.
From a development perspective it also allows the DBAs/database programmers to work on sql code without having to go through application code to work on it. (separation of responsibilities essentially).
Do stored procedures protect against injection attacks? For the most part yes. In sql server you can create stored procedures which are not effective against this, mainly by using sp_executesql. Now this doesn't main that sp_executesql is a security hole, it just means that more precaution needs to be taken when using it.
This also does not mean that stored procedures are the only way to protect against this. You can use parameritized sql to accomplish the same task of protecting against sql injection.
I do agree with other people stored procedures can be cumbersome, but they have their advantages too. Where I work, we have probably 20 different production databases for various reasons (don't ask). I work on a subset of maybe three, and my teammate and I know those three really really well. How do stored procedures help us? People come to us and when they need to grab that information out of those databases, we can get it for them. We don't have to spend hours explaining the schemas and what data is de-normalized. It's a layer of abstraction which allows us to program the most efficient code against the databases we know. If this isn't the case for you, then maybe stored procedures aren't the way to go, but in some instances they can add a lot of value.
Should queries live inside the classes that need the data? Should queries live in stored procedures in the database so that they are reusable?
In the first situation, changing your queries won't affect other people (either other code or people generating reports, etc). In the second, the queries are reusable by many and only exist in one place, but if someone breaks them, they're broken for everyone.
I used to be big on the stored proc solution but have changed my tune in the last year as databases have gotten faster and ORMs have entered the main stream. There is certainly a place for stored procs. But when it comes to simple CRUD sql: one liner insert/update/select/delete, using an ORM to handle this is the most elegant solution in my opinion, but I'm sure many will argue the other way. An ORM will take the SQL and DB connection building plumbing out of your code and make the persistence logic much more fluidly integrated with your app.
I suggest placing them as stored procedures in the database. Not only will you have the re-use advantage but also you'll make sure the same query (being a select, update, insert, etc...) it is the same because you are using the same stored procedure. If you need to change it, you'll only change it in one place. Also, you'll be taking advantage of the database server's processing power instead of the server/computer where your application resides. That is my suggestion.
Good luck!
It depends / it's situational. There are very strong arguments for and against either option. Personally, I really don't like seeing business logic get split between the database (in stored procedures) and in code, so I usually want all the queries in code to the maximum extent possible.
In the Microsoft world, there are things like Reporting Services that can retrieve data from classes/objects instead of (and in addition to) a database / stored procedures. Also, there are things like Linq that give you more strongly typed queries in code. There are also strong, mature ORMs like NHibernate that allow for writing pretty much all types of queries from code.
That said, there are still times when you are doing "rowset" types of things with your queries that work much better in a stored procedure than they work from code.
In the majority of situations, either/both options work just fine.
From my perspective, I think that stored procs are the the way to go. First, they are easier to maintain as a quick change to one means just running the script not recompiling the application. Second they are far better for security. You can set permissions at the sp level and not directly on the tables and views. This helps prevent fraud because the users cannot do anything directly to the datbase that isn't specified in a stored proc. They are easier to performance tune. When you use stored procs, you can use the database dependency metadata to help determine the affect of database changes on the code base. In many systems, not all data access or or even CRUD operations will take place through the application, having the code there seems to me to be counterproductive. If all the data access is in one place (an idea I support), it should be in the database where it is accessible to all applications or processes that might need to use it.
I've found that application programmers often don't consider the best way for a database to process information as they are focused on the application not the backend. By putting the code for the database in the database where it belongs, it is more likely to be seen and reviewed by database specialists who do consider the database and it's perfomance. We support hundreds of databases and applications here. I can look in any database and find the code that I need to find when something is slow. I don't have to upload the application code for each of hundreds of different applications just to see the part I need to do my job.