In terms of speed of execution, from webapi2 all things being equal, which method would be run faster?
OPTION 1
var result = db.Database.SqlQuery("dbo.myStoredProcedure").ToList();
OPTION 2
var query = db.table1.where(x =>x.email) == targetEmail.select(c => new PortfolioPerfomance)
Basically option1 I wrote the sql stored procedure myself, and option2 I let the models in EF do the appropriate relations. The queries are much more complicated than this but I am just trying to understand the concepts of which would be faster.
thanks!!
This is an EF thing, no matter if you are using web api or not.
But, for bulk updates where you're updating massive amounts of data, a stored procedure will always perform better than an ORM solution because you don't have to marshal the data over the wire to the ORM to perform updates.
Also, if queries are complex, stored procedures are faster than Entity Framework when it need to retrieve massive amounts of data.
Keep in mind that both options will run faster if you have a a proper DB design.
Check this article for some tips to improve WEB API performance: https://www.c-sharpcorner.com/article/Tips-And-Tricks-To-Improve-WEB-API-Performance/
Also check this one to improve stored procedures: https://www.c-sharpcorner.com/article/improve-stored-procedure-performance-in-sql-server-store-procedure-performance/
Related
I'm writing an SQL script to update the database of a deployed application (add a few tables and copy/update some data) to accommodate new functionality being added to the application (.NET using C#). I've written some code in the application to handle such updates (whenever a new update is available, run its .sql) which reusable for all DB updates (simply provide a new .sql).
The problem I'm running into now is that I am fairly new to SQL, and have a pretty short timeline to get this update ready. The more complex part of the update could easily be written in the application, but for consistency's sake I'd like to do it in SQL.
What I need to do, I've summarized as the following algorithm/pseudocode:
for each category in allCategories
Select * from unrankedEntities
where Category = category
order by (score)
take top Category.cutoff
insert each of these into rankedEntities with rank = 0
Select * from unrankedEntities
where Category = category
order by (score) DESC
take top 16 - Category.cutoff
insert each of these into rankedEntities with rank = null, belowcutoff=true
This is the general algorithm I am trying to implement (with a couple more tweaks to be made after this part is complete). Would it be more efficient to write this in SQL or just write the LINQ to do it (resulting in changing the application's database update code for each DB update)? If SQL is the best choice, where can I find a good reference/tutorial covering the topics I would need to do this (I've looked through few different ones online, and some seem pretty simple, so I would appreciate a recommendation of a good one).
Additionally, if you have any tips on improving the algorithm, I'd be happy to hear your advice.
EDIT:
I forgot to mention, but (score) is not a value stored in unrankedEntities. It needs to be calclulated for each entity by counting rows in another table that match a specific condition, and mulitplying by a constant and by numbers pulled from other tables. Additionally, it needs to be stored into rankedEntities to avoid all this calculation every time it is needed (often).
EDIT:
It has been pointed out to me that parts of the new additions to the application already contain the code necessary to do all of the data manipulation, so I will go with the simple solution of recycling that code, although the result is a less tidy update-via-sql class, as it now has an update-via-sql section and update-via-code section.
When using LINQ to SQL in debug mode you can see the SQL it's generating, which you could copy into a script, if you need to hurry and not learn all about SQL right at this moment. See http://weblogs.asp.net/scottgu/archive/2007/07/31/linq-to-sql-debug-visualizer.aspx
EDIT: I didn't realize you were using Entity Framework. See this response to a similar question: How do I get the raw SQL underlying a LINQ query when using Entity Framework CTP 5 "code only"?.
LINQ is perfectly fine for this, but inserts and updates each entity in a singular fashion, meaning in SQL you could update in bulk with one query, but you cannot in LINQ... it generates separate statements for each entity. So SQL has some efficiency gains. If you are more comfortable with LINQ, do it in LINQ. If you are talking thousands of objects, you may want to turn off change tracking (turned off in EF by changing the merge option) as the internal memory of changed objects will grow rather large.
A simple question. There are cases when I fetch the data and then process it in my BLL. But I realized that the same processing/filtering can be done in my stored procedure and the filtered results returned to BLL.
Which is better, processing at DB or processing in BLL? And Why?
consider the scenario, I want to check whether a product exists in my db and if it exists add that to the order (Example taken from answer by Nour Sabony below)now i can do this checking at my BLL or I an do this at the stored procedure as well. If I combine things to one procedure, i reduce the whole operation to one db call. Is that better?
At the database level.
Databases should be optimized for querying, so why go through all the effort to return a large dataset, and then do filtering in your application afterwards?
As a general rule: Anything that reasonably belongs to the db's responsibilities and can be done at your db, do it at your db.
well, the fatest answer is at database,
but you may consider something like Linq2Sql,
I mean to write an expression at the presentation layer, and it will be parsed as Sql Statement at Data Access Layer.
of course there are some situation BLL should get some data from DAL ,process it ,and then return it to DAL.
take an example :
PutOrder(Order value) procedure , which should check for the availability of the ordered product.
public void PutOrder(Order _order)
{
foreach (OrderDetail _orderDetail in _order.Details)
{
int count = dalOrder.GetProductCount(_orderDetail.Product.ProductID);
if (count == 0)
throw new Exception (string.Format("Product {0} is not available",_orderDetail.Product.Name));
}
dalOrder.PutOrder(_order);
}
but if you are making a Browse view, it is not a good idea (from a performance viewpoint) to bring all the data from Dal and then choose what to display in the Browse view.
may be the following could help:
public List<Product> SearchProduts(Criteria _criteria)
{
string sql = Parser.Parse(_criteria);
///code to pass the sql statement to Database procedure and get the corresponding data.
}
at the database.
I have consistently been taught that data should be handled in the data layer whenever possible. Filtering data is what a DB is specifically there to do and is optimized to do. Therefore that is where it should occur.
This also prevents the duplication of effort. If the data filtering occurs in a data layer then other code/systems could benefit from the same stored procedure instead of duplicating the work in your BLL.
I have found that people like to put the primary logic in the section that they are most comfortable working in. Regardless of a developer's strengths, data processing should go in the data layer whenever possible.
To throw a differing view point out there it also really depends upon your data abstraction model if you have a large 2nd level cache that sits on top of your database and the majority (or entirety) of your data you will be filtering on is in the cache then yes stay inside the application and use the cached data.
The difference is not very important when you consider tables that will never contain more than a few dozen rows.
Filtering at the db becomes the obvious choice when you consider tables that will grow to thousands, hundreds of thousands or millions of rows. It would make zero sense to transfer that much data to filter for a subset of records.
I asked a similar question in a SSRS class earlier this year when I wanted to know "best practice" for having a stored procedure retrieve queried data instead of executing TSQL in the report's data set. The response I received was: Allow the database engine to do the "heavy lifting." That is, do all the selecting, grouping, sorting in the stored procedure and allow SSRS to concentrate on delivering the results.
The same is true for your scenario. By all means, my vote would be to do all the filtering in your stored procedure. Call the sproc from your application and let the database do the work.
If you filter at the database, then only the data that is required by the application is actually sent on the wire between the database and the application server. For this reason, I suggest that it is better to filter at the database.
I am creating a table to summarize data that is gathered from about 8 or so queries that have very light logic/WHERE clauses and all select against different tables.
I was wondering what the best option would be to fetch the summarized data:
One query with multiple JOINS to gather all relevant information
A stored proc that encapsulates the logic and maybe executes the 8 queries and does the "joining" in some other way? This seems more modular and maintainable to me...but I'm not sure.
I am using SQL Server 2008 for this. Any suggestions?
If you can, then use usual SQL methods. Db's are optimized to run them. This "joining in some other way" would probably require the use of cursor which slows down everything. Just let the db do its job. If you need more performance then you should examine execution plan and do what has to be done there(eg. adding indexes).
Databases are pretty good at figuring out the optimal way of executing SQL. It is what they are designed to do. Using stored procedures to load the data in chunks and combining it yourself will be more complex to write, and likely to be less efficient than letting the database just do it for you.
If you are concerned about reusing a complex query in multiple places, consider creating a view of it instead.
Depending on the size of the tables, joining 8 of them could be pretty hairy. I would try it that way first - as others have said, the db is pretty good at figuring this stuff out. If the performance is not as good as you would like, I would try a stored proc which creates a table variable (or a temp table) and inserts the data from each of the 8 tables separately. Then you can return the contents of the table variable to your app.
This method also makes it a little easier to add the 9th, 10th, etc tables in the future. And it gives you an easy way to do any processing you may need on the summarized data before returning it to your app.
I am wondering which is a more efficent method to retrieve data from the database.
ex.
One particular part of my application can have well over 100 objects. Right now I have it setup to query the database twice for each object. This part of the application periodically refreshes itself, say every 2 minutes, and this application will probably end of being installed on 25-30 pc's. I am thinking that this is a large number of select statements to make from the database, and I am thinking about trying to optimize the procedure. I have to pull the information out of several tables, and both queries are using join statements.
Would it be better to rewrite the queries so that I am only executing the queries twice per update instead of 200 times? For example using a large where statement to include each object, and then do the processing of the data outside of the object, rather than inside each object?
Using SQL Server, .net No indexes on the tables, size of database is less than 10-5th
all things being equal, many statements with few rows is usually worse than few statements with many rows.
show the actual code and get better answers.
The default answer for optimization must always be: don't optimize until you have a demonstrated need for it. The followup is: once you have a demonstrated need for it, and an alternative approach: try both ways to determine which is faster. You can't predict which will be faster, and we certainly can't. KM's answer is good - fewer queries tends to be better - but the way to know is to test.
Does any one have some idea on how to run the following statement with LINQ?
UPDATE FileEntity SET DateDeleted = GETDATE() WHERE ID IN (1,2,3)
I've come to both love and hate LINQ, but so far there's been little that hasn't worked out well. The obvious solution which I want to avoid is to enumerate all file entities and set them manually.
foreach (var file in db.FileEntities.Where(x => ids.Contains(x.ID)))
{
file.DateDeleted = DateTime.Now;
}
db.SubmitChanges();
There problem with the above code, except for the sizable overhead is that each entity has a Data field which can be rather large, so for a large update a lot of data runs cross the database connection for no particular reason. (The LINQ solution is to delay load the Data property, but this wouldn't be necessary if there was some way to just update the field with LINQ to SQL).
I'm thinking some query expression provider thing that would result in the above T-SQL...
LINQ cannot perform in store updates - it is language integrated query, not update. Most (maybe even all) OR mappers will generate a select statement to fetch the data, modify it in memory, and perform the update using a separate update statement. Smart OR mappers will only fetch the primary key until additional data is required, but then they will usually fetch the whole rest because it would be much to expensive to fetch only a single attribute at once.
If you really care about this optimization, use a stored procedure or hand-written SQL statement. If you want compacter code, you can use the following.
db.FileEntities.
Where(x => ids.Contains(x.ID)).
Select(x => x.DateDeleted = DateTime.Now; return x; );
db.SubmitChanges();
I don't like this because I find it less readable, but some prefer such an solution.
LINQ to SQL is an ORM, like any other, and as such, it was not designed to handle bulk updates/inserts/deleted. The general idea with L2S, EF, NHibernate, LLBLGen and the rest is to handle the mapping of relational data to your object graphs for you, eliminating the need to manage a large library of stored procs which, ultimately, limit your flexability and adaptability.
When it comes to bulk updates, those are best left to the thing that does them best...the database server. L2S and EF both provide the ability to map stored procedures to your model, which allows your stored procs to be somewhat entity oriented. Since your using L2S, just write a proc that takes the set of identities as input, and executes the SQL statement at the beginning of your question. Drag that stored proc onto your L2S model, and call it.
Its the best solution for the problem at hand, which is a bulk update. Like with reporting, object graphs and object-relational mapping are not the best solution for bulk processes.