I am playing with Yii and having some trouble with Yii Eager Loading vs lazy loading database query. For example comparing two ways:
$comment = Comment::model()->with(array('issue'=>array('condition'=>'project_
id='.$projectId)))->findAll();
2a. $comment = Comment::model()->findAll
2b. to get Issue:
$issue = $comment->issues (assume that we have declare "issues" relation in Comment model).
I am not sure what advantages of the first approach. As I understand the 1st code will return only Comment BUT also run the get Issue query (like 2b code). However, as I see (thouth not sure there is better way to take advantage of "Eager Loading") if I want to get Issues of Comment in 1st code, I also have to run this code:
3.
$comment ->issues
If so, as I guest, we have to run 2 queries with the Eager loading approach, so it takes long time than "Lazy loading", because it run the "get Issue query" two times (fist time when we run the 1st code, and second time we run the 3rd code).
So, could you verify my though, or if it wrong could you tell me how to get all Issues from the Eager Loading approach that take advantage of its efficiency.
The matter about eager/lazy loading is not about how many lines of code you need, but how many resources it takes. When using eager loading, as in your first example, all of the elements of the 'issue' table which are associated to any comment will be loaded. They will be retrieved and preserved in memory, ready to be used. When you actually use them ($comment ->issues), they will be loaded from the memory cache, not from the database. Only one initial "big" query is performed. But if you have too many records associated and you won't use most of them, then lazy loading is the best option, as no record will be loaded from the database as long as you don't reference it. So when you reference a record, a query is performed. And that's how the story goes. So... eager loading or lazy loading, that's the question. :P
`
Related
I'm fairly new to JSF, and to coding itself. I'm working with JSF for 9 months now, and I noticed two ways of populating data tables:
-when pagination link is clicked, then new query is executed with given first and last record to show from results
-when user loads page query is executed and records are store in bean, so when pagination link is clicked, there is no query only giving records from stored result list (primefaces)
And here is my question :
Witch way is better?
I assume that second way need more memory, but it's used by primefaces, so I'm not sure that this is 'the right way'
First way need less memory - it gets only records needed to show, but isn't it to slow?
PS - BalusC - I hope you will take part in this discussion, since I see your answers very helpful, and explanatory:)
Personally when it comes to populating and managing a datatable. I feel the lazy-loading is the best practice. This is not a JSF specific way of loading the data, however; a few component libraries make it easy to implement (Primefaces, Icefaces, etc...).
Lazy-loading will help maintain large amounts of data, making sure the your page doesn't hang up and will keep a smooth user experience.
Primefaces LazyDataModel
Icefaces datatable(Already implemented)
Rich-faces Lazy Loading
This seems to be a pretty common problem: I load an NHibernate object that has a lazily loaded collection.
At some later point, I access the collection to do something.
I still have the nhibernate session open (as it's managed per view or whatever) so it does actually work but the transaction is closed so in NHprof I get 'use of implicit transactions is discouraged'.
I understand this message and since I'm using a unit of work implementation, I can fix it simply by creating a new transaction and wrapping the call to the lazy loaded collection within it.
My problem is that this doesn't feel right...
I have this great NHibernate framework that gives me nice lazy loading but I can't use it without wrapping every property access in a transaction.
I've googled this a lot, read plenty of blog posts, questions on SO, etc, but can't seem to find a complete solution.
This is what I've considered:
Turn off lazy loading. I think this is silly, it's like getting a full on sports car and then only ever driving it in eco mode. Eager loading everything would hurt performance and if I just had ids instead of references then why bother with Nhibernate at all?
Keep the transaction open longer. Transactions should not be long lived and keeping one open as long as a view is open would just be asking for trouble.
Wrap every lazy load property access in a transaction. Works but is bloaty and error prone. (i.e. if I forget to wrap an accessor then it will still work fine. Only using NHProf will tell me the problem)
Always load all the data for the properties I might need when I load the initial object. Again, this is error prone, both with loading data that you don't need (because the later call to access it has been removed at some point) or with not loading data that you do
So is there a better way?
Any help/thoughts appreciated.
I has had the same feelings when I first encountered this warning in NHProf. In web applications I think the most popular way is to have opened transaction (and unit of work) for the whole duration of request. For desktop applications managing transactions (as well as sessions) may be painful. You can use automatic transaction management frameworks (e.g. Castle) and declare with attributes service methods that should be run within transaction. With this approach you can wrap multiple operations into single transaction denending on your requirements. Also, I was using session-per-view approach with one opened session per view and manual transaction management (in this case I just ignored profiler warnings about implicit transactions).
As for your considerations: I strongly don't recommend 2) and 3). 1) and 4) are points to consider. But the general advice is: think, then try different approaches and find a solution that suits better for your particular situation.
First, please don't try to argue me out of doing the eager load - traversing the object graph and causing (by lazy loading) even more than ONE round-trip to the database is just not an option.
I have a big object graph. I want to fetch the root object, plus a subset of its children, grandchildren, great-grandchildren, etc. Currently I do this by creating multiple Future objects (with Criteria) and in each one, I do SetFetchMode("...", FetchMode.Eager) - see Ayende's post and Sam's 3rd comment here.
There are two problems:
NHibernate performs multiple select queries in the same round-trip - one for each path from root to a leaf (A.B.C.D), which is great, but uses join rather than subselect which is what I really want it to do. Using join means a ton of data needs to be sent from the database, needs to be parsed, and nhibernate needs to do a lot more work than necessary.
As a result of problem 1 - duplication of objects nested more than one level deep in some cases.
The second problem I "solved" by setting my collections to be Set, but then I lose the ordering ability - since I must specify ISet as the interface, there's no way for my code to know if the set is really an OrderedSet.
Does anyone know how to perform, in a single round-trip, eager loading of an object plus several deeply nested collections, but not using join?
I'd be extremely grateful! I've scoured the web for answers, apparently I'm not the first to hit this wall.
You can create a separate queries with only 1 call to SetFetchMode and run them in one go using MultiCriteria (or Futures or whatever you want to use). After that, only the result from the first query is relevant to you.
This will give you a single result in an single round-trip.
My members will have the ability to customise their profile page with X amount of widgets, each widget displays different data such as a list of music, list of people they are following etc.
Several widgets include:
- List of media they have uploaded
- List of people they are following
- List of people following them
- Html/Text widget
- Media Statistics (num downloads etc)
- Comments widget for other members to leave comments
Some widgets will have to page the data returned because there could be hundreds of results.
I haven't done any optimisation at the moment so it is doing lots of DB work to return all the data...what would be the most efficient way to retrieve the data...would 1 DB call per widget be acceptable? There could be around 5-20 widgets per page.
If you need more information about my situation please feel free to ask.
Paul
Short answer: It depends.
Start off from the unoptimised state, then use SQL profiler or a C# profiler like dotTrace to work out the best places to make improvements. Set a realistic goal to work towards (e.g. 'less than 800 milliseconds to load the page').
Generally I find performance starts to suffer after about 20-30 database calls in a request, but this is going to depend on your server, the location of the database etc.
There are many things that you can try: pre-caching, eager fetch using joins rather than selects etc. Nothing is going to guarantee better performance though unless it is applied intelligently.
For a page with lots of widgets, a common design pattern is to load each widget asynchronously using AJAX, rather than loading the entire page in one go.
since you've cut out out your work to widgets the proper thing to do would be for each widget to do a single query for all its required functionality. This would also be the case even if you retrieved widgets via AJAX (which as cbp noted is not a bad idea).
Secondly, i would set up some kind of mechanism for each widget to register its existence and then after all widgets have registered then i would fire a single query that would include all widget queries. (technically its again multiple queries but in a single round-trip, see MulriCriteria and MultiQueries in NH reference).
Also do not forget that lazy loads are hidden db retrievals and you could have a huge performance impact by using lazy load in a situation where an eager load is proper (for example Foo.Bar.Name where you always show the Bar.Name value when you present the Foo entity)
Performance degradation can occur even with less that 20-30 database call per request, but it depends on the size and complexity of your entities, queries, filters as well as the size of the data sets retrieved.
Most of what I hear about NHibernate's lazy-loading, is that it's better to use it, than not to use it. It seems like it just makes sense to minimize database access, in an effort to reduce bottlenecks. But few things come without trade-offs, certainly it slightly limits design by forcing you to have virtual properties. But I've also noticed that some developers turn lazy-loading off on certain often-used objects.
This makes me wonder if there are some definite situations where data-access performance is hurt by using lazy-loading.
So I wonder, when and in what situations should I avoid lazy-loading one of my NHibernate-persisted objects?
Is the downside to lazy-loading merely in additional processing time, or can nhibernate lazy-loading also increase the data-access time (for instance, by making additional round-trips to the database)?
Thanks!
There are clear performance tradeoffs between eager and lazy loading objects from a database.
If you use eager loading, you suck a ton of data in a single query, which you can then cache. This is most common on application startup. You are trading memory consumption for database round trips.
If you use lazy loading, you suck a minimal amount of data in a single query, but any time you need more information related to that initial data it requires more queries to the database and database performance hits are very often the major performance bottleneck in most applications.
So, in general, you always want to retrieve exactly the data you will need for the entire "unit of work", no more, no less. In some cases, you may not know exactly what you need (because the user is working through a wizard or something similar) and in that case it probably makes sense to lazy load as you go.
If you are using an ORM and focused on adding features quickly and will come back and optimize performance later (which is extremely common and a good way to do things), having lazy loading being the default is the correct way to go. If you later find (through performance profiling/analysis) that you have one query to get an object and then N queries to get the N objects related to that original object, you can change that piece of code to use eager loading to only hit the database once instead of N+1 times (the N+1 problem is a well known downside of using lazy loading).
The usual tradeoff for lazy loading is that you make a smaller hit on the database up front, but you end up making more hits on it long-term.
Without lazy loading, you'll grab an entire object graph up front, sucking down a large chunk of data at once. This could, potentially, cause lag in your UI, and so it is often discouraged. However, if you have a common object graph (not just single object - otherwise it wouldn't matter!) that you know will be accessed frequently, and top to bottom, then it makes sense to pull it down at once.
As an example, if you're doing an order management system, you probably won't pull down all the lines of every order, or all the customer information, on a summary screen. Lazy loading prevents this from happening.
I can't think of a good example for not using it offhand, but I'm sure there are cases where you'd want to do a big load of an object graph, say, on application initialization, in order to avoid lags in processing further down the line.
The short version is this:
Development is simpler if you use lazy loading. You just traverse object relationships in a natural OO way, and you get what you need when you ask for it.
Performance is generally better if you figure out what you need before you ask for it, and ask for it in one trip to the database.
For the past few years we've been focusing on quick development times. Now that we have a solid app and userbase, we're optimizing our data access.
If you are using a webservice between the client and server handling the database access using nhibernate it might be problematic using lazy loading since the object will be serialized and sent over the webservice and subsequent usage of "objects" further down in the object relationship needs a new trip to the database server using additional webservices. In such an instance it might not be too good using lazy loading. A word of caution, be careful in what you fetch if you turn lazy loading of, its way to easy to not think this through and through and end up fetching almost the whole database...
I have seen many performance problems aring from wrong loading behaviour configuration in Hibernate. The situation is quite the same with NHibernate I think. My recommendation is to always use lazy relations and then use eager fetching statemetns in your query - like fetch joins - . This ensures you are not loading to much data and you can avoid to many SQL queries.
It is easy to make a lazy releation eager by a query. It is nearly impossible the other way round.