I am working on a legacy application where NHibernate has been used without any apparent thought to efficiency. I am currently stepping through a method in which over 200 queries have been executed so far. This is mostly due to the N+1 problem.
Anyway, as I think about fixing this, my question is:
Given an entity with, say, 10 child collections, almost all of which will be accessed during a single operation, is there a way to lazy-load each child collection in a single call rather than have NHibernate lazy-load each individual child record as it is accessed (e.g., in a foreach loop, which is what's happening now). It seems to me that eager loading all of these child collections at once would be a massive query and not very performant. But obviously this N+1 approach is wrong. How can I tell NHibernate to do a query that loads up the whole child collection on demand? This still gives me 11 queries, but it's better than 200 and, perhaps, better than the massive query I'd have to do if I eager loaded everything.
Thanks!
You can use the method NHibernateUtil.Initialize(yourObject.ChildCollection). This method forces proxies to load their data based on their fetching strategy, that you set in your mappings. Source: NHibernate documentation
Related
I am not understanding the following scenario. I have a situation where I need to load children and grandchildren along with an entity. I have specified that collections are lazyloaded, but eagerly fetch all rows using detached criteria and setfetchmode to eager. I have the following problem:
NHibernate is taking very long to hydrate the objects - the actual queries run quickly though.
I place the list of results into .net MemoryCacheManager object and seems that after a while the grandchildren seem to drop out the object and I get a lazyload exception.
How to I ensure that the entire graph returned by the results remains in tact in the MemoryCacheManager object? Does session.Evict help me?
Ended up using QueryOver future queries and flattened the results into a DTO and ended up caching the DTO.
I have a domain object Foo that has an 1:n relation to a domain object Bar.
There are two major use cases where I need to get all foo's matching some criterion. In case A, I care about the bars attached to each foo, in case B, I don't. There are quite a lot of bars, so simply always loading the bars is not good for performance of case A. Similarly, not loading the bars eagerly will lead to an n+1 avalanche in case B. So neither tagging the realtion as #Lazy nor not tagging it is the correct choice.
Now, my question: Is it possible to tell the extbase persistence layer at query time whether to be lazy or eager? If yes, how? If no, is there another way in Extbase to avoid the n+1 problem (i.e. load all necessary bars and then hope that caching works when iterating through the foos)?
My last resort, of course, would be to load the foos with lazy loading, load the bars manually in a second query, and then manually set the relation.
Any suggestions?
I've been playing around with Extbase and delving into the internals in the last few months, and the upshot is this: It' impossible.
I suppose that closes this question, though not the way I'ld like.
Actually, even worse: Eager loading is not implemented at all, the #eager tag which according to the documentation sets eager loading for a relation is ignored.
what about leaving it lazy and converting it to an array when needed? (foo->bar->toArray())
I have entities which I would like to eagerly load , and on other ocassions lazy (or even extra lazy) load.
My mappings have no fetch mode declared in my YAML- so they use the default (lazy loading).
Currently the only way to eagerly load is to by constructing the DQL manually - and I need to update this every time I add a new entity.
Ideally I would just load the root entity and the force eager loading all the associated objects. Is there any way I can do this?
If not why (is there a reason beyond it being an unimplemented feature)?
If you want to use built-in repository methods (find(), findAll()), you're probably out of luck unless you set things to eagerly load in your annotations.
You'll probably want to use the query builder (or raw DQL) in some custom repository's method to force eager loading where you want it. Yes, you'll have to update that method as you add entities, but at least you'll always know what's going on in regards to lazy/eager loading, and you'll only need to maintain it all in one place.
I suppose the reason there's not some $eagerLoad flag to find(), etc, is because those are convenience methods for simple tasks. If you wanted to add such a flag, you'd have quickly get into situations where you'd want to limit recursive eager loading by depth. You'd also probably have to start worrying about cyclical references (any bidirectional association, for instance).
You can use setFetchMode() method of DQL to set mode.
See the documentation:
https://web.archive.org/web/20120601032806/http://readthedocs.org/docs/doctrine-orm/en/latest/reference/dql-doctrine-query-language.html
How can I get NHibernate to hook up child objects automatically or bulk load children rather than lazy loading for each parent?
I have a large number of parent objects all of the same type. Each of them has two bags of child objects. As I need to load all the parents and children objects as quickly as possible, I use NHibernate to load all the objects and then loop all child objects and add them to relevant parent in code. I'm sure NHibernate has a much better way of doing this - but what is it?
You can always use eager loading behavior of NHibernate to override its default behavior (Lazy Loading).
Here is an article that discuss lazy loading and eagerly loading
Take a look at the "Eagerly loading with HQL" part that shows how you can use HQL to eagerly load an object graph.
However using eager loading can have a negative impact on performance specially if you are working with a lot of objects.
Assuming a typical domain entity approach with SQL Server and a dbml/L2S DAL with a logic layer on top of that:
In situations where lazy loading is not an option, I have settled on a convention where getting a list of entities does not also get each item's child entities (no loading), but getting a single entity does (eager loading).
Since getting a single entity also gets children, it causes a cascading effect in which each child then gets its children too. This sounds bad, but as long as the model is not too deep, I usually don't see performance problems that outweigh the benefits of the ease of use.
So if I want to get a list in which each of the items is fully hydrated with children, I combine the GetList and GetItem methods. So I'll get a list and then loop through it getting each item with the full cascade. Even this is generally acceptable in many of the projects I've worked on - but I have recently encountered situations with larger models and/or more data in which it needs to be more efficient.
I've found that partitioning the loop and executing it on multiple threads yields excellent results. In my first experiment with a list of 50 items from one particular project, I did 5 threads of 10 items each and got a 3X improvement in time.
Of course, the mileage will vary depending on the project but all else being equal this is clearly a big opportunity. However, before I go further, I was wondering what others have done that have already been through this. What are some good approaches to parallelizing this type of thing?
Usually it is faster to make a single database call that returns a set of records.
This recordset can "hydrate" the top-level objects, then another recordset can load child objects. I'm not sure how your situation does not allow lazy-loading, but this method is essentially lazy-loading, and will surely be faster than making multiple calls to the database that returns a single record each time.
You could make asynchronous calls to the database so that multiple queries are running in parallel. If you combine this with the first strategy for each "layer" of the model, and write a somewhat more complex hydration function based on multiple-record return sets, you should see that the database handles concurrent connections very well (which is why you see a performance gain from using multiple threads).
But you don't need to explicitly create threads - check out the asynchronous methods of a SqlCommand.