Given a multi level object graph being called using Future as:
var Dads = db.Session.Query<Parent>().Where(P => P.EntityKey == Id)
.ToFuture<Parent>();
var Kids = db.Session.Query<Kid>().Where(K => K.Parent.EntityKey == Id)
.ToFuture<Kid>();
when I call var Dad = dads.ToList() I see the batch go across the wire and show in profiler.
Problem is when enumerating the collection it is still sending one off queries to the db
Eg.
for each (Kid kid in Dad.Kids) // This seems to hit the database
{
Teach(kid);
}
Sends a SQL query and hits the database to get each kid. Why is the object graph not populated? or is this expected behavior?
That behaviour is to be expected. You are simply telling NHibernate to get two collections from the database in a batch, which it is doing as told. However, you are not telling it that they are related. NH Queries with Futures do not put entities together after executing them unless they are told to do so with a join.
If you executed the separate queries without Futures you would not expect the Parent entity to suddenly have the child collection filled. Basically, Futures allow you to run things in one roundtrip. If the queries happen to have a common root with several child collections (e.g. to avoid a cartesian product), then NH is able to "combine" several collections into one entity.
Unfortunately joins with the NH LINQ Api and the ToFuture() method seem to pose a problem in the current (NH 3.0 or 3.1) implementation. You may need to use the QueryOver Api in that case.
On a side note, I think the method name is not appropriate.
Edit: After Edit of the question the method name is now ok.
Related
I'm relatively new to using NHibernate and I'm running into a shortcoming I can't seem to work myself around. I have an object tree that I wish to retrieve from the database in a single roundtrip but end up with a cartesian product.
The objects I'm trying to retrieve are called 'AccountGroup', 'Concern', 'Advertiser' and 'Product' and I only wish to get those objects where the active user has permissions for.
My initial query looked like this:
using (var session = OpenSession())
{
return session.Query<AccountGroupEntity>()
.FetchMany(a => a.Planners)
.Where(a => a.Planners.Any(p => p.Id == userId))
.FetchMany(a => a.Concerns)
.ThenFetchMany(c => c.Advertisers)
.ThenFetch(a => a.Products)
.ToList();
}
This won't work as it will return a cartesian product and the resulting entities will contain many duplicates.
However, I have NO idea how to fix this. I've seen the ToFuture() method that will allow me to execute more than one query in the same roundtrip, but I have no clue how to configure my ToFuture() query in such a way that it populates all the child collections properly.
Could anyone shine some light on how I can use ToFuture to fetch the entire tree in a single query without duplicates?
I do have an answer to this topic, solution which I do use. But it at the end means "do not use Fetch" - do it differently. So, please, take it at least as a suggestion.
Check this Q & A:
How to Eager Load Associations without duplication in NHibernate?
Small cite:
Fetching Collections is a difficult operation. It has many side effects (as you realized, when there are fetched more collections). But even with fetching one collection, we are loading many duplicated rows.
Other words, Fetching is a fragil feature, and should be used wisely in very few scenarios, I'd say. So what to use? How to solve that?
Profit from a built in NHibernate feature:
19.1.5. Using batch fetching
NHibernate can make efficient use of batch fetching, that is, NHibernate can load several uninitialized proxies if one proxy is accessed (or collections. Batch fetching is an optimization of the lazy select fetching strategy. There are two ways you can tune batch fetching: on the class and the collection level.
Batch fetching for classes/entities is easier to understand. Imagine you have the following situation at runtime: You have 25 Cat instances loaded in an ISession, each Cat has a reference to its Owner, a Person. The Person class is mapped with a proxy, lazy="true". If you now iterate through all cats and call cat.Owner on each, NHibernate will by default execute 25 SELECT statements, to retrieve the proxied owners. You can tune this behavior by specifying a batch-size in the mapping of Person:
<class name="Person" batch-size="10">...</class>
NHibernate will now execute only three queries, the pattern is 10, 10, 5.
You may also enable batch fetching of collections. For example, if each Person has a lazy collection of Cats, and 10 persons are currently loaded in the ISesssion, iterating through all persons will generate 10 SELECTs, one for every call to person.Cats. If you enable batch fetching for the Cats collection in the mapping of Person, NHibernate can pre-fetch collections:
<class name="Person">
<set name="Cats" batch-size="3">
...
</set>
My experience, this approach is pricless. The setting working for us is batch-size="25".
If you ask for any kind of Entity (via session.Get() or .QueryOver()...) - until session is open, the first time we touch related reference or collection - it is loaded in few batches... No 1 + N SELECT Issue...
Summary: Mark all your classes, and all collection with batch-size="x" (x could be 25). That will support clean queries over root Entities - until session is open, all related stuff is loaded in few SELECTS. The x could be adjusted, for some could be much more higher...
I tried to do a lot of research but I'm more of a db guy - so even the explanation in the MSDN doesn't make any sense to me. Can anyone please explain, and provide some examples on what Include() statement does in the term of SQL query?
Let's say for instance you want to get a list of all your customers:
var customers = context.Customers.ToList();
And let's assume that each Customer object has a reference to its set of Orders, and that each Order has references to LineItems which may also reference a Product.
As you can see, selecting a top-level object with many related entities could result in a query that needs to pull in data from many sources. As a performance measure, Include() allows you to indicate which related entities should be read from the database as part of the same query.
Using the same example, this might bring in all of the related order headers, but none of the other records:
var customersWithOrderDetail = context.Customers.Include("Orders").ToList();
As a final point since you asked for SQL, the first statement without Include() could generate a simple statement:
SELECT * FROM Customers;
The final statement which calls Include("Orders") may look like this:
SELECT *
FROM Customers JOIN Orders ON Customers.Id = Orders.CustomerId;
I just wanted to add that "Include" is part of eager loading. It is described in Entity Framework 6 tutorial by Microsoft. Here is the link:
https://learn.microsoft.com/en-us/aspnet/mvc/overview/getting-started/getting-started-with-ef-using-mvc/reading-related-data-with-the-entity-framework-in-an-asp-net-mvc-application
Excerpt from the linked page:
Here are several ways that the Entity Framework can load related data into the navigation properties of an entity:
Lazy loading. When the entity is first read, related data isn't retrieved. However, the first time you attempt to access a navigation property, the data required for that navigation property is automatically retrieved. This results in multiple queries sent to the database — one for the entity itself and one each time that related data for the entity must be retrieved. The DbContext class enables lazy loading by default.
Eager loading. When the entity is read, related data is retrieved along with it. This typically results in a single join query that retrieves all of the data that's needed. You specify eager loading by using the Include method.
Explicit loading. This is similar to lazy loading, except that you explicitly retrieve the related data in code; it doesn't happen automatically when you access a navigation property. You load related data manually by getting the object state manager entry for an entity and calling the Collection.Load method for collections or the Reference.Load method for properties that hold a single entity. (In the following example, if you wanted to load the Administrator navigation property, you'd replace Collection(x => x.Courses) with Reference(x => x.Administrator).) Typically you'd use explicit loading only when you've turned lazy loading off.
Because they don't immediately retrieve the property values, lazy loading and explicit loading are also both known as deferred loading.
Think of it as enforcing Eager-Loading in a scenario where your sub-items would otherwise be lazy-loading.
The Query EF is sending to the database will yield a larger result at first, but on access no follow-up queries will be made when accessing the included items.
On the other hand, without it, EF would execute separte queries later, when you first access the sub-items.
include() method just to include the related entities.
but what happened on sql is based on the relationship between those entities which you are going to include what the data you going to fetch.
your LINQ query decides what type of joins have to use, there could be left outer joins there could be inner join there could be right joins etc...
#Corey Adler
Remember that you should use .Include() and .ThenInclude() only when returning the object (NOT THE QUERYABLE) with the "other table property".
As a result, it should only be used when returning APIs' objects, not in your intra-application.
I'm a beginner in NHibernate. I have to write a complex query on say an "Employee" to populate all the associations for Employee based on the where clause. What I'm looking for is similar to this - when you do a Employee.FindById(10) should fill up OwnedDepartment, SubscribedGroups etc.
The Employee model I need to populate is really heavy (many associations with other objects)
but I need to populate only few associations. How do I achieve it using a query over? or any other approaches?
Updated
I was reading about eager loading just now, has it something to do with the loading ? In my map I have not mentioned any loading techniques, so by default all of my employee's child element are getting loaded already. There is a bunch of queries getting triggered underneath.
All the associations are lazy loaded by default. That means that the load is triggered when you access it - that's why so many queries are issued. If you want to eagerly load the data (which means either joining the tables or - rarely - doing additional select queries at once), you have to specify it in your mapping or query, depending how you fetch your data. The concept is generally called "eager fetching".
If you want to get a single Employee by ID, the standard way to do it is using session.Get<Employee>(10) - but that approach means that eager loads need to be specified in the mapping. For mapping by code it will be c.Lazy(CollectionLazy.NoLazy); for collections or c.Lazy(LazyRelation.NoProxy) for many-to-one - see here or here for details.
I prefer specifying that kind of things in the query - just where it is used, not globally for the whole entity, regardless who is fetching and what for. In LINQ provider you have FetchMany(x => x.SubscribedGroups) for collections and Fetch(x => x.OwnedDepartment) for many-to-one relations. You can find similiar options in QueryOver, if that's your choice.
Lets say I have entity A, which have many to many relationship with another entities of type A. So on entity A, I have collection of A. And lets say I have to "update" this relationships according to some external service - from time to time I receive notification that relations for certain entity has changed, and array of IDs of current related entities - some relations can be new, some existing, some of existing no longer there... How can I effectively update my database with EF ?
Some ideas:
eager load entity with its related entities, do foreach on collection of IDs from external service, and remove/add as needed. But this is not very effective - need to load possibly hundreds of related entities
clear current relations and insert new. But how ? Maybe perform delete by stored procedure, and then insert by "fake" objects
a.Related.Add(new A { Id = idFromArray })
but can this be done in transaction ? (call to stored procedure and then inserts done by SaveChanges)
or is there any 3rd way ?
Thanx.
Well, "from time to time" does not sound like a situation to think much about performance improvement (unless you mean "from millisecond to millisecond") :)
Anyway, the first approach is the correct idea to do this update without a stored procedure. And yes, you must load all old related entities because updating a many-to-many relationship goes only though EFs change detection. There is no exposed foreign key you could leverage to update the relations without having loaded the navigation properties.
An example how this might look in detail is here (fresh question from yesterday):
Selecting & Updating Many-To-Many in Entity Framework 4
(Only the last code snippet before the "Edit" section is relevant to your question and the Edit section itself.)
For your second solution you can wrap the whole operation into a manually created transaction:
using (var scope = new TransactionScope())
{
using (var context = new MyContext())
{
// ... Call Stored Procedure to delete relationships in link table
// ... Insert fake objects for new relationships
context.SaveChanges();
}
scope.Complete();
}
Ok, solution found. Of course, pure EF solution is the first one proposed in original question.
But, if performance matters, there IS a third way, the best one, although it is SQL server specific (afaik) - one procedure with table-valued parameter. All new related IDs goes in, and the stored procedure performs delete and inserts in transaction.
Look for the examples and performance comparison here (great article, i based my solution on it):
http://www.sommarskog.se/arrays-in-sql-2008.html
I'm trying to use 'adonet.batch_size' property in NHibernate. Now, I'm creating entities across multiple sessions at a large rate (hence batch inserting). So what I'm doing is creating a buffer where I keep these entities and them flush them out all at once periodically.
However I need the ID's as soon as I create the entities. So I want to create an entity (in any session) and then have its ID generated (I'm using HiLo generator). And then at a later time (and other session) I want to flush that buffer and ensure that those IDs do not change.
Is there anyway to do this?
Thanks
Guido
I find it odd that you need many sessions to do a single job. Normally a single session is enough to do all work.
That said, the Hilo generator sets the id property on the entity when calling nhSession.Save(object) without necessarily requiring a round-trip to the database and a
nhSession.Flush() will flush the inserts to the database
UPDATE ===========================================================================
This is a method i used on a specific case that made pure-sql inserts while maintaining NHibernate compatibility.
//this will get the value and update the hi-lo value repository in the datastore
public static void GenerateIdentifier(object target)
{
var targetType = target.GetType();
var classMapping = NHibernateSessionManager.Instance.Configuration.GetClassMapping(targetType);
var impl = NHibernateSessionManager.Instance.GetSession().GetSessionImplementation();
var newId = classMapping.Identifier.CreateIdentifierGenerator(impl.Factory.Dialect, classMapping.Table.Catalog, classMapping.Table.Schema,
classMapping.RootClazz).Generate(impl, target);
classMapping.IdentifierProperty.GetSetter(targetType).Set(target, newId);
}
So, this method takes your newly constructed entity like
var myEnt = new MyEnt(); //has default identifier
GenerateIdentifier(myEnt); //now has identifier injected based on nhibernate's mapping
note that this call does not place the entity in any kind of nhibernate managed space. So you still have to make a place to place your objects and make the save on each one. Also note that i used this one with pure sql inserts and unless you specify generator="assigned" (which will then require some custom hi-lo generator) in your entity mapping nhibernate may require a different mechanism to persist it.
All in all, what you want is to generate an Id for an object that will be persisted at some time in the future. This brings up some problems such as handling non-existent entries due to rollbacks and failed commits. Additionally imo nhibernate is not the tool for this particular job, you don't need nhibernate to do your bulk insert unless there is some complex entity logic that is too costly (in dev time) to implement on your own.
Also note that you are implying that you need transient detached entities which however cannot be used unless you call .nhSes.Save(obj) on the first session and flush its contents so the 2nd session when it calls Load on the transient object there will be an existing row in the database which contradicts what you want to achieve.
Imo don't be afraid of storming the database, just optimise the procedure top-to-bottom to be able to handle the volume. Using nhibernate just to do an insert seems counter-productive when you can achieve the same result with 4 times the performance using ado.net or even an isqlquery wrapped-query (and use the method i provided above)