NHibernate QueryOver.List causing N+1 - nhibernate

Having a seemingly bizzare N+1 select problem in NHibernate. I am executing a query where I'm asking for a bunch of entities where one of its linked properties is null. I don't actually need the linked property to be returned in this case by NHibernate as its only for the purpose of selecting the right data.
First Entity is a booking Window
public class BookingWindow : Entity<BookingWindow>
{
// Blah blah blah
/// <summary>
/// Gets or sets the booking order item.
/// </summary>
/// <value>
/// The booking order item.
/// </value>
public virtual BookingWindowOrderItem BookingOrderItem { get; set; }
}
And the BookingWindowOrderItem as follows
public class BookingWindowOrderItem : OrderItem
{
// Blah blah blah
public virtual BookingWindow BookingWindow { get; set; }
}
Here are the respective mappings
public BookingWindowMap()
{
this.Schema("Customer");
this.Table("BookingWindows");
this.Id(x => x.Id).GeneratedBy.Guid();
this.Component(x => x.WindowPeriod, m =>
{
m.Map(x => x.Min, "StartTime");
m.Map(x => x.Max, "EndTime");
});
this.References(window => window.BookingOrderItem).PropertyRef("BookingWindow").Column("Id").LazyLoad().Nullable().ReadOnly();
this.Map(x => x.Price);
this.References(x => x.CustomerRoom).ForeignKey("RoomId").Column("RoomId");
}
And
public BookingWindowOrderItemMap()
{
this.DiscriminatorValue(1);
this.References(x => x.BookingWindow).LazyLoad().Column("OrderItemForeignId").ForeignKey("OrderItemForeignId");
}
Now When I execute the following query I get back the correct Booking windows that don't have an order item.
Session.QueryOver<BookingWindow>().Where(w => w.CustomerRoom.Id == Guid.Parse(roomId)).Left.JoinQueryOver(bw => bw.BookingOrderItem).WhereRestrictionOn(item => item.Id).IsNull.List<BookingWindow>();
So the first query gets issued to the database like so (the order item columns are selected which is a bit annoying but the real problem comes in a minute)
SELECT this_.Id as Id2_1_, this_.Price as Price2_1_, this_.RoomId as RoomId2_1_, this_.StartTime as StartTime2_1_, this_.EndTime as EndTime2_1_, bookingwin1_.Id as Id4_0_, bookingwin1_.Price as Price4_0_, bookingwin1_.Description as Descript4_4_0_, bookingwin1_.OrderId as OrderId4_0_, bookingwin1_.OrderItemParentId as OrderIte6_4_0_, bookingwin1_.OrderItemForeignId as OrderIte7_4_0_ FROM Customer.BookingWindows this_ left outer join Payment.OrderItem bookingwin1_ on this_.Id=bookingwin1_.OrderItemForeignId and bookingwin1_.OrderItemTypeId='1' WHERE this_.RoomId = ? and bookingwin1_.Id is null
But then for each booking window returned there is an extra select for the linked order item even though I haven't asked for it or need it. This happens within the query over method so I'm not doing any kind of iterating over the returned booking windows manually.
SELECT bookingwin0_.Id as Id4_0_, bookingwin0_.Price as Price4_0_, bookingwin0_.Description as Descript4_4_0_, bookingwin0_.OrderId as OrderId4_0_, bookingwin0_.OrderItemParentId as OrderIte6_4_0_, bookingwin0_.OrderItemForeignId as OrderIte7_4_0_ FROM Payment.OrderItem bookingwin0_ WHERE bookingwin0_.OrderItemForeignId=? and bookingwin0_.OrderItemTypeId='1'
Can anyone explain to me the error I have made here. Maybe its obvious but I've struggled for a few hours and at the end of my patience :)

I see one weird part in your mapping: Using References as a one-to-one mapping style. Maybe it is intended, but this is causing that issue you have.
Firstly, as documentation says [References / many-to-one][1]
References is for creating many-to-one relationships between two
entities, and is applied on the "many side." You're referencing a
single other entity, so you use the References method. #HasMany /
one-to-many is the "other side" of the References relationship, and
gets applied on the "one side."
Other words, in the table of the BookingWindowOrderItemMap you store reference to BookingWindow. It could mean (by the DB Design), that there could be more records of OrderItem, referencing the same BookingWindow. But maybe this is what you want, and you check "uniqueness" elsewhere. The more I tried to understand your problem, I would vote for moving the reference to OrderItem in a column in the BookingWindow
Problem revealed:
To your issue. When NHibernate recieves the list of BookingWindow, the next step is to build a proxy. In this process, all valueType/string properties are set, and for references... And for references NHibernate tries to prepare the lazy load.
Simplified version is, that into each property BookingWindowOrderItem BookingOrderItem is injected a promise for an instance of the BookingWindowOrderItem, to be returned when firstly touched. In standard cases, when mapping References is used, NHibernate in this moment already loaded from the table of the BookingWindow the ReferenceId.
In your case, this ReferenceID is represented by virtual, readonly 'current item ID'. The ID which definetly exists... but the reference does not! We've selected only BookingWindows which has NULL instead of the reference.
But we do have NOT NULL Reference ID (representd by Current instance ID).
And we've used .Left.JoinQueryOver. So NHibernate is sure, that it already loaded all data in the first query... but is confused, because in his session is no OrderItem with the id equal to BookingWindow.ID/ReferenceId
That's the reason (why it tries to fix it... and does load it again)
So this is the answer, why NHibernate does "weird selects". Not a suggestion how to fix it ;) it could be another question and answer...

Related

Mapping an extension table that might not have a row

I'm working with Fluent nHibernate on a legacy database and have a main Person table and several extension tables containing additional information about the person. These extension tables are one-to-one, meaning that a person will only have one row on the extension table and the extension table should always map back to one person.
Table: Person
Columns: PersonID, FirstName, LastName, etc.
Table: PersonLogin
Columns: PersonID (FK, unique), UserName, Password, etc.
I have my mappings defined as this (with the irrelevant properties omitted):
public PersonMap()
{
Table("Person");
Id(x => x.Id, "PersonID").Not.Nullable();
References(x => x.Login, "PersonID").LazyLoad();
}
public LoginMap()
{
Table("PersonLogin");
Id(x => x.Id, "PersonID").GeneratedBy.Foreign("Person");
References(x => x.Person, "PersonID").LazyLoad();
}
This works when I have data on both tables, but I recently learned that some of the extension tables don't have data for all Person rows. This caused me to get errors during the query. So, I added .NotFound.Ignore() to my PersonMap making it look like this:
References(x => x.Login, "PersonID").LazyLoad().NotFound.Ignore();
That caused me to get unnecessary selects from the Login table due to https://nhibernate.jira.com/browse/NH-1001 when my business layer doesn't need to project any of the extension table values. It is causing the performance to be terrible in some of my search queries.
I've scoured a lot of posts, but haven't found a rock solid answer about how to address this scenario. Below are the options I've tried:
Option One:
Create rows on the extension table to ensure there is no Person without a row on the extension table and then remove the .NotFound.Ignore().
The issue with this option is that it's a legacy database and I'm not sure where I'd need to update to ensure that a PersonLogin is inserted when a Person is inserted.
Option Two:
Remove the PersonLogin reference from my PersonMap and custom load it inside my Person class. Like this:
public class Person
{
/// <summary> Gets or sets the PersonID </summary>
public virtual int Id { get; set; }
private bool loadedLogin;
private PersonLogin login;
public virtual PersonLogin Login
{
get
{
if (!loadedLogin)
{
login = SessionManager.Session().Get<PersonLogin>(Id);
loadedLogin = true;
}
return login;
}
set
{
login = value;
loadedLogin = true;
}
}
}
The issue I'm having with it is that I can't eagerly fetch the data when performing a query to pull back a large number of Person objects and their Logins.
Option Three:
I just started playing to see if I could write a custom IEntityNotFoundDelegate to not throw the exception for these objects.
private class CustomEntityNotFoundDelegate : IEntityNotFoundDelegate
{
public void HandleEntityNotFound(string entityName, object id)
{
if (entityName == "my.namespace.PersonLogin")
{
return;
}
else
{
throw new ObjectNotFoundException(id, entityName);
}
}
}
And I added this to the config
cfg.EntityNotFoundDelegate = new CustomEntityNotFoundDelegate();
It catches my scenario and returns back now instead of throwing the error, but now when I try to project those PersonLogin properties onto my business objects, it's attempting to use the Proxy object and throws this error that I'm trying to figure out if I can handle cleanly (possibly in a IPostLoadEventListener).
System.Reflection.TargetException occurred
Message = Non-static method requires a target
I think I've got this working now by keeping the .NotFound.Ignore().
I originally stated:
That caused me to get unnecessary selects from the Login table due to https://nhibernate.jira.com/browse/NH-1001 when my business layer doesn't need to project any of the extension table values. It is causing the performance to be terrible in some of my search queries.
I was able to tweak my LINQ queries to use the IQueryOver in some instances and to improve my use of LINQ in other scenarios to project only the necessary values. This appears to have resolved the queries from pulling back the extension tables since their values were not needed in the projections.
I thought that my queries weren't projecting these extension tables, but figured out that I had a method ToKeyValuePair that I was using in the projection to concatenate the ID and a Name field together of some related properties. That method was causing the objects to load completely since LINQ wasn't able to determine that the needed fields were present without joining to the extension table.

Invalid index N for this SqlParameterCollection with Count=N only when associated table has null record

I have a rather complex entity which will not save when a particular database table is missing a record. When the record exists the entity saves correctly. When the record does not I receive the exception:
Invalid index N for this SqlParameterCollection with Count=N
After reading a bunch of solutions found via Google and the most closely related questions on Stack Overflow:
What's causing “Invalid index nn for this SqlParameterCollection
with Count=nn” when a column is Null in the database?
Invalid Index n for this SqlParameterCollection with Count=n” OR “foreign key
cannot be null
I believe my issue has to do with the way I have my mapping files setup. The Customer entity has reference to the Person entity. Person maps to a table which we have read, but not write access to. It is when a record for the Person entity does not exist that I generate the exception. If the record exists no issue. I've set the reference to Person from customer to Nullable(). I have also double checked to ensure I do not have a property mapped twice from either entity.
Here is what I feel is the pertinent mapping information, but can provide more as needed:
Customer
//more mapping code...
References(x => x.Person, "snl_id").Nullable();
//more mapping code...
Person
//more mapping code...
ReadOnly();
Id(x => x.SnlId).Column("SNL_ID");
//more mapping code...
To further complicate matters we have some painful code to make NHibernate perform better when Person does not exist. I am not sure it applies here, but thought it pertinent enough to include in my question. We are using the below code because without it the NHibernate JIRA will create tons of queries. This solution is outlined in this Stack Overflow answer.
Customer's person property
public virtual Person Person
{
get
{
try
{
var snlId = per.Name;
return per;
}
catch
{
return null;
}
}
set
{
per = value;
}
}
private EPerson per;
What am I missing in my mappings that would cause this exception? Is there another piece of this problem that I am not seeing?
While Scott's solution of removing the snl_id property from the Customer class fixes the issue it causes problems that I cannot get around-- the snl_id can exist in the Customer table even there is not a corresponding Person table record. Since that is the case there are times when I will need access to the snl_id when I cannot get to it via the associated Person property.
I considered several alternative solutions but settled on creating a view of the Customer table including the Customer table primary key and the snl_id from the customer table. Then mapping that property via a join to the view.
Join("v_cust_id_snl_id", j => j.KeyColumn("cust_id").Map(x => x.SnlId, "snl_id")
This change allowed me to have my cake and eat it to. I was able to keep the SnlId property on customer, but no longer throw the exception when saving.
Do you have the snl_id referenced as a property in Customer as well as being the primary key for the child object? If so, this is causing the error you are receiving. Remove the property from Customer and use Person to get the value.

NHibernate ReferencesAny pulling back the wrong type

I've got a table called AdministratorPrivilages that has the following fields:
ID
List item
MemberId
Value
MemberType
Now, the members can be of two types (Enterprise and Express). Enterprise members live in the enterprise table. Express members live in the expressmember table. I've tried to do my fluent mapping like so.
public class AdministratorPrivilegesMapping : ClassMap<AdministratorPrivileges>
{
public AdministratorPrivilegesMapping()
{
Id(x=>x.Id);
Map(x => x.Value).Column("Value");
ReferencesAny(x => x.Member)
.EntityTypeColumn("MemberType")
.EntityIdentifierColumn("MemberId")
.IdentityType<Int32>()
.AddMetaValue<ExpressMember>("Express")
.AddMetaValue<Member>("Enterprise");
}
}
Both member tables have integer ids with ascending values. When I try to pull back the privilages associated with enterprise member 10, I'm getting the permission set associated with Express Member 10. Both other tables are mapped with the old school hbm mapping files.
Am I missing something obvious? I'm using NHibernate 2.1 and FluentNhibernate 1.1
I actually found the solution using .Where().
My situation is a little different, I have objectA and objectB.
objectA contains objectB, but objectB also contains a collection of objectBs.
"objectA":{
"someProp" : "(string)",
"objectB":{
"someProp" : "(string)",
"comeCol" : [
"(objectB)"
]
}
}
So the "Parent" property of objectB can either be of type objectB or objectA, which is why I needed to use ReferencesAny in the mapping of objectB.
Mapping looks like
ReferencesAny( x => x.Parent )
.IdentityType< int >()
.MetaType< string >()
.EntityTypeColumn( "ParentType" )
.EntityIdentifierColumn( "ParentId" )
.AddMetaValue< objectA >( "E" )
.AddMetaValue< objectB >( "S" )
.Access.Property()
.LazyLoad()
.Cascade.All();
All this works well when saving, however, my problem occured when retrieving, because the framework wasn't told what to retrieve and simply retrieved everything.
So now here is the mapping of the collection that fixed the problem:
HasMany( x => x.objectBs )
.KeyColumn( "ParentId" )
.Where( "ParentType = 'S'" )
.Cascade.All()
.LazyLoad();
Hope it will help anyone in the same situation :)
As seems to always be the case when I post on Stackoverflow, I'm being a complete goober and missing something glaringly obvious that cleared itself up with a good night's rest and some caffeine. I needed to look into the mappings for the ExpressMember and Member classes. Turns out the bag declarations there didn't filter by the type of object appropriately. Initially, I thought I had made the breaking change, when in fact it was actually a very old issue. The collision in id numbers between the two different types of members was not an issue for a very long time, as express members typically have either all the permissions or none, as do most of the older members (which were created first under an admin/plebe scheme with privileges being broken out later).

Simple relation between two tables

I started using NHibernate today, but I cannot figure out how I setup a simple relation between two tables. I don't really know what it's called, it could be one-to-many or foreign key relation (I'm not that into database design and the terms used), but here's a very simple example.
I have a table Product with attributes Id (PK), ProductName and CategoryId. Then I have a table Categories with attributes Id (PK) and CategoryName.
I created these classes:
public class Product
{
public virtual int Id { get; set; }
public virtual string ProductName { get; set; }
public virtual int CategoryId { get; set; }
public virtual Category Category { get; set; }
public virtual string CategoryName
{
get { return this.Category == null ? String.Empty : this.Category.CategoryName; }
}
}
public class Category
{
public virtual int Id { get; set; }
public virtual string CategoryName { get; set; }
}
In other words, I simply want the Product to store to which category it belongs (via the CategoryId attribute which points to an Id in the Categories table). I don't need the Category class to hold a list of related Products, if that makes it any simpler.
To make it even more clear what I'm after, this is the SQL that I'm expecting:
SELECT Products.*, Categories.*
FROM Products INNER JOIN Categories ON Products.CategoryId = Categories.Id
at least that's what I think it should be (again, I'm not that good at database design or queries).
I can't figure out which kind of mapping I need for this. I suppose I need to map it in the Product.hbm.xml file. But do I map the CategoryId as well? And how do I map the Category property?
It seems like I would need a 'one-to-many' relation since I have ONE category per product (or is this reasoning backward?) but it seems like there is no one-to-many mapping...
Thanks for any help!
Addition:
I tried to add the many-to-one relation in the Person mapping, but I keep getting an exception saying "Creating proxy failed", and in the inner exception "Ambiguous match found".
I should maybe mention I am using an old version of NHibernate (1.2 I think) because that is the only one I got running with MS Access due to it not finding the JetDriver in newer versions.
I've put the mapping files, classes, and code where the error occurs in screenshots because I can't figure out how to post XML code here... It keeps reading it as html tags and skipping half of it. Anyway.
The mappings:
http://www.nickthissen.nl/Images/tmp7B5A.png
The classes:
http://www.nickthissen.nl/Images/tmpF809.png
The loading code where the error occurs:
http://www.nickthissen.nl/Images/tmp46B6.png
(As I said, the inner exception says "Ambiguous match found".
(Product in my example has been replaced by Person)
The Person and Category classes inherit Entity which is an abstract base class and defines the Id, Deleted, CreatedTime and UpdatedTime properties.
The code where the error occurs is in a generic 'manager' class (type parameter TEntity which must inherit Entity). It is simply supposed to load all entities with the Deleted attribute false. In this case, TEntity is 'Person'.
It works fine if I leave out the many-to-one Category mapping in the Person mapping, but then obviously the Category property is always null.
Oh yeah, sorry about the mix between C# and VB, the C# code is in a generic framework I use for multiple projects while the VB part is the actual implementation of that framework on my website and I just happened to use VB for that.
Help? Thanks!
In your Product class only needs to contain a Category object, you don't need a CategoryId property. Then in your Product mapping you need to have this entry
<many-to-one name="Category" column="CategoryId" />
UPDATE:
Your mappings appear to be missing the fully qualified name of the mapped class in the tag. See http://nhibernate.info/doc/nh/en/index.html#mapping-declaration-class
UPDATE 2:
See if this helps you NHibernate 1.2 in a .NET 4.0 solution
The 'Ambiguous match found' exception was caused by the project targeting .NET Framework 4, which does not seem to be compatible with NHibernate 1.2.1. I switched to 3.5 and that seems to solve that particular issue.
Now on to the next. As you can see, the Person class has a CategoryName property that should return the name of the current Category object, or an empty string if the category happens to be null. This is so I can databind a collection of Person objects to a grid, specifying 'CategoryName' as a property to bind a column to.
Apparently this does not work with NHibernate. Whenever I try to databind my collection of persons, I get this exception:
"Property accessor 'CategoryName' on object 'NHibernateWebTest.Database.Person' threw the following exception:'Could not initialize proxy - the owning Session was closed.'"
This occurs on the 'DataBind' method call in this code:
public virtual void LoadGrid()
{
if (this.Grid == null) return;
this.Grid.DataSource = this.Manager.Load();
this.Grid.DataBind();
}
(This is an ASP.NET project and 'Grid' is a GridView)
'this.Manager' returns an existing instance of NHibernateEntityManager, and I've already shown its Load method before, it contains this:
public virtual EntityCollection Load()
{
using (ISession session = this.GetSession())
{
var entities = session
.CreateCriteria(typeof (TEntity))
.Add(Expression.Eq("Deleted", false))
.List();
return new EntityCollection(entities);
}
}
(THere's some generic type parameters in there but this website seems to hide them (due to the html like tags I guess)... Sorry about that).
This may have something to do with NHibernate itself, as I said I'm completely new to this. When I call my Load method I would expect it to return an EntityCollection(Of Person) with all its properties already set. It seems I have to keep the ISession open while I am databinding for some reason..? That seems a little strange...
Can I get around this? Can I make my Load method simply return a collection of persons already fully loaded, so that I can access CategoryName whenever I want?
Wait... Is this lazy loading perhaps?

nhibernate - sproutcore : How to only retrieve reference ID's and not load the reference/relation?

I use as a front-end sproutcore, and as back-end an nhibernate driven openrasta REST solution.
In sproutcore, references are actualy ID's / guid's. So an Address entity in the Sproutcore model could be:
// sproutcore code
App.Address = App.Base.extend(
street: SC.Record.attr(String, { defaultValue: "" }),
houseNumber: SC.Record.attr(String),
city: SC.Record.toOne('Funda.City')
);
with test data:
Funda.Address.FIXTURES = [
{ guid: "1",
street: "MyHomeStreet",
houseNumber: "34",
city: "6"
}
]
Here you see that the reference city has a value of 6. When, at some point in your program, you want to use that reference, it is done by:
myAddress.Get("city").MyCityName
So, Sproutcore automatically uses the supplied ID in a REST Get, and retrieves the needed record. If the record is available in de local memory of the client (previously loaded), then no round trip is made to the server, otherwise a http get is done for that ID : "http://servername/city/6". Very nice.
Nhibernate (mapped using fluent-nhibernate):
public AddressMap()
{
Schema(Config.ConfigElement("nh_default_schema", "Funda"));
Not.LazyLoad();
//Cache.ReadWrite();
Id(x => x.guid).Unique().GeneratedBy.Identity();
Table("Address");
Map(x => x.street);
Map(x => x.houseNumber);
References(x => x.city,
"cityID").LazyLoad().ForeignKey("fk_Address_cityID_City_guid");
}
Here i specified the foreign key, and to map "cityID" on the database table. It works ok.
BUT (and these are my questions for the guru's):
You can specify to lazy load / eager load a reference (city). Off course you do not want to eager load all your references. SO generally your tied to lazy loading.
But when Openrast (or WCF or ...) serializes such an object, it iterates the properties, which causes all the get's of the properties to be fired, which causes all of the references to be lazy loaded.
SO if your entity has 5 references, 1 query for the base object, and 5 for the references will be done. You might better be off with eager loading then ....
This sucks... Or am i wrong?
As i showed how the model inside sproutcore works, i only want the ID's of the references. So i Don't want eagerloading, and also not lazy loading.
just a "Get * from Address where ID = %" and get that mapped to my Address entity.
THen i also have the ID's of the references which pleases Sproutcore and me (no loading of unneeded references). But.... can NHibernate map the ID's of the references only?
And can i later indicate nHibernate to fully load the reference?
One approach could be (but is not a nice one) to load all reference EAGER (with join) (what a waste of resources.. i know) and in my Sever-side Address entity:
// Note: NOT mapped as Datamember, is NOT serialized!
public virtual City city { get; set; }
Int32 _cityID;
[Datamember]
public virtual Int32 cityID
{
get
{
if (city != null)
return city .guid;
else
return _cityID;
}
set
{
if (city!= null && city.guid != value)
{
city= null;
_cityID = value;
}
else if (city == null)
{
_cityID = value;
}
}
}
So i get my ID property for Sproutcore, but on the downside all references are loaded.
A better idea for me???
nHibernate-to-linq
3a. I want to get my address without their references (but preferably with their id's)
Dao myDao = new Dao();
from p in myDao.All()
select p;
If cities are lazy loading in my mapping, how can i specify in the linq query that i want it also to include my city id only?
3b.
I want to get addresses with my cities loaded in 1 query: (which are mapped as lazyloaded)
Dao myDao = new Dao();
from p in myDao.All()
join p.city ???????
select p;
My Main Question:
As argued earlier, with lazy loading, all references are lazy loaded when serializing entities. How can I prevent this, and only get ID's of references in a more efficient way?
Thank you very much for reading, and hopefully you can help me and others with the same questions. Kind regards.
as a note you wrote you do this
myAddress.Get("city").MyCityName
when it should be
myAddress.get("city").get("MyCityName")
or
myAddress.getPath("city.MyCityName")
With that out of the way, I think your question is "How do I not load the city object until I want to?".
Assuming you are using datasources, you need to manage in your datasource when you request the city object. So in retrieveRecord in your datasource simply don't fire the request, and call dataSourceDidComplete with the appropriate arguments (look in the datasource.js file) so the city record is not in the BUSY state. You are basically telling the store the record was loaded, but you pass an empty hash, so the record has no data.
Of course the problem with this is at some point you will need to retrieve the record. You could define a global like App.WANTS_CITY and in retrieveRecords only do the retrieve when you want the city. You need to manage the value of that trigger; statecharts are a good place to do this.
Another part of your question was "How do I load a bunch of records at once, instead of one request for each record?"
Note on the datasource there is a method retrieveRecords. You can define your own implementation to this method, which would allow you to fetch any records you want -- that avoids N requests for N child records -- you can do them all in one request.
Finally, personally, I tend to write an API layer with methods like
getAddress
and
getCity
and invoke my API appropriately, when I actually want the objects. Part of this approach is I have a very light datasource -- I basically bail out of all the create/update/fetch methods depending on what my API layer handles. I use the pushRetrieve and related methods to update the store.
I do this because the store uses in datasources in a very rigid way. I like more flexibility; not all server APIs work in the same way.