AutoMapper is great, saves a lot of time, but when I started looking at the performance of my application AutoMapper is responsible for performance loss.
I'm using lazy loading with NHibernate. Most of the time a need parent entity without needing to access child entities at all. In reality what happens is that AutoMapper tries to map as many relationships as possible causing NHibernate to lazy load all the child entities (I'm seeing SELECT N+1 happening all the time).
Is there way to limit how deep AutoMapper goes or is it possible for AutoMapper to map child entities lazily?
You could use the ignore method for associations you don't need to have loaded.
Mapper.CreateMap<User, UserDto>()
.ForMember(dest => dest.LazyCollection, opt => opt.Ignore())
.ForMember(dest => dest.AnotherLazyCollection, opt => opt.Ignore())
Mapper.CreateMap<UserProperty, UserPropertyDto>()
.ForMember(dest => dest.PropertyLazyReference, opt => opt.Ignore());
return Mapper.Map<User, UserDto>(user);
For associations you know you will need in your dto, you should look at ways of fetching these more efficiently with the initial query, but that is a whole new problem.
Perhaps you should consider using two different dtos; one that includes the child entities, and one that doesn't. You can then return the proper dto from your service layer depending upon the context.
I'm using pre conditions to prevent data from being mapped.
CreateMap<Team, TeamDto>()
.ForMember(dto => dto.Users, options =>
{
options.PreCondition(ctx => !ctx.Items.ContainsKey(AutoMapperItemKeys.SKIP_TEAM_USERS));
options.MapFrom(t => t.TeamUsers.Where(tu => tu.IsDeleted == false));
})
.ReverseMap();
When Map() is called I feed the Items dictionary with skip keys for the properties I don't want mapped.
this.mapper.Map<IEnumerable<Team>, IEnumerable<TeamDto>>(teams, opts =>
{
opts.Items.Add(AutoMapperItemKeys.SKIP_TEAM_USERS, true);
});
Advantages:
you can fine-grain which properties not to map
prevents from mapping to deep with nested objects
no need for duplicate dto's
no duplicate mapping profiles
Related
When querying nhibernate, I'm seeing some odd behavior
When I write a query like this
Repository.QueryOver<Entity>()
.Fetch(x => x.Child1).Eager
.Fetch(x => x.child2).Eager
It will eagerly grab child1 and child2 entities, but there are grandchildren for child1 and child2 that aren't lazily loaded. I'm a bit confused on how to achieve this.
In my nhibernate mappings, it seems to have no affect on the laziness or eagerness of grandchildren and I require at least some entities be eagerly loaded to avoid the N+1 query problem.
I'm also wondering how I could eagerly load grandchildren entities under my original entity.
Any help or ideas are appreciated!
I would suggest to use the batch-fetching. As discussed here, the fluent syntax is:
1) the collection setting
HasMany<MyEntity>(x => x.Entities)
.BatchSize(100);
2) the class level setting
public MyEntityMap()
{
Id(x => x....
...
BatchSize(100);
This setting should be applied on every collection and every class. To do that with fluent - we can use Conventions - see more e.g. here
'IClassConvention' - Use to alter your ClassMaps values. You can't change properties, collections, etc with this convention, only alter the settings such as Lazy and BatchSize.
I have an entity with a binary column that's set to lazy load in the mapping. In some cases, though, we want to get the entity along with the binary data at the same time. I tried using Linq.Fetch(x => x.BinaryData), but that gives an invalid join exception. Understandable, considering it shouldn't be a join in the first place. Is there a way to get this working? I'm using NHibernate 3.1
This is the mapping:
Map(x => x.BinaryData)
.CustomSqlType("image")
.Length(int.MaxValue)
.Not.Nullable()
.LazyLoad(); // Wanna make sure we don't kill the app by loading the image data when we don't need it.
This is the fetching:
Linq.Where(x => x.Id == id).Fetch(x => x.BinaryData).FirstOrDefault();
This looks like to be not possible at the moment : https://nhibernate.jira.com/browse/NH-2888
So, You have to use HQL :
var post = session.CreateQuery("from Post fetch all properties")
.SetMaxResults(1)
.UniqueResult<Post>();
Source : http://ayende.com/blog/4377/nhibernate-new-feature-lazy-properties
In HQL you can use fetch all properties to eagerly load lazy property. But in NH3.1 it is not yet implemented for Linq queries. As I know this bug is in NHibernate Jira so you can check if it is resolved or you can fix it yourself. For our company prototype i fixed this bug, but I did so in very brute-force way so i didn't send patch to NHibernate project
I am trying to use NHibernate to map a graph built with the following entites to the relational database (The code is incomplete and just for demo purpose, which should look straightforward). Potentially Node and Edge classes may have subclases and there are already a string of subclasses defined inherited from Node class. For all the inheritance relationships in this model, the mapping type used is joined-subclass (table-per-subclass);
class GraphObject { ... }
class Node : GraphObject {
List<Edge> IngoingEdges;
List<Edge> OutgoingEdges;
}
class Edge : GraphObject {
Node StartNode { get; set; }
Node EndNode { get; set; }
}
For connections between nodes and edges a dual many-to-one mapping is used as follows,
many-to-one from Edge.StartNode to nodes(.OutgoingEdges);
many-to-one from Edge.EndNode to nodes(.IngoingEdges)
Since there's a need to work with huge volume data in our project (millions of nodes and edges) and we would like to both keep having the benefits NHibernate provides and minimize the performance issues. Unfortunately it seems to take nearly an hour to save or load such a model. What I'm currently doing is trying to figure out a way to finish loading in one statement and see how long it takes. I made a few attempts and I used NHibernate Profiler to keep track of the SQL statements generated by the NHibernate framework when doing things like loading the entire graph from the data persistence, but so far I haven't managed to eliminate that huge amount of individual queries apparently for determining which are the start and end nodes for specific edges which look like
select ...StartNode as .., ..Id as .., ... from Link link where link.StartNode=10 (a number indicating node id)
which means I am kind of suffering from the so-called N+1 issues.
So is there anyone wo has come across a similar problem and can give me some idea, either in native SQLs or improving performance for this particular case by other approaches. I would really appreciate that. Any questions regarding points unclear are also welcome.
some optimisations come to mind:
disable lazyloading of StartNode and EndNode (get Egde in 1 Query instead 3)
EagerLoad the collections of an edge (http://ayende.com/blog/4367/eagerly-loading-entity-associations-efficiently-with-nhibernate)
this would give something along the lines of
// initialize the collections efficiently
session.QueryOver<Node>()
.Where(n => n.Id == nodeId)
.Fetch(n => n.IngoingEdges)
.ToFuture();
firstNode = session.QueryOver<Node>()
.Where(n => n.Id == nodeId)
.Fetch(n => n.OutgoingEdges)
.ToFuture().Value;
var egdeIds = firstNode
.SelectMany(n => n.IngoingEdges)
.SelectMany(edge => new [] { edge.StartNode.Id, edge.EndNode.Id });
EagerLoadNode(nodeIds);
void EagerLoadNode(IEnumerable<int> nodeIds)
{
// initialize the collections efficiently
session.QueryOver<Node>()
.Where(n => n.Id.IsIn(nodeIds))
.Fetch(n => n.IngoingEdges)
.ToFuture();
firstNode = session.QueryOver<Node>()
.Where(n => n.Id.IsIn(nodeIds))
.Fetch(n => n.OutgoingEdges)
.ToFuture();
}
I've got a pretty complex object graph that I want to load in one fell
swoop.
Samples have Daylogs which have Daylog Tests which have Daylog
Results
Daylog Tests have Testkeys, Daylog Results have Resultkeys, and
TestKeys have Resultkeys.
I'm using the QueryOver API and Future to run these all as one query,
and all the data that NHibernate should need to instantiate the entire
graph IS being returned, verfied by NHProf.
public static IList<Daylog> DatablockLoad(Isession sess,
ICollection<int> ids)
{
var daylogQuery = sess.QueryOver<Daylog>()
.WhereRestrictionOn(dl => dl.DaylogID).IsIn(ids.ToArray())
.Fetch(dl => dl.Tests).Eager
.TransformUsing(Transformers.DistinctRootEntity)
.Future<Daylog>();
sess.QueryOver<DaylogTest>()
.WhereRestrictionOn(dlt =>
dlt.Daylog.DaylogID).IsIn(ids.ToArray())
.Fetch(dlt => dlt.Results).Eager
.Inner.JoinQueryOver<TestKey>(dlt => dlt.TestKey)
.Fetch(dlt => dlt.TestKey).Eager
.Inner.JoinQueryOver<ResultKey>(tk => tk.Results)
.Fetch(dlt => dlt.TestKey.Results).Eager
.Future<DaylogTest>();
sess.QueryOver<DaylogResult>()
.Inner.JoinQueryOver(dlr => dlr.DaylogTest)
.WhereRestrictionOn(dlt =>
dlt.Daylog.DaylogID).IsIn(ids.ToArray())
.Fetch(dlr => dlr.ResultKey).Eager
.Fetch(dlr => dlr.History).Eager
.Future<DaylogResult>();
var daylogs = daylogQuery.ToList();
return daylogs;
}
However, I still end up with proxies to represent the relationship
between Testkey and ResultKey, even though I'm specifically loading
that relationship.
I think this entire query is probably representative of a poor
understanding of the QueryOver API, so I would like any and all advice
on it, but primarily, I'd like to understand why I get a proxy and not
a list of results when later I try to get
daylogresult.resultkey.testkey.results.
Any help?
The answer was to call NHibernateUtil.Initialize on the various objects. Simply pulling the data down does not mean that NHibernate will hydrate all the proxies.
You have to load all your entities in one QueryOver clause to get rid of proxies. But in this case you will have a lot of joins in your query, so I recommend to use lazy loading with batching.
I have a product table that has a many-to-many relation to itself (using a two-column many-to-many table) and I have set it up in Fluent NHibernate with the following code:
public class ProductConfiguration : ClassMap<Product>
{
public ProductConfiguration()
{
Table("Product");
Id(p => p.Id).GeneratedBy.Guid();
Map(p => p.Name).Not.Nullable().Length(254);
Map(p => p.Description).Not.Nullable().Length(1000);
Map(p => p.CreatedAt).Not.Nullable();
HasManyToMany(p => p.CrossSell)
.Table("ProductCrossSell")
.ParentKeyColumn("Id")
.ChildKeyColumn("ProductId");
}
}
My MVC application has two pages that uses this setup:
Index - Uses a generic repository GetAll method to display all products.
Detail - Uses a generic repository GetById method to display one product and any related cross sell products setup in the many-to-many realation.
It looks like NHibernate is set to LazyLoad the many-to-many by default so when I fire up the application and watch it in profiler I can see that it does LazyLoad the many-to-many with the following alert "Use of implicit transactions is discouraged".
How do I get rid of this alert? I couldn't find any information on how to wrap a LazyLoad inside a transaction to get rid the alert. Is it even possible?
Is there a way to not lazyload this by telling NHibernate that whenever I ask for GetById make sure to join the tables a get everything in one query? I tried using .Fetch.Join() in the many-to-many mapping but that also affected my GetAll query which now displays a joined result set as well which is incorrect.
What is the best apprach for this kind of simple scenario?
Thanks
The way to get rid of the warning is to access the object graph and fully populate the UI elements inside a single transaction.
Not by configuration. You can create an HQL query that eager fetches the association and use that query for a specific view. I would stick with lazy loading and not make that optimization unless needed. The HQL would be:
return session.CreateQuery("from ProductionConfiguration pc join fetch pc.CrossSell where pc.Id = ?")
.SetGuid(0, id)
.List<ProductConfiguration>();
All collections are lazily loaded in NHibernate by default.
You must be triggering loading with a call of some kind (maybe even with the debugger watches)