I was just wondering whether when loading an entity which contains a collection e.g. a Post which may contain 0 -> n Comments if you can define how many comments to return.
At the moment I have this:
public IList<Post> GetNPostsWithNCommentsAndCreator(int numOfPosts, int numOfComments)
{
var posts = Session.Query<Post>().OrderByDescending(x => x.CreationDateTime)
.Take(numOfPosts)
.Fetch(z => z.Comments)
.Fetch(z => z.Creator).ToList();
ReleaseCurrentSession();
return posts;
}
Is there a way of adding a Skip and Take to Comments to allow a kind of paging functionality on the collection so you don't end up loading lots of things you don't need.
I'm aware of lazy loading but I don't really want to use it, I'm using the MVC pattern and want my object to return from the repositories loaded so I can then cache them. I don't really want my views causing select statements.
Is the only real way around this is to not perform a fetch on comments but to perform a separate Select on Comments to Order By Created Date Time and then Select the top 5 for example and then place the returned result into the Post object?
Any thoughts / links on this would be appreciated.
Thanks,
Jon
A fetch simple does a left-outer join on the associated table so that it can hydrate the collection entities with data. What you are looking to do will require a separate query on the specific entities. From there you can use any number of constructs to limit your result set (skip/take, setmaxresults, etc)
Related
My SQL and Entity Framework knowledge is a somewhat limited. In one Entity Framework (4) application, I notice it takes forever (about 2 minutes) to complete one of my method calls. The first queries do not take much time, but when I loop through the Entity Framework objects returned by the queries, even though I am only reading (not modifying) the data I supposedly got, it takes forever to complete the nested loops, even though there are only dozens of entries in each list and a few levels of looping.
I expect the example below could be re-written with a fancier query that could probably include all of the filtering I am doing in my loops with some SQL words I don't really know how to use, so if someone could show me what the equivalent SQL expression would be, that would be extremely educational to me and probably solve my current performance problem.
Moreover, since other parts of this and other applications I develop often want to do more complex computations on SQL data, I would also like to know a good way to retrieve data from Entity Framework to local memory objects that do not have huge delays in reading them. In my LINQ-to-SQL project there was a similar performance problem, and I solved it by refactoring the whole application to load all SQL data into parallel objects in RAM, which I had to write myself, and I wonder if there isn't a better way to either tell Entity Framework to not keep doing whatever high-latency communication it is doing, or to load into local RAM objects.
In the example below, the code gets a list of food menu items for a member (i.e. a person) on a certain date via a SQL query, and then I use other queries and loops to filter out the menu items on two criteria: 1) If the member has a rating of zero for any group id which the recipe is a member of (a many-to-many relationship) and 2) If the member has a rating of zero for the recipe itself.
Example:
List<PFW_Member_MenuItem> MemberMenuForCookDate =
(from item in _myPfwEntities.PFW_Member_MenuItem
where item.MemberID == forMemberId
where item.CookDate == onCookDate
select item).ToList();
// Now filter out recipes in recipe groups rated zero by the member:
List<PFW_Member_Rating_RecipeGroup> ExcludedGroups =
(from grpRating in _myPfwEntities.PFW_Member_Rating_RecipeGroup
where grpRating.MemberID == forMemberId
where grpRating.Rating == 0
select grpRating).ToList();
foreach (PFW_Member_Rating_RecipeGroup grpToExclude in ExcludedGroups)
{
List<PFW_Member_MenuItem> rcpsToRemove = new List<PFW_Member_MenuItem>();
foreach (PFW_Member_MenuItem rcpOnMenu in MemberMenuForCookDate)
{
PFW_Recipe rcp = GetRecipeById(rcpOnMenu.RecipeID);
foreach (PFW_RecipeGroup group in rcp.PFW_RecipeGroup)
{
if (group.RecipeGroupID == grpToExclude.RecipeGroupID)
{
rcpsToRemove.Add(rcpOnMenu);
break;
}
}
}
foreach (PFW_Member_MenuItem rcpToRemove in rcpsToRemove)
MemberMenuForCookDate.Remove(rcpToRemove);
}
// Now filter out recipes rated zero by the member:
List<PFW_Member_Rating_Recipe> ExcludedRecipes =
(from rcpRating in _myPfwEntities.PFW_Member_Rating_Recipe
where rcpRating.MemberID == forMemberId
where rcpRating.Rating == 0
select rcpRating).ToList();
foreach (PFW_Member_Rating_Recipe rcpToExclude in ExcludedRecipes)
{
List<PFW_Member_MenuItem> rcpsToRemove = new List<PFW_Member_MenuItem>();
foreach (PFW_Member_MenuItem rcpOnMenu in MemberMenuForCookDate)
{
if (rcpOnMenu.RecipeID == rcpToExclude.RecipeID)
rcpsToRemove.Add(rcpOnMenu);
}
foreach (PFW_Member_MenuItem rcpToRemove in rcpsToRemove)
MemberMenuForCookDate.Remove(rcpToRemove);
}
You can use EFProf http://www.hibernatingrhinos.com/products/EFProf to track see exactly what EF is sending to SQL. It can also show you how many queries you are sending and how many unique queries. It also provides you some analysis of each query (e.g. is it unbound etc). Entity Framework with its navigation properties, it is quite easy to not realize you are making a db request. When you are in a loop, and have a navigation property, you get in to the N + 1 problem.
You could use the Keyword Virtual on your List parts of your model if you are using code first to enable proxying, that way you will not have to get all the data back at once, only as you need it.
Also consider NoTracking for read only data
context.bigTable.MergeOption = MergeOption.NoTracking;
A small briefing on what I am trying to do.
I have three tables Content(contentId, body, timeofcreation), ContentAttachmentMap(contentId, attachmentId) and Attachment(attachmentId, resourceLocation).
The reason I adopted to create the mapping table because in future application the attachment can also be shared with different content.
Now I am using HQL to get data. My objectives is as follows:
Get All contents with/without Attachments
I have seen some examples in the internet like you can create an objective specific class (not POJO) and put the attribute name from the select statement within its constructor and the List of that Class object is returned.
For e.g. the HQL will be SELECT new com.mydomain.myclass(cont.id, cont.body) ..... and so on.
In my case I am looking for the following SELECT new com.mydomain.contentClass(cont.id, cont.body, List<Attachment>) FROM ...`. Yes, I want to have the resultList contain contentid, contentbody and List of its Attachments as a single result List item. If there are no attachments then it will return (cont.id, contentbody, null).
Is this possible? Also tell me how to write the SQL statements.
Thanks in advance.
I feel you are using Hibernate in a fundamentally wrong way. You should use Hibernate to view your domain entity, not to use it as exposing the underlying table.
You don't need to have that contentClass special value object for all these. Simply selecting the Content entity serves what you need.
I think it will be easier to have actual example.
In your application, you are not seeing it as "3 tables", you should see it as 2 entities, which is something look like:
#Entity
public class Content {
#Id
Long id;
#Column(...)
String content;
#ManyToMany
#JoinTable(name="ContentAttachmentMap")
List<Attachment> attachments;
}
#Entity
public class Attachment {
#Id
Long id;
#Column(...)
String resourceLocation
}
And, the result you are looking for is simply the result of HQL of something like
from Content where attachments IS EMPTY
I believe you can join fetch too in order to save DB access:
from Content c left join fetch c.attachments where c.attachments IS EMPTY
I would like to return a DTO from my data layer which would also contain child collections...such as this:
Audio
- Title
- Description
- Filename
- Tags
- TagName
- Comments
- PersonName
- CommentText
Here is a basic query so far, but i'm not sure how to transform the child collections from my entity to the DTO.
var query = Session.CreateCriteria<Audio>("audio")
.SetProjection(
Projections.ProjectionList()
.Add(Projections.Property<Audio>(x => x.Title))
.Add(Projections.Property<Audio>(x => x.Description))
.Add(Projections.Property<Audio>(x => x.Filename))
).SetResultTransformer(new AliasToBeanResultTransformer(typeof(AudioDto)))
.List<AudioDto>();
Is this even possible, or is there another reccomended way of doing this?
UPDATE:
Just want to add a little more information about my scenario...I want to return a list of Audio items to the currently logged in user along with some associated entities such as tags, comments etc...these are fairly straight forward using MultiQuery / Future.
However, when displaying the audio items to the user, i also want to display 3 other options to the user:
Weather they have added this audio item to their list of favourites
Weather they have given this audio the 'thumbs up'
Weather the logged in user is 'Following' the owner of this audio
Favourites : Audio -> HasMany -> AudioUserFavourites
Thumbs Up : Audio -> HasManyToMany -> UserAccount
Following Owner : Audio -> References -> UserAccount ->
ManyToMany -> UserAccount
Hope this makes sense...if not i'll try and explain again...how can I eager load these extra details for each Audio entity returned...I need all this information in pages of 20 also.
I looked at Batch fetching, but this appears to fetch ALL thumbs ups for each Audio entity, rather than checking if only the logged in user has thumbed it.
Sorry for rambling :-)
Paul
If you want to fetch your Audio objects with both the Tags collection and Comments collections populated, have a look at Aydende Rahien's blog: http://ayende.com/blog/4367/eagerly-loading-entity-associations-efficiently-with-nhibernate.
You don't need to use DTOs for this; you can get back a list of Audio with its collections even if the collections are lazily loaded by default. You would create two future queries; the first will fetch Audio joined to Tags, and the second will fetch Audio joined to Comments. It works because by the time the second query result is being processed, the session cache already has the Audio objects in it; NHibernate grabs the Audio from the cache instead of rehydrating it, and then fills in the second collection.
You don't need to use future queries for this; it still works if you just execute the two queries sequentially, but using futures will result in just one round trip to the database, making it faster.
I've just checked the man page of CDbCriteria, but there is not enough info about it.
This property is available since v1.1.7 and I couldn't find any help for it.
Is it for dynamically changing Model->scopes "on-the-fly"?
Scopes are an easy way to create simple filters by default. With a scope you can sort your results by specific columns automatically, limit the results, apply conditions, etc. In the links provided by #ldg there's a big example of how cool they are:
$posts=Post::model()->published()->recently()->findAll();
Somebody is retrieving all the recently published posts in one single line. They are easier to maintain than inline conditions (for example Post::model()->findAll('status=1')) and are encapsulated inside each model, which means big transparency and ease of use.
Plus, you can create your own parameter based scopes like this:
public function last($amount)
{
$this->getDbCriteria()->mergeWith(array(
'order' => 't.create_time DESC',
'limit' => $amount,
));
return $this;
}
Adding something like this into a Model will let you choose the amount of objects you want to retrieve from the database (sorted by its create time).
By returning the object itself you allow method chaining.
Here's an example:
$last3posts=Post::model()->last(3)->findAll();
Gets the last 3 items. Of course you can expand the example to almost any property in the database. Cheers
Yes, scopes can be used to change the attributes of CDbCriteria with pre-built conditions and can also be passed parameters. Before 1.1.7 you could use them in a model() query and can be chained together. See:
http://www.yiiframework.com/doc/guide/1.1/en/database.ar#named-scopes
Since 1.1.7, you can also use scopes as a CDbCriteria property.
See: http://www.yiiframework.com/doc/guide/1.1/en/database.arr#relational-query-with-named-scopes
Starting with a List of entities and needing all dependent entities through an association, is there a way to use the corresponding navigation-propertiy to load all child-entities with one db-round-trip? Ie. generate a single WHERE fkId IN (...) statement via navigation property?
More details
I've found these ways to load the children:
Keep the set of parent-entities as IQueriable<T>
Not good since the db will have to find the main set every time and join to get the requested data.
Put the parent-objects into an array or list, then get related data through navigation properties.
var children = parentArray.Select(p => p.Children).Distinct()
This is slow since it will generate a select for every main-entity.
Creates duplicate objects since each set of children is created independetly.
Put the foreign keys from the main entities into an array then filter the entire dependent-ObjectSet
var foreignKeyIds = parentArray.Select(p => p.Id).ToArray();
var children = Children.Where(d => foreignKeyIds.Contains(d.Id))
Linq then generates the desired "WHERE foreignKeyId IN (...)"-clause.
This is fast but only possible for 1:*-relations since linking-tables are mapped away.
Removes the readablity advantage of EF by using Ids after all
The navigation-properties of type EntityCollection<T> are not populated
Eager loading though the .Include()-methods, included for completeness (asking for lazy-loading)
Alledgedly joins everything included together and returns one giant flat result.
Have to decide up front which data to use
It there some way to get the simplicity of 2 with the performance of 3?
You could attach the parent object to your context and get the children when needed.
foreach (T parent in parents) {
_context.Attach(parent);
}
var children = parents.Select(p => p.Children);
Edit: for attaching multiple, just iterate.
I think finding a good answer is not possible or at least not worth the trouble. Instead a micro ORM like Dapper give the big benefit of removing the need to map between sql-columns and object-properties and does it without the need to create a model first. Also one simply writes the desired sql instead of understanding what linq to write to have it generated. IQueryable<T> will be missed though.