I'm trying to implement this thing: https://ocramius.github.io/blog/doctrine-orm-optimization-hydration
So, for instance, I have a User entity and User ManyToMany Initiatives. This way any User can participate in any Initiative.
All goes well when there are some Initiatives User already joined.
But the problem arises when the User didn't join any of the Initiatives, so the User's MtM Initiatives collection should be empty, so the Initiatives collection stays uninitialized when there are no query results.
The question is: how to initialize all user collections even when there are no query results?
Sample code:
// User initialized without relations hydration
$customer = $this->createQueryBuilder('customer')
->andWhere('customer.id=:id')
->setParameter('id', $id)
->getQuery()->getSingleResult();
// Hydrate Participation relation (User MtM Initiative), here is no results, so User->Participation is NOT initialized and empty. BUT I need it to be empty AND initialized!
$this->createQueryBuilder('customer')
->addSelect('participation')
->join('customer.participation', 'participation')
->andWhere('customer.id=:id')
->setParameter('id', $id)
->getQuery()->getResult();
// Hydration of Following (User MtM Initiative) goes well because we have some following Initiatives in DB
$this->createQueryBuilder('customer')
->addSelect('following')
->join('customer.following', 'following')
->andWhere('customer.id = :id')
->setParameter('id', $id)
->getQuery()->getResult();
So when we have nested relation collections NOT initialized, this later leads to N+1 lazy loading queries in my case when I access these not initialized collections, which I want to also avoid.
Related
I have a lot of queries in my profiler, So I try to add join() and select() in my repository for reduce the number of queries like usual
But this time, I reload the page and I have more queries than before
/**
* #return ForumDiscussion[] Returns an array of Page objects
*/
public function findBySortUpdatedAt()
{
return $this->createQueryBuilder('f')
// Doesn't work, make more queries
// ->select(array('f', 'user', 'forumCategory'))
// ->leftJoin('f.user', 'user')
// ->leftJoin('f.forumCategory', 'forumCategory')
->orderBy('f.updatedAt', 'DESC')
->setMaxResults(10)
->getQuery()
->getResult()
;
}
Someone know why it's happen ?
Thank you
Your User entity must have relations to other Entities. By default Doctrine uses the "LAZY" fetch mode when fetching entities.
The "EXTRA_LAZY" will reduce the number of queries by returning dummy objects instead of actual hydrated entities concerning the User related objects.
To use the "EXTRA_LAZY" fetch mode in the User entity you can just pass it as a parameter of your association annotations.
Here is a link to the official documentation: https://www.doctrine-project.org/projects/doctrine-orm/en/2.11/tutorials/extra-lazy-associations.html
If I have a document Shop that has many Activities defined as ReferenceMany, is there a way I can directly query for the list of Activities for a Shop without hydrating a Shop instance?
For example:
{
"_id": fd390j09afj09dfj,
"activities": [
...
]
}
All I want is to be able to say "get me the array activities where _id is fd390j09afj09dfj, and hydrate them as Activity instances.
Here's the first solution I came up with:
/**
* Gets all activities configured for a shop.
*
* #param string $shopId
* #return \BikeShed\Domain\Activity[]|\Doctrine\Common\Collections\ArrayCollection
*/
public function findByShopId($shopId) {
/** #var \BikeShed\Domain\Repository\Shop $shopRepository */
$shopRepository = $this->dm->getRepository('BikeShed\Domain\Shop');
$shop = $shopRepository->findOneById($shopId);
return $shop->getActivities();
}
It's simply fetching the Shop and then getting all the Activities via the defined relation.
Here's a working example of how you would implement jmikola's last suggestion:
/**
* #param string $shopId
* #return ActivityModel[]
*/
public function findByShopId($shopId) {
$partialShopData = $this->dm->getRepository('BikeShed\Domain\Shop')->createQueryBuilder()
->hydrate(false)
->field('activities')
->getQuery()
->getSingleResult()
;
$activityIds = [];
if(!empty($partialShopData['activities']))
foreach($partialShopData['activities'] as $activity)
if(!empty($activity['$id']))
$activityIds[] = $activity['$id'];
return $this->createQueryBuilder()
->field('id')
->in($activityIds)
->getQuery()
->toArray()
;
}
You cannot directly query the Shop collection or (or ODM repository) and receive Activity instances; however, you can use the Query Builder API to specify a projection with select('activities'). The executed query will still return Shop instances, but the activities field should be the only thing hydrated (as a PersistentCollection of Activity instances). In this case, you shouldn't modify any of the non-hydrated Shop fields, as ODM will detect any non-null value as a change.
It should be trivial to add a convenience method on ShopRepository that issues the above query with its select() and returns the collection (or an array) of Activity documents instead of the Shop. Keeping the Shop inaccessible should also protect you from inadvertently modifying other non-hydrated fields within it.
The down-side with this method is that the Activities will be proxy objects and lazily loaded. You can mitigate this with reference priming. With priming, you'll end up doing two queries (one for the Shop and one for all referenced Activity documents).
Regarding your follow-up question about putting this method on the Activity repository, you do have another option. Firstly, I agree that ActivityRepository::findByShopId() is preferable to calling a method on ShopRepository that returns Activity objects.
Each repository has a reference to the document manager, which you can use to access other repositories via the getRepository() method. An ActivityRepository::findByShopId() could do the following:
Access the Shop repository through the document manager
Query for the Shop by its ID, projecting only the activities field and disabling hydration completely
Collect the identifiers from the activities array. Depending on whether the Activity references are simple or not, the elements in that array may be the raw _id values or DBRef objects.
Execute a query for all Activity objects (easy, since we're already in that repository) where the ID is $in the array of identifiers
I have a User, Role & Page setup, all with many-to-many relationships and the pivot tables setup in the usual fashion (role_user, page_role), along with the eloquent methods to attach a model to the pivot tables.
My idea is to allow a user to have many roles, and a page can be accessed by many roles.
However now I'd like to return a collection where I have my users details and then the pages they're allowed to access.
The closest I have got is:
return User::find( Auth::user()->id )->with('roles.pages')->first()->roles;
Now this returns each role the user has, and each page that the role can access. Which is correct, however I have duplication on the pages part.
How would I go about getting only a list of pages the user is able to access with no duplication?
Cheers
Read that answer to get you on the track: HasManyThrough with one-to-many relationship
Only for your setup you need to adjust the query - join 2 pivot tables (and make sure they represent real data, ie no rows referencing non-existing models):
// User model
// accessor so you can access it like any relation: $user->pages;
public function getPagesAttribute()
{
if ( ! array_key_exists('pages', $this->relations)) $this->loadPages();
return $this->getRelation('pages');
}
// here you load the pages and set the collection as a relation
protected function loadPages()
{
$pages = Page::join('page_role as pr', 'pr.page_id', '=', 'pages.id')
->join('role_user as ru', 'ru.role_id', '=', 'pr.role_id')
->where('ru.user_id', $this->id)
->distinct()
->get(['pages.*', 'user_id']);
$hasMany = new Illuminate\Database\Eloquent\Relations\HasMany(Page::query(), $this, 'user_id', 'id');
$hasMany->matchMany(array($this), $pages, 'pages');
return $this;
}
One more thing: I hardcoded tables and columns names for sake of simplicity, but in real life I suggest you rely on the relationships and their getters, like: $relation->getTable(), $relation->getForeignKey() etc.
Now suggestion about your code:
return User::find( // 2. query to get the same user
Auth::user()->id // 1. query to get the user and his id
)->with('roles.pages')
->first() // 3. query to get ANOTHER user (or the same, luckily..)
->roles;
Use Auth::id() instead of Auth::user()->id (for Laravel ver 4.1.25+) to avoid redundant query
find() and first() are methods that execute the query, so you just returned the user with id = Auth::user()->id and moment later you fetch another one, who comes first() from the users table..
You don't need to use User::whatever for authenticated user, use Auth::user() instead.
So the code with suggested solution would look like this:
Auth::user()->pages; // collection of Page models with unique entries
I am trying get a record updated from database with QueryOver.
My code initially creates an entity and saves in database, then the same record is updated on database externally( from other program, manually or the same program running in other machine), and when I call queryOver filtering by the field changed, the query gets the record but without latest changes.
This is my code:
//create the entity and save in database
MyEntity myEntity = CreateDummyEntity();
myEntity.Name = "new_name";
MyService.SaveEntity(myEntity);
// now the entity is updated externally changing the name property with the
// "modified_name" value (for example manually in TOAD, SQL Server,etc..)
//get the entity with QueryOver
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
The previous statement gets a collection with only one record(good), BUT with the name property established with the old value instead of "modified_name".
How I can fix this behaviour? First Level cache is disturbing me? The same problem occurs with
CreateCriteria<T>();
The session in my NhibernateHelper is not being closed in any moment due application framework requirements, only are created transactions for each commit associated to a session.Save().
If I open a new session to execute the query evidently I get the latest changes from database, but this approach is not allowed by design requirement.
Also I have checked in the NHibernate SQL output that a select with a WHERE clause is being executed (therefore Nhibernate hits the database) but don´t updates the returned object!!!!
UPDATE
Here's the code in SaveEntity after to call session.Save: A call to Commit method is done
public virtual void Commit()
{
try
{
this.session.Flush();
this.transaction.Commit();
}
catch
{
this.transaction.Rollback();
throw;
}
finally
{
this.transaction = this.session.BeginTransaction();
}
}
The SQL generated by NHibernate for SaveEntity:
NHibernate: INSERT INTO MYCOMPANY.MYENTITY (NAME) VALUES (:p0);:p0 = 'new_name'.
The SQL generated by NHibernate for QueryOver:
NHibernate: SELECT this_.NAME as NAME26_0_
FROM MYCOMPANY.MYENTITY this_
WHERE this_.NAME = :p0;:p0 = 'modified_name' [Type: String (0)].
Queries has been modified due to company confidential policies.
Help very appreciated.
As far as I know, you have several options :
have your Session as a IStatelessSession, by calling sessionFactory.OpenStatelesSession() instead of sessionFactory.OpenSession()
perform Session.Evict(myEntity) after persisting an entity in DB
perform Session.Clear() before your QueryOver
set the CacheMode of your Session to Ignore, Put or Refresh before your QueryOver (never tested that)
I guess the choice will depend on the usage you have of your long running sessions ( which, IMHO, seem to bring more problems than solutions )
Calling session.Save(myEntity) does not cause the changes to be persisted to the DB immediately*. These changes are persisted when session.Flush() is called either by the framework itself or by yourself. More information about flushing and when it is invoked can be found on this question and the nhibernate documentation about flushing.
Also performing a query will not cause the first level cache to be hit. This is because the first level cache only works with Get and Load, i.e. session.Get<MyEntity>(1) would hit the first level cache if MyEntity with an id of 1 had already been previously loaded, whereas session.QueryOver<MyEntity>().Where(x => x.id == 1) would not.
Further information about NHibernate's caching functionality can be found in this post by Ayende Rahien.
In summary you have two options:
Use a transaction within the SaveEntity method, i.e.
using (var transaction = Helper.Session.BeginTransaction())
{
Helper.Session.Save(myEntity);
transaction.Commit();
}
Call session.Flush() within the SaveEntity method, i.e.
Helper.Session.Save(myEntity);
Helper.Session.Flush();
The first option is the best in pretty much all scenarios.
*The only exception I know to this rule is when using Identity as the id generator type.
try changing your last query to:
var result = NhibernateHelper.Session
.QueryOver<MyEntity>()
.CacheMode(CacheMode.Refresh)
.Where(param => param.Name == "modified_name")
if that still doesn't work, try add this after the query:
NhibernateHelper.Session.Refresh(result);
After search and search and think and think.... I´ve found the solution.
The fix: It consist in open a new session, call QueryOver<T>() in this session and the data is succesfully refreshed. If you get child collections not initialized you can call HibernateUtil.Initialize(entity) or sets lazy="false" in your mappings. Take special care about lazy="false" in large collections, because you can get a poor performance. To fix this problem(performance problem loading large collections), set lazy="true" in your collection mappings and call the mentioned method HibernateUtil.Initialize(entity) of the affected collection to get child records from database; for example, you can get all records from a table, and if you need access to all child records of a specific entity, call HibernateUtil.Initialize(collection) only for the interested objects.
Note: as #martin ernst says, the update problem can be a bug in hibernate and my solution is only a temporal fix, and must be solved in hibernate.
People here do not want to call Session.Clear() since it is too strong.
On the other hand, Session.Evict() may seem un-applicable when the objects are not known beforehand.
Actually it is still usable.
You need to first retrieve the cached objects using the query, then call Evict() on them. And then again retrieve fresh objects calling the same query again.
This approach is slightly inefficient in case the object was not cached to begin with - since then there would be actually two "fresh" queries - but there seems to be not much to do about that shortcoming...
By the way, Evict() accepts null argument too without exceptions - this is useful in case the queried object is actually not present in the DB.
var cachedObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>();
foreach (var obj in cachedObjects)
NhibernateHelper.Session.Evict(obj);
var freshObjects = NhibernateHelper.Session
.QueryOver<MyEntity>()
.Where(param => param.Name == "modified_name")
.List<T>()
I'm getting something very similar, and have tried debugging NHibernate.
In my scenario, the session creates an object with a couple children in a related collection (cascade:all), and then calls ISession.Flush().
The records are written into the DB, and the session needs to continue without closing. Meanwhile, another two child records are written into the DB and committed.
Once the original session then attempts to re-load the graph using QueryOver with JoinAlias, the SQL statement generated looks perfectly fine, and the rows are being returned correctly, however the collection that should receive these new children is found to have already been initialized within the session (as it should be), and based on that NH decides for some reason to completely ignore the respective rows.
I think NH makes an invalid assumption here that if the collection is already marked "Initialized" it does not need to be re-loaded from the query.
It would be great if someone more familiar with NHibernate internals could chime in on this.
I couldn't find an answer to this issue so I assume it is something I am doing wrong.
I have a PersistenceModel set up where I have set a convention as follows: -
persistenceModel.Conventions.Add(DefaultLazy.Always());
However, for one of the HasManyToMany relationships in one of my entities I want eager loading to take place which I am setting up as follows: -
HasManyToMany(x => x.Affiliates).Not.LazyLoad();
Intuitively, I expect eager loading to take place as I am overriding the lazy load default that I have specified as a convention but it still lazy loads. If I set the DefaultLazy convention to never and then set LazyLoad on an individual relationship it doesn't work either.
Any ideas?
When you set Not.LazyLoad(), you tell NHibernate to load Affiliates when the parent loads. NHibernate will do this by performing another select on the Affliates many-to-many table regardless of whether you access the Affiliates collection or not. NHibernate is using another select because that is the default fetching mode. You want to override fetching mode as well, either in the query or in the mapping. To do it in the mapping, add the following:
HasManyToMany(x => x.Affiliates)
.Not.LazyLoad()
.Fetch.Join();
You might also want to include a ".Cascade.AllDeleteOrphan()" if you want NHibernate to persist new Affiliaites added to the collection and delete orphaned ones. If you do not do this, you will have to explicitly call session.Save(newAffiliate). Otherwise you'll receive a TransientObjectException when your Affiliates collection contains a new Affiliate.
It may be one stupid thing to ask, but have you execute the query inside your session? Say,
Using(var session = OpenSession())
{
session.Query<Entity>().ToList();
}
I had this problem before, and finally realized the objects that I was accessing hadn't been queried before disposing the session.