I have a lot of queries in my profiler, So I try to add join() and select() in my repository for reduce the number of queries like usual
But this time, I reload the page and I have more queries than before
/**
* #return ForumDiscussion[] Returns an array of Page objects
*/
public function findBySortUpdatedAt()
{
return $this->createQueryBuilder('f')
// Doesn't work, make more queries
// ->select(array('f', 'user', 'forumCategory'))
// ->leftJoin('f.user', 'user')
// ->leftJoin('f.forumCategory', 'forumCategory')
->orderBy('f.updatedAt', 'DESC')
->setMaxResults(10)
->getQuery()
->getResult()
;
}
Someone know why it's happen ?
Thank you
Your User entity must have relations to other Entities. By default Doctrine uses the "LAZY" fetch mode when fetching entities.
The "EXTRA_LAZY" will reduce the number of queries by returning dummy objects instead of actual hydrated entities concerning the User related objects.
To use the "EXTRA_LAZY" fetch mode in the User entity you can just pass it as a parameter of your association annotations.
Here is a link to the official documentation: https://www.doctrine-project.org/projects/doctrine-orm/en/2.11/tutorials/extra-lazy-associations.html
Related
I am using TYPO3 8. In my extension I have a database table "company" in which I store for each company the total number of places (number_places) and the number of occupied places (occupied_places).
Now I want to limit the search to companies which have available places left.
In MySQL it would be like this:
SELECT * FROM company WHERE number_places > occupied_places;
How can I create this query in the extbase repository?
I tried to introduce the virtual property placesLeft in my model but it did not work.
I don't want to use a raw SQL statement as mentioned below, because I already have implemented a filter which uses plenty of different constraints.
Extbase query to compare two fields in same table
You can do it like this in your repository class, please note the comments inside the code:
class CompanyRepository extends \TYPO3\CMS\Extbase\Persistence\Repository
{
public function findWithAvailablePlaces(bool $returnRawQueryResult = false)
{
// Create a QueryBuilder instance
$queryBuilder = $this->objectManager->get(\TYPO3\CMS\Core\Database\ConnectionPool::class)
->getConnectionForTable('company')->createQueryBuilder();
// Create the query
$queryBuilder
->select('*')
->from('company')
->where(
// Note: this string concatenation is needed, because TYPO3's
// QueryBuilder always escapes the value in the ExpressionBuilder's
// methods (eq(), lt(), gt(), ...) and thus render it impossible to
// compare against an identifier.
$queryBuilder->quoteIdentifier('number_places')
. \TYPO3\CMS\Core\Database\Query\Expression\ExpressionBuilder::GT
. $queryBuilder->quoteIdentifier('occupied_places')
);
// Execute the query
$result = $queryBuilder->execute()->fetchAll();
// Note: this switch is not needed in fact. I just put it here, if you
// like to get the Company model objects instead of an array.
if ($returnRawQueryResult) {
$dataMapper = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper::class);
return $dataMapper->map($this->objectType, $result);
}
return $result;
}
}
Notes:
If you have lots of records to deal with, I would - for performance reasons - not use the data mapping feature and work with arrays.
If you want to use the fluid pagination widget, be sure you don't and build your own pagination. Because of the way this works (extbase-internally), you'd get a huge system load overhead when the table grows. Better add the support for limited db queries to the repository method, for example:
class CompanyRepository extends \TYPO3\CMS\Extbase\Persistence\Repository
{
public function findWithAvailablePlaces(
int $limit = 10,
int $offset = 0,
bool $returnRawQueryResult = false
) {
// ...
$queryBuilder
->setMaxResults($limit)
->setFirstResult($offset);
$result = $queryBuilder->execute()->fetchAll();
// ...
}
}
I think you cant do this using the default Extbase Query methods like equals() and so on. You may use the function $query->statement() for your specific queries like this.
You also can use the QueryBuilder since TYPO3 8 which has functions to compare fields to each other:
https://docs.typo3.org/typo3cms/CoreApiReference/latest/ApiOverview/Database/QueryBuilder/Index.html#quoteidentifier-and-quoteidentifiers
It's fine to use this QueryBuilder inside Extbase repositories. After this you can use the DataMapper to map the query results to Extbase models.
In case of using "statement()" be aware of escaping every value which may cause any kind of SQL injections.
Based on the current architecture of TYPO3, the data structure is such that comparing of two tables or, mixing results from two tables ought to be done from within the controller, by injecting the two repositories. Optionally, you can construct a Domain Service that can work on the data from the two repositories from within the action itself, in the case of a routine. The service will also have to be injected.
Note:
If you have a foreign relation defined in your table configuration, the results of that foreign relation will show in your defined table repository. So, there's that too.
If I have a document Shop that has many Activities defined as ReferenceMany, is there a way I can directly query for the list of Activities for a Shop without hydrating a Shop instance?
For example:
{
"_id": fd390j09afj09dfj,
"activities": [
...
]
}
All I want is to be able to say "get me the array activities where _id is fd390j09afj09dfj, and hydrate them as Activity instances.
Here's the first solution I came up with:
/**
* Gets all activities configured for a shop.
*
* #param string $shopId
* #return \BikeShed\Domain\Activity[]|\Doctrine\Common\Collections\ArrayCollection
*/
public function findByShopId($shopId) {
/** #var \BikeShed\Domain\Repository\Shop $shopRepository */
$shopRepository = $this->dm->getRepository('BikeShed\Domain\Shop');
$shop = $shopRepository->findOneById($shopId);
return $shop->getActivities();
}
It's simply fetching the Shop and then getting all the Activities via the defined relation.
Here's a working example of how you would implement jmikola's last suggestion:
/**
* #param string $shopId
* #return ActivityModel[]
*/
public function findByShopId($shopId) {
$partialShopData = $this->dm->getRepository('BikeShed\Domain\Shop')->createQueryBuilder()
->hydrate(false)
->field('activities')
->getQuery()
->getSingleResult()
;
$activityIds = [];
if(!empty($partialShopData['activities']))
foreach($partialShopData['activities'] as $activity)
if(!empty($activity['$id']))
$activityIds[] = $activity['$id'];
return $this->createQueryBuilder()
->field('id')
->in($activityIds)
->getQuery()
->toArray()
;
}
You cannot directly query the Shop collection or (or ODM repository) and receive Activity instances; however, you can use the Query Builder API to specify a projection with select('activities'). The executed query will still return Shop instances, but the activities field should be the only thing hydrated (as a PersistentCollection of Activity instances). In this case, you shouldn't modify any of the non-hydrated Shop fields, as ODM will detect any non-null value as a change.
It should be trivial to add a convenience method on ShopRepository that issues the above query with its select() and returns the collection (or an array) of Activity documents instead of the Shop. Keeping the Shop inaccessible should also protect you from inadvertently modifying other non-hydrated fields within it.
The down-side with this method is that the Activities will be proxy objects and lazily loaded. You can mitigate this with reference priming. With priming, you'll end up doing two queries (one for the Shop and one for all referenced Activity documents).
Regarding your follow-up question about putting this method on the Activity repository, you do have another option. Firstly, I agree that ActivityRepository::findByShopId() is preferable to calling a method on ShopRepository that returns Activity objects.
Each repository has a reference to the document manager, which you can use to access other repositories via the getRepository() method. An ActivityRepository::findByShopId() could do the following:
Access the Shop repository through the document manager
Query for the Shop by its ID, projecting only the activities field and disabling hydration completely
Collect the identifiers from the activities array. Depending on whether the Activity references are simple or not, the elements in that array may be the raw _id values or DBRef objects.
Execute a query for all Activity objects (easy, since we're already in that repository) where the ID is $in the array of identifiers
I have a User, Role & Page setup, all with many-to-many relationships and the pivot tables setup in the usual fashion (role_user, page_role), along with the eloquent methods to attach a model to the pivot tables.
My idea is to allow a user to have many roles, and a page can be accessed by many roles.
However now I'd like to return a collection where I have my users details and then the pages they're allowed to access.
The closest I have got is:
return User::find( Auth::user()->id )->with('roles.pages')->first()->roles;
Now this returns each role the user has, and each page that the role can access. Which is correct, however I have duplication on the pages part.
How would I go about getting only a list of pages the user is able to access with no duplication?
Cheers
Read that answer to get you on the track: HasManyThrough with one-to-many relationship
Only for your setup you need to adjust the query - join 2 pivot tables (and make sure they represent real data, ie no rows referencing non-existing models):
// User model
// accessor so you can access it like any relation: $user->pages;
public function getPagesAttribute()
{
if ( ! array_key_exists('pages', $this->relations)) $this->loadPages();
return $this->getRelation('pages');
}
// here you load the pages and set the collection as a relation
protected function loadPages()
{
$pages = Page::join('page_role as pr', 'pr.page_id', '=', 'pages.id')
->join('role_user as ru', 'ru.role_id', '=', 'pr.role_id')
->where('ru.user_id', $this->id)
->distinct()
->get(['pages.*', 'user_id']);
$hasMany = new Illuminate\Database\Eloquent\Relations\HasMany(Page::query(), $this, 'user_id', 'id');
$hasMany->matchMany(array($this), $pages, 'pages');
return $this;
}
One more thing: I hardcoded tables and columns names for sake of simplicity, but in real life I suggest you rely on the relationships and their getters, like: $relation->getTable(), $relation->getForeignKey() etc.
Now suggestion about your code:
return User::find( // 2. query to get the same user
Auth::user()->id // 1. query to get the user and his id
)->with('roles.pages')
->first() // 3. query to get ANOTHER user (or the same, luckily..)
->roles;
Use Auth::id() instead of Auth::user()->id (for Laravel ver 4.1.25+) to avoid redundant query
find() and first() are methods that execute the query, so you just returned the user with id = Auth::user()->id and moment later you fetch another one, who comes first() from the users table..
You don't need to use User::whatever for authenticated user, use Auth::user() instead.
So the code with suggested solution would look like this:
Auth::user()->pages; // collection of Page models with unique entries
I was just wondering whether when loading an entity which contains a collection e.g. a Post which may contain 0 -> n Comments if you can define how many comments to return.
At the moment I have this:
public IList<Post> GetNPostsWithNCommentsAndCreator(int numOfPosts, int numOfComments)
{
var posts = Session.Query<Post>().OrderByDescending(x => x.CreationDateTime)
.Take(numOfPosts)
.Fetch(z => z.Comments)
.Fetch(z => z.Creator).ToList();
ReleaseCurrentSession();
return posts;
}
Is there a way of adding a Skip and Take to Comments to allow a kind of paging functionality on the collection so you don't end up loading lots of things you don't need.
I'm aware of lazy loading but I don't really want to use it, I'm using the MVC pattern and want my object to return from the repositories loaded so I can then cache them. I don't really want my views causing select statements.
Is the only real way around this is to not perform a fetch on comments but to perform a separate Select on Comments to Order By Created Date Time and then Select the top 5 for example and then place the returned result into the Post object?
Any thoughts / links on this would be appreciated.
Thanks,
Jon
A fetch simple does a left-outer join on the associated table so that it can hydrate the collection entities with data. What you are looking to do will require a separate query on the specific entities. From there you can use any number of constructs to limit your result set (skip/take, setmaxresults, etc)
I'm looking at using doctrine for an application I'm working on - but after reading the documentation I'm having trouble conceptualizing how to represent the database structure we have in terms of entities.
I have many tables which have partner tables which hold translation data like the following....
Where I would like to have one Entity (Navigation Element) which had access to the 'label' field depending on what Language I set in my application. The following from the Doctrine documentation seems to suggest that you need to define one (single) table which is used to persist an entity
http://www.doctrine-project.org/docs/orm/2.0/en/reference/basic-mapping.html
By default, the entity will be
persisted to a table with the same
name as the class name. In order to
change that, you can use the #Table
annotation as follows:
Or do I need to define two entities and link them (or allow the translation table to inherit from the element table).
And what strategy would I use to always insert a language_id clause to the Join (to ensure I'm pulling the right label for the currently set language). Is this something I would define in the entity itself, or elsewhere?
This seems to suit a One-To-Many Bidirectional association. This is the scenario from that page translated to your situation:
/** #Entity */
class NavigationElement
{
// ...
/**
* #OneToMany(targetEntity="NavigationElementTranslation", mappedBy="navigationElement")
*/
private $translations;
// ...
public function __construct() {
$this->translations = new \Doctrine\Common\Collections\ArrayCollection();
}
}
/** #Entity */
class NavigationElementTranslation
{
// ...
/**
* #ManyToOne(targetEntity="NavigationElement", inversedBy="translations")
* #JoinColumn(name="navigation_element_id", referencedColumnName="id")
*/
private $navigationElement;
// ...
}
You could add a getLabel($languageId) method to the NavigationElement entity that searches through the translations to get the correct label:
public function getLabel($languageId) {
foreach($this->translations as $trans) {
if($trans->languageId == $languageId)
return $trans->label;
}
throw new InvalidArgumentException();
}
And you could use the following DQL to ensure you only load the translation you want into the $translations property:
$query = $em->createQuery(
"SELECT ne, net
FROM Entity\NavigationElement ne
JOIN ne.translations net WITH net.languageId = :langId"
);
$query->setParameter('langId', $languageId);
$navigationElements = $query->execute();
This situation sounds like one where you would want to cache aggressively. Make sure you look into Doctrine 2's caching mechanisms too.
Also, internationalization can be handled reasonably well in PHP with gettext if you find join tables for translations start to become unmanageable.
I would also direct anyone who has to tackle this same problem to take a look at the following doctrine extension.
http://www.gediminasm.org/article/translatable-behavior-extension-for-doctrine-2