Is it possible to prevent counting in the data provider, or at least cache the result?
This is my current code, and the count is done on every call:
$dataProvider = new CActiveDataProvider(PProjectJob::model()->cache(3600, null, 2), array(
'criteria' => $searchForm->search(),
'pagination' => array(
'pageSize' => Yii::app()->user->getState('pageSize', Yii::app()->params['defaultPageSize']),
),
'sort' => RedeliverySearchForm::getSort(),
'totalItemCount' => null
));
Initial query is always the same, and caching a count result will for 24 hours will have a great impact on performance.
You can pass a 'totalItemCount' value into the CActiveDataProvider constructor, which you have passed in as null in your current code
The data provider will only run a count query if its 'totalItemCount' is null
You could happily calculate, or fetch from cache, the count result yourself, based on the $searchForm->search() criteria, and pass it in
A second option is to create your own extension of CActiveDataProvider, override calculateTotalItemCount(), and provide a caching option for the count result where it currently runs the count() query
Related
Using the relationship field from Advanced Custom Fields I link artists to events. Both these artist and events are custom post types. For each event, the post IDs of the related artists are stored in an array as a custom meta field (lineup_artists).
On each event page I list all the artists. When you click on an artist, I'd like to show all the events where you can find this artist (through an AJAX call which shows the results in a bootstrap modal). I've tested the AJAX call and it's working, but there's something wrong with the query (takes very long to complete).
In my function I have:
$ArtistID = $_POST['ArtistID']; // Gets the ArtistID from the AJAX call
$meta_query = array(
'key' => 'lineup_artists',
'value' => '"' . $ArtistID .'"',
'compare' => 'LIKE'
);
$args = array(
'post_type' => 'events',
'meta_query' => array($meta_query),
'posts_per_page' => 5,
'post_status' => 'publish',
);
If I dump the results of wp_query, I get the following sql query:
SELECT SQL_CALC_FOUND_ROWS yt_posts.ID FROM yt_posts
INNER JOIN yt_postmeta ON (yt_posts.ID = yt_postmeta.post_id)
WHERE 1=1
AND yt_posts.post_type = 'events'
AND (yt_posts.post_status = 'publish')
AND ( (yt_postmeta.meta_key = 'lineup_artists' AND CAST(yt_postmeta.meta_value AS CHAR) LIKE '%\"17497\"%') )
GROUP BY yt_posts.ID ORDER BY yt_posts.post_date DESC LIMIT 0, 5
When I paste this query in phpmyadmin it takes very long to complete (I've never seen it finish because it takes so long).
Is this because the artist IDs are stored as an array? Someone told me that this is not very efficient and that these relations should be stored in a separate table (which I don't know how to do). Is there something wrong with my query or is this a very inefficient way of querying relations?
I'm using discriminators to deal with item states. I'm using the NHibernate Profiler and noticing that my application is going quite bonkers over selecting my states. I have 8 states in total and NHibernate keeps querying like so:
SELECT state0_.State_id as State1_10_0_,
state0_.Name as Name10_0_
FROM States state0_
WHERE state0_.State_id = **3** /* #p0 */
What triggers this SELECT+1 is when I select all items where the state is for example equal to 1, 2, 3, 4 or 5. This would issue one query for my items as well 5 queries for each state. Since I'm querying for items (with state) very often I'm executing an awful lot of queries.
Now, looking at the profiler I'm not sure it's much of an issue because the first state I query takes 6ms to query and 6ms to materialize. All other states display as being taken 0ms to query and 0ms to materialize. NHibernate Profiler still displays this as the notorious SELECT+1 problem.
From my aggregate root mapping I have:
References(x => x.State)
.Not.Nullable()
.Not.LazyLoad();
For my state discriminator mapping I have:
Id(x => x.Id, "State_id").GeneratedBy.Assigned();
Map(x => x.Name).Not.Nullable();
DiscriminateSubClassesOnColumn("State_id");
Is there something I'm not understanding regarding discriminators?
Here is the query causing trouble:
var items =
session.QueryOver(() => itemAlias)
.SelectList(x => x
.Select(xx => xx.Id).WithAlias(() => sentTo.Id)
.Select(xx => xx.State).WithAlias(() => sentTo.State)
.Select(xx => xx.Status).WithAlias(() => sentTo.Status)
)
.JoinAlias(() => itemAlias.State, () => stateAlias)
.WhereRestrictionOn(() => stateAlias.Id).IsInG(filters)
.Where(() => itemAlias.Status == Status.InProgress)
.TransformUsing(Transformers.AliasToBean<SentToDto>())
.List<SentToDto>();
Solution
The culprit was
.WhereRestrictionOn(() => stateAlias.Id).IsInG(filters)
To resolve my SELECT+1 issue I simply need to query by a list of discriminator objects instead of individual id's.
.Where(() => itemAlias.State.IsIn(new []{State.FooState, State.BarState}))
It sounds like you eager load the states themselves, so your first query is something like:
SELECT * FROM Items where StateID IN (1,2,3,4,5)
And then you have five queries one for each state:
SELECT Name FROM State WHERE StateID = 1
SELECT Name FROM State WHERE StateID = 2
....
It does that to eager load the name of the state, so to avoid it you should in your first query specify a join from your "items" to the "states" so you get the name there, otherwise set the State object to be lazy loaded if you don't need other info than the identifier in your result.
I have the following code where I am aggregating auction statistics from a statistics table and returning the Auction as well as the summed counts (there may be multiple rows in the statistics table per auction...)
var stats = _session.QueryOver<AuctionStatistic>()
.Select(
Projections.Group<AuctionStatistic>(s => s.Auction),
Projections.Sum<AuctionStatistic>(s => s.BidCount),
Projections.Sum<AuctionStatistic>(s => s.ViewCount),
Projections.Sum<AuctionStatistic>(s => s.SearchCount)
)
.OrderBy(Projections.Sum<AuctionStatistic>(s => s.ApplicationCount)).Desc
.Fetch(x => x.Auction).Eager
.Take(take)
.List<object[]>();
The query itself seems to work fine - except that the Auction that is returned is evaluated lazily, causing a SELECT N+1 scenario. Can anyone provide suggestions as to how this field can be evaluated eagerly?
Thanks in advance
JP
The brute force way would be (similar to a sub-select):
_session.QueryOver<Auction>().WhereRestrictionOn(a => a.Id).IsIn(stats.SelectMany(s => s).OfType<Auction>().Select(a => a.Id).ToArray());
I am using WCF calls to the server to retrive records.
This is done through Nhibernate.
How do I limit the query result that I am getting to say only return the first 20 records.
Depending on how you query, with QueryOver:
var rates = session
.QueryOver<ExchangeRate>()
.OrderBy(r => r.ExchangeDate).Desc
.ThenBy(r => r.CurrencyId.CurrencyId).Asc
.Take(20)
.List();
With the ICreateria interface you can do it by making a createria
I have a model structure: Category hasMany Product hasMany Stockitem belongsTo Warehouse, Manufacturer.
I fetch data with this code, using containable to be able to filter deeper in the associated models:
$this->Category->find('all', array(
'conditions' => array('Category.id' => $category_id),
'contain' => array(
'Product' => array(
'Stockitem' => array(
'conditions' => array('Stockitem.warehouse_id' => $warehouse_id),
'Warehouse',
'Manufacturer',
)
)
),
)
);
Data structure is returned just fine, however, I get multiple repeating queries like, sometimes hundreds of such queries in a row, based on dataset.
SELECT `Warehouse`.`id`, `Warehouse`.`title` FROM `beta_warehouses` AS `Warehouse` WHERE `Warehouse`.`id` = 2
Basically, when building data structure Cake is fetching data from mysql over and over again, for each row. We have datasets of several thousand rows, and I have a feeling that it's going to impact performance. Is it possible to make it cache results and not repeat same queries?
Try this:
$this->Product->find('all', array(
'conditions' => array('Category.id' => $category_id, 'Stockitem.warehouse_id' => $warehouse_id),
'contain' => array(
'Category'
, 'Stockitem' => array(
'Warehouse'
, 'Manufacturer'
)
),
));
If you remove Warehouse and Manufacturer you will find that Cakephp only executes one query.
Alternatively you can create database views for complex queries.