UPDATED: see the end of the question
I'm working with Yii (and RESTFullYii in particular but I doubt that is relevant to the question)
There is a CDbCriteria for a model:
$criteria = new CDbCriteria(
array(
'together' => true,
'with' => array(
'roles'=> array(
'having' => "roles.role IN ($userRoles)"
))
)
);
$count = $model->count($criteria);
$result= $model->findAll($criteria);
While the findAll() method returns only 3 records (which is good) the count() method returns 13 which is the total number of records in the table represented by the $model
I've enabled query logging in MySQL and I found out that the two query generated by Yii is completely different
SELECT `t`.`id` AS `t0_c0`,
`t`.`name` AS `t0_c1`,
`t`.`description` AS `t0_c2`,
`t`.`enabled` AS `t0_c3`,
`t`.`issuegroup_id` AS `t0_c4`,
`t`.`role_id_exec` AS `t0_c5`,
`t`.`require_attachment` AS `t0_c6`,
`roles`.`id` AS `t1_c0`,
`roles`.`role` AS `t1_c1`,
`roles`.`enabled` AS `t1_c2`,
`roles`.`description` AS `t1_c3`
FROM `issuetype` `t`
LEFT OUTER JOIN `role_has_issuetype` `roles_roles` ON
(`t`.`id`=`roles_roles`.`issuetype_id`)
LEFT OUTER JOIN `role` `roles` ON
(`roles`.`id`=`roles_roles`.`role_id`)
HAVING (roles.role IN ('user'))
LIMIT 100
The other query:
SELECT COUNT(DISTINCT `t`.`id`)
FROM `issuetype` `t`
LEFT OUTER JOIN `role_has_issuetype` `roles_roles` ON
(`t`.`id`=`roles_roles`.`issuetype_id`)
LEFT OUTER JOIN `role` `roles` ON
(`roles`.`id`=`roles_roles`.`role_id`)
Is this the normal behavior for the findAll() and count() methods or did I do something I shouldn't have done or is this a bug in Yii?
And how to get the actual count of the records properly?
count($model->findAll($criteria)) seems to be working fine but is this the correct solution or is it just a workaround?
(From a performance viewpoint I think it might be better than the actual count() because I'm running the same query twice which is cached by the MySQL server)
UPDATE:
I've asked the same question on GitHub and Paul Klimov kindly pointed out that it is unnecessary for the 'having' and 'group' clauses to be in the joined table and it is perfectly OK to move it out of the 'with' see it here: https://github.com/yiisoft/yii/issues/3297
Yii had some problem with Having criteria while using count method from ActiveRecord, but it is fixed in newer Yii versions: https://github.com/yiisoft/yii/pull/2167
You need to clone your model before count() or findAll() method applying:
$result= $model->findAll($criteria);
$modelClone = clone $model;
$count = $model->count($criteria);
Related
How do I translate this into orm that uses the value form the main query in the sub query
Here is my sql subquery
(select ups.username from users as ups where ups.id=student_id) as StudentName
My main query gets the student_id from an inner join
Here is my Orm Subquery
->select([
'StudentName'=>'Users.username'])
->where(['Users.id'=>'students.studentid']);
When I run the query StudentName shows up blank, but if I manually set the value like this it returns a result
->select([
'StudentName'=>'Users.username'])
->where(['Users.id'=>'55']);
When using the key => value syntax, the right hand side value will always be subject to binding/escaping, unless it's an expression object. So your condition will bind Students.student_id as a literal string, or possibly as an integer, ie you end up with SQL like:
WHERE Users.id = 'Students.student_id'
or
WHERE Users.id = 0
Either pass a \Cake\Database\Expression\IdentifierExpression object, like:
->where(['Users.id' => $mainQuery->identifier('Students.student_id')])
or in older CakePHP versions:
->where(['Users.id' => new \Cake\Database\Expression\IdentifierExpression('Students.student_id')])
Or use the expression builder, which has support for comparing fields:
->where(function(\Cake\Database\Expression\QueryExpression $exp) {
return $exp->equalFields('Users.id', 'Students.student_id');
})
There's an example for your use case in the advanced conditions docs (see the exists() example).
See also
Cookbook > Database Access & ORM > Query Builder > Advanced Conditions
I have a mildly-complex ActiveRecord query in Rails 3.2 / Postgres that returns documents that are related and most relevant to all documents a user has favorited in the past.
The problem is that despite specifying uniq my query does not return distinct document records:
Document.joins("INNER JOIN related_documents ON
documents.docid = related_documents.docid_id")
.select("documents.*, related_documents.relevance_score")
.where("related_documents.document_id IN (?)",
some_user.favorited_documents)
.order("related_documents.relevance_score DESC")
.uniq
.limit(10)
I use a RelatedDocument join table, ranking each relation by a related_document.relevance_score which I use to order the query result before sampling the top 10. (See this question for schema description.)
The problem is that because I select("documents.*, related_documents.relevance_score"), the same document record returned multiple times with different relevance_scores are considered unique results. (i.e. if the document is a related_document for multiple favorited-documents.)
How do I return unique Documents regardless of the related_document.relevance_score?
I have tried splitting the select into two seperate selects, and changing the position of uniq in the query with no success.
Unfortunately I must select("related_documents.relevance_score") so as to order the results by this field.
Thanks!
UPDATE - SOLUTION
Thanks to Jethroo below, GROUP BY is the needed addition, giving me the follow working query:
Document.joins("INNER JOIN related_documents ON
documents.docid = related_documents.docid_id")
.select("documents.*, max(related_documents.relevance_score)")
.where("related_documents.document_id IN (?)",
some_user.favorited_documents)
.order("related_documents.relevance_score DESC")
.group("documents.id")
.uniq
.limit(10)
Have you tried to group it by documents.docid see http://guides.rubyonrails.org/active_record_querying.html#group?
I am making scopes for a model that looks something like this:
class PressRelease < ActiveRecord::Base
has_many :publications
end
What I want to get is all press_releases that does not have publications, but from a scope method, so it can be chained with other scopes. Any ideas?
Thanks!
NOTE: I know that there are methods like present? or any? and so on, but these methods does not return an ActiveRecord::Relation as scope does.
NOTE: I am using RoR 3
Avoid eager_loading if you do not need it (it adds overhead). Also, there is no need for subselect statements.
scope :without_publications, -> { joins("LEFT OUTER JOIN publications ON publications.press_release_id = press_releases.id").where(publications: { id: nil }) }
Explanation and response to comments
My initial thoughts about eager loading overhead is that ActiveRecord would instantiate all the child records (publications) for each press release. Then I realized that the query will never return press release records with publications. So that is a moot point.
There are some points and observations to be made about the way ActiveRecord works. Some things I had previously learned from experience, and some things I learned exploring your question.
The query from includes(:publications).where(publications: {id: nil}) is actually different from my example. It will return all columns from the publications table in addition to the columns from press_releases. The publication columns are completely unnecessary because they will always be null. However, both queries ultimately result in the same set of PressRelease objects.
With the includes method, if you add any sort of limit, for example chaining .first, .last or .limit(), then ActiveRecord (4.2.4) will resort to executing two queries. The first query returns IDs, and the second query uses those IDs to get results. Using the SQL snippet method, ActiveRecord is able to use just one query. Here is an example of this from one of my applications:
Profile.includes(:positions).where(positions: { id: nil }).limit(5)
# SQL (0.8ms) SELECT DISTINCT "profiles"."id" FROM "profiles" LEFT OUTER JOIN "positions" ON "positions"."profile_id" = "profiles"."id" WHERE "positions"."id" IS NULL LIMIT 5
# SQL (0.8ms) SELECT "profiles"."id" AS t0_r0, ..., "positions"."end_year" AS t1_r11 FROM "profiles" LEFT OUTER JOIN "positions" ON "positions"."profile_id" = "profiles"."id" # WHERE "positions"."id" IS NULL AND "profiles"."id" IN (107, 24, 7, 78, 89)
Profile.joins("LEFT OUTER JOIN positions ON positions.profile_id = profiles.id").where(positions: { id: nil }).limit(5)
# Profile Load (1.0ms) SELECT "profiles".* FROM "profiles" LEFT OUTER JOIN positions ON positions.profile_id = profiles.id WHERE "positions"."id" IS NULL LIMIT 5
Most importantly
eager_loading and includes were not intended to solve the problem at hand. And for this particular case I think you are much more aware of what is needed than ActiveRecord is. You can therefore make better decisions about how to structure the query.
you can de the following in your PressRelease:
scope :your_scope, -> { where('id NOT IN(select press_release_id from publications)') }
this will return all PressRelease record without publications.
Couple ways to do this, first one requires two db queries:
PressRelease.where.not(id: Publications.uniq.pluck(:press_release_id))
or if you don't want to hardcode association foreign key:
PressRelease.where.not(id: PressRelease.uniq.joins(:publications).pluck(:id))
Another one is to do a left join and pick those without associated elements - you get a relation object, but it will be tricky to work with it as it already has a join on it:
PressRelease.eager_load(:publications).where(publications: {id: nil})
Another one is to use counter_cache feature. You will need to add publication_count column to your press_releases table.
class Publications < ActiveRecord::Base
belongs_to :presss_release, counter_cache: true
end
Rails will keep this column in sync with a number of records associated to given mode, so then you can simply do:
PressRelease.where(publications_count: [nil, 0])
I'm trying to set up a subquery in ZendFramework 2 and I got an issue with the limit function for a Select object. Whatever I do, numeric value is put between quotes and makes my query fails : I should get LIMIT 1 and instead I get LIMIT '1'.
Seems this is not the first time this issue has been encountered, I saw some have asked about this issue before (like 8 months ago) but without getting any proper answer.
I also saw this issue has been marker as resolved in 2012 (https://github.com/zendframework/zf2/pull/2775) so I really don't understand what's happening there.
Here's my code in ZF2 :
$resultSet = $this->tableGateway->select( function (Select $select) use ($params) {
$sub = new Select();
$sub->from(array('temp' => 'scores'))
->columns(array(new \Zend\Db\Sql\Expression("id AS id")))
->where(array('temp.glitch' => array('None', 'Glitch')))
->where('temp.zone=scores.zone')
->order('temp.multi DESC, temp.score DESC')
->limit(1);
$select->join('players', 'player=players.id', array('player_name' => 'name', 'player_url' => 'name_url'))
->join('countries', 'players.country=countries.id', array('country_name' => 'name', 'country_iso' => 'iso'))
->join('cars', 'car=cars.id', array('car_name' => 'name'), 'left')
->join('zones', 'zone=zones.id', array('zone_name' => 'name'));
$select->where(array('scores.id' => $sub));
$select->order('scores.zone ASC');
print_r($select->getSqlString());
});
This should render the following query (which I get right except LIMIT '1' instead of LIMIT 1) :
SELECT "scores".*, "players"."name" AS "player_name", "players"."name_url" AS "player_url", "countries"."name" AS "country_name", "countries"."iso" AS "country_iso", "cars"."name" AS "car_name", "zones"."name" AS "zone_name"
FROM "scores" INNER JOIN "players" ON "player"="players"."id"
INNER JOIN "countries" ON "players"."country"="countries"."id"
LEFT JOIN "cars" ON "car"="cars"."id"
INNER JOIN "zones" ON "zone"="zones"."id"
WHERE "scores"."id" = (SELECT id AS id FROM "scores" AS "temp" WHERE "temp"."glitch" IN ('None', 'Glitch')
AND temp.zone=scores.zone ORDER BY "temp"."multi" DESC, "temp"."score" DESC LIMIT 1)
ORDER BY "scores"."zone" ASC
Since this doesn't seem to work this way, is there another way I could proceed to get my limit (using Mysql 5 database) ?
EDIT :
Thanks for your help. Finally I figured out a way to get things done the way I want and to remove the quotes by simply remove the subquery construction and to write it directly in the where function :
$select->where('scores.id = (SELECT id FROM scores AS lookup WHERE lookup.zone = scores.zone ORDER BY multi DESC , score DESC LIMIT 1)');
Although I can continue my dev with this, I feel more like using a poor trick to get rid of this issue and so I will let this question unanswered until someone comes with a real solution there.
Anyway there might be no solution at all, since it might be an issue in ZF2 core itself.
Change the line -
$select->where(array('scores.id' => $sub));
with
$select->where(array('scores.id' => new \Zend\Db\Sql\Expression("({$sub->getSqlString($this->tableGateway->adapter->getPlatform())})"));
Try with just above change.
And if it still doesn't work then make changes to the core Select class file located at -
PROJECT_FOLDER/vendor/zendframework/zendframework/library/Zend/Db/Sql/Select.php
Line No. 921 -
Change $sql = $platform->quoteValue($limit); with $sql = $limit;
Line No. 940 -
Change return array($platform->quoteValue($offset)); with return array($offset);
I have come across the issue from github and wondered as why it is still not working with the latest ZF2 files. I know the solution given above doesn't look like the proper one but I had to somehow make it work. I have tried it and it works.
Its only a quick fix before the actual solution comes into picture.
I'm wondering if it's possible to specify additional JOIN ON criteria using ActiveRecord includes?
ex: I'm fetching a record and including an association with some conditions
record.includes(:other_record).where(:other_record => {:something => :another})
This gives me (roughly):
select * from records
left outer join other_records on other_records.records_id = records.id
where other_records.something = another
Does anyone know how I can specify an extra join condition so I could achieve something like.
select * from records
left outer join other_records on other_records.records_id = records.id
and other_records.some_date > now()
where other_records.something = another
I want my includes to pull in the other_records but I need additional criteria in my join. Anything using ARel would also be great, I've just never known how to plug a left outer join from ARel into and ActiveRecord::Relation
I can get you close with ARel. NOTE: My code ends up calling two queries behind the scenes, which I'll explain.
I had to work out LEFT JOINs in ARel myself, recently. Best thing you can do when playing with ARel is to fire up a Rails console or IRB session and run the #to_sql method on your ARel objects to see what kind of SQL they represent. Do it early and often!
Here's your SQL, touched up a bit for consistency:
SELECT *
FROM records
LEFT OUTER JOIN other_records ON other_records.record_id = records.id
AND other_records.some_date > now()
WHERE other_records.something = 'another'
I'll assume your records model is Record and other_records is OtherRecord. Translated to ARel and ActiveRecord:
class Record < ActiveRecord::Base
# Named scope that LEFT JOINs OtherRecords with some_date in the future
def left_join_other_in_future
# Handy ARel aliases
records = Record.arel_table
other = OtherRecord.arel_table
# Join criteria
record_owns_other = other[:record_id].eq(records[:id])
other_in_future = other[:some_date].gt(Time.now)
# ARel's #join method lets you specify the join node type. Defaults to InnerJoin.
# The #join_sources method extracts the ARel join node. You can plug that node
# into ActiveRecord's #joins method. If you call #to_sql on the join node,
# you'll get 'LEFT OUTER JOIN other_records ...'
left_join_other = records.join(other, Arel::Nodes::OuterJoin).
on(record_owns_other.and(other_in_future)).
join_sources
# Pull it together back in regular ActiveRecord and eager-load OtherRecords.
joins(left_join_other).includes(:other_records)
end
end
# MEANWHILE...
# Elsewhere in your app
Record.left_join_other_in_future.where(other_records: {something: 'another'})
I bottled the join in a named scope so you don't need to have all that ARel mixed in with your application logic.
My ARel ends up calling two queries behind the scenes: the first fetches Records using your JOIN and WHERE criteria, the second fetches all OtherRecords "WHERE other_records.record_id IN (...)" using a big list of all the Record IDs from the first query.
Record.includes() definitely gives you the LEFT JOIN you want, but I don't know of a way to inject your own criteria into the join. You could use Record.joins() instead of ARel if you wanted to write the SQL yourself:
Record.joins('LEFT OUTER JOIN other_records' +
' ON other_records.record_id = records.id' +
' AND other_records.some_date > NOW()')
I really, really prefer to let the database adapter write my SQL, so I used ARel.
If it were me, I'd consider putting the additional join criterion in the WHERE clause. I assume you're asking because putting the additional criterion on the join makes the query's EXPLAIN look better or because you don't want to deal with NULLs in the other_records.some_date column when there aren't any related other_records.
If you have a simple (equality) extra join condition it could simply be
record.includes(:other_record).where(:other_record => {:something => :another,
:some_date => Time.now})
But if you need the greater than comparison the following should do it.
record.includes(:other_record).where([
'other_records.something = ? and other_records.some_date > ?',
another, Time.now])
Hope that helps.