How do I use a main query value as a filter in a Subquery in cake php - sql

How do I translate this into orm that uses the value form the main query in the sub query
Here is my sql subquery
(select ups.username from users as ups where ups.id=student_id) as StudentName
My main query gets the student_id from an inner join
Here is my Orm Subquery
->select([
'StudentName'=>'Users.username'])
->where(['Users.id'=>'students.studentid']);
When I run the query StudentName shows up blank, but if I manually set the value like this it returns a result
->select([
'StudentName'=>'Users.username'])
->where(['Users.id'=>'55']);

When using the key => value syntax, the right hand side value will always be subject to binding/escaping, unless it's an expression object. So your condition will bind Students.student_id as a literal string, or possibly as an integer, ie you end up with SQL like:
WHERE Users.id = 'Students.student_id'
or
WHERE Users.id = 0
Either pass a \Cake\Database\Expression\IdentifierExpression object, like:
->where(['Users.id' => $mainQuery->identifier('Students.student_id')])
or in older CakePHP versions:
->where(['Users.id' => new \Cake\Database\Expression\IdentifierExpression('Students.student_id')])
Or use the expression builder, which has support for comparing fields:
->where(function(\Cake\Database\Expression\QueryExpression $exp) {
return $exp->equalFields('Users.id', 'Students.student_id');
})
There's an example for your use case in the advanced conditions docs (see the exists() example).
See also
Cookbook > Database Access & ORM > Query Builder > Advanced Conditions

Related

nhibernate querying over other query results

Hello i'm trying to get all users which have had payments at least 6 months over the given period (which must be a year). I've written SQL which works fine, but i have difficulties trying to convert it to nhibernate.
SQL:
SELECT COUNT(UserId) AS paidMonthsCount, UserId FROM (
SELECT DISTINCT UserId,
YEAR(PayDate) as _year,
MONTH(PayDate) as _month
FROM Payments
WHERE PayDate >= '2014-04-02T00:00:00' AND PayDate < '2015-04-02T23:59:00'
)result GROUP BY result.UserId
i have converted inner SQL:
var subQuery = Session.QueryOver(() => paymentAlias)
.SelectList(list => list
.Select(Projections.Distinct(Projections.Property<VelferdPayment>(p => p.Client.Id)).WithAlias(() => userWithHelp.Id))
.Select(p => p.AssignmentYear).WithAlias(() => userWithHelp.AssignmentDate)
)
.WhereRestrictionOn(p => p.AssignmentDate)
.IsBetween(parameters.FromDate)
.And(parameters.ToDate);
which selects the distinct part and i have the other part which is selecting from result:
var query = Session.QueryOver(() => userWithHelp).
SelectList(list => list
.SelectCount(p=> p.Id).WithAlias(()=> userWithHelpCount.Count)
.SelectGroup(p => p.Id).WithAlias(() => userWithHelpCount.Id)
)
.TransformUsing(Transformers.AliasToBean<UserWithHelpCount>())
.List<UserWithHelpCount>();
How can i queryover the subQuery results or is it possible to write single request to SQL. Working for a long time please help.
In general, with NHibernate we can only (or mainly) query Entities, not TABLES. Other words, we firstly map tables or views or even some <subselect>s into entities. The below mapping of the User (C# object User)
<class name="user" table="[dbo].[user_table]" ...
Will allow us to create query over C# User.
session.QueryOver<User>()...
Behind the scene it will generate FROM clause, which will contain the content of table attribute, i.e. FROM [dbo].[user_table]
That's it. There is no other way how to set the generated FROM clause. Just by mapping.
But there is a way which allow us to use existing ADO.NET connection to create custom query and even convert its result to some entity, or DTO. It is CreateSQLQuery() API:
17.1.5. Returning non-managed entities
It is possible to apply an IResultTransformer to native sql queries. Allowing it to e.g. return non-managed entities.
sess.CreateSQLQuery("SELECT NAME, BIRTHDATE FROM CATS")
.SetResultTransformer(Transformers.AliasToBean(typeof(CatDTO)))
This query specified:
the SQL query string
a result transformer
The above query will return a list of CatDTO which has been instantiated and injected the values of NAME and BIRTHNAME into its corresponding properties or fields.
So, we can use native SQL SELECT statements to get any results. We can even create some custom DTO and let NHibernate to transform result into them...

Yii model findAll() and count() return different number of results

UPDATED: see the end of the question
I'm working with Yii (and RESTFullYii in particular but I doubt that is relevant to the question)
There is a CDbCriteria for a model:
$criteria = new CDbCriteria(
array(
'together' => true,
'with' => array(
'roles'=> array(
'having' => "roles.role IN ($userRoles)"
))
)
);
$count = $model->count($criteria);
$result= $model->findAll($criteria);
While the findAll() method returns only 3 records (which is good) the count() method returns 13 which is the total number of records in the table represented by the $model
I've enabled query logging in MySQL and I found out that the two query generated by Yii is completely different
SELECT `t`.`id` AS `t0_c0`,
`t`.`name` AS `t0_c1`,
`t`.`description` AS `t0_c2`,
`t`.`enabled` AS `t0_c3`,
`t`.`issuegroup_id` AS `t0_c4`,
`t`.`role_id_exec` AS `t0_c5`,
`t`.`require_attachment` AS `t0_c6`,
`roles`.`id` AS `t1_c0`,
`roles`.`role` AS `t1_c1`,
`roles`.`enabled` AS `t1_c2`,
`roles`.`description` AS `t1_c3`
FROM `issuetype` `t`
LEFT OUTER JOIN `role_has_issuetype` `roles_roles` ON
(`t`.`id`=`roles_roles`.`issuetype_id`)
LEFT OUTER JOIN `role` `roles` ON
(`roles`.`id`=`roles_roles`.`role_id`)
HAVING (roles.role IN ('user'))
LIMIT 100
The other query:
SELECT COUNT(DISTINCT `t`.`id`)
FROM `issuetype` `t`
LEFT OUTER JOIN `role_has_issuetype` `roles_roles` ON
(`t`.`id`=`roles_roles`.`issuetype_id`)
LEFT OUTER JOIN `role` `roles` ON
(`roles`.`id`=`roles_roles`.`role_id`)
Is this the normal behavior for the findAll() and count() methods or did I do something I shouldn't have done or is this a bug in Yii?
And how to get the actual count of the records properly?
count($model->findAll($criteria)) seems to be working fine but is this the correct solution or is it just a workaround?
(From a performance viewpoint I think it might be better than the actual count() because I'm running the same query twice which is cached by the MySQL server)
UPDATE:
I've asked the same question on GitHub and Paul Klimov kindly pointed out that it is unnecessary for the 'having' and 'group' clauses to be in the joined table and it is perfectly OK to move it out of the 'with' see it here: https://github.com/yiisoft/yii/issues/3297
Yii had some problem with Having criteria while using count method from ActiveRecord, but it is fixed in newer Yii versions: https://github.com/yiisoft/yii/pull/2167
You need to clone your model before count() or findAll() method applying:
$result= $model->findAll($criteria);
$modelClone = clone $model;
$count = $model->count($criteria);

Query: getting the last record for each member

Given a table ("Table") as follows (sorry about the CSV style since I don't know how to make it look like a table with the Stack Overflow editor):
id,member,data,start,end
1,001,abc,12/1/2012,12/31/2999
2,001,def,1/1/2009,11/30/2012
3,002,ghi,1/1/2009,12/31/2999
4,003,jkl,1/1/2012,10/31/2012
5,003,mno,8/1/2011,12/31/2011
If using Ruby Sequel, how should I write my query so I will get the following dataset in return.
id,member,data,start,end
1,001,abc,12/1/2012,12/31/2999
3,002,ghi,1/1/2009,12/31/2999
4,003,jkl,1/1/2012,10/31/2012
I get the most current (largest end date value) record for EACH (distinct) member from the original table.
I can get the answer if I convert the table to an Array, but I am looking for a solution in SQL or Ruby Sequel query, if possible. Thank you.
Extra credit: The title of this post is lame...but I can't come up with a good one. Please offer a better title if you have one. Thank you.
The Sequel version of this is a bit scary. The best I can figure out is to use a subselect and, because you need to join the table and the subselect on two columns, a "join block" as described in Querying in Sequel. Here's a modified version of Knut's program above:
require 'csv'
require 'sequel'
# Create Test data
DB = Sequel.sqlite()
DB.create_table(:mytable){
field :id
String :member
String :data
String :start # Treat as string to keep it simple
String :end # Ditto
}
CSV.parse(<<xx
1,"001","abc","2012-12-01","2999-12-31"
2,"001","def","2009-01-01","2012-11-30"
3,"002","ghi","2009-01-01","2999-12-31"
4,"003","jkl","2012-01-01","2012-10-31"
5,"003","mno","2011-08-01","2011-12-31"
xx
).each{|x|
DB[:mytable].insert(*x)
}
# That was all setup, here's the query
ds = DB[:mytable]
result = ds.join(ds.select_group(:member).select_append{max(:end).as(:end)}, :member=>:member) do |j, lj, js|
Sequel.expr(Sequel.qualify(j, :end) => Sequel.qualify(lj, :end))
end
puts result.all
This gives you:
{:id=>1, :member=>"001", :data=>"abc", :start=>"2012-12-01", :end=>"2999-12-31"}
{:id=>3, :member=>"002", :data=>"ghi", :start=>"2009-01-01", :end=>"2999-12-31"}
{:id=>4, :member=>"003", :data=>"jkl", :start=>"2012-01-01", :end=>"2012-10-31"}
In this case it's probably easier to replace the last four lines with straight SQL. Something like:
puts DB[
"SELECT a.* from mytable as a
join (SELECT member, max(end) AS end FROM mytable GROUP BY member) as b
on a.member = b.member and a.end=b.end"].all
Which gives you the same result.
What's the criteria for your result?
If it is the keys 1,3 and 4 you may use DB[:mytable].filter( :id => [1,3,4]) (complete example below)
For more information about filtering with sequel, please refer the sequel documentation, especially Dataset Filtering.
require 'csv'
require 'sequel'
#Create Test data
DB = Sequel.sqlite()
DB.create_table(:mytable){
field :id
field :member
field :data
field :start #should be date, not implemented in example
field :end #should be date, not implemented in example
}
CSV.parse(<<xx
id,member,data,start,end
1,001,abc,12/1/2012,12/31/2999
2,001,def,1/1/2009,11/30/2012
3,002,ghi,1/1/2009,12/31/2999
4,003,jkl,1/1/2012,10/31/2012
5,003,mno,8/1/2011,12/31/2011
xx
).each{|x|
DB[:mytable].insert(*x)
}
#Create Test data - end -
puts DB[:mytable].filter( :id => [1,3,4]).all
In my opinion, you're approaching the problem from the wrong side. ORMs (and Sequel as well) represent a nice, DSL-ish layer above the database, but, underneath, it's all SQL down there. So, I would try to formulate the question and the answer in a way to get SQL query which would return what you need, and then see how it would translate to Sequel's language.
You need to group by member and get the latest record for each member, right?
I'd go with the following idea (roughly):
SELECT t1.*
FROM table t1
LEFT JOIN table t2 ON t1.member = t2.member AND t2.end > t1.end
WHERE t2.id IS NULL
Now you should see how to perform left joins in Sequel, and you'll need to alias tables as well. Shouldn't be that hard.

ActiveRecord/ARel modify `ON` in a left out join from includes

I'm wondering if it's possible to specify additional JOIN ON criteria using ActiveRecord includes?
ex: I'm fetching a record and including an association with some conditions
record.includes(:other_record).where(:other_record => {:something => :another})
This gives me (roughly):
select * from records
left outer join other_records on other_records.records_id = records.id
where other_records.something = another
Does anyone know how I can specify an extra join condition so I could achieve something like.
select * from records
left outer join other_records on other_records.records_id = records.id
and other_records.some_date > now()
where other_records.something = another
I want my includes to pull in the other_records but I need additional criteria in my join. Anything using ARel would also be great, I've just never known how to plug a left outer join from ARel into and ActiveRecord::Relation
I can get you close with ARel. NOTE: My code ends up calling two queries behind the scenes, which I'll explain.
I had to work out LEFT JOINs in ARel myself, recently. Best thing you can do when playing with ARel is to fire up a Rails console or IRB session and run the #to_sql method on your ARel objects to see what kind of SQL they represent. Do it early and often!
Here's your SQL, touched up a bit for consistency:
SELECT *
FROM records
LEFT OUTER JOIN other_records ON other_records.record_id = records.id
AND other_records.some_date > now()
WHERE other_records.something = 'another'
I'll assume your records model is Record and other_records is OtherRecord. Translated to ARel and ActiveRecord:
class Record < ActiveRecord::Base
# Named scope that LEFT JOINs OtherRecords with some_date in the future
def left_join_other_in_future
# Handy ARel aliases
records = Record.arel_table
other = OtherRecord.arel_table
# Join criteria
record_owns_other = other[:record_id].eq(records[:id])
other_in_future = other[:some_date].gt(Time.now)
# ARel's #join method lets you specify the join node type. Defaults to InnerJoin.
# The #join_sources method extracts the ARel join node. You can plug that node
# into ActiveRecord's #joins method. If you call #to_sql on the join node,
# you'll get 'LEFT OUTER JOIN other_records ...'
left_join_other = records.join(other, Arel::Nodes::OuterJoin).
on(record_owns_other.and(other_in_future)).
join_sources
# Pull it together back in regular ActiveRecord and eager-load OtherRecords.
joins(left_join_other).includes(:other_records)
end
end
# MEANWHILE...
# Elsewhere in your app
Record.left_join_other_in_future.where(other_records: {something: 'another'})
I bottled the join in a named scope so you don't need to have all that ARel mixed in with your application logic.
My ARel ends up calling two queries behind the scenes: the first fetches Records using your JOIN and WHERE criteria, the second fetches all OtherRecords "WHERE other_records.record_id IN (...)" using a big list of all the Record IDs from the first query.
Record.includes() definitely gives you the LEFT JOIN you want, but I don't know of a way to inject your own criteria into the join. You could use Record.joins() instead of ARel if you wanted to write the SQL yourself:
Record.joins('LEFT OUTER JOIN other_records' +
' ON other_records.record_id = records.id' +
' AND other_records.some_date > NOW()')
I really, really prefer to let the database adapter write my SQL, so I used ARel.
If it were me, I'd consider putting the additional join criterion in the WHERE clause. I assume you're asking because putting the additional criterion on the join makes the query's EXPLAIN look better or because you don't want to deal with NULLs in the other_records.some_date column when there aren't any related other_records.
If you have a simple (equality) extra join condition it could simply be
record.includes(:other_record).where(:other_record => {:something => :another,
:some_date => Time.now})
But if you need the greater than comparison the following should do it.
record.includes(:other_record).where([
'other_records.something = ? and other_records.some_date > ?',
another, Time.now])
Hope that helps.

How to simplify this LINQ to Entities Query to make a less horrible SQL statement from it? (contains Distinct,GroupBy and Count)

I have this SQL expression:
SELECT Musclegroups.Name, COUNT(DISTINCT Workouts.WorkoutID) AS Expr1
FROM Workouts INNER JOIN
Series ON Workouts.WorkoutID = Series.WorkoutID INNER JOIN
Exercises ON Series.ExerciseID = Exercises.ExerciseID INNER JOIN
Musclegroups ON Musclegroups.MusclegroupID = Exercises.MusclegroupID
GROUP BY Musclegroups.Name
Since Im working on a project which uses EF in a WCF Ria LinqToEntitiesDomainService, I have to query this with LINQ (If this isn't a must then pls inform me).
I made this expression:
var WorkoutCountPerMusclegroup = (from s in ObjectContext.Series1
join w in ObjectContext.Workouts on s.WorkoutID equals w.WorkoutID
where w.UserID.Equals(userid) && w.Type.Equals("WeightLifting")
group s by s.Exercise.Musclegroup into g
select new StringKeyIntValuePair
{
TestID = g.Select(n => n.Exercise.MusclegroupID).FirstOrDefault(),
Key = g.Select(n => n.Exercise.Musclegroup.Name).FirstOrDefault(),
Value = g.Select(n => n.WorkoutID).Distinct().Count()
});
The StringKeyIntValuePair is just a custom Entity type I made so I can send down the info to the Silverlight client. Also this is why I need to set an "TestID" for it, as it is an entity and it needs one.
And the problem is, that this linq query produces this horrible SQL statement:
http://pastebay.com/144532
I suppose there is a better way to query this information, a better linq expression maybe. Or is it possible to just query with plain SQL somehow?
EDIT:
I realized that the TestID is unnecessary because the other property named "Key" (the one on which Im grouping) becomes the key of the group, so it will be a key also. And after this, my query looks like this:
var WorkoutCountPerMusclegroup = (from s in ObjectContext.Series1
join w in ObjectContext.Workouts on s.WorkoutID equals w.WorkoutID
where w.UserID.Equals(userid) && w.Type.Equals("WeightLifting")
group w.WorkoutID by s.Exercise.Musclegroup.Name into g
select new StringKeyIntValuePair
{
Key = g.Key,
Value = g.Select(n => n).Distinct().Count()
});
This produces the following SQL: http://pastebay.com/144545
This seems far better then the previous sql statement of the half-baked linq query.
But is this good enough? Or this is the boundary of LinqToEntities capabilities, and if I want even more clear sql, I should make another DomainService which operates with LinqToSQL or something else?
Or the best way would be using a stored procedure, that returns Rowsets? If so, is there a best practice to do this asynchronously, like a simple WCF Ria DomainService query?
I would like to know best practices as well.
Compiling of lambda expression linq can take a lot of time (3–30s), especially using group by and then FirstOrDefault (for left inner joins meaning only taking values from the first row in the group).
The generated sql excecution might not be that bad but the compilation using DbContext which cannot be precompiled with .NET 4.0.
As an example 1 something like:
var q = from a in DbContext.A
join b ... into bb from b in bb.DefaultIfEmtpy()
group new { a, b } by new { ... } into g
select new
{
g.Key.Name1
g.Sum(p => p.b.Salary)
g.FirstOrDefault().b.SomeDate
};
Each FirstOrDefault we added in one case caused +2s compile time which added up 3 times = 6s only to compile not load data (which takes less than 500ms). This basically destroys your application's usability. The user will be waiting many times for no reason.
The only way we found so far to speed up the compilation is to mix lambda expression with object expression (might not be the correct notation).
Example 2: refactoring of previous example 1.
var q = (from a in DbContext.A
join b ... into bb from b in bb.DefaultIfEmtpy()
select new { a, b })
.GroupBy(p => new { ... })
.Select(g => new
{
g.Key.Name1
g.Sum(p => p.b.Salary)
g.FirstOrDefault().b.SomeDate
});
The above example did compile a lot faster than example 1 in our case but still not fast enough so the only solution for us in response-critical areas is to revert to native SQL (to Entities) or using views or stored procedures (in our case Oracle PL/SQL).
Once we have time we are going to test if precompilation works in .NET 4.5 and/or .NET 5.0 for DbContext.
Hope this helps and we can get other solutions.