I'm experiencing a SQL error from a Core Data-generated query. Here's the data model:
I want to get all of the drivers for whose busses have active passengers. This is simply described by the following fetch request and predicate, which works:
NSFetchRequest(entityName: "Driver")
NSPredicate(format: "ANY bus.passengers.active == YES")
However, let's say I want to do a more complex query on the passenger, like they are both active and a balance above 100. To do such a query, I'd have to use a SUBQUERY, like so:
NSPredicate(format: "SUBQUERY(bus.passengers, $passenger, $passenger.active == YES
AND $passenger.balance > 100).#count > 0")
I would expect SUBQUERY to accept a keypathed collection as its first argument. It generates the following SQL query:
SELECT 0,
t0.z_pk,
t0.z_opt,
t0.zbus
FROM zdriver t0
WHERE (SELECT Count(t2.z_3passengers)
FROM zbus t1
JOIN z_1passengers t2
ON t1.z_pk = t2.z_1busses
WHERE ( t0.zbus = t1.z_pk
AND (( t2.zactive = ?
AND t2.zbalance > ? )) )) > ?
And the error: 'no such column: t2.ZACTIVE'. It looks like it's missing a JOIN on the ZPASSENGER table.
Is this a Core Data bug or is there a different way to write this predicate?
Update: Here's a sample project that reproduces this issue.
Try fetching the Bus entity, avoiding the key path in the first argument in the SUBQUERY.
As was pointed out, this can be viewed as a limitation or a bug.
Update:
Further research suggests that your subquery predicate should actually work. I would recommend to check your NSManagedObject subclasses and make sure passengers returns a collection and active is properly listed as an attribute.
Related
I have the following SQL-code in my (SAP IdM) Application:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
I already found the solution for the ORA-01841 problem, as it could either be a solution similar to MSSQLs try_to_date as mentioned here: How to handle to_date exceptions in a SELECT statment to ignore those rows?
or a solution where I change the code to something like this, to work soly on strings:
Select mcmskeyvalue as MKV,v1.searchvalue as STARTDATE, v2.avalue as Running_Changes_flag
from idmv_entry_simple
inner join idmv_value_basic_active v1 on mskey = mcmskey and attrname = 'Start_of_company_change'
and mcentrytype = 'MX_PERSON' and v1.searchvalue<= to_char(sysdate+3,'YYYY-MM-DD')
left join idmv_value_basic v2 on v2.mskey = mcmskey and v2.attrname = 'Running_Changes_flag'
where mcmskey not in (Select mskey from idmv_value_basic_active where attrname = 'Company_change_running_flag')
So for the actually problem I have a solution.
But now I came into discussion with my customers and teammates why the error happens at all.
Basically for all entries of idmv_value_basic_activ that comply to the requirement of "attrname = 'Start_of_company_change'" we can be sure that those are dates. In addition, if we execute the query to check all values that would be delivered, all are in a valid format.
I learned in university that the DB-Engine could decide in which order it will run individual segments of a query. So for me the most logical explanation would be that, on the development environment (where we face the problem), the section " to_date(v1.searchvalue,'YYYY-MM-DD')<= sysdate+3” is executed before the section “attrname = 'Start_of_company_change'”
Whereas on the productive environment, where everything works like a charm, the segments are executed in the order that is descripted by the SQL Statement.
Now my Question is:
First: do I remember that right, since the teacher said that only once and at that time I could not really make sense out of it
And Second: Is this assumption of mine correct or is there another reason for the problem?
Borderinformation:
The Tool uses a kind of shifted data structure which is why there can be quite a few different types in the actual “Searchvalue” column of the idmv_value_basic_activ view. The datatype on the database layer is always a varchar one.
"the DB-Engine could decide in which order it will run individual segments of a query"
This is correct. A SQL query is just a description of the data you want and where it's stored. Oracle will calculate an execution plan to retrieve that data as best it can. That plan will vary based on any number of factors, like the number of actual rows in the table and the presence of indexes, so it will vary from environment to environment.
So it sounds like you have an invalid date somewhere in your table, so to_date raises an exception. You can use validate_conversion to find it.
I have a query in the DB:
SELECT GreenInventoryBlendGradeID,bgx.blendgradeid,
bgX.GreenBlendGradeTypeID,[Description]
FROM [GreenInventory] gi
INNER JOIN [GreenInventoryBlendGradeXref] bgX
ON bgX.[GreenInventoryID] = gi.[GreenInventoryID]
INNER JOIN [BlendGrade] bg
ON bg.[BlendGradeID]=bgx.[BlendGradeID]
That returns 3 records:
TypeID Desc
1 XR
2 XR
1 XF2
The LINQ:
var GreenInventory = (from g in Session.GreenInventory
.Include("GreenInventoryBlendGradeXref")
.Include("GreenInventoryBlendGradeXref.BlendGrade")
.Include("GreenInventoryBlendGradeXref.GreenBlendGradeType")
.Include("GreenInventoryWeightXref")
.Where(x => x.GreenInventoryID == id && x.GreenInventoryBlendGradeXref.Any(bg=>bg.GreenBlendGradeTypeID > 0) )
select g);
I have tried different Where clauses including the simple - (x => x.GreenInventoryID == id)
but always have only the first 2 records returned.
Any Ideas?
If I try the following:
var GreenInventory = (from gi in Session.GreenInventory.Where(y => y.GreenInventoryID == id)
join bgX in Session.GreenInventoryBlendGradeXref.DefaultIfEmpty() on gi.GreenInventoryID equals bgX.GreenInventoryID
join bg in Session.BlendGrade.DefaultIfEmpty() on bgX.BlendGradeID equals g.BlendGradeID
select new { GreenInventory = gi, GreenInventoryBlendGradeXref = bgX, BlendGrade = bg });
I Get back 3 of each objects and the correct information is in the BlendGrade objects. It looks like the 3 GreenInventory objects are the same. They each include 2 of the GreenInventoryBlendGradeXref objects which show the the same 2 records as before.
So I not clear on what the original problem was. Also dont know if this is the best way to resolve it.
Thanks for the answers. If anyone has a further thoughts please let us know.
Based on the few details you present, I would assume that you are missing a join. I have no experience with EntityFramework (I assume that you use this ORM), but as far as I know, the ".Include" tries to ensure that the set of root entities will not change and will not contain duplicates.
Your manually created query seems to indicate that there is at least one 1:n relationship in the model. The result you get from LINQ show that only distinct GreenInventory entities are returned.
Therefore you need to adjust your query and explicitly declare that you want all results (and not only distinct root entities) - I would assume that with an explicit join EntityFramework will yield all expected results - or you need to adjust your mapping.
The first place I'd look in would be your model and joins you have defined between the entities. You might also want to check your generated SQL statement:
Trace.WriteLine(GreenInventory.Provider.ToString())
or use Visual Studio IntelliTrace to investigate what was sent to the database.
In my app a Position has_many Questions. In my position edit view, I create a set of checkboxes by looping over an array of Questions and checking those that the current position already includes.
Looks like this:
- #questions.each do |question|
= check_box_tag :question_ids, question.id, #position.questions.include?(question), name: 'position[question_ids][]', id: "question_check_#{question.id}"
= label_tag "question_check_#{question.id}", question.text
I noticed loading this page took an unusually long time and my logs had these (I've snipped it for space):
LOG: duration: 827.370 ms statement: SELECT 1 FROM "questions" INNER JOIN "required_questions" ON "questions"."id" = "required_questions"."question_id" WHERE "required_questions"."position_id" IS NULL AND "questions"."id" = 1 ORDER BY account_id desc, "questions".id asc LIMIT 1
LOG: duration: 821.666 ms statement: SELECT 1 FROM "questions" INNER JOIN "required_questions" ON "questions"."id" = "required_questions"."question_id" WHERE "required_questions"."position_id" IS NULL AND "questions"."id" = 2 ORDER BY account_id desc, "questions".id asc LIMIT 1
LOG: duration: 713.379 ms statement: SELECT 1 FROM "questions" INNER JOIN "required_questions" ON "questions"."id" = "required_questions"."question_id" WHERE "required_questions"."position_id" IS NULL AND "questions"."id" = 3 ORDER BY account_id desc, "questions".id asc LIMIT 1
I tracked the problem to this code:
#position.questions.include?(question)
I thought this was a simple Array.include? check so I was surprised by my findings. I was also surprised when I fixed it by adding an explicit .to_a call:
#position.questions.to_a.include?(question)
I was surprised a third time when I tested the original code in the Rails console and no SQL query was generated.
At this point I'm just curious - why did my original code generate a SQL query for each iteration when the same code in Rails console (albeit without the iteration and inside Rails console) did not? Why didn't it just do an Array.include? check?
This happens because #position.questions is not an array. It's actually an ActiveRecord::Relation. The problem is that the console actually behaves differently from your server application.
For example, in your console position.questions returns an array. That is because the console is actually evaluating this expression as position.questions.all which is equivalent to position.questions.to_a. In your server application the to_a or all is not called until you actually use the query. This is a good thing because it means you can continue to build up a query and it will only actually get executed when it's called.
For instance:
query = #position.questions
query.first
query.last
Will actually generate two queries that return two records in your server application because the query variable will be assigned an ActiveRecord::Relation instead of an Array. In your console this will generate a single query, but the query will load all of the questions into an Array and assign that to query and then select the first and last elements.
The all, first, last, count and even include? keywords are all triggers that actually execute the query, so in your application when you call #position.questions.include? you are executing a single query on the #position.questions relation. Adding the to_a causes this query to be executed immediately.
In my project, I have two entity, first PaperEntity contains several properties (consisting of value types and also reference types -reference to other entities-) and second is PaperStatusEntity.
PaperEntity has a property named Result of type PaperStatusEntity (and also a property named locked of type bool)
Imagine you have near 500 papers and just 8 paper status defined in database.
I want to find how much every status is used? for example status1 used 58 times and status2 used 130 times and so on.
I write below HQL
select paper.Result, Count(paper.Result) from PaperEntity paper group by paper.Result
this hql generates below error:
Column 'Conference_PaperStatusesTable.Id' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.
generated sql is:
select paperentit0_.Result as col_0_0_, count(paperentit0_.Result) as col_1_0_, paperstatu1_.Id as Id48_, paperstatu1_.Version as Version48_, paperstatu1_.CreationTime as Creation3_48_, paperstatu1_.Portal as Portal48_, paperstatu1_.TitleCodeName as TitleCod5_48_, paperstatu1_.Enabled as Enabled48_, paperstatu1_.RefereeChoice as RefereeC7_48_, paperstatu1_.OrderIndex as OrderIndex48_, paperstatu1_.ContactMessageTemplate as ContactM9_48_ from Conference_PapersTable paperentit0_ inner join Conference_PaperStatusesTable paperstatu1_ on paperentit0_.Result=paperstatu1_.Id, Conference_PaperStatusesTable paperstatu2_ where paperentit0_.Result=paperstatu2_.Id group by paperentit0_.Result
If I try to group data with a value type property like 'Locked' (that is bool), no problems and all things are ok
also If I use Criteria instead of HQL, works truly:
IList result = NHibernateSessionManager.Instance.CurrentSession.CreateCriteria(typeof(PaperEntity))
.SetProjection(Projections.ProjectionList().Add(Projections.RowCount()).Add(Projections.GroupProperty("Result"))).List();
foreach (var item in result) {
object[] value = item as object[];
yield return new Pair<PaperStatusEntity, int>(value[1] as PaperStatusEntity, (int)value[0]);
}
HQL group by is not smart enough to project all the properties of an entity when grouping by it. You have to either:
Specify all the properties
Select just those you need (id, description, whatever)
Use a subquery to get the count
The reason why it's working with Criteria is that it selects just the id when you use Projections.GroupProperty, creating uninitialized proxies.
This can create a SELECT N+1 problem.
You need to provide a simple data type for the database engine to group by. If paper.Result is another entity you'll probably need to join to the PaperStatusEntity table.
Something like:
select paper.Result, Count(paper.Result)
from PaperEntity as paper
join paper.Result as status
group by status.Id
If PaperStatus is mapped as a component you can also access Result.Id directly.
You can try:
SELECT paper.Result, Count(paper.Result)
FROM PaperEntity paper
JOIN paper.Result as status
GROUP BY paper.Result
Note that you would rather use a "FROM PaperStatusEntity" and join the papers
Because with the actual thing you are trying to do, you may not get a count for a PaperStatusEntity if no paper use this status (this means you don't get a count=0, you just get no count)
You can also use the #Formula annotation so that, for a given status, you can do paperStatusEntity.getPaperNumber()
Just add something like
#Formula("select count() from PaperEntity paper where paper.result_id = id");
public int getPaperNumber();
(should be adapted)
(I use Hibernate but it should be the same?)
I am currently working on a project using NHiberate as the DAL with .NET 2.0 and NHibernate 2.2.
Today I came to a point where I had to join a bunch of entities/collections to get what I want. That is fine.
What got me was that I do not want the query to return a list of objects of a certain entity type but rather the result would include various properties from different entities.
The following query is not what I am doing but it is kind of query that I am talking about here.
select order.id, sum(price.amount), count(item)
from Order as order
join order.lineItems as item
join item.product as product,
Catalog as catalog
join catalog.prices as price
where order.paid = false
and order.customer = :customer
and price.product = product
and catalog.effectiveDate < sysdate
and catalog.effectiveDate >= all (
select cat.effectiveDate
from Catalog as cat
where cat.effectiveDate < sysdate
)
group by order
having sum(price.amount) > :minAmount
order by sum(price.amount) desc
My question is, in this case what type result is supposed to be returned? It is certainly not of type Order, neither is of type LineItems.
Thanks for your help!
John
you can always use List of object[] for returning data and it will work fine.
This is called a projection, and it happens any time you specify an explicit select clause that contains rows from various tables (or even aggregate / summary data from a single table).
Using LINQ you can create anonymous objects to store these rows of data, like this:
var crunchies = (from foo in bar
where foo.baz == quux
select new { foo.corge, foo.grault }).ToList();
Then you can do crunchies[0].corge for example to pull out the rows & columns.
If you are using NHibernate.Linq this will "just work".
If you're using HQL or Criteria API, then what Fahad mentioned will work. You'll get a List<object[]> as a result, and the index of the array references the order of the columns that you returned in your select clause.