Doctrine - Get items based on the count of a many to many - sql

I have the following tables, User, Project and Images.
A User has a one to many relationship with Projects and Images. Each Project and Image is owned by a User.
A Project has a many to many relationship with Images. So each Project can have many Images, and an Image can appear within many Projects.
I want to write a DQL query to get all Images, for a specific User that are not included in any Projects. This I can write in SQL.
Is writing in SQL the best way to go?
Or should I be using DQL to do this?
I have tried writing the DQL but its hard work!
Edit
From within my Image Repo I am now doing this
$qb = $this->createQueryBuilder("i");
$qb->select("i")
->from('MyBundleName:User','u')
->innerJoin('u.images', 'user_images')
->where('IDENTITY(u.id) = :user_id')
->andWhere('NOT EXISTS (
SELECT p
FROM MyBundleName:Project p
WHERE user_images MEMBER of p.images
)')
->setParameter('user_id', $user_id);
I have replace the / syntax with : for my classes as they failed when using /
I am still getting this error though;
[Semantical Error] line 0, col 131 near 'id) = :user_id': Error:
Invalid PathExpression. Must be a SingleValuedAssociationField.
The function createQueryBuilder requires an alias, I am passing it "i" - is that right?
I then give it "i" again when calling select?
If I remove the innerJoin then it works, but the results are wrong. It returns all images even if they do exist within an project.

I can't say how difficult it would be in DQL, but I can tell you that in SQL it sounds pretty simple:
SELECT I.*
FROM IMAGES I
INNER JOIN USERS U ON (I.USER_ID = U.USER_ID)
WHERE NOT EXISTS (
SELECT *
FROM PROJECTS P, PROJECT_IMAGES PI
WHERE P.USER_ID = U.USER_ID
AND PI.PROJECT_ID = P.PROJECT_ID
AND I.IMAGE_ID = PI.IMAGE_ID
)
Images owned by a user which does not exist in any project that the user owns.

I don't know what your entities look like but given the relations something like this should work. The key is combining NOT EXISTS and MEMBER OF (you want to make sure that for all returned images no project exists that the image is a member of).
$qb->select("i")
->from('MyBundle\Entity\User','u')
->innerJoin('u.images','i')
->where('IDENTITY(u) = :user_id')
->andWhere('NOT EXISTS (
SELECT p
FROM MyBundle\Entity\Project p
WHERE i MEMBER of p.images
)')
->setParameter('user_id', $user_id);

Related

PyPika control order of with clauses

I am using PyPika (version 0.37.6) to create queries to be used in BigQuery. I am building up a query that has two WITH clauses, and one clause is dependent on the other. Due to the dynamic nature of my application, I do not have control over the order in which those WITH clauses are added to the query.
Here is example working code:
a_alias = AliasedQuery("a")
b_alias = AliasedQuery("b")
a_subq = Query.select(Term.wrap_constant("1").as_("z")).select(Term.wrap_constant("2").as_("y"))
b_subq = Query.from_(a_alias).select("z")
q = Query.with_(a_subq, "a").from_(a_alias).select(a_alias.y)
q = q.with_(b_subq, "b").from_(b_alias).select(b_alias.z)
sql = q.get_sql(quote_char=None)
That generates a working query:
WITH a AS (SELECT '1' z,'2' y) ,b AS (SELECT a.z FROM a) SELECT a.y,b.z FROM a,b
However, if I add the b WITH clause first, then since a is not yet defined, the resulting query:
WITH b AS (SELECT a.z FROM a), a AS (SELECT '1' z,'2' y) SELECT a.y,b.z FROM a,b
does not work. Since BigQuery does not support WITH RECURSIVE, that is not an option for me.
Is there any way to control the order of the WITH clauses? I see the _with list in the QueryBuilder (the type of variable q), but since that's a private variable, I don't want to rely on that, especially as new versions of PyPika may not operate the same way.
One way I tried to do this is to always insert the first WITH clause at the beginning of the _with list, like this:
q._with.insert(0, q._with.pop())
Although this works, I'd like to use a PyPika supported way to do that.
In a related question, is there a supported way within PyPika to see what has already been added to the select list or other parts of the query? I noticed the q.selects member variable, but selects is not part of the public documentation. Using q.selects did not actually work for me when using our project's Python version (3.6) even though it did work in Python 3.7. The code I was trying to use is:
if any(field.name == "date" for field in q.selects if isinstance(field, Field))
The error I got was as follows:
def __getitem__(self, item: slice) -> "BetweenCriterion":
if not isinstance(item, slice):
> raise TypeError("Field' object is not subscriptable")
Thank you in advance for your help.
I could not figure out how to control the order of the WITH clauses after calling query.with_() (except for the hack already noted). As a result, I restructured my application to get around this problem. I am now calling query.with_() before building up the rest of the query.
This also made my related question moot, because I no longer need to see what I've already added to the query.

Oracle spatial request working on one instance and not on another

I have this statement that is generated by Geoserver
SELECT
shape AS shape
FROM
(
SELECT
c.chantier_id id,
sdo_geom.sdo_buffer(c.shape, m.diminfo, 1) shape,
c.datedebut datedebut,
c.datefin datefin,
o.nom operation,
c.brouillon brouillon,
e.code etat,
u.utilisateur_id utilisateur,
u.groupe_id groupe
FROM
user_sdo_geom_metadata m, lyv_chantier c
JOIN lyv_utilisateur u ON c.createur_id = u.utilisateur_id
JOIN lyv_etat e ON c.etat_id = e.etat_id
JOIN lyv_operation o ON c.operation = o.id
WHERE
m.table_name = 'LYV_CHANTIER'
AND m.column_name = 'SHAPE'
) vtable
WHERE
( brouillon = 0
AND ( etat != 'archive'
OR etat IS NULL )
AND sdo_filter(shape, mdsys.sdo_geometry(2003, 4326, NULL, mdsys.sdo_elem_info_array(1, 1003, 1), mdsys.sdo_ordinate_array(
2.23365783691406, 48.665657043457, 2.23365783691406, 48.9341354370117, 2.76649475097656, 48.9341354370117, 2.76649475097656, 48.665657043457, 2.23365783691406, 48.665657043457)), 'mask=anyinteract querytype=WINDOW') = 'TRUE' );
On my local instance (dockerized if that can explain anything) it works fine, but on another instance I get an error :
ORA-13226: interface not supported without a spatial index
I guess that the SDO_FILTER is applied to the result of SDO_BUFFER which is therefore not indexed.
But why is it working on my local instance ?!
Is there some kind of weird configuration shenanigan that could explain the different behavior maybe ?
EDIT : The idea behind this is to get around a bug in Geoserver with Oracle databases where it renders only the first point of MultiPoint geometries, but works fine with MutltiPolygon.
I am using a SQL view as layer in Geoserver (hence the subselect I guess).
First, you need to do some debugging here.
Connect to each instance, on the same user as your Geoserver's datasource, and run the sql. From the same connections (in each instance) you must also verify that the user's metadata view (user_sdo_geom_metadata) have an entry for the table and the table has a spatial index - whose owner is the same user as the one you connect.
Also, your query ( select ... from 'vtable') has a column 'shape' which is a buffer of the column lyv_chantier.shape. The sdo_filter, in this sql, expects a spatial index on the vtable.shape - which cannot exist. You should try to use a different alias (e.g. buf_shape) and sdo_filter(buf_shape,...) - to see if the sql fails in both instances, as it should.
I'm in a bit of a hurry right now, so my instructions are summarized. If you want, do this debugging and post the results. We then can go into details.
EDIT: Judging from your efforts, I'd say that the simplest approach is: 1) add a second geometry column to lyv_chantier (e.g. buf_shp). 2) update lyv_chantier set buf_shp = sdo_geom.sdo_buffer(shape,...). 3) insert into user_sdo_geom_metadata the values (lyv_chantier, buf_shp, ...). 4) create a spatial index on column buf_shp. You may need to consider a trigger to update buf_shp whenever shape changes...
This is a very practical approach but you don't provide any info about your case (what is the oracle version, how many rows does the table have, how is it used, why do you want to use sdo_buffer, etc), so that's my recommendation for now.
Also, since you are, most likely, using an sql view as layer in Geoserver (you don't say anything about that, either), you could also consider using pure GS functionality to achieve your goal.
At the end, without describing your goal, it's difficult to provide anything more tailor-made.

QueryDsl: Exception "argument type mismatch" with projection bean and oneToMany or manyToMany association

I have an association manyToMany between User and Role entities (User >---< Role)
I wanted to perform this query:
createQuery()
.from(qUser)
.leftJoin(qUser.roles, qRole)
.where(qUser.login.eq(login))
.singleResult(
Projections.bean(User.class,
qUser.id,
qUser.login,
qUser.password,
GroupBy.set(Projections.bean(Role.class,
qRole.id,
qRole.code
)).as(qUser.roles)
)
);
The generated query looks like this, for me it's perfect:
SELECT user0_.ID AS col_0_0_,
user0_.LOGIN AS col_1_0_,
user0_.PASSWORD AS col_2_0_,
role2_.ID AS col_4_0_,
role2_.CODE AS col_5_0_
FROM public.USER user0_
LEFT OUTER JOIN public.USER_ROLE roles1_ ON user0_.ID=roles1_.USER_ID
LEFT OUTER JOIN public.ROLE role2_ ON roles1_.ROLE_ID=role2_.ID
WHERE user0_.LOGIN=? LIMIT ?
But I have a java.lang.IllegalArgumentException: argument type mismatch.
I debugged and I found out that data from database id loaded without problem. This is when QueryDsl/Hibernate did some introspection to create and initialise my entities that the exception is throwed.
The problem is that the User.setRoles(Set<Role>) method has called with a long parameter: The ID of the first Role entity list of the User. Instead of create a Set of Role an then associate these roles to the User.
Is there a problem with the query? Or is it not supported by QueryDsl?
I am using the QueryDsl 3.6.6 (I tested with 3.7.4: same result)
I guess that the java.lang.IllegalArgumentException: argument type mismatch are thrown not by the JOIN comparing and you can verify that by verifing the ID type of the 3 tables UTILISATEUR, ROLE and USER_ROLE.
If there is a difference on type between
UTILISATEUR.ID=USER_ROLE.USER_ID
or
USER_ROLE..ROLE_ID=ROLE.ID
so that's the probleme.
If there is not a problem , the exception is thrown by the test of equality of the login.
But, I suspect that the USER_ROLE join table did not connect with the correct table. You may have two tables USER and UTILISATEUR . Unless you renamed the join table.
I have had same error as you,
I try different ways and it took a lot of time.Eventually I founded this way,
My classes are:
Lesson
LessonScores for saving user scores
LessonScoresModel for returning data and
LessonScoresModel.ScoresModel is a static nested class
I need some information about user scores in a class.
I use below query for returning data.
I use exact related models for retuning data but it's gives the beatiful "argument type mismatch" error.
So I developed a static class for returning data.
JPAQuery query = new JPAQuery(em);
//select from which model you need
query.from(QLessonScores.lessonScores);
//use your condition
query.where(predicate);
//use query and returning data
Map<Lesson,List<LessonScoresModel.ScoresModel>> map = (Map<Lesson,List<LessonScoresModel.ScoresModel>>)
//use tranform for making group by
query.transform(GroupBy.groupBy(QLessonScores.lessonScores.lesson).as(
//return list of data which column we need
GroupBy.list(Projections.fields(LessonScoresModel.ScoresModel.class,
QLessonScores.lessonScores.score.as("score"),
QLessonScores.lessonScores.scoresType.as("scoresType"),
QLessonScores.lessonScores.success.as("success")
))
));
you will find more information in these links,
codata
programcreek

How to use LINQ to Entities to make a left join using a static value

I've got a few tables, Deployment, Deployment_Report and Workflow. In the event that the deployment is being reviewed they join together so you can see all details in the report. If a revision is going out, the new workflow doesn't exist yet new workflow is going into place so I'd like the values to return null as the revision doesn't exist yet.
Complications aside, this is a sample of the SQL that I'd like to have run:
DECLARE #WorkflowID int
SET #WorkflowID = 399 -- Set to -1 if new
SELECT *
FROM Deployment d
LEFT JOIN Deployment_Report r
ON d.FSJ_Deployment_ID = r.FSJ_Deployment_ID
AND r.Workflow_ID = #WorkflowID
WHERE d.FSJ_Deployment_ID = 339
The above in SQL works great and returns the full record if viewing an active workflow, or the left side of the record with empty fields for revision details which haven't been supplied in the event that a new report is being generated.
Using various samples around S.O. I've produced some Entity to SQL based on a few multiple on statements but I feel like I'm missing something fundamental to make this work:
int Workflow_ID = 399 // or -1 if new, just like the above example
from d in context.Deployments
join r in context.Deployment_Reports.DefaultIfEmpty()
on
new { d.Deployment_ID, Workflow_ID }
equals
new { r.Deployment_ID, r.Workflow_ID }
where d.FSJ_Deployment_ID == fsj_deployment_id
select new
{
...
}
Is the SQL query above possible to create using LINQ to Entities without employing Entity SQL? This is the first time I've needed to create such a join since it's very confusing to look at but in the report it's the only way to do it right since it should only return one record at all times.
The workflow ID is a value passed in to the call to retrieve the data source so in the outgoing query it would be considered a static value (for lack of better terminology on my part)
First of all don't kill yourself on learning the intricacies of EF as there are a LOT of things to learn about it. Unfortunately our deadlines don't like the learning curve!
Here's examples to learn over time:
http://msdn.microsoft.com/en-us/library/bb397895.aspx
In the mean time I've found this very nice workaround using EF for this kind of thing:
var query = "SELECT * Deployment d JOIN Deployment_Report r d.FSJ_Deployment_ID = r.Workflow_ID = #WorkflowID d.FSJ_Deployment_ID = 339"
var parm = new SqlParameter(parameterName="WorkFlowID" value = myvalue);
using (var db = new MyEntities()){
db.Database.SqlQuery<MyReturnType>(query, parm.ToArray());
}
All you have to do is create a model for what you want SQL to return and it will fill in all the values you want. The values you are after are all the fields that are returned by the "Select *"...
There's even a really cool way to get EF to help you. First find the table with the most fields, and get EF to generated the model for you. Then you can write another class that inherits from that class adding in the other fields you want. SQL is able to find all fields added regardless of class hierarchy. It makes your job simple.
Warning, make sure your filed names in the class are exactly the same (case sensitive) as those in the database. The goal is to make a super class model that contains all the fields of all the join activity. SQL just knows how to put them into that resultant class giving you strong typing ability and even more important use-ability with LINQ
You can even use dataannotations in the Super Class Model for displaying other names you prefer to the User, this is a super nice way to keep the table field names but show the user something more user friendly.

How to accomplish this query in RavenDB

I have something like this in RavenDb:
Path [hasMany] Goals [hasMany] Achievements
So I have one document that contains goals and achievements.
If it where a traditional database, I could make this query
select g.id, count(a.id)
from path p
join goals g on p.id = g.pathId
join achievement a on a.id = g.id
group by goal.id
How do I do this query in ravendb?
Right now I'm doing it like this:
Path path = RavenSession.Load<Path>(pathId);
path.Goals.Select(x => new
{
Id = x.Id,
Name = x.Name,
TOnCourse = x.Achievements.Where(y => y.Resolution == Resolution.OnCourse).Count(),
TAstray = x.Achievements.Where(y => y.Resolution == Resolution.Astray).Count()
});
I don't want to do it this way, because I think that I'm loading all the "Goals" and "Achievements" and what I only want is to make the aggregation in the DB (ravendb)
If you want to know what Path, Goals and Achievements really are in my model here is my "pet" project http://dpath.apphb.com/
EDIT
I think I will get those "calculated" values and put them in Goal as properties, because I think the calculation is going to be very common, so keeping it simple since an achievement cannot be deleted, and the user will have a "relation" with goal and the quantity of achievements in order to get "My Stats"
Nevertheless, I would like to know how would you accomplish that query in RavenDb
Thanks!
You are loading just a single document for this, so that is likely to not be an issue.
You aren't loading anything else.
You can do this in an index, and then load the stored fields from there, but I don't think you need this unless your document is very large.