How to chain mutiples IN AND on WHERE in SQL? - sql

I want to do something like :
SELECT "recipes"."id"
FROM "recipes"
INNER JOIN "groups" ON "groups"."recipe_id" = "recipes"."id"
INNER JOIN "steps" ON "steps"."group_id" = "groups"."id"
INNER JOIN "steps_ingredients_memberships" ON "steps_ingredients_memberships"."step_id" = "steps"."id"
INNER JOIN "ingredients" ON "ingredients"."id" = "steps_ingredients_memberships"."ingredient_id"
WHERE (ingredients.id IN (5, 6) AND ingredients.id IN (10, 11))
LIMIT 10
But this request return 0 rows...
I know this request can't work but i can't find a solution
I want to get "Recipes" that have ingredients 5 OR 6 AND 10 OR 11.
Think like :
5 = tomatoes
6 = big tomatoes
10 = potatoes
11 = big potatoes
I want "Recipes" with tomatoes OR big tomatoes AND potatoes OR big potatoes.
Solution adopted
Because ingredients can be random, I wrote a scope :
scope :ingredients_filtering, lambda { |ingredients|
return if ingredients.nil?
queries = []
ingredients.each do |ingredient|
related_ids = ingredient.related_ids.uniq.join(', ')
queries << "BOOL_OR(ingredients.id IN (#{related_ids}))"
end
group(:id).having(queries.join(' AND '))
}
Thanks you to all of you ! :)

To expand on #MrYoshiji's comment Here we can make this more "railsy" as follows:
ingredients_table = Ingredient.arel_table
ingredient_list1 = [5,6]
ingredient_list2 = [10,11]
condition = Arel::Nodes::NamedFunction.new(
'BOOL_OR',[ingredients_table[:id].in(ingredient_list1)]
).and(
Arel::Nodes::NamedFunction.new(
'BOOL_OR',[ingredients_table[:id].in(ingredient_list2)]
)
)
recipes = Recipe.joins(groups: {
steps: {
steps_ingredients_memberships: :ingredients
}
})
.group('recipes.id')
.having(condition)
This will produce SQL similar to the linked answer: (Provided by #ThorstenKettner) e.g.
SELECT recipes.*
FROM recipes
JOIN groups ON groups.recipe_id = recipes.id
JOIN steps ON steps.group_id = groups.id
JOIN steps_ingredients_memberships ON steps_ingredients_memberships.step_id = steps.id
JOIN ingredients ON ingredients.id = steps_ingredients_memberships.ingredient_id
GROUP BY
recipes.id
HAVING
BOOL_OR(ingredients.id IN (5, 6)) AND
BOOL_OR(ingredients.id IN (10, 11))
However it offers the flexibility to change ingredient_list1 and ingredient_list2 in a far more efficient manner without concern for escaping and the like.
BTW to address your comment:
"I put the ruby-on-rails tag because if there is a solution in ruby-on-rails it's better but I don't think so ..."
arel can assemble any query you can imagine (it can assemble invalid/fragmented SQL too). If it is valid SQL ActiveRecord can run it so if you are using rails and have query questions definitely include the ruby-on-rails tag like you did.
Update (based on your posted solution) but using arel rather than string concatenation
scope :ingredients_filtering, lambda { |ingredients|
return unless ingredients
ingredients_table = Ingredient.arel_table
conditions = ingredients.map do |ingredient|
Arel::Nodes::NamedFunction.new(
'BOOL_OR',[ingredients_table[:id].in(ingredient.related_ids)]
)
end.reduce(&:and)
group(:id).having(conditions)
}

You want aggregation. Group by recipe and see if it has the desired ingredients.
SELECT r.id
FROM recipes r
JOIN groups g ON g.recipe_id = r.id
JOIN steps s ON s.group_id = g.id
JOIN steps_ingredients_memberships sim ON sim.step_id = s.id
JOIN ingredients i ON i.id = sim.ingredient_id
GROUP BY r.id
HAVING BOOL_OR(i.id IN (5, 6)) AND BOOL_OR(i.id IN (10, 11));

use GROUP BY and HAVING:
SELECT "recipes"."id"
FROM "recipes"
INNER JOIN "groups" ON "groups"."recipe_id" = "recipes"."id"
INNER JOIN "steps" ON "steps"."group_id" = "groups"."id"
INNER JOIN "steps_ingredients_memberships" ON "steps_ingredients_memberships"."step_id" = "steps"."id"
INNER JOIN "ingredients" ON "ingredients"."id" = "steps_ingredients_memberships"."ingredient_id"
GROUP BY "recipes"."id"
HAVING COUNT(CASE WHEN ingredients.id IN (5, 6) THEN 1 END) > 0
AND COUNT(CASE WHEN ingredients.id IN (10, 11) THEN 1 END) > 0

Related

Long query execution - SSP Datatables

I use Datables with SSP Class. And i have query with result 2000 lines.
But when I try running query I got error 500/504 error (if I have 504 table is didn't load)
I use OVH CloudDB with MariadDB 10.2 and on server I have php 7.2
My joinQuery looks like:
FROM data_platforms p
LEFT JOIN game_platforms gp ON gp.platform_id = p.platform_id
LEFT JOIN games g ON g.game_id = gp.game_id
LEFT JOIN tastings t ON t.tasting_game_id = g.game_id
LEFT JOIN notes n ON n.note_game_id = g.game_id
LEFT JOIN ratings r ON r.rating_game_id = g.game_id
LEFT JOIN images i ON i.image_type_id = g.game_id AND (i.image_type = 2 || i.image_type = 1)
LEFT JOIN game_generes gg ON gg.game_id = g.game_id
LEFT JOIN generes gen ON gen.id = gg.genere_id
And extraWhere
p.platform_id = '.$platformID.' AND g.game_status = 1
And groupBy
gp.game_id
Is there any way to be able to optimize this query?
Am I doomed to fail at this point, or should I use a different SSP class?
Don't use LEFT unless you really expect the 'right' table to be missing.
Don't "over-normalize". For example, I would expect genre to simply be a column, not a many-to-many mapping table plus a genre table.
(i.image_type = 2 || i.image_type = 1) --> i.image_type IN (1,2) might optimizer better.
See this for a likely improvement in many-to-many table indexes: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Please provide SHOW CREATE TABLE so we can check other indexes, such as on platform_id.
Let's see the entire query, plus EXPLAIN SELECT .... GROUP BY gp.game_id may be invalid (unless everything else is dependent on it).

Why does this query in this n-n relation return nothing?

I'm trying to do the following query but nothing happends. I get no results.
SELECT * FROM "product"
LEFT OUTER JOIN (
"product_filter"
INNER JOIN "filter_value"
ON "filter_value"."filterValueId" = "product_filter"."filterValueId")
ON "product"."productId" = "product_filter"."productId"
WHERE "product"."isAdult" = false
AND ("filter_value"."filterValueId" = '4'
AND "filter_value"."filterValueId" = '8');
I know that the problem comes from the filterValueId that has to be equal to 4 and 8. If I remove one of them, I get the result. But I need the result of both.
filter_value is a relationnal table for a n-n relation.
The query looks correct to me but nothing come out.
What is worse is that when I execute the LEFT OUTER JOIN query, I get results:
"product_filter"
INNER JOIN "filter_value"
ON "filter_value"."filterValueId" = "product_filter"."filterValueId"
I use sequelize to built the following relation:
db.product.belongsToMany(
db.filterValue,
{
through: db.productFilter,
as: 'productFilters',
foreignKey: 'productId'
});
db.filterValue.belongsToMany(
db.product,
{
through: db.productFilter,
as: 'productFilters',
foreignKey: 'filterValueId'
});
I've check tables content multiple times and everything is correct.
I really don't understand why this query returns nothing?
You seem to understand why it returns nothing. The filter value cannot be both 4 and 8 at the same time.
I think you want something like this:
select p.*
from "product" p
where p.ProductId in (select pf.ProductId
from product_filter pf
where pf.filterValueId in (4, 8)
group by pf.ProductId
having count(distinct pf.filterValueId = 2)
);
Note: You do not need the filter table at all, because the filter value is in product_filter. (This assumes it is not needed for some type of filtering.)
You might find the use of two exists subqueries faster:
select p.*
from "product" p
where exists (select pf.ProductId
from product_filter pf
where pf.ProductId = p.ProductId and
pf.filterValueId = 4
) and
exists (select pf.ProductId
from product_filter pf
where pf.ProductId = p.ProductId and
pf.filterValueId = 8
);
"filter_value"."filterValueId" = '4' AND "filter_value"."filterValueId" = '8
filterValueId cannot be 4 and 8 at the same time.
Maybe you want
"filter_value"."filterValueId" = '4' OR "filter_value"."filterValueId" = '8

Why LINQ Count() returning multiple rows instead one?

I would like to translate the following SQL into LINQ:
select count(p.ID) as NumPosts,
count(t.Trustee_ID)as TrusteePost,
count(pat.ID)as PatientPost,
count(s.ID) as SpecialistPost
from [dbo].[Posts] as p
left join [dbo].[Trusteehips] as t
on p.Autor_ID = t.Trustee_ID
left join [dbo].[Patients] as pat
on p.Autor_ID = pat.ID
left join [dbo].[Specialists] as s
on p.Autor_ID = s.ID
where p.Deleted = 0
I I've tried this:
var res = from p in context.Posts
join t in context.Trusteeships
on p.Autor.ID equals t.Trustee.ID into tGroup
join pat in context.Patients
on p.Autor.ID equals pat.ID into patGroup
join s in context.Specialists
on p.Autor.ID equals s.ID into sGroup
select new NumUserPosts
{
//CountAllPosts = ?
TrusteePost = tGroup.Count(),
PatientPost = patGroup.Count(),
SpecialistPost = sGroup.Count()
};
But result is this:
1 0 0
0 0 1
0 0 1
0 1 0
and etc.
I expect result
TrusteePost PatientPost SpecialistPost
1000 2000 3000
Why when i try to count group return this result?
SQL query is correct. I would like to translate into LINQ.
The query returns 0 or 1 records per joined Trustee, etc. because you outer join by the unique primary key. So a join into (which is a GroupJoin in fluent syntax) produces a group of 0 or 1 records. If you run the generated SQL query and view the raw query result you'd probably understand better what's going on.
The problem is, there is no LINQ equivalent for count(t.Trustee_ID), etc. Therefore it's impossible to do what you want in one query without "hacking".
Hacking it into one query could be done like so:
(from p in context.Posts.Take(1)
select new
{
TrusteePost = context.Posts
.Count(p1 => context.Trusteeships.Any(x => x.ID == p1.Autor.ID)),
PatientPost = context.Posts
.Count(p2 => context.Patients.Any(x => x.ID == p2.Autor.ID)),
SpecialistPost = context.Posts
.Count(p3 => context.Specialists.Any(x => x.ID == p3.Autor.ID))
})
.AsEnumerable()
.Select(x => new NumUserPosts
{
CountAllPosts = x.TrusteePost + x.PatientPost + x.SpecialistPost,
x.TrusteePost,
x.PatientPost,
x.SpecialistPost
}
The SQL query will be much more elaborate than the original SQL (for example, it involves cross joins), but it will probably still perform pretty well. AsEnumerable prevents the second part from being executed as SQL, which would bloat the SQL statement even more. It simply runs in memory.
I consider this a hack because the first part, context.Posts.Take(1) doesn't really have any meaning, it's only there to serve as a wrapper for the three separate queries. It's poor man's query packaging.
It looks like you're doing group joins instead of left outer joins (see this page).
A left outer join looks more like:
var res = from p in context.Posts
join t in context.Trusteeships
on p.Autor.ID equals t.Trustee.ID into tGroup
from tJoin in tGroup.DefaultIfEmpty()
join pat in context.Patients
on p.Autor.ID equals pat.ID into patGroup
from patJoin in patGroup.DefaultIfEmpty()
join s in context.Specialists
on p.Autor.ID equals s.ID into sGroup
from sJoin in sGroup.DefaultIfEmpty()
select ...
Unfortunately, it doesn't seem that Linq can create a query to count elements in each column.
If you don't mind using multiple queries, you could count each separately, for example:
var trusteePost = (from p in context.Posts
join t in context.Trusteeships on p.Autor.ID equals t.Trustee.ID
select t).Count()

MySQL, SQL Select Statement, Where with OR... What's wrong with this?

I'm looking for help with my query below. which is never returning anything for veggie... Is the way I have my WHERE statement written valid?
SELECT *
FROM newsfeed INNER JOIN newsfeedaction ON newsfeed.newsfeedactionid = newsfeedaction.newsFeedActionID
INNER JOIN person ON newsfeed.personID = person.personID
LEFT OUTER JOIN food ON newsfeed.foodID = food.foodID
LEFT OUTER JOIN veggie ON newsfeed.veggieID = veggie.veggieID
WHERE
(
newsfeed.veggieID IS NOT NULL
AND veggie.deleted = 'N'
)
OR
(
newsfeed.foodID IS NOT NULL
AND food.deleted = 'N')
The where clause is incomplete. The second set of conditions are to be completed.
Unless the VEGGIE.veggieid can be null (likewise for FOOD.foodid), use:
SELECT *
FROM NEWSFEED nf
JOIN NEWSFEEDACTION nfa ON nfa.newfeedactionid = nf.newsfeedactionid
JOIN PERSON p ON p.personid = nf.personid
JOIN FOOD f ON f.foodid = nf.foodid
AND f.deleted = 'N'
JOIN VEGGIE v ON v.veggieid = nf.veggieid
AND v.deleted = 'N'
The query is otherwise correct, but requires that NEWSFEED records must have supporting records in both the VEGGIE and FOOD tables.
A Union between the two sets of data did the trick and I read online that UNION between two selects is faster than a WHere with 2 groups

MySQL 4.0 query help double join in lieu of subqueries

select u.user, g.group, u2g.something
from users, groups, u2g
where users.u = u2g.u and groups.g = u2g.g
that returns data like this:
user, group, something
----------------------
1 , 3, a
1 , 5, b
2 , 3, c
3 , 3, d
4 , 5, e
now I would like to limit this query in such a way that it only shows users which are both in groups 3 and 5 - so it would only return {1,3, a} , {1,5, b} for my example data.
edit: I added another column to the data because there may be an incorrect solution using a group by.
edit2: sorry, I was misled by documentation. MySQL 4.0 does not support subqueries :(
edit3: This SQL will be generated programatically for any number of groups (well, up to 20 in current specification) so I would like to avoid solutions that give me too much additional coding to do. If a solution will not be found, I will just modify the resulting .Net 1.1 DataTable, but I would like to avoid that if possible.
edit4: any new idea? Perhaps one without subqueries that includes IN (3,5)?
Using double join with groups-table will give you the correct result:
select u.user, u2g.something
from users
INNER JOIN u2g ON users.u = u2g.u
INNER JOIN groups g1 ON u2g.g = g1.g AND g1.group = 3
INNER JOIN groups g2 ON u2g.g = g2.g AND g2.group = 5
/* try this for two rows, one for each group */
INNER JOIN groups ON u2g.g = groups.g
However, this doesn't exactly match your request, that you want two rows, one for each group. This will only give you one row, you might be able to join it once more with groups therefor rendering two rows.
Another example (if you're selecting using the same groupID that you map against ):
SELECT u.uID, gm.something
FROM cdcms_users u
inner join cdcms_group_memberships gm1 on gm1.uID = u.uID AND gm1.gID = 32
inner join cdcms_group_memberships gm2 on gm2.uID = u.uID AND gm2.gID = 33
select u.user, g.group, u2g.something
from users u, groups g, u2g
where u.user = u2g.user and g.group = u2g.group
where exists
(select 1
from u2g u2g2
where u2g2.user=u.user and u2g2.group in(3,5))
Something along these lines?
select u.[user], g.group
from u
inner join ug on ug.userid = u.id
inner join g on g.id = ug.groupid
inner join
(
select ug.userid
from ug
where ug.groupid in (1,2)
group by ug.userid
having count(*) = 2
) sub on sub.userid = u.id
-Edoode
Quite hideous non-general solution that results in two rows in Oracle:
select users.u, groups.g
from users , groups, u2g, groups g2, u2g u2g2
where users.u = u2g.u
and users.u = u2g2.u
and groups.g = u2g.g
and g2.g = u2g2.g
and (groups.g in (3,5) and g2.g in (3,5) and groups.g <> g2.g)
;
Why is groups used in the query? Its only accessed field (g) exists in u2g. I imagine you probably want to bring back a boatload of stuff from there as well.
In order to get the sort of resultset you describe without the use of subqueries you end up with a real mess: a quadratic explosion of query text!
You'll need something of the following form:
select users.u, groups.g, u2g0.something
from users u, groups g, u2g u2g0, u2g u2g1
where groups.g = 3
and users.u = u2g0.u
and u2g0.g = 3
and users.u = u2g1.u
and u2g1.g = 5
union all
select users.u, groups.g, u2g1.something
from users u, groups g, u2g u2g0, u2g u2g1
where groups.g = 5
and users.u = u2g0.u
and u2g0.g = 3
and users.u = u2g1.u
and u2g1.g = 5
Since this is a programmatically generated query, I'll use a web-page-scripting-like notation here to describe the query construction. I will also make the rash and unwarranted simplifying assumption that the group identifiers are not a potential SQL-injection attack vector. :-)
<% for(int i = 0; i < requiredGroups.Length; i++) { %>
<% if(i > 0) { %>
union all
<% } %>
select users.u, groups.g, u2g<%=i%>.something
from users u, groups g
<% for(int j = 0; j < requiredGroups.Length; j++) { %>
, u2g u2g<%=j%>
<% } %>
where groups.g = <%=requiredGroups[i]%>
<% for(int j = 0; j < requiredGroups.Length; j++) { %>
and users.u = u2g<%=j%>.u
and u2g<%=j>.g = <%=requiredGroups[j]%>
<% } %>
<% } %>