Slow SQL view using several subqueries - sql

There is probably a much better way to create these views. I have limited SQL experience so this is the way I designed it, I am hoping some of you SQL gurus can point me in a more efficient direction.
I essentially have 3 tables (sometimes 4) in my view, here is the essential structure:
Table USER
USER_ID | EMAIL | PASSWORD | CREATED_DATE
(Indexes: USER_ID)
Table USER_META
ID | USER_ID | NAME | VALUE
(Indexes: ID,USER_ID,NAME)
Table USER_SCORES
ID | USER_ID | GAME_ID | SCORE | CREATED_DATE
(Indexes: ID,USER_ID)
All the tables use the first ID column as an auto-increment primary key.
The second table "USER_META" is where I keep all the contact info and other misc. Primarily it is first_name,last_name, street,city, etc. - Depending on the user this could be 4 items or 140, which is why I use this table instead of having 150 columns in my USER table.
For reports, searching and editing I need about 20 values from USER_META, so I have views that look like this:
View V_USR_META
select USER_ID,EMAIL,
(select VALUE from USER_META
where NAME = 'FIRST_NAME' and USER_ID = u.USER_ID) as first_name,
(select VALUE from USER_META
where NAME = 'LAST_NAME' and USER_ID = u.USER_ID) as last_name,
(select VALUE from USER_META
where NAME = 'CITY' and USER_ID = u.USER_ID) as city,
(select VALUE from USER_META
where NAME = 'STATE' and USER_ID = u.USER_ID) as state,
(select VALUE from USER_META
where NAME = 'ZIP' and USER_ID = u.USER_ID) as zip,
/* 10 more selects for different meta values here */
(select max(SCORE) from USER_SCORES
where USER_ID = u.USER_ID) as high_score,
(select top (1) CREATED_DATE from USER_SCORES
where USER_ID = u.USER_ID
order by id desc) as last_game
from USER u
This get's pretty slow, and there are actually many more sub queries, this is just to illustrate the query. I also have to query a few other tables to get misc. info about the user.
I use the view when searching for a user, searches use name or userid or email or score, etc. I also use it to populate the user information screen when I present all the data in one place.
So - Is there a better way to write the view?

An alternative to all of those correlated subqueries would be to use max with case:
select u.USER_ID,
u.EMAIL,
max(case when um.name = 'FIRST_NAME' then um.value end) first_name,
max(case when um.name = 'LAST_NAME' then um.value end) last_name
...
from USER u
left join USER_META um
on u.user_id = um.user_id
group by u.user_id, u.email
Then you could add the user_scores results:
select u.USER_ID,
u.EMAIL,
max(case when um.name = 'FIRST_NAME' then um.value end) first_name,
max(case when um.name = 'LAST_NAME' then um.value end) last_name
...,
max(us.score) maxscore,
max(us.created_date) maxcreateddate
from USER u
left join USER_META um
on u.user_id = um.user_id
left join USER_SCORES us
on u.user_id = us.user_id
group by u.user_id, u.email

WITH Meta AS (
SELECT USER_ID
,FIRST_NAME
,LAST_NAME
,CITY
,STATE
,ZIP
FROM USER_META
PIVOT (
MAX(VALUE) FOR NAME IN (FIRST_NAME, LAST_NAME, CITY, STATE, ZIP)
) AS p
)
,MaxScores AS (
SELECT USER_ID
,MAX(SCORE) AS Score
FROM USER_SCORES
GROUP BY USER_ID
)
,LastGames AS (
SELECT USER_ID
,MAX(CREATED_DATE) AS GameDate
FROM USER_SCORES
GROUP BY USER_ID
)
SELECT USER.USER_ID
,USER.EMAIL
,Meta.FIRST_NAME
,Meta.LAST_NAME
,Meta.CITY
,Meta.STATE
,Meta.ZIP
,MaxScores.Score
,LastGames.GameDate
FROM USER
INNER JOIN Meta
ON USER.USER_ID = Meta.USER_ID
LEFT JOIN MaxScores
ON USER.USER_ID = MaxScores.USER_ID
LEFT JOIN LastGames
ON USER.USER_ID = LastGames.USER_ID

Related

join table on condition

I have 3 tables user, student_data, teacher_data. A user can be either student or a teacher. If it is the teacher I want to join user and teacher_data. And if it is a student then I want to join user with student_data.
How I can do this join with the condition.
I'd combine the two data tables in a sub-query, and then join the users to that.
SELECT
*
FROM
usr u
LEFT JOIN
(
SELECT user_id, datum, xxx, NULL AS yyy FROM student_data
UNION ALL
SELECT user_id, datum, NULL, yyy FROM teacher_data
)
d
ON d.user_id = u.id
https://dbfiddle.uk/?rdbms=oracle_21&fiddle=9b801ea739d42fe50c00ef4e17eaf143
NOTES:
The columns selected from the two data tables must match
Any unmatched columns must either be skipped or filled with NULL
Please don't call a table user, it's a reserved keyword and Oracle won't allow it.
You can write it like this:
select u.user_id,
s.student_id,
t.teacher_id
from usr u
left join student_data s on u.user_id=s.student_id
left join teacher_data t on u.user_id=t.teacher_id
where s.student_id is not null or t.teacher_id is not null
order by u.user_id
For every user_id check if he is a student or teacher, if he is student get his student column values else null, if he is a teacher get his teacher column values else null.
maybe try a union - something like this
select user_id, user_other_stuff
from user, student_data
where user.user_id = student_data.user_id
UNION
select user_id, user_other_stuff
from user, teacher_data
where user.user_id = teacher_data.user_id

Join 2 tables on foreign key while using count() in SQL

So I have two tables: Please see the ER diagram here
I want to use SELECT to create one table with "name" from the USER table, "id" as the foreign key for the two tables, and the count of friend_id as the number of friends each user has.
Here is my code:
SELECT name, id, (SELECT count(friend_id) as number
FROM friend
GROUP BY user_id)
FROM user
ORDER BY number DESC
I'm wondering what's the problem with these lines. Thank you!
You can use a subquery to calculate the count.
SELECT name, id, COALESCE(f.Count, 0) AS friend_count
FROM user u
LEFT JOIN (
SELECT user_id, COUNT(DISTINCT friend_id) AS Count
FROM friend
GROUP BY user_id
) f ON f.user_id = u.id
ORDER BY friend_count DESC
I used a LEFT JOIN so that if a user doesn't have a row in friend, it will still return a row with a friend count of 0 (thanks to COALESCE). I also added a DISTINCT so that if the friend has duplicates the friend is counted only one, might not be necessary especially if you have a UNIQUE INDEX setup on columns user_id, friend_id
Just add where to find only one id and remove group by because you have only one id for one or more friends as your diagram says.
SELECT name, id, (SELECT count(friend_id) as number
FROM friend
WHERE user_id = user.id)
FROM user
ORDER BY number DESC
I think this will be correct for you puprose
CREATE TABLE #user(
id VARCHAR(22),
[name] VARCHAR(255),
)
CREATE TABLE #friend(
user_id VARCHAR(22),
friend_id VARCHAR(22)
)
SELECT name, id, (SELECT COALESCE(COUNT(friend_id), 0)
FROM #friend f
WHERE f.user_id = u.id
GROUP BY user_id) as number
FROM #user u
ORDER BY number DESC
--Same query with join:
SELECT u.[name], u.id, COALESCE(COUNT(f.friend_id),0) number
FROM #user u
LEFT JOIN #friend f ON f.user_id = u.id
GROUP BY u.[name], u.id
ORDER BY number

Select count field of rows that are related to another table

I'm struggling with this issue from a long time and don't know how to solve it. It's hard for me to describe, so please be patient. There are two tables:
Table "Users"
UserId PK
Gender
Table "Forms"
FormId PK
UserId1 FK
UserId2 FK
Type
Forms are always related to two users, but not all users have related forms. Now I want to count specified gender only of those users, who have related forms.
So as a result, I want to have sth. like this:
# | Gender | GenderCount
1 | male | 43
2 | female | 12
3 | trans | 2
I tried the following SQL-Script but the result isn't distinct (the sum of all GenderCount is greater then the actual number of users)
SELECT u.Gender AS 'Gender', COUNT(u.Gender) AS 'GenderCount'
FROM Users u, Forms f
WHERE ((f.UserId1 = u.UserId)
OR (f.UserId2 = u.UserId))
AND (Type = 'Foo')
GROUP BY Gender
ORDER BY GenderCount
DESC
Any tips to solve this?
Let's take a look at what you want:
How many of each gender answered any form?
Note: each user should only be counted once, no matter how many forms they've filled out.
Phrased like this, the answer becomes fairly obvious, at least in pseudo-code:
SELECT
u.Gender,
COUNT(u.Gender)
FROM
Users u
WHERE
[User has answered a form]
GROUP BY
u.Gender
The easiest way to determine if a user has answered a form depends on the specific flavour of SQL being used. You'll need to use a subquery. There are a couple of options for how to access it.
IN is the most common method:
SELECT
u.Gender Gender,
COUNT(u.Gender) GenderCount
FROM
Users u
WHERE
u.id IN (
SELECT f.UserId1 user_id FROM Forms f WHERE Type = 'Foo'
UNION
SELECT f.UserId2 user_id FROM Forms f WHERE Type = 'Foo'
)
GROUP BY
Gender
ORDER BY
GenderCount DESC
Where available, EXISTS is more natural to read, and is sometimes faster:
SELECT
u.Gender Gender,
COUNT(u.Gender) GenderCount
FROM
Users u
WHERE
EXISTS(
SELECT '1'
FROM Forms f
WHERE
(f.UserId1 = u.id OR f.UserId2 = u.id)
AND Type = 'Foo'
)
GROUP BY
Gender
ORDER BY
GenderCount DESC
Regarding speed: The query optimiser will often convert IN to EXISTS where possible, to avoid selecting extra rows unnecessarily. However, the use of multiple columns necessitates either an OR or a UNION, so it may be pretty even in this case. ie: neither OR nor UNION play nicely with indexes.
SELECT u1.Gender AS 'Gender', COUNT(*) AS 'GenderCount'
FROM
Users u1
INNER JOIN
(SELECT DISTINCT u.UserId
FROM
Users u
INNER JOIN Forms f ON ((f.UserId1 = u.UserId)
OR (f.UserId2 = u.UserId))
AND (f.Type = 'Foo')) T ON T.UserId = u1.UserId
GROUP BY Gender
ORDER BY GenderCount DESC
Skip the join which is generating multiple rows per user:
SELECT Gender, COUNT(Gender) AS 'GenderCount'
FROM Users
WHERE UserId IN (SELECT UserId1 FROM Forms WHERE Type = 'Foo'
UNION
SELECT UserId2 FROM Forms WHERE Type = 'Foo')
GROUP BY Gender
ORDER BY GenderCount DESC
Or if you prefer to avoid a UNION (which is perfectly valid in this scenario BTW) you can use OR like this:
SELECT Gender, COUNT(Gender) AS 'GenderCount'
FROM Users
WHERE UserId IN (SELECT UserId1 FROM Forms WHERE Type = 'Foo')
OR UserId IN (SELECT UserId2 FROM Forms WHERE Type = 'Foo')
GROUP BY Gender
ORDER BY GenderCount DESC
As others have pointed out, there are ways to do this using a JOIN as well. However, a JOIN adds needless complexity for the DBMS engine as it will first need to match up the rows, and then reduce to DISTINCT values.
You should use
count(distinct u.UserId)
that way users only get counted once: count(distinct field_name) counts the number of unique values contained in field_name, so counting distinct on the primary key gives you the number of unique users, which is what you're looking for.
Also, instead of joining, you probably would be better off using an in clause like this
select Gender, count(distinct UserId) as GenderCount
from Users
where u.UserId in (select UserId1 from Forms) or u.UserId in (select UserId2 from Forms)
It's probably also going to be slightly faster.

query with count subquery, inner join and group

I'm definitely a noob with SQL, I've been busting my head to write a complex query with the following table structure in Postgresql:
CREATE TABLE reports
(
reportid character varying(20) NOT NULL,
userid integer NOT NULL,
reporttype character varying(40) NOT NULL,
)
CREATE TABLE users
(
userid serial NOT NULL,
username character varying(20) NOT NULL,
)
The objective of the query is to fetch the amount of report types per user and display it in one column. There are three different types of reports.
A simple query with group-by will solve the problem but display it in different rows:
select count(*) as Amount,
u.username,
r.reporttype
from reports r,
users u
where r.userid=u.userid
group by u.username,r.reporttype
order by u.username
SELECT
username,
(
SELECT
COUNT(*)
FROM reports
WHERE users.userid = reports.userid && reports.reporttype = 'Type1'
) As Type1,
(
SELECT
COUNT(*)
FROM reports
WHERE users.userid = reports.userid && reports.reporttype = 'Type2'
) As Type2,
(
SELECT
COUNT(*)
FROM reports
WHERE users.userid = reports.userid && reports.reporttype = 'Type3'
) As Type3
FROM
users
WHERE
EXISTS(
SELECT
NULL
FROM
reports
WHERE
users.userid = reports.userid
)
SELECT
u.username,
COUNT(CASE r.reporttype WHEN 1 THEN 1 END) AS type1Qty,
COUNT(CASE r.reporttype WHEN 2 THEN 1 END) AS type2Qty,
COUNT(CASE r.reporttype WHEN 3 THEN 1 END) AS type3Qty
FROM reports r
INNER JOIN users u ON r.userid = u.userid
GROUP BY u.username
If your server's SQL dialect requires the ELSE branch to be present in CASE expressions, add ELSE NULL before every END.
If you're looking for the "amountof report types per user", you'll be expecting to see a number, either 1, 2 or 3 (given that there are three different types of reports) against each user. You won't be expecting the reporttype (it'll just be counted not displayed), so you don't need reporttype in either the SELECT or the GROUP BY part of the query.
Instead, use COUNT(DISTINCT r.reporttype) to count the number of different reporttypes that are used by each user.
SELECT
COUNT(DISTINCT r.reporttype) as Amount
,u.username
FROM users u
INNER JOIN reports r
ON r.userid=u.userid
GROUP BY
u.username
ORDER BY u.username

MySQL- complex data query in a single statement

Consider the following structure :
alt text http://aeon-dev.org/pap/pap_db.png
Ignore the table user_token.
Now, imagine that you need to get all the roles related to an user, wich may be through it's related groups or directly related to him. In case the same role appears related to a group and the user directly, the role related to the user will prevail over the role given by the group.
Is there any chance this could be done in a single query?
Cheers!
Use:
SELECT DISTINCT r.name AS role_name
FROM USER u
LEFT JOIN USER_HAS_ROLE uhr ON uhr.user_id = u.id
LEFT JOIN USER_HAS_GROUP uhg ON uhg.user_id = u.id
LEFT JOIN GROUP_HAS_ROLE ghr ON ghr.group_id = uhg.group_id
LEFT JOIN ROLE r ON r.id = uhr.role_id
OR r.id = ghr.role_id
WHERE u.username = ?
try this:
Select role_id
From user_has_role
Where userId = #UserId
Union
Select role_id
From user_has_group g
Join Group_has_Role gr
On gr.GroupId = g.GroupId
Where userId = #UserId
This query will get every role a single user has either directly or given by a group
SELECT * FROM ROLE WHERE ID IN (
SELECT ROLE_ID
FROM USER_HAS_ROLE
WHERE USER_ID = 1
UNION
SELECT ROLE_ID
FROM USER_HAS_GROUP UG
INNER JOIN GROUP_HAS_ROLE GR ON UG.GROUP_ID = GR.GROUP_ID
WHERE USER_ID = 1
)
You could find all distinct roles for user 1 like:
select distinct role_id
from (
select uhr.user_id
, uhr.role_id
from user_has_role uhr
union all
select uhg.user_id
, ghr.role_id
from user_has_group uhg
join group_has_role ghr
on ghr.group_id = uhg.group_id
where not exists
(
select *
from user_has_role uhr
where uhr.user_id = uhg.user_id
and uhr.role_id = ghr.role_id
)
) user2role
where user_id = 1
Not sure how a role related to a user should "prevail", but you can assign priorities to them in the union.