I have written a PostgresSQL query that is relatively performant at scale and gives me the dataset I want back, but I am wondering if it is the simplest/best way to write the query. It seems like there should be a simpler join operation that satisfies the conditions I need.
EDIT: I do need this to be performant on large tables. In the example given below, pets is 150 million rows, food is roughly 100k rows. My solution at the bottom clocks in at about 0.6ms. Both tables have an index on id and user_id. The food table also includes an index on pet_id.
I have two tables that are related in my system that have one guaranteed shared attribute - the user_id. Here is an example that in essence shows my problem:
Pets
+------+-------+---------+
| id | type | user_id |
+------+-------+---------+
| 1234 | dog | 1 |
| 1235 | cat | 1 |
| 1236 | gecko | 1 |
+------+-------+---------+
Food
+------+-----------+---------+--------+
| id | name | user_id | pet_id |
+------+-----------+---------+--------+
| 4321 | hamburger | 1 | NULL |
| 4322 | dog food | 1 | 1234 |
| 4323 | cat food | 1 | 1235 |
+------+-----------+---------+--------+
Desired Results
+------+------+
| p.id | f.id |
+------+------+
| NULL | 4321 | --no pet, hamburger
| 1234 | 4322 | --dog, dog food
| 1235 | 4323 | --cat, cat food
| 1236 | NULL | --gecko, no food
+------+------+
Now with an example to refer to, I'll make sure it's clear what the result is. The result contains all rows from both sides that belong to my user_id (imagine that the table could contain thousands of other rows that don't belong to user_id 1). I want these result rows to include exactly ONE copy of each row matched to the other table.
An example of a full outer join that I tried to make this work:
SELECT p.id, f.id
FROM pets p FULL OUTER JOIN food f ON p.user_id = f.user_id
WHERE p.user_id = 1;
There's a bit of a problem in this query because
It excludes NULLs from the left side of the query. I need those.
Because the user_id is essentially the constant here, I end up with plenty of duplicates because it matches on user_id. Every row from the left gets matched to every row from the right. Not what I need. I need a one-to-one match.
I could fix #1 by including an OR in the WHERE filter:
SELECT p.id, f.id
FROM pets p FULL OUTER JOIN food f ON p.user_id = f.user_id
WHERE p.user_id = 1 OR f.user_id = 1;
For reasons I'm not completely sure of, it makes the query take a very long time. In our system, both tables have an index on user_id, so it isn't the lack of an index.
To solve my issue, I landed on the following query (really two combined):
SELECT p.id, f.id
FROM pets p LEFT JOIN food f
ON p.id = f.pet_id AND f.user_id = 1
WHERE p.user_id = 1
UNION
SELECT p.id, f.id FROM pets p RIGHT JOIN food f
ON p.id = f.pet_id
WHERE f.user_id = 1 AND p.id IS NULL;
So my question is this: Is there a simpler way to execute this as a single query?
SQL DEMO
SELECT p.id, f.id
FROM pets p
FULL OUTER JOIN food f
ON p.user_id = f.user_id
AND p.id = f.pet_id
AND p.user_id = 1;
OUTPUT
| id | id |
|--------|--------|
| 1234 | 4322 |
| 1235 | 4323 |
| 1236 | (null) |
| (null) | 4321 |
NOTE:
You should add a composite index on (user_id, pet_id) for both tables.
You're just overthinking this a bit. You want to join on P.ID = F.PET_ID:
SELECT P.ID, F.ID
FROM PETS P
FULL OUTER JOIN FOOD F ON P.ID = F.PET_ID
AND P.USER_ID = F.USER_ID
AND P.USER_ID = 1 --optional
ORDER BY P.ID
Related
I'm trying to use the SQL join function to grab information from multiple tables.
My issue is I can't seem to get the desired result.
select a.DRINKER, sum(C.PRICE)
from DRINKERS a
left join ORDERS b on a.DRINKER = b.DRINKER
join SERVES c on b.PUB = c.PUB and d.DRINK = c.DRINK
group by a.DRINKER;
This gives the following results
----------------------
|DRINKER|sum(C.PRICE)|
----------------------
| BOB | 200.10 |
| NED | 172.50 |
| JOE | 270.90 |
| TIM | 80.10 |
----------------------
However I want this to be giving all of the people in a.DRINKER like such:
----------------------
|DRINKER|sum(C.PRICE)|
----------------------
| BOB | 200.10 |
| NED | 172.50 |
| JOE | 270.90 |
| TIM | 80.10 |
| PAT | null |
| ANN | null |
----------------------
Any guidance would be appreciated and if you could also explain the logic behind the changes that would be greatly appreciated as I wanna learn what I should be doing! Thanks in advance!
Even though you got a left join between DRINKERS and ORDERS, the join between ORDERS and SERVES, will filter out any nulls obtained in the first left join.
To fix this you could try by further left joining the tables
select a.DRINKER, sum(C.PRICE)
from DRINKERS a
left join ORDERS b on a.DRINKER = b.DRINKER
left join SERVES c on b.PUB = c.PUB and d.DRINK = c.DRINK
group by a.DRINKER;
I'm trying to display left over records after matching one-to-one rows. How do I display extra/left over records after joining two tables?
Suppose I have two tables, A and B. They both display the the same transactions at the end of the day. However, Table A has more detail about the records but is late in getting updated. Table B, on the other hand, has limited information about transactions but is updated several hours before Table A.
I need a query that can return which records have yet to appear in Table A from Table B.
TABLE A
+-------+-----+---------+----------+---------------------------+
| NAME | ID | AMOUNT | TYPE | PROCESSED TIMESTAMP |
+-------+-----+---------+----------+---------------------------+
| ABC | 123 | -420.07 | PURCHASE | 2018-09-06-08.26.32.000000|
| ABC | 123 | 420.07 | REFUND | 2018-09-06-07.12.18.000000|
| BBC | 456 | -5.00 | PURCHASE | 2018-09-06-10.25.13.000000|
+-------+-----+---------+----------+---------------------------+
TABLE B
+----+----------+---------------------------+
| ID | AMOUNT | RECEIVED TIMESTAMP |
+----+----------+---------------------------+
|123 | -420.07 | 2018-09-05-09.26.15.000000|
|123 | 420.07 | 2018-09-05-08.12.03.000000|
|123 | -420.07 | 2018-09-05-08.40.00.000000|
|456 | -5.00 | 2018-09-05-08.45.00.000000|
+----+----------+---------------------------+
QUERY RESULTS
+----+----------+
| ID | AMOUNT |
+----+----------+
|123 | -420.07 |
+----+----------+
I can manage to find all the records related to the ID that is "off balance" but I need only the specific records that are extra:
SELECT * FROM b
WHERE id
IN
(SELECT d.id AS id
FROM
(SELECT * FROM
(SELECT id, ROUND(SUM(amount),2) AS balance FROM a GROUP BY id) c
RIGHT JOIN
(SELECT id, ROUND(SUM(amount),2) AS balance FROM b GROUP BY id) d
ON c.id = d.id
WHERE c.balance <> d.balance))
Yields...
+----+----------+
| ID | AMOUNT |
+----+----------+
|123 | -420.07 |
|123 | 420.07 |
|123 | -420.07 |
+----+----------+
You need to read up more on joins . There are 3 basic joins which can make life much simpler.
INNER JOIN: First, this is not asked, but the query you have provided for finding off balance items is too complicated. It can be simplified by an inner join.
Inner join is a set operation which will basically get data from both tables (set) which match the condition.
select * from
(
(select id, sum(amount) from a group by id) group_A
INNER JOIN (select id, sum(amount) from b group by id) group_B
ON group_A.id = group_B.id
WHERE group_A.balance != group_B.balance
)
LEFT/RIGHT OUTER JOIN: Left outer join is an operation which will return all the data that is present in both sets and also the data that is in left set but not the right set. Right join is essential same operation on the right set. Important to notice that the extra records pulled here would be null for the side where they do not exist.
Since you want records which are present in table B but not in A, there are multiple ways of achieving this, one would be to get records present in both tables (inner join) and then get all the records in table B but not in the inner join done earlier. Using definition of group_A/group_B from the inner join example.
select id from b where id not in (
select id from group_A INNER JOIN group_B on group_A.id = group_B.id)
Or you can do a right outer join and then using the property of that fields fetched from table A would be coming as null, can filter out the required ID.
select group_B.id from group_A RIGHT OUTER JOIN group_B ON group_A.id = group_B.id
where group_A.id is null
Please use the primary keys on the joins to get the correct results as mentioned by user #ComputerVersteher
I think, you should add PK col.
I can't match data with table A and B, and can't seperate 2 rows at table B.
+----+----------+---------------------------+
| ID | AMOUNT | RECEIVED TIMESTAMP |
+----+----------+---------------------------+
|123 | -420.07 | 2018-09-05-09.26.15.000000|<-
|123 | 420.07 | 2018-09-05-08.12.03.000000|
|123 | -420.07 | 2018-09-05-08.40.00.000000|<-
|456 | -5.00 | 2018-09-05-08.45.00.000000|
+----+----------+---------------------------+
I add new col(deal_no) and made it.
https://www.db-fiddle.com/f/3GfZoQwGhBLf7YWf2RucBF/4
select tmp_B.deal_no, tmp_B.id, tmp_B.amount, tmp_A.deal_no
from tmp_B
left outer join tmp_A
on tmp_A.deal_no = tmp_B.deal_no
where tmp_A.deal_no is null
I am trying to figure out the best way to use a JOIN in MSSQL in order to do the following:
I have two tables. One table contains technician IDs and an example of one data set would be as follows:
+--------+---------+---------+---------+---------+
| tagid | techBid | techPid | techFid | techMid |
+--------+---------+---------+---------+---------+
| 1-1001 | 12 | 0 | 11 | 6 |
+--------+---------+---------+---------+---------+
I have another table that stores the names of these technicians:
+------+-----------+
| TTID | SHORTNAME |
+------+-----------+
| 11 | Steven |
| 12 | Mark |
| 6 | Pierce |
+------+-----------+
If the ID of a technician in the first table is 0, there is no technician of that type for that row (types are either B, P, F, or M).
I am trying to come up with a query that will give me a result that contains all of the data from table 1 along with the shortnames from table 2 IF there is a matching ID, so the result would look something like the following:
+--------+---------+---------+---------+---------+----------------+----------------+----------------+----------------+
| tagid | techBid | techPid | techFid | techMid | techBShortName | techPShortName | techFShortName | techMShortName |
+--------+---------+---------+---------+---------+----------------+----------------+----------------+----------------+
| 1-1001 | 12 | 0 | 11 | 6 | Mark | NULL | Steven | Pierce |
+--------+---------+---------+---------+---------+----------------+----------------+----------------+----------------+
I am trying to use a JOIN to do this, but I cannot figure out how to join on multiple columns multiple times to where it would look something like
Select table1.tagid, table1.techBid, table1.techPid, table1.techFid, table1.techMid, table2.shortname
FROM table1
INNER JOIN table2 on //Dont know what to put here
You need to use left joins like this:
Select table1.tagid, table1.techBid, table1.techPid, table1.techFid, table1.techMid,
t2b.shortname, t2p.shortname, t2f.shortname, t2m.shortname,
FROM table1
LEFT JOIN table2 t2b on table1.techBid = t2b.ttid
LEFT JOIN table2 t2p on table1.techPid = t2p.ttid
LEFT JOIN table2 t2f on table1.techFid = t2f.ttid
LEFT JOIN table2 t2m on table1.techMid = t2m.ttid
you just do mutiple left join
select tech.techPid, techPname.SHORTNAME
, tech.techFid, techFname.SHORTNAME
from tech
left join techName as techPname
on tech.techPid = techPname.TTID
left join techName as techFname
on tech.techFid = techFname.TTID
I'm working on a sql database with three tables.
PageIds
Links
Ranks
I want to query the database with a page name and find all links for the given page name, furthermore I want to get Links sorted by a Score.
The PageName used for the query either exists as PageA or as PageB, and if PageA then I want PageBs PageName and PageBs PageScore - and vice versa.
I know I have to use inner joins, but how should I handle 'or' in inner join and how do get PageId in PageIds, to query other table and then PageName in PageIds.
This what I have tried, but it take quite some time and then I will not get the opposite PageName
select * from PageIds
inner join Links
on Links.PageAId = PageIds.PageId or Links.PageBId = PageIds.PageId
inner join Ranks
on Ranks.PageId = Links.PageBId
where PageIds.PageName = #searchternm
PageIds
Get PageId for PageName matching #searchterm
+--------+----------+
| PageId | PageName |
+--------+----------+
| 1234 | NameA |
| 4321 | NameB |
| 3321 | NameC |
+--------+----------+
Links
if previously found PageId = PageAId,
then get PageBId and PageName for PageBId,
else get PageAId and PageName for PageAId
+---------+---------+
| PageAId | PageBId |
+---------+---------+
| 1234 | 3321 |
| 4321 | 3321 |
| 1234 | 4321 |
+---------+---------+
Ranks
if previously found PageId = PageAId,
then get score for PageBId,
else get score for PageAId
+--------+-----------+
| PageId | PageScore |
+--------+-----------+
| 1234 | 1 |
| 3321 | 4 |
| 4321 | 2 |
+--------+-----------+
If input is NameB then the output should be:
PageName, Score
NodeC, 4
NodeA, 1
Thanks for your help.
If I understand your requirements correctly you'll want to join the PageIDs table twice, once as PageAIDs and once as PageBIDs.
Then have one query pick up pageA details where pageB matches, then union it with a reversed query for pageB details where pageA matches.
SELECT PageA.PageName, RanksA.Score FROM Links
INNER JOIN PageIDs [PageA] on Links.PageAID = PageA.PageID
INNER JOIN PageIDs [PageB] on Links.PageBID = PageB.PageID
INNER JOIN Ranks [RanksA] on Links.PageAID = RanksA.PageID
WHERE PageB.PageName = #PageName
UNION
SELECT PageB.PageName, RanksB.Score FROM Links
INNER JOIN PageIDs [PageA] on Links.PageAID = PageA.PageID
INNER JOIN PageIDs [PageB] on Links.PageBID = PageB.PageID
INNER JOIN Ranks [RanksB] on Links.PageBID = RanksB.PageID
WHERE PageA.PageName = #PageName
My scenario: There are 3 tables for storing tv show information; season, episode and episode_translation.
My data: There are 3 seasons, with 3 episodes each one, but there is only translation for one episode.
My objetive: I want to get a list of all the seasons and episodes for a show. If there is a translation available in a specified language, show it, otherwise show null.
My attempt to get serie 1 information in language 1:
SELECT
season_number AS season,number AS episode,name
FROM
season NATURAL JOIN episode
NATURAL LEFT JOIN episode_trans
WHERE
id_serie=1 AND
id_lang=1
ORDER BY
season_number,number
result:
+--------+---------+--------------------------------+
| season | episode | name |
+--------+---------+--------------------------------+
| 3 | 3 | Episode translated into lang 1 |
+--------+---------+--------------------------------+
expected result
+-----------------+--------------------------------+
| season | episode| name |
+-----------------+--------------------------------+
| 1 | 1 | NULL |
| 1 | 2 | NULL |
| 1 | 3 | NULL |
| 2 | 1 | NULL |
| 2 | 2 | NULL |
| 2 | 3 | NULL |
| 3 | 1 | NULL |
| 3 | 2 | NULL |
| 3 | 3 | Episode translated into lang 1 |
+--------+--------+--------------------------------+
Full DB dump
http://pastebin.com/Y8yXNHrH
I tested the following on MySQL 4.1 - it returns your expected output:
SELECT s.season_number AS season,
e.number AS episode,
et.name
FROM SEASON s
JOIN EPISODE e ON e.id_season = s.id_season
LEFT JOIN EPISODE_TRANS et ON et.id_episode = e.id_episode
AND et.id_lang = 1
WHERE s.id_serie = 1
ORDER BY s.season_number, e.number
Generally, when you use ANSI-92 JOIN syntax you need to specify the join criteria in the ON clause. In MySQL, I know that not providing it for INNER JOINs results in a cross join -- a cartesian product.
LEFT JOIN episode_trans
ON episode_trans.id_episode = episode.id_episode
AND episode_trans.id_lang = 1
WHERE id_serie=1
You probably need to move the id_lang = 1 into the LEFT JOIN clause instead of the WHERE clause. Think of it this way... for all of those rows with no translation the LEFT JOIN gives you back NULLs for all of those translation columns. Then in the WHERE clause you are checking to see if that is equal to 1 - which of course evaluates to FALSE.
It would probably be easier if you included your code in the question next time instead of in a link.
Can you try using
LEFT OUTER JOIN
instead of
NATURAL LEFT JOIN