I'm working on enhancing a query for a DB2 database and I'm having some problems getting acceptable performance due to the number of joins across large tables that need to be performed to get all of the data and I'm hoping that there's a SQL function or technique that can simplify and speed up the process.
To break it down, let's say there are two tables: People and Groups. Groups contain multiple people, and a person can be part of multiple groups. It's a many-to-many, but bear with me. Basically, there's a subquery that will return a set of groups. From that, I can join to People (which requires additional joins across other tables) to get all of the people from those groups. However, I also need to know all of the groups that those people are in, which means joining back to the Groups table again (several more joins) to get a superset of the original subquery. There are additional joins in the query as well to get other pieces of relevant data, and the cost is adding up in a very ugly way. I also need to return information from both tables, so that rules out a number of techniques.
What I'd like to do is be able to start with the People table, join it to Groups, and then compare Groups with the subquery. If the Groups attached to one person has one match in the subquery, it should return ALL Group items associated with that person.
Essentially, let's say that Bob is part of Group A, B, and C. Currently, I start with groups, and let's say that only Group A comes out of the subquery. Then I join A to Bob, but then I have to come back and join Bob to Group again to get B and C. SQL example:
SELECT p.*, g2.*
FROM GROUP g
JOIN LINKA link
ON link.GROUPID = g.GROUPID
JOIN LINKB link1
ON link1.LISTID = link.LISTID
JOIN PERSON p
ON link1.PERSONID = p.PERSONID
JOIN LINKB link2
ON link2.PERSONID = p.PERSONID
JOIN LINKA link3
ON link2.LISTID = link3.LISTID
JOIN GROUP g2
ON link3.GROUPID = g2.GROUPID
WHERE
g.GROUPID IN (subquery)
Yes, the linking tables aren't ideal, but they're basically normalized tables containing additional information that is not relevant to the query I'm running. We have to start with a filtered Group set, join to People, then come back to get all of the Groups that the People are associated to.
What I'd like to do is start with People, join to Group, and if ANY Group that Bob is in returns from the subquery, ALL should be returned, so if we have Bob joined to A, B, and C, and A is in the subquery, it will return three rows of Bob to A, B, and C as there was at least one match. In this way, it could be treated as a one-to-many relationship if we're only concerned with the Groups for each Person and not the other way around. SQL example:
SELECT p.*, g.*
FROM PEOPLE p
JOIN LINKB link
ON link.PERSONID = p.PERSONID
JOIN LINKA link1
ON link.LISTID = link1.LISTID
JOIN GROUP g
ON link1.GROUPID = g.GROUPID
WHERE
--SQL function, expression, or other method to return
--all groups for any person who is part of any group contained in the subquery
The number of joins in the first query make it largely unusable as these are some pretty big tables. The second would be far more ideal if this sort of thing is possible.
From the question, I think you are querying hierarchical data. DB2 provides facility to deal with such data. There are two clauses Start with and Connect by in DB2 which will be useful. They are explained here.
Related
I need help in understanding how left join is working in below query.
There are total three tables and two left joins.
So my question is, the second left join is between customer and books table or between result of first join and books table?
SELECT c.id, c.first_name, c.last_name, s.date AS sale,
b.name AS book, b.genre
FROM customers c
LEFT JOIN sales s
ON c.id = s.customer_id
LEFT JOIN books b
ON s.book_id = b.id;
Good question.
When it comes to outer-joined tables, it depends on the predicates in the ON clause. The engine is free to reorder the fetch and scans on indexes or tables as long as the predicates are respected.
In this particular case there are three tables:
customers (c)
sales (s)
books (b)
customers is inner joined so it becomes the driving table; there are other considerations, but for simplicity you can consider that this is the table that is read first. Now, which one is second? sales or books?
The first join predicate c.id = s.customer_id doesn't establish any relationship between the secondary tables; therefore it doesn't affect which table is joined first.
The second join predicate s.book_id = b.id makes books dependent on sales. Therefore, it decides sales is the second table, and books is the last one.
A final note: if you understand the concept of dependency there are several dirty tricks you can use to force the engine to walk the tables in the order you want. I would not recommend to do this to a novice, but if at some point you realise the engine is not doing what you want, you can tweak the queries.
The second join statement specifies to join on s.book_id = b.id where s is sales and b is books. However, a record in the books table will not be returned unless it has a corresponding record in the sales AND customers tables, which is what a left join does by definition https://www.w3schools.com/sql/sql_join_left.asp. put another way, this query will return all books that have been purchased by at least one customer (and books that have been purchased multiple times will appear in the results multiple times).
I'm creating an archive for Academic Papers. Each paper may have one author, or multiple authors. I've created the tables in the following manner:
Table 1: PaperInfo - Each row contains information on the paper
Table 2: PaperAuthor - Only Two Columns: contains PaperID, and AuthorID
Table 3: AuthorList - Contains Author Information.
There is also a Table 4 which is linked to Table 4, which contains a list of Universities which the author belongs to, but I'm going to leave it out for now in case it gets too complicated.
I wish to have a Query which will link all three tables together, and display Paper Information of the recordset in a table, with columns such as these:
Paper Title
Paper Authors
The column "Paper Authors" is going to contain more than one authors in some cases.
I've wrote the following query:
SELECT a.*,b.*,c.*
FROM PaperInfo a, PaperAuthor b, AuthorList c
WHERE a.PaperID = b.PaperID AND b.AuthorID = c.AuthorID
So far, the results I've been getting for each row is one author per row. I wish to contain more authors in one column. Can this be done in anyway?
Note: I'm using Access 2010 as my database.
In straight SQL the answer unfortunately is that it isn't possible. You would need to use a processing language in order to get the result you are after.
Since you mention you are using Access 2010 please refer to this question: is there a group_concat function in ms-access?
Particularly, read the post which points to http://www.rogersaccesslibrary.com/forum/generic-function-to-concatenate-child-records_topic16&SID=453fabc6-b3z9-34z6zb14-a78f832z-19z89a2c.html
You probably need to implement a custom function but the 2nd url does what you are looking for.
This functionality is not part of the SQL standard, but different vendors have solutions for it, see for instance Pivot Table with many to many table, MySQL pivot table.
If you know the maximum number of authors per paper (for example 3 or 4), you could get away with a triple or quadruple left join.
What you are after is an inner join.
An SQL JOIN clause is used to combine rows from two or more tables, based on a common field between them.
The most common type of join is: SQL INNER JOIN (simple join). An SQL INNER JOIN return all rows from multiple tables where the join
condition is met.
http://www.w3schools.com/sql/sql_join.asp
You may want to combine the inner join with a group to give you 1 paper to many authors in your results.
The GROUP BY statement is used in conjunction with the aggregate
functions to group the result-set by one or more columns.
http://www.w3schools.com/sql/sql_groupby.asp
I find myself in a bit of an unworkable situation with a SQL query and I'm hoping that I'm missing something or might learn something new. The structure of the DB2 database I'm working with isn't exactly built for this sort of query, but I'm tasked with this...
Let's say we have Table People and Table Groups. Groups can contain multiple people, and one person can be part of multiple groups. Yeah, it's already messy. In any case, there are a couple of intermediary tables linking the two. The problem is that I need to start with a list of groups, get all of the people in those groups, and then get all of the groups with which the people are affiliated, which would be a superset of the initial group set. This would mean starting with groups, joining down to the people, and then going BACK and joining to the groups again. I need information from both tables in the result set, too, so that rules out a number of techniques.
I have to join this with a number of other tables for additional information and the query is getting enormous, cumbersome, and slow. I'm wondering if there's some way that I could start with People, join it to Groups, and then specify that if a person has one group that is in the supplied set of groups (which is done via a subquery), then ALL groups for that person should be returned. I don't know of a way to make this happen, but I'm thinking (hoping) that there's a relatively clean way to make this happen in SQL.
A quick and dirty example:
SELECT ...
FROM GROUPS g
JOIN LINKING_A a
ON g.GROUPID = a.GROUPID
AND GROUPID IN (subquery)
JOIN LINKING_B b
ON a.GROUPLIST = b.GROUPLIST
JOIN PEOPLE p
ON b.PERSONID = p.PERSONID
--This gets me all people affiliated with groups,
-- but now I need all groups affiliated with those people...
JOIN LINKING_B b2
ON p.PERSONID = b2.PERSONID
JOIN LINKING_A a2
ON b2.GROUPLIST = a.GROUPLIST
JOIN GROUPS g2
ON a2.GROUPID = g.GROUPID
And then I can return information from p and g2 in the result set. You can see where I'm having trouble. That's a lot of joining on some large tables, not to mention a number of other joins that are performed in this query as well. I need to be able to query by joining PEOPLE to GROUPS, then specify that if any person has an associated group that is in the subquery, it should return ALL groups affiliated with that entry in PEOPLE. I'm thinking that GROUP BY might be just the thing, but I haven't used that one enough to really know. So if Bill is part of group A, B, and C, and our subquery returns a set containing Group A, the result set should include Bill along with groups A, B, and C.
The following is a shorter way to get all the groups that people in the supplied group list are in. Does this help?
Select g.*
From Linking_B b
Join Linking_B b2
On b2.PersonId = b.PersonId
Join Group g
On g.GroupId = b2.GroupId
Where b.Groupid in (SubQuery)
I'm not clear why you have both Linking_A and Linking_B. Generally all you should need to represent a many-to-many relationship between two master tables is a single association table with GroupID and PersonId.
I often recommend using "common table expressions" [CTE's] in order to help you break a problem up into chunks that can be easier to understand. CTE's are specified using a WITH clause, which can contain several CTE's before starting the main SELECT query.
I'm going to assume that the list of groups you want to start with is specified by your subquery, so that will be the 1st CTE. The next one selects people who belong to those groups. The final part of the query then selects groups those people belong to, and returns the columns from both master tables.
WITH g1 as
(subquery)
, p1 as
(SELECT p.*
from g1
join Linking a1 on g1.groupID=a1.groupID
join People p on p.personID=a1.personID )
SELECT p1.*, g2.*
from p1
join Linking a2 on p2.personID=a2.personID
join Groups g2 on g2.groupID=a2.groupID
I think I'd build the list of people you want to pull records for first, then use that to query out all the groups for those people. This will work across any number of link tables with the appropriate joins added:
with persons_wanted as
(
--figure out which people are in a group you want to include
select p.person_key
from person p
join link l1
on p.person_key = l1.person_key
join groups g
on l1.group_key = g.group_key
where g.group name in ('GROUP_I_WANT_PEOPLE_FROM', 'THIS_ONE_TOO')
group by p.person_key --we only want each person_key once
)
--now pull all the groups for the list of people in at least one group we want
select p.name as person_name, g.name as group_name, ...
from person p
join link l1
on p.person_key = l1.person_key
join groups g
on l1.group_key = g.group_key
where p.person_key in (select person_key from persons_wanted);
I have a database with two tables
One with games
and one with participants
A game is able to have more participants and these are in a different table.
Is there a way to combine these two into one query?
Thanks
You can combine them using the JOIN operator.
Something like
SELECT *
FROM games g
INNER JOIN participants p ON p.gameid = g.gameid
Explanation on JOIN operators
INNER JOIN - Match rows between the two tables specified in the INNER
JOIN statement based on one or more
columns having matching data.
Preferably the join is based on
referential integrity enforcing the
relationship between the tables to
ensure data integrity.
o Just to add a little commentary to the basic definitions
above, in general the INNER JOIN
option is considered to be the most
common join needed in applications
and/or queries. Although that is the
case in some environments, it is
really dependent on the database
design, referential integrity and data
needed for the application. As such,
please take the time to understand the
data being requested then select the
proper join option.
o Although most join logic is based on matching values between
the two columns specified, it is
possible to also include logic using
greater than, less than, not equals,
etc.
LEFT OUTER JOIN - Based on the two tables specified in the join
clause, all data is returned from the
left table. On the right table, the
matching data is returned in addition
to NULL values where a record exists
in the left table, but not in the
right table.
o Another item to keep in mind is that the LEFT and RIGHT OUTER
JOIN logic is opposite of one another.
So you can change either the order of
the tables in the specific join
statement or change the JOIN from left
to right or vice versa and get the
same results.
RIGHT OUTER JOIN - Based on the two tables specified in the join
clause, all data is returned from the
right table. On the left table, the
matching data is returned in addition
to NULL values where a record exists
in the right table but not in the left
table.
Self -Join - In this circumstance, the same table is
specified twice with two different
aliases in order to match the data
within the same table.
CROSS JOIN - Based on the two tables specified in the join clause, a
Cartesian product is created if a
WHERE clause does filter the rows.
The size of the Cartesian product is
based on multiplying the number of
rows from the left table by the number
of rows in the right table. Please
heed caution when using a CROSS JOIN.
FULL JOIN - Based on the two tables specified in the join clause,
all data is returned from both tables
regardless of matching data.
example
table Game has columns (gameName, gameID)
table Participant has columns (participantID, participantName, gameID)
the GameID column is the "link" between the 2 tables. you need a common column you can join between 2 tables.
SELECT gameName, participantName
FROM Game g
JOIN Participat p ON g.gameID = p.gameID
This will return a data set of all games and the participants for those games.
The list of games will be redundant unless you structure it some other way due to multiple participants to that game.
sample data
WOW Bob
WOW Jake
StarCraft2 Neal
Warcraft3 James
Warcraft3 Rich
Diablo Chris
It seems to me that there are two scenarios in which to use JOINs:
When data would otherwise be duplicated
When data from one query would otherwise be used in another query
Are these scenarios right? Are there any other scenarios in which to use JOIN?
EDIT: I think I've miscommunicated. I understand how a JOIN works, what I'm not so sure about is when to use one.
JOINS are used to JOIN tables together with related information.
Tipical situations are where you have lets say
A user table where the user has specific security settings. The join would be used such that you can determine which settings the user has.
Users
-UserID
-UserName
UserSecurityRoles
-UserID
-SecurityRoleID
SecurityRoles
-SecurityRoleID
-SecurityRole
SELECT *
FROM Users u INNER JOIN
UserSecurityRoles usr ON u.UserID = usr.UserID INNER JOIN
SecurityRoles sr ON usr.SecurityRoleID = sr.SecurityRoleID
WHERE sr.SecurityRole = 'Admin'
LEFT joins will be used in the cases where you wish to retrieve all the data from the table in the left hand side, and only data from the right that match.
JOINS are used when you start normalizing your table structure. You can crete a table with 100s on columns, where a lot of the data could possibly be NULL, or you can normalize the tables, such that you avoid having too many columns with null values, where you group the appropriate data into table structures.
The answer to this Question has a VERY good link that has graphical display of using JOINs
JOINS are used to return data that is related in a relational database. Data can be related in 3 ways
One to Many relationship (A Person can have many Transactions)
Many to Many relationship (A Doctor can have many Patients, but a Patient can have more than one Doctor)
One to One relationship (One Person can have exactly one Passport number)
JOINS come in various flavours:
AN INNER JOIN will return data from both tables where the keys in each table match
A LEFT JOIN or RIGHT JOIN will return all the rows from one table and matching data from the other table
A CROSS JOIN will return the product of each table
You use joins when you need information from more than one table :)