I already know what a self-join does. Thank you, I also read all the other computerised operational descriptions on stack overflow, so I know this is not actually a duplicate question, so please do not give me tables or join lists.
What I am seeking to why it would be done (and please, not just the self-referencing employee-manager example).
In plain English, what would I seek to achieve from a self join?
My usage is in a university course, and coming from a Relational Algebra angle. I have done some SQL for a few years but the instructor loves to do self-joins on tables (after renaming one or more fields). Not something often done in SQL, so I'm wondering what the action is that he is trying to perform but he seems pretty keen about doing it frequently.
I thought I'd ask here, as many others have asked for this information but get marked as "already answered" but all the answers give operational descriptions NOT the "why is this being done".
The reason why the employee-manager example is so common, is because it hits the nail on the head. A self join on a table looks for pairs of rows, like any join, but with both rows coming from the same table. Nothing special really.
The database designer gives each base table a predicate (sentence template parameterized by column names).
Parent(person, child) -- person PERSON is parent of person CHILD
Likes(person, food) -- person PERSON likes food FOOD
Relational algebra is designed so that the value of a relational expression (base table name or operator call) holds the rows that make a true proposition (statement) from its predicate.
/* (PERSON, CHILD) rows where
person PERSON is parent of person CHILD
*/
Parent
The predicate of an expression that is a call to operator NATURAL JOIN is the AND of the predicates of its inputs.
/* (PERSON, CHILD, FOOD) rows where
person PERSON is parent of person CHILD AND person PERSON likes food FOOD
*/
Parent NATURAL JOIN Likes
Ditto for UNION & OR, MINUS & AND NOT, PROJECT column(s) & EXISTS other column(s), RESTRICT condition & AND condition and RENAME of a column & rename of a parameter.
/* (CHILD, FOOD) rows where
there EXISTS a value for PERSON such that
person PERSON is parent of person CHILD AND person CHILD likes food FOOD
*/
PROJECT child, food (Parent NATURAL JOIN (RENAME person:=child Likes))
So every query expression's value holds the rows that make its predicate into a true statement.
Suppose we define algebraic self-join of a table as NATURAL JOIN of two tables got from an original via sequences of zero or more renamings. Per above we NATURAL JOIN for rows that satisfy the AND of predicates. A self-join arises when we want the rows that satisfy a result predicate expressed via predicates that differ only in parameters/columns.
/* (PERSON, FOOD, CHILD) rows where
person PERSON likes food FOOD AND person CHILD likes food FOOD
*/
Likes NATURAL JOIN (RENAME person:=child Likes)
There's nothing special about a self-join arising in a given query in a given application other than that.
SQL SELECT DISTINCT statements can be described via algebraic operators. They also calculate query predicates. First FROM table columns are RENAMEd by prefixing a table alias (correlation name) & a dot. (SQL NATURAL JOIN doesn't dot common columns.) The new tables are NATURAL JOINed. ON and WHERE RESTRICT per a condition. Then the SELECT DISTINCT clause RENAMES to remove dots from returned columns & PROJECTS away unwanted dotted columns.
We can convert SQL to predicates directly: Dotting input columns renames. NATURAL/CROSS/INNER JOIN, ON & WHERE give AND. Each dot-free result column gives an AND that it equals its dotted version. Finally dropping all dotted columns gives EXISTS.
/* same as above */
/* (PERSON, FOOD, CHILD) rows where
there EXISTS values for P.* & C.* such that
PERSON = P.PERSON AND CHILD = C.person AND FOOD = P.FOOD
AND person P.CHILD likes food P.FOOD
AND person C.CHILD likes food C.FOOD
AND P.FOOD = C.FOOD
*/
SELECT DISTINCT p.person AS person, c.person AS child, p.food AS food
FROM Likes p INNER JOIN Likes c
ON p.food = c.food
Again: In SQL we say there is a self-join when multiple table aliases of a JOIN are associated with the same table value; in application terms that means we can express a query meaning in terms of predicates differing in some parameters/columns; there's nothing special about applications or table meanings for this to arise.
See this re query semantics which happens to include a link to this re self-join semantics in particular.
Related
Whenever there is any description of query in front of us, we try to apply heuristics and brainstorming to construct the query.
Is there any systematic step-by-step or mathematical way to construct SQL query from a given human-readable description?
For instance, how to determine that, whether a SQL query would need a join rather than a subquery, whether it would require a group by, whether it would require a IN clause, etc....
For example, whoever studied Digital Electronics would be aware of the methods like Karnaugh Map or Quin McClausky method. These, are some systematic approaches to simplify digital logic.
If there any method like these to analyze sql queries manually to avoid brainstorming each time?
Is there any systematic step-by-step or mathematical way to construct an SQL query from a given human-readable description?
Yes, there is.
It turns out that natural language expressions and logical expressions and relational algebra (and calculus) expressions and SQL expressions (a hybrid of the last two) correspond in a rather direct way. (What follows is for no duplicate rows & no nulls.)
Each table (base or query result) has an associated predicate--a natural language fill-in-the-(named-)blanks statement template parameterized by column names.
[liker] likes [liked]
A table holds every row that, using the row's column values to fill in the (named) blanks, makes a true statement aka proposition. Here's a table with that predicate & its rows' propositions:
liker | liked
--------------
Bob | Dex /* Bob likes Dex */
Bob | Alice /* Bob likes Alice */
Alice | Carol /* Alice likes Carol */
Each proposition from filling a predicate with the values from a row in a table is true. And each proposition from filling a predicate with the values from a row not in a table is false. Here's what that table says:
/*
Alice likes Carol
AND NOT Alice likes Alice
AND NOT Alice likes Bob
AND NOT Alice likes Dex
AND NOT Alice likes Ed
...
AND Bob likes Alice
AND Bob likes Dex
AND NOT Bob likes Bob
AND NOT Bob likes Carol
AND NOT Bob likes Ed
...
AND NOT Carol likes Alice
...
AND NOT Dex likes Alice
...
AND NOT Ed likes Alice
...
*/
The DBA gives the predicate for each base table. The SQL syntax for a table declaration is a lot like the traditional logic shorthand for the natural language version of a given predicate. Here's a declaration of a base table to hold our value:
/* (person, liked) rows where [liker] likes [liked] */
/* (person, liked) rows where Likes(liker, liked) */
CREATE TABLE Likes (
liker ...,
liked ...
);
Say r is the predicate of R and s is the predicate of S.
An SQL query (sub)expression transforms argument table values to a new table value holding the rows that make a true statement from a new predicate. The new table predicate can be expressed in terms of the argument table predicate(s) according to the (sub)expression's relational/table operators. A query is an SQL expression whose predicate is the predicate for the table of rows we want.
When we give a table & (possibly implicit) alias A to be joined, the operator acts on a value & predicate like the table's but with columns renamed from C,... to A.C,.... Then
R , S & R CROSS JOIN S are rows where r AND s
R WHERE condition is rows where r AND condition
R INNER JOIN S ON condition is rows where r AND s AND condition
R LEFT JOIN S ON condition is rows where (for S-only columns S1,...)
r AND s AND condition
OR
r
AND NOT FOR SOME values for S1,... [s AND condition]
AND S1 IS NULL AND ...
SELECT DISTINCT A.C AS D,... FROM R (maybe with implicit A. and/or implicit AS D) is rows where
FOR SOME values for A.*,... [A.C = D AND ... AND r] (This can be less compact but looks more like the SQL.)
if there are no dropped columns, r with A.C,... replaced by D,...
if there are dropped columns, FOR SOME values for the dropped columns [ r with A.C,... replaced by D,... ]
(X,...) IN (R) means
r with columns C,... replaced by X,...
(X,...) IN R
Example: Natural language for (person, liked) rows where [person] is Bob and Bob likes someone who likes [liked] but who doesn't like Ed:
/* (person, liked) rows where
FOR SOME value for x,
[person] likes [x]
and [x] likes [liked]
and [person] = 'Bob'
and not [x] likes 'Ed'
*/
Rewrite using shorthand predicates:
/* (person, liked) rows where
FOR SOME value for x,
Likes(person, x)
AND Likes(x, liked)
AND person = 'Bob'
AND NOT Likes(x, 'Ed')
*/
Rewrite using only shorthand predicates of base & aliased tables:
/* (person, liked) rows where
FOR SOME values for l1.*, l2.*,
person = l1.liker AND liked = l2.liked
AND Likes(l1.liker, l1.liked)
AND Likes(l2.liker, l2.liked)
AND l1.liked = l2.liker
AND person = 'Bob'
AND NOT (l1.liked, 'Ed') IN Likes
*/
Rewrite in SQL:
SELECT DISTINCT l1.liker AS person, l2.liked AS liked
/* (l1.liker, l1.liked, l2.liker, l2.liked) rows where
Likes(l1.liker, l1.liked)
AND Likes(l2.liker, l2.liked)
AND l1.liked = l2.liker
AND l1.liker = 'Bob'
AND NOT (l1.liked, 'Ed') IN Likes
*/
FROM Likes l1
INNER JOIN Likes l2
ON l1.liked = l2.liker
WHERE l1.liker = 'Bob'
AND NOT (l1.liked, 'Ed') IN (SELECT * FROM Likes)
Similarly,
R UNION CORRESPONDING S is rows where r OR s
R UNION S is rows where r OR s with the columns of S replaced by the columns of R
VALUES (X,...), ... with columns C,... is rows where C = X AND ... OR ...
Example:
/* (person) rows where
FOR SOME value for liked, Likes(person, liked)
OR person = 'Bob'
*/
SELECT liker AS person
FROM Likes
UNION
VALUES ('Bob')
So if we express our desired rows in terms of given base table natural language statement templates that rows make true or false (to be returned or not) then we can translate to SQL queries that are nestings of logic shorthands & operators and/or table names & operators. And then the DBMS can convert totally to tables to calculate the rows making our predicate true.
See How to get matching data from another SQL table for two different columns: Inner Join and/or Union? re applying this to SQL. (Another self-join.)
See Relational algebra for banking scenario for more on natural language phrasings. (In a relational algebra context.)
See Null in Relational Algebra for another presentation of relational querying.
Here's what I do in non-grouped queries:
I put into the FROM clause the table of which I expect to receive zero or one output row per row in the table. Often, you want something like "all customers with certain properties". Then, the customer table goes into the FROM clause.
Use joins to add columns and filter rows. Joins should not duplicate rows. A join should find zero or one rows, never more. That keeps it very intuitive because you can say that "a join adds columns and filters out some rows".
Subqueries are to be avoided if a join can replace them. Joins look nicer, are more general and often are more efficient (due to common query optimizer weaknesses).
How to use WHERE and projections is easy.
I'm working on enhancing a query for a DB2 database and I'm having some problems getting acceptable performance due to the number of joins across large tables that need to be performed to get all of the data and I'm hoping that there's a SQL function or technique that can simplify and speed up the process.
To break it down, let's say there are two tables: People and Groups. Groups contain multiple people, and a person can be part of multiple groups. It's a many-to-many, but bear with me. Basically, there's a subquery that will return a set of groups. From that, I can join to People (which requires additional joins across other tables) to get all of the people from those groups. However, I also need to know all of the groups that those people are in, which means joining back to the Groups table again (several more joins) to get a superset of the original subquery. There are additional joins in the query as well to get other pieces of relevant data, and the cost is adding up in a very ugly way. I also need to return information from both tables, so that rules out a number of techniques.
What I'd like to do is be able to start with the People table, join it to Groups, and then compare Groups with the subquery. If the Groups attached to one person has one match in the subquery, it should return ALL Group items associated with that person.
Essentially, let's say that Bob is part of Group A, B, and C. Currently, I start with groups, and let's say that only Group A comes out of the subquery. Then I join A to Bob, but then I have to come back and join Bob to Group again to get B and C. SQL example:
SELECT p.*, g2.*
FROM GROUP g
JOIN LINKA link
ON link.GROUPID = g.GROUPID
JOIN LINKB link1
ON link1.LISTID = link.LISTID
JOIN PERSON p
ON link1.PERSONID = p.PERSONID
JOIN LINKB link2
ON link2.PERSONID = p.PERSONID
JOIN LINKA link3
ON link2.LISTID = link3.LISTID
JOIN GROUP g2
ON link3.GROUPID = g2.GROUPID
WHERE
g.GROUPID IN (subquery)
Yes, the linking tables aren't ideal, but they're basically normalized tables containing additional information that is not relevant to the query I'm running. We have to start with a filtered Group set, join to People, then come back to get all of the Groups that the People are associated to.
What I'd like to do is start with People, join to Group, and if ANY Group that Bob is in returns from the subquery, ALL should be returned, so if we have Bob joined to A, B, and C, and A is in the subquery, it will return three rows of Bob to A, B, and C as there was at least one match. In this way, it could be treated as a one-to-many relationship if we're only concerned with the Groups for each Person and not the other way around. SQL example:
SELECT p.*, g.*
FROM PEOPLE p
JOIN LINKB link
ON link.PERSONID = p.PERSONID
JOIN LINKA link1
ON link.LISTID = link1.LISTID
JOIN GROUP g
ON link1.GROUPID = g.GROUPID
WHERE
--SQL function, expression, or other method to return
--all groups for any person who is part of any group contained in the subquery
The number of joins in the first query make it largely unusable as these are some pretty big tables. The second would be far more ideal if this sort of thing is possible.
From the question, I think you are querying hierarchical data. DB2 provides facility to deal with such data. There are two clauses Start with and Connect by in DB2 which will be useful. They are explained here.
I have a very simple problem I can't wrap my head around in regards to SQLite. I am trying to learn more about SQL by reading through a book and following the examples. One of the exercises is to write a query that can find which pet out of a pet table is the parent of another pet. Both the pet.id and pet.parent columns use INTEGER data types. So for one pet the id would be 2 and another pet would be 3... The pet schema has pet (id, name, breed, parent); type of structure. So for example: INSERT INTO pet VALUES(2, "scraps", "lolcat", 3);
would make this pet.id=2,pets.name=scraps,pet.breed=lolcat, pet.parent=3. In this table some of the pets will also be parents of other pets. So their pet.id would also match the parent.id of some other pet...(It seems kinda complex)
I wrote a query that I thought makes since but it does not return any results or throw any errors. That query is:
SELECT pet.name, pet.breed FROM pet WHERE
pet.parent = pet.id;
If you're talking about exercise 13: it actually says:
Write a query that can find the pets that are children of a given pet.
So you have one specific parent pet, and its ID, and you are searching for all pets that have that ID as their parent ID:
SELECT * FROM pet WHERE parent = 3
To find out all pets in child/parent relationships, you have to do a query that involves two different pets at the same time; this requires a join.
In this case, there is the additional wrinkle that these records come from the same table, so it is necessary to use aliases (AS) to rename one or both tables so that we can address their columns unambiguously:
SELECT parent.name,
parent.breed,
child.name,
child.breed
FROM pet AS parent
JOIN pet AS child
ON parent.id = child.parent
WHERE ... -- if needed
http://www.codinghorror.com/blog/2007/10/a-visual-explanation-of-sql-joins.html
and
https://web.archive.org/web/20120621231245/http://www.khankennels.com/blog/index.php/archives/2007/04/20/getting-joins/
have been very helpful in learning the basics of joins using Venn diagrams. But I am wondering how you would apply that same thinking to a query that has more than one join.
Let's say I have 3 tables:
Employees
EmployeeID
FullName
EmployeeTypeID
EmployeeTypes (full time, part time, etc.)
EmployeeTypeID
TypeName
InsuranceRecords
InsuranceRecordID
EmployeeID
HealthInsuranceNumber
Now, I want my final result set to include data from all three tables, in this format:
EmployeeID | FullName | TypeName | HealthInsuranceNumber
Using what I learned from those two sites, I can use the following joins to get all employees, regardless of whether or not their insurance information exists or not:
SELECT
Employees.EmployeeID, FullName, TypeName, HealthInsuranceNumber
FROM Employees
INNER JOIN EmployeeTypes ON Employees.EmployeeTypeID = EmployeeTypes.EmployeeTypeID
LEFT OUTER JOIN InsuranceRecords ON Employees.EmployeeID = InsuranceRecords.EmployeeID
My question is, using the same kind of Venn diagram pattern, how would the above query be represented visually? Is this picture accurate?
I think it is not quite possible to map your example onto these types of diagrams for the following reason:
The diagrams here are diagrams used to describe intersections and unions in set theory. For the purpose of having an overlap as depicted in the diagrams, all three diagrams need to contain elements of the same type which by definition is not possible if we are dealing with three different tables where each contains a different type of (row-)object.
If all three tables would be joined on the same key then you could identify the values of this key as the elements the sets contain but since this is not the case in your example these kind of pictures are not applicable.
If we do assume that in your example both joins use the same key, then only the green area would be the correct result since the first join restricts you to the intersection of Employees and Employee types and the second join restricts you the all of Employees and since both join conditions must be true you would get the intersection of both of the aforementioned sections which is the green area.
Hope this helps.
That is not an accurate set diagram (either Venn or Euler). There are no entities which are members of both Employees and Employee Types. Even if your table schema represented some kind of table-inheritance, all the entities would still be in a base table.
Jeff's example on the Coding Horror blog only works with like entities i.e. two tables containing the same entities - technically a violation of normalization - or a self-join.
Venn diagrams can accurately depict scenarios like:
-- The intersection lens
SELECT *
FROM table
WHERE condition1
AND condition2
-- One of the lunes
SELECT *
FROM table
WHERE condition1
AND NOT condition2
-- The union
SELECT *
FROM table
WHERE condition1
OR condition2
I need a way to represent existential relations in a database. For instance I have a bio-historical table (i.e. a family tree) that stores a parent id and a child id which are foreign keys to a people table. This table is used to describe arbitrary family relationships. Thus I’d like to be able to say that X and Y are siblings without having to know exactly who the parents of X and Y are. I just want to be able to say that there exists two different people A and B such that A and B are each parents of X and Y. Once I do know who A and/or B are I’d need to be able to reconcile them.
The simplest solution I can think of is to store existential people with negative integer user ids. Once I know who the people are, I’d need to cascade update all of the IDs. Are there any well-known techniques for this?
Does existential mean "non existant"?
They don't have to be negative. You could just add a record to People table with no last/first name and perhaps a flag "unknown person". Or existential if you like.
Then when you know something (e.g. like last name but not first) you update this record.
Reconciling duplicate people could be more difficult. I guess you could just update FamilyTree set parent_id=new_id where parent_id=old_id, etc. But this means for instance that the same person could end up with too many parents, so you'll need to perform a number of complex checks before doing that.
I would document only the known relationships in a link table which links your Person table to itself with:
FK Person1ID
FK Person2ID
RelationshipTypeID (Sibling, Father, Mother, Step-Father, Step-Mother, etc.)
With some appropriate constraints on that table (or multiple tables, one for each relationship type if that makes the constraints more logical)
Then when other relationships can possibly (a half-sibling will only share one parent) be inferred (by running an exception query) but are missing, create them.
For instance, people who are siblings who don't have all their parents identified:
SELECT *
FROM People p1
INNER JOIN Relationship r_sibling
ON r_sibling.Person1ID = p1.PersonID
AND r_sibling.RelationshipType = SIBLING_TYPE_CONSTANT
INNER JOIN People p2
ON r_sibling.Person2ID = p2.PersonID
WHERE EXISTS (
-- p1 has a father
SELECT *
FROM Relationship r_father
ON r_father.RelationshipType = FATHER_TYPE_CONSTANT
AND r_father.Person2ID = p1.PersonID
)
AND NOT EXISTS (
-- p2 (p1's sibling) doesn't have a father yet
SELECT *
FROM Relationship r_father
ON r_father.RelationshipType = FATHER_TYPE_CONSTANT
AND r_father.Person2ID = p2.PersonID
)
You might need to UNION the reverse of this query depending on how you want your relationships constrained (siblings are always commutative, unlike other relationships) and then handle mothers similarly.
Hmmm, come to think of it, I guess I need a general way to reconcile duplicate people anyway and I can use it for this purpose. Thoughts?