I have a table with IDs that represent Individuals (call it id_table) and another table with characteristics an individual can have (call it char_table).
Now I want to add a column to the id_table containing certain values from the char_table. The difficulty, for me, is, that some ID shall get only one characteristic (simple case) and some ID shall get several characteristics. For that reason the rows with these ID that get several characteristics must be reproduced so often until I have matched every characteristic to that ID.
For Example:
ID '001' shall get the characteristics 'a', 'b', and 'c'. So the row ID='001' must be reproduced 2 times (to get the same row 3 times) and each of these rows shall get one of these 3 characteristics.
I hope I explained intelligible enough.
Anyone an idea how to do this?
Thanks.
In almost all cases, you want to do this with an association or junction table. This table would like like:
create table IndividualCharacteristicsr (
individualId int not null,
characterId int not null
);
(It might also have a unique id itself.)
Each characteristic an individual has would be on a separate row. So, three characteristics mean three rows. A typical query using this information would join all three tables:
from id_table i left outer join
IndividualCharacteristicsr ic
on i.individualId = ic.individualId left outer join
char_table c
on c.characteristicId = ic.characteristicId
(The left outer join includes individuals with no characteristics.)
In Postgres, you could store the characteristics in a string or an array. Neither is very convenient for joining back to id_char. The better approach is an additional table.
Related
I have two data sources that are loaded into Azure Synapse. Both raw data sources contain an 'Apple' table.
I merge these into a single 'Apple' table in my Enriched data store.
SELECT * FROM datasource1.apple JOIN datasource2.apple on datasource1.apple.id = datasource2.apple.id
However, both data sourecs also contain a one to many relation AppleColours.
Please could someone help me understand the correct approach to creating a single AppleColours table in my enriched zone?
You need data from both sources when you want to merge them. JOIN(INNER JOIN) will bring only the apple.id that is in both datasource1 and datasource2.
You should try a CROSS JOIN
For the AppleColours 1 to many relation there are 2 methods:
You could put direct the color in the Apple table in this case there is no need for a separate AppleColours table
Apple
ID| Color
1 | red
2 | green
To get data into Color column make another JOIN this time with the AppleColours on the colorID from Apple table and AppleColours
Create a separate table AppleColours with ID and color. IN this table import both AppleColours tables from both datasources using a CROSS JOIN
Add a column in Apple table named AppleColorId which has the id's from AppleColours
If you want an Applet table that has all the data and don't need any join's to determine the apple color use method 1.
If you want a 'slim' apple table which has minimal data inside use method 2.
In this case to get the apple color you have to make an extra JOIN(INNER JOIN) to the AppleColour table
Maybe including a subquery making an UNION (you will get only one of each), but your problem still will be that, as each table has its own relationship with colours and you are joining both, same item can give you two different colours. My proposal: to make a switch to choose only one, If first is null, choose second, and if second is also null, a default value (some colour code). Other options are to use the lower id, because it was early created, or higher because it was the last...
Something like that
SELECT datasource1.*, datasource2.*, Q.Name, Q.Value FROM datasource1.apple
JOIN datasource2.apple on datasource1.apple.id = datasource2.apple.id
JOIN
(SELECT ColourID, Name, Value FROM datasource1.AppleColours UNION SELECT ColourID, Name, Value FROM datasource2.AppleColours) Q
ON Q.ColourID = COALESCE(datasource1.ColourID, datasource2.ColourID, {DefaultColor})
Are the two data sources supposed to represent slices of the same real population?
I.e., if full joining datasource1 with datasource2 on apple.id is logically consistent, then full joining AppleColours between the 2 datasources should be logically correct as well.
The one-to-many then logically preserves the information from the two datasets, and remains correctly one-to-many. If there are any relationships cardinality violations as the results of this join - those weren't the right cardinalities to begin with.
(btw, should be a full join)
I'm trying to achieve 2 joins. If I run the 1st join alone it pulls 4 lots of results, which is correct. However when I add the 2nd join which queries the same reference table using the results from the select statement it pulls in additional results. Please see attached. The squared section should not be being returned
So I removed the 2nd join to try and explain better. See pic2. I'm trying to get another column which looks up InvolvedInternalID against the initial reference table IRIS.Practice.idvClient.
Your database is simply doing as you tell it. When you add in the second join (confusingly aliased as tb1 in a 3 table query) the database is finding matching rows that obey the predicate/truth statement in the ON part of the join
If you don't want those rows in there then one of two things must be the case:
1) The truth you specified in the ON clause is faulty; for example saying SELECT * FROM person INNER JOIN shoes ON person.age = shoes.size is faulty - two people with age 13 and two shoes with size 13 will produce 4 results, and shoe size has nothing to do with age anyway
2) There were rows in the table joined in that didn't apply to the results you were looking for, but you forgot to filter them out by putting some WHERE (or additional restriction in the ON) clause. Example, a table holds all historical data as well as current, and the current record is the one with a NULL in the DeletedOn column. If you forget to say WHERE deletedon IS NULL then your data will multiply as all the past rows that don't apply to your query are brought in
Don't alias tables with tbX, tbY etc.. Make the names meaningful! Not only do aliases like tbX have no relation to the original table name (so you encounter tbX, and then have to go searching the rest of the query to find where it's declared so you can say "ah, it's the addresses table") but in this case you join idvclient in twice, but give them unhelpful aliases like tb1, tb3 when really you should have aliased them with something that describes the relationship between them and the rest of the query tables
For example, ParentClient and SubClient or OriginatingClient/HandlingClient would be better names, if these tables are in some relationship with each other.
Whatever the purpose of joining this table in twice is, alias it in relation to the purpose. It may make what you've done wriong easier to spot, for example "oh, of course.. i'm missing a WHERE parentclient.type = 'parent'" (or WHERE handlingclient.handlingdate is not null etc..)
The first step to wisdom is by calling things their proper names
I have two tables. One gives me basic information about demographics. One of the categories in my demographics table is a subset of people, which is housed in ATID 530 (there are several hundred different ATIDs) of this table:
As you can see the PK of this table is ADefID. My other table uses this as a FK. It houses indexes to additional definitions for records in the original table. However those additional definitions are also just records in the original table. The second table just provides pointers.
So if we pick a record, let's say ADefID=4684423, and look it up in the second table, we are returned this:
The CategoryADefID will then point back to the original table's ADefID for another record:
(note the ATID of this ADefID differs from the original ADefID that this is related to)
So. Let's say I want to pull out a set of records from the first row, say
WHERE ATID = 530 AND CycleID = 9600
But I also want to pull the ADesc (and maybe ADEValue) from the related definition as a separate field.
So the end result would look sort of like this:
I understand enough to make the join to the second table and return the CategoryADefID, but I dont know how to use that to call back to another ADefID in the original table. The other limitation is that I would use the ATID field in the WHERE clause (ATID=530) and the related definition will have a different ATID.
Just add another join back to the original table:
Select *
From tableA a
join tableB b on b.ADefID=a.ADefID
join tableA a2 on a2.ADefID = b.CategoryADefID
Where a.ADefID = 4684423
When I have to select a number of fields from different tables:
do I always need to join tables?
which tables do I need to join?
which fields do I have to use for the join/s?
do the joins effects reflect on fields specified in select clause or on where conditions?
Thanks in advance.
Think about joins as a way of creating a new table (just for the purposes of running the query) with data from several different sources. Absent a specific example to work with, let's imagine we have a database of cars which includes these two tables:
CREATE TABLE car (plate_number CHAR(8),
state_code CHAR(2),
make VARCHAR(128),
model VARCHAR(128),);
CREATE TABLE state (state_code CHAR(2),
state_name VARCHAR(128));
If you wanted, say, to get a list of the license plates of all the Hondas in the database, that information is already contained in the car table. You can simply SELECT * FROM car WHERE make='Honda';
Similarly, if you wanted a list of all the states beginning with "A" you can SELECT * FROM state WHERE state_name LIKE 'A%';
In either case, since you already have a table with the information you want, there's no need for a join.
You may even want a list of cars with Colorado plates, but if you already know that "CO" is the state code for Colorado you can SELECT * FROM car WHERE state_code='CO'; Once again, the information you need is all in one place, so there is no need for a join.
But suppose you want a list of Hondas including the name of the state where they're registered. That information is not already contained within a table in your database. You will need to "create" one via a join:
car INNER JOIN state ON (car.state_code = state.state_code)
Note that I've said absolutely nothing about what we're SELECTing. That's a separate question entirely. We also haven't applied a WHERE clause limiting what rows are included in the results. That too is a separate question. The only thing we're addressing with the join is getting data from two tables together. We now, in effect, have a new table called car INNER JOIN state with each row from car joined to each row in state that has the same state_code.
Now, from this new "table" we can apply some conditions and select some specific fields:
SELECT plate_number, make, model, state_name
FROM car
INNER JOIN state ON (car.state_code = state.state_code)
WHERE make = 'Honda'
So, to answer your questions more directly, do you always need to join tables? Yes, if you intend to select data from both of them. You cannot select fields from car that are not in the car table. You must first join in the other tables you need.
Which tables do you need to join? Whichever tables contain the data you're interested in querying.
Which fields do you have to use? Whichever fields are relevant. In this case, the relationship between cars and states is through the state_code field in both table. I could just as easily have written
car INNER JOIN state ON (state.state_code = car.plate_number)
This would, for each car, show any states whose abbreviations happen to match the car's license plate number. This is, of course, nonsensical and likely to find no results, but as far as your database is concerned it's perfectly valid. Only you know that state_code is what's relevant.
And does the join affect SELECTed fields or WHERE conditions? Not really. You can still select whatever fields you want and you can still limit the results to whichever rows you want. There are two caveats.
First, if you have the same column name in both tables (e.g., state_code) you cannot select it without clarifying which table you want it from. In this case I might write SELECT car.state_code ...
Second, when you're using an INNER JOIN (or on many database engines just a JOIN), only rows where your join conditions are met will be returned. So in my nonsensical example of looking for a state code that matches a car's license plate, there probably won't be any states that match. No rows will be returned. So while you can still use the WHERE clause however you'd like, if you have an INNER JOIN your results may already be limited by that condition.
Very broad question, i would suggest doing some reading on it first but in summary:
1. joins can make life much easier and queries faster, in a nut shell try to
2. the ones with the data you are looking for
3. a field that is in both tables and generally is unique in at least one
4. yes, essentially you are createing one larger table with joins. if there are two fields with the same name, you will need to reference them by table name.columnname
do I always need to join tables?
No - you could perform multiple selects if you wished
which tables do I need to join?
Any that you want the data from (and need to be related to each other)
which fields do I have to use for the
join/s?
Any that are the same in any tables within the join (usually primary key)
do the joins effects reflect on fields specified in select clause or on where conditions?
No, however outerjoins can cause problems
(1) what else but tables would you want to join in mySQL?
(2) those from which you want to correlate and retrieve fields (=data)
(3) best use indexed fields (unique identifiers) to join as this is fast. e.g. join
retrieve user-email and all the users comments in a 2 table db
(with tables: tableA=user_settings, tableB=comments) and both having the column uid to indetify the user by
select * from user_settings as uset join comments as c on uset.uid = c.uid where uset.email = "test#stackoverflow.com";
(4) both...
I'm always discouraged from using one, but is there a circumstance when it's the best approach?
It's rare, but I have a few cases where it's used. Typically in exception reports or ETL or other very peculiar situations where both sides have data you are trying to combine.
The alternative is to use an INNER JOIN, a LEFT JOIN (with right side IS NULL) and a RIGHT JOIN (with left side IS NULL) and do a UNION - sometimes this approach is better because you can customize each individual join more obviously (and add a derived column to indicate which side is found or whether it's found in both and which one is going to win).
I noticed that the wikipedia page provides an example.
For example, this allows us to see
each employee who is in a department
and each department that has an
employee, but also see each employee
who is not part of a department and
each department which doesn't have an
employee.
Note that I never encountered the need of a full outer join in practice...
I've used full outer joins when attempting to find mismatched, orphaned data, from both of my tables and wanted all of my result set, not just matches.
Just today I had to use Full Outer Join. It is handy in situations where you're comparing two tables. For example, the two tables I was comparing were from different systems so I wanted to get following information:
Table A has any rows that are not in Table B
Table B has any rows that are not in Table A
Duplicates in either Table A or Table B
For matching rows whether values are different (Example: The table A and Table B both have Acct# 12345, LoanID abc123, but Interest Rate or Loan Amount is different
In addition, I created an additional field in SELECT statement that uses a CASE statement to 'comment' why I am flagging this row. Example: Interest Rate does not match / The Acct doesn't exist in System A, etc.
Then saved it as a view. Now, I can use this view to either create a report and send it to users for data correction/entry or use it to pull specific population by 'comment' field I created using a CASE statement (example: all records with non-matching interest rates) in my stored procedure and automate correction, etc.
If you want to see an example, let me know.
The rare times i have used it has been around testing for NULLs on both sides of the join in case i think data is missing from the initial INNER JOIN used in the SQL i'm testing on.
They're handy for finding orphaned data but I rarely use then in production code. I wouldn't be "always discouraged from using one" but I think in the real world they are less frequently the best solution compared to inners and left/right outers.
In the rare times that I used Full Outer Join it was for data analysis and comparison purpose such as when comparing two customers tables from different databases to find out duplicates in each table or to compare the two tables structures, or to find out null values in one table compared to the other, or finding missing information in one tables compared to the other.
For example, suppose you have two tables: one containing customer data and another containing order data. A full outer join would allow you to see all customers and all orders, even if some customers have no orders or some orders have no corresponding customer. This can help you identify any gaps in the data and ensure that all relevant information is included in the result set.
It's important to note that a full outer join can produce a huge result set since it includes all rows from both tables. This can be inefficient in terms of performance, so it's best to use a full outer join only when it is necessary to include all rows from both tables.
SELECT *
FROM table1
FULL OUTER JOIN table2
ON table1.column_name = table2.column_name;
This will return all rows from both table1 and table2, filling in NULL values for missing matches on either side.