JOIN the same table on two columns - sql

I use JOINs to replace country and product IDs in import and export data with actual country and products names stored in separate tables. In the data source table (data), there are two columns with country IDs, for origin and destination, both of which I am replacing with country names.
The code I have come up with refers to the country_names table twice – as country_names, and country_names2, – which doesn’t seem to be very elegant. I expected to be able to refer to the table just once, by a single name. I would be grateful if someone pointed me to a more elegant and maybe more efficient way to achieve the same result.
SELECT
country_names.name AS origin,
country_names2.name AS dest,
product_names.name AS product,
SUM(data.export_val) AS export_val,
SUM(data.import_val) AS import_val
FROM
OEC.year_origin_destination_hs92_6 AS data
JOIN
OEC.products_hs_92 AS product_names
ON
data.hs92 = product_names.hs92
JOIN
OEC.country_names AS country_names
ON
data.origin = country_names.id_3char
JOIN
OEC.country_names AS country_names2
ON
data.dest = country_names2.id_3char
WHERE
data.year > 2012
AND data.export_val > 1E8
GROUP BY
origin,
dest,
product
The table to convert product IDs to product names has 6K+ rows. Here is a small sample:
id hs92 name
63215 3215 Ink
2130110 130110 Lac
21002 1002 Rye
2100200 100200 Rye
52706 2706 Tar
20902 902 Tea
42203 2203 Beer
42302 2302 Bran
178703 8703 Cars
The table to convert country IDs to country names (which is the table I have to JOIN on twice) has 264 rows for all countries in the world. (id_3char is the column used.) Here is a sample:
id id_3char name
euchi chi Channel Islands
askhm khm Cambodia
eublx blx Belgium-Luxembourg
eublr blr Belarus
eumne mne Montenegro
euhun hun Hungary
asmng mng Mongolia
nabhs bhs Bahamas
afsen sen Senegal
And here is a sample of data from the import and export data table with a total of 205M rows that has the two columns origin and dest that I am making a join on:
year origin dest hs92 export_val import_val
2009 can isr 300410 2152838.47 3199.24
1995 chn jpn 590190 275748.65 554154.24
2000 deu gmb 100610 1573508.44 1327.0
2008 deu jpn 540822 10000.0 202062.43
2010 deu ukr 950390 1626012.04 159423.38
2006 esp prt 080530 2470699.19 125291.33
2006 grc ind 844859 8667.0 3182.0
2000 ltu deu 630399 6018.12 5061.96
2005 usa zaf 290219 2126216.52 34561.61
1997 ven ecu 281122 155347.73 1010.0

I think you already have it done such that it can be considered good enough to just use as is :o)
Meantime, If for some reason you really-really want to avoid two joins on that country table - what you can do is to materialize below select statement into let's say `OEC.origin_destination_pairs` table
SELECT
o.id_3char o_id_3char,
o.name o_name,
d.id_3char d_id_3char,
d.name d_name
FROM `OEC.country_names` o
CROSS JOIN `OEC.country_names` d
Then you can just join on that new table as below
SELECT
country_names.o_name AS origin,
country_names.d_name AS dest,
product_names.name AS product,
SUM(data.export_val) AS export_val,
SUM(data.import_val) AS import_val
FROM OEC.year_origin_destination_hs92_6 AS data
JOIN OEC.products_hs_92 AS product_names
ON data.hs92 = product_names.hs92
JOIN OEC.origin_destination_pairs AS country_names
ON data.origin = country_names.o_id_3char
AND data.dest = country_names2.d_id_3char
WHERE data.year > 2012
AND data.export_val > 1E8
GROUP BY
origin,
dest,
product
The motivation behind above is cost of storing and querying in your particular case
Your `OEC.country_names` table is just about 10KB in size
Each time you query it you pay as if it is 10MB (Charges are rounded to the nearest MB, with a minimum 10 MB data processed per table referenced by the query, and with a minimum 10 MB data processed per query.)
So, if you will materialize above mentioned table - it will still be less than 10MB so no difference in querying charges
Similar situation with storing that table - no visible changes in charges
You can check more about pricing here

Related

PostgreSQL dynamic query comparing row from table to column name of another table

I work with QGIS and PostgreSQL with PostGIS. I need help with dynamic queries for PostgreSQL.
Information is structured in tables that contain votes for parties, and other types of information like geographic area or election date.
I need to work with “tidy” data in plotly for pie charts. The desired end table must have one row per observation.
So for example for the given table “Election Results”:
Country
PartyA
PartyB
PartyC
Argentina
100
10
20
Uruguay
3
5
1
Chile
40
200
50
Values for columns to be treated like parties, are stored in table “Parties”:
Party
PartyA
PartyB
PartyC
PartyD
PartyE
I need to separate one observation of voting results per row as follows (“Ending Table”):
Country
Party
Votes
Argentina
PartyA
100
Argentina
PartyB
10
Argentina
PartyC
20
Uruguay
PartyA
3
Uruguay
PartyB
5
Uruguay
PartyC
1
Chile
PartyA
40
Chile
PartyB
200
Chile
PartyC
50
The query should work with any number of parties stored in the “Parties” table. The “parties” table could include some rows not present in the election results table, but all parties in the election results table will exist in the “parties” table.
I understand it should be done iterating over the columns in the “Election results” table. If the name of a column matches the value of a row in “party table”, then we “untidy” data as in the “ending table”.
You can "iterate" over the columns using Postgres's JSON functions:
Something like this:
select er."Country",
pv.*
from election_results er
cross join jsonb_each_text(to_jsonb(er) - 'Country') as pv(party, votes)
join parties p on p.party = pv.party
;
Online example

Duplicate rows because 1 column has multiple distinct values

I'm running a SELECT query to get data across multiple tables in the same server instance. However I've just noticed that the rows pulled on some data get duplicated because the main table I'm pulling from has a few different values in one of the columns. Here's the query:
SELECT DISTINCT BIF030.C_ACCOUNT AS ACCOUNTNUMBER,
BIF003.C_ACCOUNTTYPE AS ACCOUNTTYPECODE,
CON013.C_DESCRIPTION AS ACCOUNTTYPE,
BIF003.C_DIVISION AS ZONE_DIVISONCODE,
CON028.C_DESCRIPTION AS ZONE_DIVISION,
BIF030.C_METER as METERNUMBER,
BIF005.C_METERCUSTOM1 AS REGISTERNUMBER,
CONVERT(DECIMAL(20,2), BIF030.N_CONSUMP) AS CONSUMPTION,
CON007.C_DESCRIPTION AS UNITS,
BIF030.T_READDATE AS READINGDATE,
MONTH(BIF030.T_READDATE) AS READINGMONTH,
DAY(BIF030.T_READDATE) AS READINGDAY,
YEAR(BIF030.T_READDATE) AS READINGYEAR,
BIF030.I_DAYS AS READINGDAYSCOUNT
FROM ADVANCED.BIF030
LEFT JOIN ADVANCED.CON007 ON CON007.C_UNITS=BIF030.C_UNITS
LEFT JOIN ADVANCED.BIF005 ON BIF005.C_METER=BIF030.C_METER
LEFT JOIN ADVANCED.BIF003 ON BIF003.C_ACCOUNT=BIF030.C_ACCOUNT
LEFT JOIN ADVANCED.CON013 ON CON013.C_ACCOUNTTYPE=BIF003.C_ACCOUNTTYPE
LEFT JOIN ADVANCED.CON028 ON CON028.C_DIVISION=BIF003.C_DIVISION
WHERE T_READDATE > '01-01-2014'
ORDER BY ACCOUNTNUMBER, READINGDATE ASC
I know SELECT DISTINCT is frowned upon, but I get even more rows without it. Here's a sample of what the data looks like when pulled:
ACCOUNTNUMBER
ACCOUNTTYPECODE
ACCOUNTTYPE
ZONE_DIVISIONCODE
ZONE_DIVISION
METERNUMBER
REGISTERNUMBER
CONSUMPTION
UNITS
READINGDATE
READINGMONTH
READINGDAY
READINGYEAR
READINGDAYSCOUNT
1234567
SP
ACCOUNT TYPE 1
00
00-NO ZONE
123456789
987654321
3.00
Thousands of Gallons
2014-01-16 00:00:00.00
1
16
2014
30
1234567
MF
ACCOUNT TYPE 2
02
02-GRAVITY
123456789
987654321
3.00
Thousands of Gallons
2014-01-16 00:00:00.00
1
16
2014
30
1234567
SR
ACCOUNT TYPE 3
02
02-GRAVITY
123456789
987654321
3.00
Thousands of Gallons
2014-01-16 00:00:00.00
1
16
2014
30
I also know the column that is messing this up is the "AccountTypeCode" because other accounts that don't have multiple codes associated with the "AccountNumber" only show 1 set of rows. So this one specifically (and probably others) is tripling the amount of rows pulled when it should only pull one for each "ReadingDate".
Also if anyone knows a good way to optimize the query I'd be happy to learn. I know just enough SQL to be dangerous, but not enough to figure this out. Thanks.
Ok. So good news and I want to add this in case it helps anyone else in the future. I found out that since the ACCOUNTTYPECODE and ZONE_DIVISIONCODE were coming from the table BIF003 I needed to add more in the WHERE statement. This is what fixed it for me:
AND BIF030.C_CUSTOMER = BIF003.C_CUSTOMER
Because the C_CUSTOMER column was different (it's a column in the BIF003 and BIF030 tables) which lead to the separate ACCOUNTTYPECODE results I need to check it in the WHERE statement.
Thanks everyone for kick starting my brain on this one.

DB2 Select from two tables when one table requires sum

In a DB2 Database, I want to do the following simple mathematics using a SQL query:
AvailableStock = SupplyStock - DemandStock
SupplyStock is stored in 1 table in 1 row, let's call this table the Supply table.
So the Supply table has this data:
ProductID | SupplyStock
---------------------
109 10
244 7 edit: exclude this product from the search
DemandStock is stored in a separate table Demand, where demand is logged as each customer logs demand during a customer order journey. Example data from the Demand table:
ProductID | DemandStock
------------------------
109 1
244 4 edit: exclude this product
109 6
109 2
So in our heads, if I want to calculate the AvailableStock for product '109', Supply is 10, Demand for product 109 totals to 9, and so Available stock is 1.
How do I do this in one select query in DB2 SQL?
The knowledge I have so far of some of the imagined steps in PseudoCode:
I select SupplyStock where product ID = '109'
I select sum(DemandStock) where product ID = '109'
I subtract SupplyStock from DemandStock
I present this as a resulting AvailableStock
The results will look like this:
Product ID | AvailableStock
109 9
I'd love to get this selected in one SQL select query.
Edit: I've since received an answer (that was almost perfect) and realised the question missed out some information.
This information:
We need to exclude data from products we don't want to select data for, and we also need to specifically select product 109.
My apologies, this was omitted from the original question.
I've since added a 'where' to select the product and this works for me. But for future sake, perhaps the answer should include this information too.
You do this using a join to bring the tables together and group by to aggregate the results of the join:
select s.ProductId, s.SupplyStock, sum(d.DemandStock),
(s.SupplyStock - sum(d.DemandStock)) as Available
from Supply s left join
Demand d
on s.ProductId = d.ProductId
where s.ProductId = 109
group by s.ProductId, s.SupplyStock;

Display the related attributes of a MAX in an ACCESS query

I have 2 tables joined with political results and I need to have the votes SUM per county, and then the MAX of the vote counts per county, with the Party that relates to the MAX in another column. I'm having trouble getting the Party into the Query results without messing up the SUM and MAX columns.
This Table I can get with the Following SQL
County Name SumOfVoteCount MaxOfVoteCount OfficeID
Baker 7253 4008 S
SELECT NY_Race.[County Name], Sum(NY_Results.VoteCount) AS SumOfVoteCount, Max(NY_Results.VoteCount) AS MaxOfVoteCount
FROM NY_Race INNER JOIN NY_Results ON NY_Race.RaceCountyID = NY_Results.RaceCountyID
GROUP BY NY_Race.[County Name], NY_Race.OfficeID
HAVING (((NY_Race.OfficeID)="S"));
What I need is for the Party that has that 4008 vote total to be included in the query results, but when I try to select Party to be added, it shows all of them and messes up the SUM of the vote count, and I end of with this:
County Name SumOfVoteCount MaxOfVoteCount1 Party OfficeID
Baker 2927 2927 Dem S
Baker 4008 4008 GOP S
Baker 101 101 Lib S
Baker 53 53 Prg S
Baker 164 164 WF S
This is the SQL code I am using that gets the above Table:
SELECT NY_Race.[County Name], Sum(NY_Results.VoteCount) AS SumOfVoteCount, Max(NY_Results.VoteCount) AS MaxOfVoteCount, NY_Results.Party
FROM NY_Race INNER JOIN NY_Results ON NY_Race.RaceCountyID = NY_Results.RaceCountyID
GROUP BY NY_Race.[County Name], NY_Race.OfficeID, NY_Results.Party
HAVING (((OR_Race.OfficeID)="S"));
How can I get this table in the query results?
County Name SumOfVoteCount MaxOfVoteCount Party OfficeID
Baker 7253 4008 GOP S
I can't help but think I'm missing a WHERE claus somewhere that compares Party to MAXofVoteCount
One way to approach these is to have a nested subquery that gets the MAX() for the field of interest. Then, only select the record with that MAX(). Here's the structure:
select COUNTY_NAME, R1.*
, (select sum(votecount) from results R2 where R1.COUNTY_ID=R2.COUNTY_ID and R1.OFFICE_ID=R2.OFFICE_ID)
from RESULTS R1
join RACE on R1.COUNTY_ID=RACE.COUNTY_ID and R1.OFFICE_ID=RACE.OFFICE_ID
where R1.office_id = 'S'
and voteCount =
(select max(votecount) from results R3 where R1.COUNTY_ID=R3.COUNTY_ID and R1.OFFICE_ID=R3.OFFICE_ID)
I created a demo on SQLFiddle.
One issue: what if two get exactly the same number of votes. That's a functional issue you will have to resolve.

SQL SUM with Repeating Sub Entries - Best Practice?

I hit this issue regularly but here is an example....
I have a Order and Delivery Tables. Each order can have one to many Deliveries.
I need to report totals based on the Order Table but also show deliveries line by line.
I can write the SQL and associated Access Report for this with ease ....
SELECT xxx
FROM
Order
LEFT OUTER JOIN
Delivery on Delivery.OrderNO = Order.OrderNo
until I get to the summing element. I obviously only want to sum each Order once, not the 1-many times there are deliveries for that order.
e.g. The SQL might return the following based on 2 Orders (ignore the banalness of the report, this is very much simplified)
Region OrderNo Value Delivery Date
North 1 £100 12-04-2012
North 1 £100 14-04-2012
North 2 £73 01-05-2012
North 2 £73 03-05-2012
North 2 £73 07-05-2012
South 3 £50 23-04-2012
I would want to report:
Total Sales North - £173
Delivery 12-04-2012
Delivery 14-04-2012
Delivery 01-05-2012
Delivery 03-05-2012
Delivery 07-05-2012
Total Sales South - £50
Delivery 23-04-2012
The bit I'm referring to is the calculation of the £173 and £50 which the first of which obviously shouldn't be £419!
In the past I've used things like MAX (for a given Order) but that seems like a fudge.
Surely there must be a regular answer to this seemingly common problem but I can't find one.
I don't necessarily need the code - just a helpful point in the right direction.
Many thanks,
Chris.
A roll up operator may not look pretty. However, it would do the regular aggregates that you see now, and it show the subtotals of the order. This is what you're looking for.
SELECT xxx
FROM
Order
LEFT OUTER JOIN
Delivery on Delivery.OrderNO = Order.OrderNo
GROUP BY xxx
WITH ROLLUP;
I'm not exactly sure how the rest of your query is set up, but it would look something like this:
Region OrderNo Value Delivery Date
North 1 £100 12-04-2012
North 1 £100 14-04-2012
North 2 £73 01-05-2012
North 2 £73 03-05-2012
North 2 £73 07-05-2012
NULL NULL f419 NULL
I believe what you want is called a windowing function for your aggregate operation. It looks like the following:
SELECT xxx, SUM(Value) OVER (PARTITION BY Order.Region) as OrderTotal
FROM
Order
LEFT OUTER JOIN
Delivery on Delivery.OrderNO = Order.OrderNo
Here's the MSDN article. The PARTITION BY tells the SUM to be done separately for each distinct Order.Region.
Edit: I just noticed that I missed what you said about orders being counted multiple times. One thing you could do is SUM() the values before joining, as a CTE (guessing at your schema a bit):
WITH RegionOrders AS (
SELECT Region, OrderNo, SUM(Value) OVER (PARTITION BY Region) AS RegionTotal
FROM Order
)
SELECT Region, OrderNo, Value, DeliveryDate, RegionTotal
FROM RegionOrders RO
INNER JOIN Delivery D on D.OrderNo = RO.OrderNo