I need to map a many-to-many relationship between two flat tables. Table A contains a list of possible configurations (where each column is a question and the cell value is the answer). NULL values denote that the answer does not matter. Table B contains actual configurations with the same columns.
Ultimately, I need the final results to show which configurations are mapped between table B and A:
Example
ActualId | ConfigId
---------+---------
5 | 1
6 | 1
8 | 2
. | .
. | .
. | .
N | M
To give a simple example of the tables and data I'm working with, the first table would look like such:
Table A
--------
ConfigId | Size | Color | Cylinders | ... | ColumnN
---------+------+-------+-----------+-----+--------
1 | 3 | | 4 | ... | 5
2 | 4 | 5 | 5 | ... | 5
3 | | 5 | | ... | 5
And Table B would look like this:
Table B
-------
ActualId | Size | Color | Cylinders | ... | ColumnN
---------+------+-------+-----------+-----+--------
1 | 3 | 1 | 4 | ... | 5
2 | 3 | 8 | 4 | ... | 5
3 | 4 | 5 | 5 | ... | 5
4 | 7 | 5 | 6 | ... | 5
Since the NULL values denote that any value can work, the expected result would be:
Expected
---------
ActualId | ConfigId
---------+---------
1 | 1
2 | 1
3 | 2
3 | 3
4 | 3
I'm trying to figure out the best way to go about matching the actual data which has over a hundred columns. I know trying to check each and every column for NULL values is absolutely wrong and will not perform well. I'm really fascinated with this problem and would love some help to find the best way to tackle this.
So, this joins table a on size, color and cylinders.
The size match will be A against B:
If A.SIZE is null, the compare will B.SIZE=B.SIZE which will always return true.
If A.SIZE is not null, the compare will be A.SIZE=B.SIZE which will only be true if they match.
The matching on color and cylinders are similar.
SELECT * FROM TABLEA A
INNER JOIN TABLEB B ON ISNULL(A.SIZE, B.SIZE)=B.SIZE
AND ISNULL(A.COLOR, B.COLOR)=B.COLOR
AND ISNULL(A.CYLINDERS, B.CYLINDERS)=B.CYLINDERS
The following schema is used to create simple algebraic formulas. variables is used to create formulas such as x=3+4y. variables_has_sub_variables is used to combine the previous mentioned formulas and uses the sign column (will be +1 or -1 only) to determine whether the formula should be added or subtracted to the combination.
For instance, variables table might have the following data where the Implied Formulas column is not really in the table but just for illustrative purposes only.
variables table
+-----------+-----------+-------+------------------+
| variables | intercept | slope | Implied Formula |
+-----------+-----------+-------+------------------+
| 1 | 2.86 | -0.82 | Y1=+2.86-0.82*X1 |
| 2 | 2.96 | -3.49 | Y2=+2.96-3.49*X2 |
| 3 | 2.56 | 2.81 | Y3=+2.56+2.81*X3 |
| 4 | 3.04 | -3.43 | Y4=+3.04-3.43*X4 |
| 5 | -1.94 | 4.11 | Y5=-1.94+4.11*X5 |
| 6 | -1.21 | -0.62 | Y6=-1.21-0.62*X6 |
| 7 | 0.88 | -0.61 | Y7=+0.88-0.61*X7 |
| 8 | -2.77 | -0.34 | Y8=-2.77-0.34*X8 |
| 9 | 1.81 | 1.65 | Y9=+1.81+1.65*X9 |
+-----------+-----------+-------+------------------+
Then, given the below variables_has_sub_variables data, the variables combined resulting in X7=+Y1-Y2+Y3, X8=+Y4+Y5-Y7, and X9=+Y6-Y7+Y8. Next Y7, Y8, and Y9 can be derived using the variables table resulting in Y7=+0.88-0.61*X7, etc. Note that the application will prevent an endless loop such as inserting a record where variables equals 7 and sub_variables equals 9 as variable 9 is based on variable 7.
variables_has_sub_variables table
+-----------+---------------+------+
| variables | sub_variables | sign |
+-----------+---------------+------+
| 7 | 1 | 1 |
| 7 | 2 | -1 |
| 7 | 3 | 1 |
| 8 | 4 | 1 |
| 8 | 5 | 1 |
| 8 | 7 | -1 |
| 9 | 6 | 1 |
| 9 | 7 | -1 |
| 9 | 8 | 1 |
+-----------+---------------+------+
My objective is given any variable (i.e. 1 to 9), determine the constants and root variables where a root variable is defined as not being in variables_has_sub_variables.variables (I can also easily a root column to variables if needed), and these root variables includes 1 through 6 using my above example data.
Doing so for a root variable is easier as there are no sub_variables and is simply Y1=+2.86-0.82*X1.
Doing so for variable 7 is a little trickier:
Y7=+0.88-0.61*X7
=+0.88-0.61*(+Y1-Y2+Y3)
=+0.88-0.61*(+(+2.86-0.82*X1)-(+2.96-3.49*X2)+( +2.56+2.81*X3))
= -0.62 + 0.50*X1 - 2.13*X2 - 1.71*X3
Now the SQL. Below is how I created the tables:
CREATE DATABASE algebra;
USE algebra;
CREATE TABLE `variables` (
`variables` INT NOT NULL,
`slope` DECIMAL(6,2) NOT NULL DEFAULT 1,
`intercept` DECIMAL(6,2) NOT NULL DEFAULT 0,
PRIMARY KEY (`variables`))
ENGINE = InnoDB;
CREATE TABLE `variables_has_sub_variables` (
`variables` INT NOT NULL,
`sub_variables` INT NOT NULL,
`sign` TINYINT NOT NULL,
PRIMARY KEY (`variables`, `sub_variables`),
INDEX `fk_variables_has_variables_variables1_idx` (`sub_variables` ASC),
INDEX `fk_variables_has_variables_variables_idx` (`variables` ASC),
CONSTRAINT `fk_variables_has_variables_variables`
FOREIGN KEY (`variables`)
REFERENCES `variables` (`variables`)
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_variables_has_variables_variables1`
FOREIGN KEY (`sub_variables`)
REFERENCES `variables` (`variables`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB;
INSERT INTO variables(variables,intercept,slope) VALUES (1,2.86,-0.82),(2,2.96,-3.49),(3,2.56,2.81),(4,3.04,-3.43),(5,-1.94,4.11),(6,-1.21,-0.62),(7,0.88,-0.61),(8,-2.77,-0.34),(9,1.81,1.65);
INSERT INTO variables_has_sub_variables(variables,sub_variables,sign) VALUES (7,1,1),(7,2,-1),(7,3,1),(8,4,1),(8,5,1),(8,7,-1),(9,6,1),(9,7,-1),(9,8,1);
And now the queries. XXXX is 7, 8, and 9 for the following results. Before each query, I show my expected results.
WITH RECURSIVE t AS (
SELECT v.variables, v.slope, v.intercept
FROM variables v
WHERE v.variables=XXXX
UNION ALL
SELECT v.variables, vhsv.sign*t.slope*v.slope slope, vhsv.sign*t.slope*v.intercept intercept
FROM t
INNER JOIN variables_has_sub_variables vhsv ON vhsv.variables=t.variables
INNER JOIN variables v ON v.variables=vhsv.sub_variables
)
SELECT variables, SUM(slope) constant FROM t GROUP BY variables
UNION SELECT 'intercept' variables, SUM(intercept) intercept FROM t;
Variable 7 Desired
+-----------+----------+
| variables | constant |
+-----------+----------+
| 1 | 0.50 |
| 2 | -2.13 |
| 3 | -1.71 |
| intercept | -0.6206 |
+-----------+----------+
Variable 7 Actual
+-----------+----------+
| variables | constant |
+-----------+----------+
| 1 | 0.50 |
| 2 | -2.13 |
| 3 | -1.71 |
| 7 | -0.61 |
| intercept | -0.61 |
+-----------+----------+
5 rows in set (0.00 sec)
Variable 8 Desired
+-----------+-----------+
| variables | constant |
+-----------+-----------+
| 1 | 0.17 |
| 2 | -0.72 |
| 3 | -0.58 |
| 4 | 1.17 |
| 5 | -1.40 |
| intercept | -3.355004 |
+-----------+-----------+
Variable 8 Actual
+-----------+----------+
| variables | constant |
+-----------+----------+
| 1 | 0.17 |
| 2 | -0.73 |
| 3 | -0.59 |
| 4 | 1.17 |
| 5 | -1.40 |
| 7 | -0.21 |
| 8 | -0.34 |
| intercept | -3.36 |
+-----------+----------+
8 rows in set (0.00 sec)
Variable 9 Desired
+-----------+------------+
| variables | constant |
+-----------+------------+
| 1 | -0.54 |
| 2 | 2.32 |
| 3 | 1.87 |
| 4 | 1.92 |
| 5 | -2.31 |
| 6 | -1.02 |
| intercept | -4.6982666 |
+-----------+------------+
Variable 9 Actual
+-----------+----------+
| variables | constant |
+-----------+----------+
| 1 | -0.55 |
| 2 | 2.33 |
| 3 | 1.88 |
| 4 | 1.92 |
| 5 | -2.30 |
| 6 | -1.02 |
| 7 | 0.67 |
| 8 | -0.56 |
| 9 | 1.65 |
| intercept | -4.67 |
+-----------+----------+
10 rows in set (0.00 sec)
All I need to do is detect which variables are not the root variables and filter them out. How should this be accomplished?
In response to JNevill's answer:
For v.variables of 9
+-----------+-------+-------+----------+
| variables | depth | path | constant |
+-----------+-------+-------+----------+
| 1 | 3 | 9>7>1 | -0.55 |
| 2 | 3 | 9>7>2 | 2.33 |
| 3 | 3 | 9>7>3 | 1.88 |
| 4 | 3 | 9>8>4 | 1.92 |
| 5 | 3 | 9>8>5 | -2.30 |
| 6 | 2 | 9>6 | -1.02 |
| 7 | 2 | 9>7 | 0.67 |
| 8 | 2 | 9>8 | -0.56 |
| 9 | 1 | 9 | 1.65 |
| intercept | 1 | 9 | -4.67 |
+-----------+-------+-------+----------+
10 rows in set (0.00 sec)
I'm not going to attempt to fully wrap my head around what you are doing, and I would agree with #RickJames up in the comments that this feels like maybe not the best use-case for a database. I too am a little obsessive though. I get it.
There are couple of things that I almost always track in a recursive CTE.
The "Path". If I'm going to let a query head down a rabbit hole, I want to know how it got to the end point. So I track a path so I know which primary key was selected through each iteration. In the recursive seed (top portion) I use something like SELECT CAST(id as varchar(500)) as path... and in the recursive member (bottom portion) I do something like recursiveCTE.path + '>' + id as path...
The "Depth". I want to know how deep the iterations went to get to the resulting record. This is tracked by adding SELECT 1 as depth to the recursive seed and recursiveCTE + 1 as depth to the recursive member. Now I know how deep each record is.
I believe number 2 will solve your issue:
WITH RECURSIVE t
AS (
SELECT v.variables,
v.slope,
v.intercept,
1 as depth
FROM variables v
WHERE v.variables = XXXX
UNION ALL
SELECT v.variables,
vhsv.sign * t.slope * v.slope slope,
vhsv.sign * t.slope * v.intercept intercept,
t.depth + 1
FROM t
INNER JOIN variables_has_sub_variables vhsv ON vhsv.variables = t.variables
INNER JOIN variables v ON v.variables = vhsv.sub_variables
)
SELECT variables,
SUM(slope) constant
FROM t
WHERE depth > 1
GROUP BY variables
UNION
SELECT 'intercept' variables,
SUM(intercept) intercept
FROM t;
The WHERE clause here will restrict records in your recursive result set that have a depth of 1, meaning they were brought in from the recursive seed portion of the recursive CTE (That they are a root).
It wasn't clear if you required that the root be removed from your second UNION of your t CTE. If so, the same logic applies; just toss that WHERE clause on to restrict depth records of 1
While it may not be helpful here, an example of your recursive cte with PATH would be:
WITH RECURSIVE t
AS (
SELECT v.variables,
v.slope,
v.intercept,
1 as depth,
CAST(v.variables as CHAR(30)) as path
FROM variables v
WHERE v.variables = XXXX
UNION ALL
SELECT v.variables,
vhsv.sign * t.slope * v.slope slope,
vhsv.sign * t.slope * v.intercept intercept,
t.depth + 1,
CONCAT(t.path,'>', v.variables)
FROM t
INNER JOIN variables_has_sub_variables vhsv ON vhsv.variables = t.variables
INNER JOIN variables v ON v.variables = vhsv.sub_variables
)
SELECT variables,
SUM(slope) constant
FROM t
WHERE depth > 1
GROUP BY variables
UNION
SELECT 'intercept' variables,
SUM(intercept) intercept
FROM t;
Might be very simple, but I've been digging fow a few days now... I just can't figure out how to make this SQL query in Access...
In reference to the tables below, i'm looking for the query that can extract all the ITEMS for a specific Shop (ie 1:Alpha) from a specific GROUP (ie 1:Tools), that are NOT in the report for 2014... in this case ITEMS.IDs 6, 8, 9 and 10!
Tables:
Years
ID | Year
-----------------------------------------------
1 | 2014
2 | 2015
Shops
ID | ShopName
-----------------------------------------------
1 | Alpha
2 | Bravo
Items
ID | StockNbr | Description | GroupID
-----------------------------------------------
1 | 00-1200 | Ratchet 1/4 | 1
2 | 00-1201 | Ratchet 1/2 | 1
3 | 00-1300 | Screwdriver Philips No1 | 1
4 | 01-5544 | Banana | 2
5 | 00-4457 | Apple | 2
6 | 21-8887 | Hammer | 1
7 | 21-6585 | Drill | 1
8 | 21-4499 | Multimeter | 1
9 | 21-5687 | Digital Caliper | 1
10 | 22-7319 | File Set | 1
...
Groups
ID | GroupName
-----------------------------------------------
1 | Tools
2 | Fruits
REPORTS
ID | YearID | ShopID | ItemID
-----------------------------------------------
1 | 1 | 1 | 1
2 | 1 | 1 | 2
3 | 1 | 1 | 3
4 | 1 | 1 | 4
5 | 1 | 1 | 7
6 | 1 | 2 | 5
7 | 1 | 2 | 8
8 | 1 | 2 | 10
I've tried this, but then I realize it doesn't take the shops into consideration, it'll list all items that are not listed in reports, so if reports has an item for shop 2, it won't list it either...
SELECT Items.ID, Items.StockNbr, Items.Description, Items.GroupID, Reports.YearID, Reports.ShopID
FROM Reports
RIGHT JOIN Items ON Reports.ItemID = Items.ID
WHERE (((Items.GroupID)=1) AND ((Reports.UnitID) Is Null))
ORDER BY Items.StockNbr;
Thank you!
I think you're looking for an anti-join. There are several ways to do this. Here's one using not in.
select i.* from items i
where i.GroupId = 1
and i.ID NOT IN (
select ItemID from reports r
where r.ShopID = 1
and r.YearID = 2014
)
If the table Reports does not reference Items.ID then there is no available relationship ShopID or YearID
select *
from items
left join reports on items.id = reports.itemid
where reports.itemid IS NULL
Assume I have this schema (tested on postgresql) where the 'Scorelines' relation contains results of sport matches. (kickoff is a TIMESTAMP but replaced by INT for readability)
SQLFiddle here: http://sqlfiddle.com/#!12/52475/3
CREATE TABLE Scorelines (
team TEXT,
kickoff INT,
scored INT,
conceded INT
);
Now I want to produce another column 'three_matches_scored' that contains the sum of the points scored
over the 3 preceding game (determined by kickoff) of the same team. I have this:
SELECT team, kickoff, scored, conceded, SUM(scored) OVER three_matches AS three_matches_scored
FROM Scorelines
WINDOW three_matches AS
(PARTITION BY team ORDER BY kickoff
ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING)
ORDER BY kickoff;
This works beautifully so far, except that I get values starting from the second game. Example:
| TEAM | KICKOFF | SCORED | CONCEDED | THREE_MATCHES_SCORED |
|------|---------|--------|----------|----------------------|
| A | 1 | 1 | 0 | (null) |
| B | 2 | 1 | 1 | (null) |
| A | 3 | 1 | 1 | 1 |
| A | 4 | 3 | 0 | 2 |
| B | 4 | 1 | 4 | 1 |
| A | 6 | 0 | 2 | 5 |
| B | 6 | 4 | 2 | 2 |
| B | 8 | 1 | 2 | 6 |
| B | 10 | 1 | 1 | 6 |
| A | 11 | 2 | 1 | 4 |
I want the column 'three_matches_scored' to be (null) for the first 3 games because there are no 3 results to sum up. How can I achieve this?
I'd prefer simple understandable solutions, performance is not critical for this particular case.
My only idea right now, is to define a stored function SUM3, that results in (null) with less than 3 values to add up. But I never defined a function in SQL and can't seem to figure it out.
You can use a case statement to null the rows where there are less than 3 games:
SELECT team, kickoff, scored, conceded,
CASE WHEN COUNT(scored) OVER three_matches = 3
THEN SUM(scored) OVER three_matches
ELSE NULL
END AS three_matches_scored
FROM Scorelines
WINDOW three_matches AS
(PARTITION BY team ORDER BY kickoff
ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING)
ORDER BY kickoff;
Output:
team | kickoff | scored | conceded | three_matches_scored
------+---------+--------+----------+----------------------
A | 1 | 1 | 0 |
B | 2 | 1 | 1 |
A | 3 | 1 | 1 |
A | 4 | 3 | 0 |
B | 4 | 1 | 4 |
A | 6 | 0 | 2 | 5
B | 6 | 4 | 2 |
B | 8 | 1 | 2 | 6
B | 10 | 1 | 1 | 6
A | 11 | 2 | 1 | 4
(10 rows)
See harmics answer above.
(my first solution, just for reference)
Solution with user defined aggregate:
CREATE TYPE intermediate_sum AS (
sum INT,
count INT
);
CREATE FUNCTION sum_sfunc(intermediate_sum, INTEGER) RETURNS intermediate_sum AS
$$ SELECT $2 + $1.sum AS sum, $1.count - 1 AS count $$ LANGUAGE SQL;
CREATE FUNCTION sum_ffunc(intermediate_sum) RETURNS INTEGER AS
$$ SELECT (CASE WHEN $1.count > 1 THEN null
WHEN $1.count = 0 THEN $1.sum
END)
$$ LANGUAGE SQL;
CREATE AGGREGATE sum3(INTEGER) (
sfunc = sum_sfunc,
finalfunc = sum_ffunc,
stype = intermediate_sum,
initcond = '(0,3)'
);
The aggregate SUM3 wants at least 3 values, otherwise it returns (null). One can define other aggreates like SUM4 by changing the initcond, for example to '(0,4)'.
An example is the easiest way to explain what I'm looking to do:
GIVEN:
~move~
id | from | to
--------------
1 | 1 | 2
2 | 1 | 2
3 | 2 | 3
4 | 3 | 1
~locations~
id | name
---------
1 | home
2 | work
3 | out
How can I get:
id | from | to
----------------
1 | home | work
2 | home | work
3 | work | out
4 | out | home
That is, the human-readable name for both the from and to columns.
Select
Move.ID,
[From] = FromLocation.Name,
[To] = ToLocation.Name
From
Move
Inner Join Location As FromLocation On Move.[From] = FromLocation.ID
Inner Join Location As ToLocation On Move.[To] = ToLocation.ID