Is there a validate the cardinality of SQL joins? - sql

Question

When using SQL (snowflake) I often join tables.
I can never be 100% sure that the join is a one-to-one, one-to-many, many-to-many, etc...
In Python pandas this is a setting in merge statements that will assert that the join is of the expected kind.
Is there an equivalent in SQL?
EDIT: This is the pandas API (which I like)
table1 = pd.DataFrame({
'userId': [1,2,3],
'age':[20,30,40]
})
#| userId | age |
#|---------:|------:|
#| 1 | 20 |
#| 2 | 30 |
#| 3 | 40 |
table2 = pd.DataFrame({
'userId': [1,2,2,3],
'gender':['M','M','F', 'F'],
'gender_valid_to_date': [None, '2020-01-01', None, None]
})
#| userId | gender | gender_valid_to_date |
#|---------:|:---------|:-----------------------|
#| 1 | M | |
#| 2 | M | 2020-01-01 |
#| 2 | F | |
#| 3 | F | |
pd.merge(table1, table2, on='userId', how='left', validate='one_to_one')
# This raises a merge error
MergeError: Merge keys are not unique in right dataset; not a one-to-one merge
You can imagine it's easy to naively go for a LEFT JOIN here assuming it'd just add the gender column, but you know what they say about assuming...
In this case the "right" solution to get a table and the user's current gender is
SELECT
userId,
age,
gender AS current_gender
FROM table1 t1
LEFT JOIN (
SELECT userId, gender FROM table2 WHERE gender_valid_to_date is Null
) t2 ON t2.userId = t1.userId
However, you can see how this can get complicated to "check" constantly.

Answer 1

Give a try with ERROR_ON_NONDETERMINISTIC_MERGE which helps to raise an error in case if there are dup's which comes from join and try to merge. May be this could help you.
CREATE TABLE T1(USERID NUMBER, NAME STRING);
CREATE TABLE T2(USERID NUMBER, NAME STRING);
INSERT INTO
T1 VALUES(1, 'E'),(1, 'D'),(2, 'C');
INSERT INTO
T2 VALUES(1, 'A'),(1, 'B'),(2, 'C');
MERGE INTO T2 USING (
Select
T1.USERID,
T1.NAME
FROM
T1
) AS T1 ON T1.USERID = T2.USERID(+)
WHEN MATCHED THEN UPDATE SET
T2.USERID = T1.USERID,
T2.NAME = T1.NAME;

Answer 2

The immediate answer to this question is that Snowflake itself does not provide this feature built-in, although IMO it would be an interesting and useful new feature (I might even call it an "innovation") in data warehouses.
In order to make this job easier, save all non-trivial query results to a view or temporary table.
The first solution is to craft ad-hoc validity checks to be run on the resulting view or table.
You can do this in SQL directly by putting each check into a field or row of a table and visually ensuring that they are all TRUE.
In the example you gave, if can save the results to joined_1_2, then you can run a query like the following:
SELECT
NOT EXISTS (
SELECT
user_id, count(*) as n
FROM joined_1_2
GROUP BY user_id
HAVING n > 1
) AS no_duplicates
;
Depending on how fancy and/or automated you want things to be, can write stored procedures, Python scripts, etc. to run such queries and provide various outputs based on their results. The Python option could be particularly interesting, as you can use native Python assert statements or even a test framework like Pytest.
If these joins are being automated in some kind of recurring ETL/ELT pipeline, you might also want to consider using a tool like Great Expectations to implement data quality and correctness checks.
Another strategy is to impose constraints on the temporary tables that you create.
For example, you can impose a PRIMARY KEY constraint on the table that is intended to hold the results of a 1:1 join. Then if the data being inserted into that table results in duplicated primary keys, then you know that there is a problem in the source tables.
I think this technique is the most direct analogy of setting validate= in Pandas, but it requires a bit more setup and forethought. Unfortunately, I don't think Snowflake supports combining constraints (e.g. foreign key) with create-table-as-select syntax, so you might need to run a separate CREATE TABLE command for every query output, which can be very annoying if you have a lot of columns and you only want to check a few of them.

Answer 3

No, there isn’t (not that I’m aware of). A SQL join is normally 1 to many, is less commonly 1 to 1 and there is no such thing as a many to many join (hence the need for intersection tables to model many to many relationships).
Is there a specific reason why you need to know the join type or why you don’t know it anyway - I’m not sure how you would write almost any SQL query if you didn’t already know the data model?