How could I join 4 large tables in BigQuery with Salesforce tables? SQL - sql

Question

What I want to do is to join each table with the key field sendID which is not unique (I use the GROUP BY function to make it unique)
FYI: These tables just have information from year 2022.
I have 4 tables from SalesForce.
Table 1: salesforce_sent (emails sent to different destinations and the largest table)
Table 2: salesforce_open (destinations which open the email and it's also a pretty large table)
Table 3: salesforce_clicks (destinations which opens the email and clicks the link to a website)
Table 4: salesforce_sendjobs (Helps to link the information from Salesforce and Google Analytics)
Each table has different columns. I already tried using LEFT JOIN and INNER JOIN in my queries but the time running the query it's insane (I've been up to 2-3 hours waiting and then cancel the run).
What I tried is this (I guess using INNER JOIN it is better than a LEFT JOIN because it's less heavy):
WITH SENT AS (
SELECT
sendid as sendid,
EXTRACT(date FROM eventdate) as sent_date,
lower(emailaddress) as emailaddress,
COUNT(*) as sent,
FROM salesforce_sent
group by 1,2,3
),
CLICKS AS (
SELECT
sendid as sendid,
EXTRACT(DATE from eventdate) as click_date,
url as url,
regexp_extract(url, r'utm_source=([^&]+)') as source,
regexp_extract(url, r'utm_medium=([^&]+)') as medium,
regexp_extract(url, r'utm_campaign=([^&]+)') as campaign,
regexp_extract(url, r'utm_content=([^&]+)') as ad_content,
isunique as isunique_click,
COUNT(*) as clicks,
FROM salesforce_clicks
group by 1,2,3,4,5,6,7,8
),
OPEN AS (
SELECT
sendid as sendid,
EXTRACT(date FROM eventdate) as open_date,
isunique as isunique_open,
COUNT(*) as open
FROM salesforce_opens
group by 1,2,3
),
SENDJOBS AS (
SELECT
sendid as sendid,
EXTRACT(date FROM senttime) as sent_date,
LOWER(emailname) as emailname,
LOWER(SPLIT(emailname, '-')[SAFE_OFFSET(1)]) AS pos,
FROM salesforce_sendjobs
group by 1,2,3,4
)
SELECT
a.sendid as sendid,
a.sent_date,
c.open_date,
d.click_date,
a.emailaddress,
b.emailname,
d.url as url,
b.pos,
d.source,
d.medium,
d.campaign,
d.ad_content,
sum(a.sent) as sent,
sum(c.open) as open,
sum(d.clicks) as clicks
FROM SENT a
INNER JOIN SENDJOBS b ON a.sendid = b.sendid
INNER JOIN OPEN c ON a.sendid = c.sendid
INNER JOIN CLICKS d ON a.sendid = d.sendid
WHERE 1=1
GROUP BY 1,2,3,4,5,6,7,8,9,10,11,12
I also tried using UNION ALL but it's not what I'm searching for. Because you won't have the complete information row when it comes to information that you have in table A but not in table B.
What I want is to have a merge between all this 4 tables using the sendID as the key. It would be really nice to make lighter the query somehow. This query process 100 GB when run (is not that much).
I also see that the compute problem it's in the join.