Group by based on 3 tables that are joined using left join - sql

Question

Below is my query that fetches the fields based on the left join of 3 tables. My requirement is to get all the fields based on the recent SystemDateTime in table Debug.T. For example, if i try it for HardwareId = 550803413, it returns 2 records with 2 different SystemDateTime. I need to filter it so that I get only 1 record for all HardwareIds based on recent SystemDateTime. Data is stored in Google Big Query.
Any help would be appreciated.
SELECT HardwareId, e.Carrier, max(d.SystemDateTime) as DateTime,
CASE
WHEN lower(DebugData) LIKE 'veri%' THEN 'Verizon'
WHEN REGEXP_MATCH(lower(DebugData),'\\d+') THEN c.Network
END
AS ActualData
FROM (
SELECT
HardwareId, SystemDateTime, max(SystemDateTime) as max_date,
INTEGER(RTRIM(SUBSTR(REGEXP_REPLACE(REGEXP_REPLACE(DebugData,'\\"',' '), '\\?',' ') ,0,3))) AS d1,
INTEGER(RTRIM(SUBSTR(REGEXP_REPLACE(DebugData,'[^a-zA-Z0-9]',' '),4,LENGTH(DebugData)-3))) AS d2
FROM TABLE_DATE_RANGE([Debug.T],TIMESTAMP('2016-05-16'),TIMESTAMP('2016-05-16'))
GROUP BY HardwareId, DebugReason, DebugData, SystemDateTime
HAVING DebugReason = 31) AS d
LEFT JOIN
(
SELECT Mcc, Mnc as Mnc, Network from [Debug.Carrier]
) As c
ON c.Mcc = d.d1 and c.Mnc = d.d2
INNER JOIN
(
SELECT VehicleId, APNCarrier FROM [Info_20160516]
) As e
ON d.HardwareId = e.VehicleId
GROUP BY HardwareId, ActualData, e.Carrier
HAVING HardwareId = 550803413
Current output:
HardwareId DebugReason DebugData e_APNCarrier DateTime ActualDebugData
550473814 50013 23430"? Unknown 2016-05-16 08:09:09.534597 Everyth. Ev.wh./T-Mobile
550473814 50013 23410"? Unknown 2016-05-16 07:50:48.526288 O2 Ltd.
550473814 50013 23415"? Unknown 2016-05-16 23:54:37.487154 Vodafone
Expected output:
Since the recent SystemDateTime is 23:54:37.487154, query should filter the records based on the recent SystemDateTime and provide the result.
HardwareId DebugReason DebugData e_APNCarrier DateTime ActualDebugData
550473814 50013 23415"? Unknown 2016-05-16 23:54:37.487154 Vodafone

Answer 1

so you just want the latest record per HardwareId based on DateTime? Try this:
SELECT * FROM (
SELECT HardwareId, e.Carrier, d.SystemDateTime as DateTime,
CASE
WHEN lower(DebugData) LIKE 'veri%' THEN 'Verizon'
WHEN REGEXP_MATCH(lower(DebugData),'\\d+') THEN c.Network
END
AS ActualData,
ROW_NUMBER() OVER (PARTITION BY HARDWAREID ORDER BY d.SystemDateTime desc) RN
FROM (
SELECT
HardwareId, SystemDateTime, max(SystemDateTime) as max_date,
INTEGER(RTRIM(SUBSTR(REGEXP_REPLACE(REGEXP_REPLACE(DebugData,'\\"',' '), '\\?',' ') ,0,3))) AS d1,
INTEGER(RTRIM(SUBSTR(REGEXP_REPLACE(DebugData,'[^a-zA-Z0-9]',' '),4,LENGTH(DebugData)-3))) AS d2
FROM TABLE_DATE_RANGE([Debug.T],TIMESTAMP('2016-05-16'),TIMESTAMP('2016-05-16'))
GROUP BY HardwareId, DebugReason, DebugData, SystemDateTime
HAVING DebugReason = 31) AS d
LEFT JOIN
(
SELECT Mcc, Mnc as Mnc, Network from [Debug.Carrier]
) As c
ON c.Mcc = d.d1 and c.Mnc = d.d2
INNER JOIN
(
SELECT VehicleId, APNCarrier FROM [Info_20160516]
) As e
ON d.HardwareId = e.VehicleId
HAVING HardwareId = 550803413
)
WHERE RN = 1