How to count changes within each column and in SQL - sql

Question

This is how the table is looking like:
id
city
address
steps
date
1
null
null
a
2021-11-01
1
NY
null
b
2021-11-04
1
Chicago
null
c
2021-11-05
2
SF
33, ABC colony
x
2021-12-01
2
SF
33, ABC colony
y
2021-12-04
2
SF
44, Kang Street
z
2021-12-05
3
Austin
null
i
2022-01-01
3
Austin
12, Bridgetown
j
2022-01-04
3
Austin
null
k
2022-01-05
What I want is total count of times that for any 'id' there was an update in fields city and address only but excluding null. We dont care about the column steps and any updates there.
For id = 1, the city was changed from null to NY to Chicago. However, the address remained null, but the given the dates I count it as 2. Changing from null to NY is not supposed to be counted as an update.
For id = 2, the city was never changed it was always SF. But, there is a change in address but only once and thus we count the update as 2 again.
For id = 3, the city was never changed but the address changed from null to an address back to null. We don't count the first null because the customer may not have the info but if he/she changes it back to null that has to be counted. Here also update count will be 2.
I am expecting the results as:
id
change_count
1
2
2
2
3
2
Can I know how to do this via sql? The major problem is to not count "null" as I rank the id in ascending order of when the record came but count when it is changed back to "null" is where I am mainly confused.
Any help is appreciated. I am working on it and if I get the SQL finalized, I will share it here too.

Answer 1

Can this work for you?
WITH
-- your input, do not use in query ...
indata(id,city,addr,steps,dt) AS (
SELECT 1,NULL ,NULL ,'a',DATE '2021-11-01'
UNION ALL SELECT 1,'NY' ,NULL ,'b',DATE '2021-11-04'
UNION ALL SELECT 1,'Chicago',NULL ,'c',DATE '2021-11-05'
UNION ALL SELECT 2,'SF' ,'33, ABC colony' ,'x',DATE '2021-12-01'
UNION ALL SELECT 2,'SF' ,'33, ABC colony' ,'y',DATE '2021-12-04'
UNION ALL SELECT 2,'SF' ,'44, Kang Street','z',DATE '2021-12-05'
UNION ALL SELECT 3,'Austin' ,NULL ,'i',DATE '2022-01-01'
UNION ALL SELECT 3,'Austin' ,'12, Bridgetown' ,'j',DATE '2022-01-04'
UNION ALL SELECT 3,'Austin' ,NULL ,'k',DATE '2022-01-05'
)
-- end of your input
-- real query starts here, replace following comma with "WITH" ...
,
olap AS (
SELECT
id
-- a NULL is not COUNTed DISTINCT, but an empty string is
, CASE WHEN city IS NULL AND LAG(city) OVER w IS NOT NULL THEN '' ELSE city END AS city
, CASE WHEN addr IS NULL AND LAG(addr) OVER w IS NOT NULL THEN '' ELSE addr END AS addr
FROM indata
WINDOW w AS (PARTITION BY id ORDER BY dt)
)
SELECT
id
, GREATEST(COUNT(DISTINCT city),COUNT(DISTINCT addr)) AS changecount
FROM olap
GROUP BY 1
ORDER BY 1
;
-- out id | changecount
-- out ----+-------------
-- out 1 | 2
-- out 2 | 2
-- out 3 | 2

Answer 2

I tired using combination of window-function lag and coalesce method and I finally got the answer but if someone has a better solution, do suggest. :)
My sql:
with cte1 as(
select *,
row_number over(partition by id order by date) as rn
from main_table),
cte2 as (
select * from cte1 where (rn =1 and city <> null or address <> null)),
cte3 as (
SELECT id,
case when coalesce(city,'-1')=COALESCE(lag(city,1) over(partition by id order by date), city,'-1') then 0 else 1 end as cityChange,
case when coalesce(address,'-1')=COALESCE(lag(address,1) over(partition by id order by date), address,'-1') then 0 else 1 end as addressChange
from cte2)
select id,
sum(cityChange) as cityChangeCount,
sum(addressChange) as addressChangeCount
from cte3
group by id