Adding Missing Days in Data -SQL - sql

Question

I have a table that's only showing weekday values. It's grabbing these from a file that's imported only on the weekdays. I'm needing to also add in the weekend (or holidays) with the previously known day's value. I have asked this question when I was needing it to be used in MS Access. I'm now moving this database to SQL Server.
If you're wanting to see what worked for me in Access, you're more than welcome to check out the link.
I have attempted to adapt the MS Access SQL to SQL Server with:
SELECT a1.IDNbr, a1.Balance, CONVERT(int, DAY(a1.BalDate)) + 3
FROM tblID a1 INNER JOIN tblID a2 ON (CONVERT(int, DAY(a1.BalDate)) + 4 = a2.BalDate) AND (a1.IDNbr = a2.IDNbr)
WHERE NOT EXISTS (
SELECT *
FROM tblID a3
WHERE a3.IDNbr = a1.IDNbr AND a3.BalDate = CONVERT(int, DAY(a1.BalDate)) + 3) AND (DATEPART(W, a1.BalDate) = 6
);
However, I'm getting the Error:
Msg 206, Level 16, State 2, Line 4
Operand type clash: date is incompatible with int
Question: How can I get this query (which I will be turning into an INSERT statement) to show all the missing days within my data and to assign the value of the last known day to the missing days?
Data that I have(starting on Friday):
+-------------------------------------+
|ID | IDNbr | Balance | BalDate |
+-------------------------------------+
|001| 91 | 529 | 1/5/2018 |
|002| 87 | 654 | 1/5/2018 |
|003| 45 | 258 | 1/5/2018 |
|004| 91 | 611 | 1/8/2018 |
|005| 87 | 753 | 1/8/2018 |
|006| 45 | 357 | 1/8/2018 |
|...| .. | ... | ........ |
+-------------------------------------+
'BalDate then skips past 1/6/2018 and 1/7/2018 to 1/8/2018
Data that I'm needing:
+-------------------------------------+
|ID | IDNbr | Balance | BalDate |
+-------------------------------------+
|001| 91 | 529 | 1/5/2018 |
|002| 87 | 654 | 1/5/2018 |
|003| 45 | 258 | 1/5/2018 |
|004| 91 | 529 | 1/6/2018 |
|005| 87 | 654 | 1/6/2018 |
|006| 45 | 258 | 1/6/2018 |
|007| 91 | 529 | 1/7/2018 |
|008| 87 | 654 | 1/7/2018 |
|009| 45 | 258 | 1/7/2018 |
|010| 91 | 611 | 1/8/2018 |
|011| 87 | 753 | 1/8/2018 |
|012| 45 | 357 | 1/8/2018 |
|...| .. | ... | ........ |
+-------------------------------------+
'I'm needing it to add the Saturday(1/6/2018) and Sunday(1/7/2018) before continuing on to 1/8/2018
Any help would be appreciated. Thank you in advance!
If there are downvotes, I ask that you please explain why you are downvoting so I may correct it!

Answer 1

Ok, you're going to need the CalTable() function from Bernd's answer. We're going to use it to create a list of all calendar dates between the MIN(BalDate) and the MAX(BalDate) in tblID. We're also going to CROSS JOIN that with the list of DISTINCT IDNbr values, which I assume is the PK of tblID.
Let's create some sample data.
CREATE TABLE #tblID (ID VARCHAR(3), IDNbr INT, Balance INT, BalDate DATE)
INSERT INTO #tblID
(
ID
,IDNbr
,Balance
,BalDate
)
VALUES
('001',91,529,'1/5/2018'),
('002',87,654,'1/5/2018'),
('003',45,258,'1/5/2018'),
('004',91,611,'1/8/2018'),
('005',87,753,'1/8/2018'),
('006',45,357,'1/8/2018')
Next, we're going to INSERT new records into #tblID for the missing days. The magic here is in the LAG() function, which can looks at a previous row's data. We give it an expression for the offset value, based on the difference between missing date and the last date with data.
;WITH IDs AS
(
SELECT DISTINCT
IDNbr
FROM #tblID
)
,IDDates AS
(
SELECT
BalDate = c.[Date]
,i.IDNbr
FROM [CalTable]((SELECT MIN(BalDate) FROM #tblID), (SELECT MAX(BalDate) FROM #tblID)) c
CROSS APPLY IDs i
)
,FullResults AS
(
SELECT
i.BalDate
,i.IDNbr
,Balance = CASE WHEN t.Balance IS NOT NULL THEN t.Balance
ELSE LAG(t.Balance,
DATEDIFF(
DAY
,(SELECT MAX(t1.BalDate) FROM #tblID t1 WHERE t1.IDNbr = i.IDNbr AND t1.BalDate <= i.BalDate GROUP BY t1.IDNbr)
,i.BalDate
)
) OVER (PARTITION BY i.IDNbr ORDER BY i.BalDate ASC)
END
FROM IDDates i
LEFT JOIN #tblID t ON t.BalDate = i.BalDate AND t.IDNbr = i.IDNbr
)
INSERT INTO #tblID
(
IDNbr
,Balance
,BalDate
)
SELECT
f.IDNbr
,f.Balance
,f.BalDate
FROM FullResults f
LEFT JOIN #tblID t ON t.IDNbr = f.IDNbr AND t.BalDate = f.BalDate
WHERE t.IDNbr IS NULL
At this point, if we didn't care about the ID field, which appears to be a 3-character string representation of the row number, we'd be good. However, while I don't think it's a good practice to use a string in this manner, I'm also not one to comment on someone else's business requirements that I am not privy to.
So let's assume we have to update the ID field to match the expected output. We can do that like this:
;WITH IDUpdate AS
(
SELECT
ID = RIGHT('000' + CAST(ROW_NUMBER() OVER (ORDER BY BalDate ASC, IDNbr DESC) AS VARCHAR), 3)
,t.IDNbr
,t.Balance
,t.BalDate
FROM #tblID t
)
UPDATE t
SET t.ID = i.ID
FROM #tblID t
INNER JOIN IDUpdate i ON i.IDNbr = t.IDNbr AND i.BalDate = t.BalDate
Now if you query your updated table, you'll get the following:
SELECT
ID
,IDNbr
,Balance
,BalDate
FROM #tblID
ORDER BY BalDate ASC, IDNbr DESC
Output:
ID | IDNbr | Balance | BalDate
------------------------------
001 | 91 | 529 | 2018-01-05
002 | 87 | 654 | 2018-01-05
003 | 45 | 258 | 2018-01-05
004 | 91 | 529 | 2018-01-06
005 | 87 | 654 | 2018-01-06
006 | 45 | 258 | 2018-01-06
007 | 91 | 529 | 2018-01-07
008 | 87 | 654 | 2018-01-07
009 | 45 | 258 | 2018-01-07
010 | 91 | 611 | 2018-01-08
011 | 87 | 753 | 2018-01-08
012 | 45 | 357 | 2018-01-08

Answer 2

Here is a samples for the linked function:
create FUNCTION [dbo].[CalTable]
(
#startDate date,
#endDate date
)
RETURNS
#calender TABLE
(
[Date] date not null primary key CLUSTERED,
isMondayToFriday bit not null
)
AS
BEGIN
declare #currentday date = #startDate;
declare #isMondayToFriday bit;
while (#currentday<=#endDate)
begin
-- respect DATEFIRST depending on language settings
if (DATEPART(dw, #currentday)+##DATEFIRST-2)%7+1>5
set #isMondayToFriday = 0
else
set #isMondayToFriday = 1
insert into #calender values (#currentday, #isMondayToFriday);
set #currentday = DATEADD(D, 1, #currentday);
end
RETURN
END
GO
select * from [CalTable]({d'2018-01-01'}, {d'2018-02-03'});
use this for find the gaps.