I have a table that records daily sales data. However, there are days when no sale is made and hence there is no record on the database for those dates. Is it possible to extract data out from the table that returns null for these dates when no sale was made
Referring to the image attached, it is seen there is no sales done on Jan 4 and Jan 8. I would like to write a SQL query that would return all dates from Jan 1 - Jan 10 but for Jan 4 and Jan 8, it should return 0 since there is no row for those dates (no sale done)
My date starts from Mar 1, 2018 and should go on for the next few quarters.
Yes. In Postgres, you can use generate_series() to generate dates or numbers within a range.
Then, you can use a cross join to generate the rows and then a left join to bring in the data:
select s.seller, gs.dte, t.count
from (select generate_series(mindate::timestamp, maxdate::timestamp, interval '1 day')::date
from (select min(date) as mindate, max(date) as maxdate
from t
) x
) gs(dte) cross join
(select distinct seller from t) s left join
t
on t.date = gs.dte and t.seller = s.seller
CTE is also an alternative here,
DECLARE #FDATE DATE = '2018-01-01'
,#TDATE DATE = '2018-01-10'
;WITH CTE_DATE
AS (
SELECT #FDATE AS CDATE
UNION ALL
SELECT DATEADD(DAY,1,CDATE)
FROM CTE_DATE
WHERE DATEADD(DAY,1,CDATE) <= #TDATE
)
SELECT C.CDATE AS [DATE],COUNT(*) AS [COUNT]
FROM CTE_DATE AS C
LEFT OUTER JOIN [MY_TABLE] AS M ON C.CDATE = M.[DATE] --*[your table here]*
GROUP BY C.CDATE
OPTION ( MAXRECURSION 0 );
Related
Below is a subset of my table (for the first id)
date
id
value
01/01/2022
1
5
08/01/2022
1
2
For each id, the dates are not consecutive (e.g., for id 1, the min date is 01/01/2022 and the max date is 08/01/2022)--there are 7 days in between both dates. I want to insert rows to make the dates for each id consecutive and contiguous - the value for the value field/column to be filled with 0s so that the updated table looks like:
date
id
value
01/01/2022
1
5
02/01/2022
1
0
03/01/2022
1
0
04/01/2022
1
0
05/01/2022
1
0
06/01/2022
1
0
07/01/2022
1
0
08/01/2022
1
2
Any SQL code on how to implement this would be highly appreciated. I have a calendar table but am unsure how to join it with the above table so that I fill in missing dates dynamically for each id with 0s.
My calendar table looks like:
date
01/01/2022
02/01/2022
03/01/2022
04/01/2022
Considering you state you have a calendar table, it seems what you need to do with JOIN to it with the MIN and MAX dates from your other table, and the LEFT JOIN back to your table:
WITH MinMax AS(
SELECT ID,
MIN(date) AS MinDate,
MAX(date) AS MaxDate
FROM dbo.YourTable
GROUP BY ID),
Dates AS(
SELECT MM.ID,
C.CalendarDate AS [Date]
FROM MinMax MM
JOIN dbo.CalendarTable C ON MM.MinDate <= C.CalendarDate
AND MM.MaxDate >= C.CalendarDate)
SELECT D.ID,
D.[Date],
ISNULL(YT.[Value],0) AS [Value]
FROM Dates D
LEFT JOIN dbo.YourTable YT ON D.ID = YT.ID
AND D.[Date] = YT.[Date];
SET DATEFORMAT DMY
-- CREATE A TABLE WITH OUR INPUT DATA
DROP TABLE IF EXISTS #TheData
GO
CREATE TABLE #TheData
(TheDate DATE, id INT, TheValue INT)
INSERT INTO #TheData
(TheDate,id,Thevalue)
VALUES
('01/01/2022',1,5),
('08/01/2022',1,2),
('17/01/2022',2,7),
('25/01/2022',2,7),
('15/02/2022',2,7)
-- CREATE A CALENDAR CTE
DECLARE #StartDate date = '20210101';
DECLARE #CutoffDate date = DATEADD(DAY, -1, DATEADD(YEAR, 2, #StartDate));
;WITH DateSeq(TheDate) AS
(
SELECT #StartDate
UNION ALL
SELECT DATEADD(dd,1,TheDate) FROM DateSeq
WHERE TheDate < #CutoffDate
)
-- CROSS JOIN OUR CALENDAR CTE TO OUR SOURCE DATA. DERIVED TABLE TO GET FIRST AND LAST OF EACH RANGE TO USE FOR JOIN
SELECT
ds.*
,SourceDataRangesByID.ID
,ISNULL(td.TheValue,0) AS TheValue
FROM
DateSeq ds
CROSS JOIN
(
SELECT
d.ID
,MIN(d.TheDate) AS MinDatePerID
,MAX(d.TheDate) AS MaxDatePerID
FROM #TheData d
GROUP BY d.ID
) SourceDataRangesByID
LEFT JOIN #TheData td ON td.id = SourceDataRangesByID.ID AND td.TheDate = ds.TheDate
WHERE ds.TheDate >= SourceDataRangesByID.MinDatePerID
AND ds.TheDate <= SourceDataRangesByID.MaxDatePerID
OPTION (MAXRECURSION 0);
try the generate_series to create a date table then right join with it and coalesce for the non null value
SELECT generate_series('2016-01-01', -- series start date
'2018-06-30', -- series end date
'1 day'::interval)::date AS day) AS daily_series
from mytable
See Generate_Series for TSQL
https://dba.stackexchange.com/questions/255165/does-ms-sql-server-have-generate-series-function
(Sql server 2022)
https://learn.microsoft.com/en-us/sql/t-sql/functions/generate-series-transact-sql?view=sql-server-ver16
I have data that looks like this where there is a monthly count of a particular animal for each month. By default, it aggregates in the month where there is data.
However, I would like to like to have a default set of dates for each animal up to the current month date with 0 if there's no data. Desired Result -
Is there a way to handle with a on sql server and not in Excel?
Much appreciated in advance.
You can generate the months you want using a numbers table or recursive CTE (or calendar table). Then cross join with the animals to generate the rows and use left join to bring in the existing data:
with dates as (
select min(date) as dte
from t
union all
select dateadd(month, 1 dte)
from dates
where dte < getdate()
)
select a.animal, d.dte, coalesce(t.monthly_count, 0) as monthly_count
from dates d cross join
(select distinct animal from t) a left join
data t
on t.date = d.dte and t.animal = a.animal
order by a.animal, d.dte;
I am querying the counts of logs that appear on particular days. However on some days, no log records I'm searching for are recorded. How can I set count to 0 for these days and return a result with the full set of dates in a date range?
SELECT r.LogCreateDate, r.Docs
FROM(
SELECT SUM(TO_NUMBER(REPLACE(ld.log_detail, 'Total Documents:' , ''))) AS Docs, to_char(l.log_create_date,'YYYY-MM-DD') AS LogCreateDate
FROM iwbe_log l
LEFT JOIN iwbe_log_detail ld ON ld.log_id = l.log_id
HAVING to_char(l.log_create_date , 'YYYY-MM-DD') BETWEEN '2020-01-01' AND '2020-01-07'
GROUP BY to_char(l.log_create_date,'YYYY-MM-DD')
ORDER BY to_char(l.log_create_date,'YYYY-MM-DD') DESC
) r
ORDER BY r.logcreatedate
Current Result - Id like to include the 01, 04, 05 with 0 docs.
LOGCREATEDATE
Docs
2020-01-02
7
2020-01-03
3
2020-01-06
6
2020-01-07
1
You need a full list of dates first, then outer join the log data to that. There are several ways to generate the list of dates but now common table expressions (cte) are an ANSI standard way to do this, like so:
with cte (dt) as (
select to_date('2020-01-01','yyyy-mm-dd') as dt from dual -- start date here
union all
select dt + 1 from cte
where dt + 1 < to_date('2020-02-01','yyyy-mm-dd') -- finish (the day before) date here
)
select to_char(cte.dt,'yyyy-mm-dd') as LogCreateDate
, r.Docs
from cte
left join (
SELECT SUM(TO_NUMBER(REPLACE(ld.log_detail, 'Total Documents:' , ''))) AS Docs
, trunc(l.log_create_date) AS LogCreateDate
FROM iwbe_log l
LEFT JOIN iwbe_log_detail ld ON ld.log_id = l.log_id
HAVING trunc(l.log_create_date) BETWEEN to_date('2020-01-01','yyyy-mm-dd' AND to_date('2020-01-07','yyyy-mm-dd')
GROUP BY trunc(l.log_create_date)
) r on cte.dt = r.log_create_date
order by cte.dt
also, when dealing with dates I prefer to not convert them to strings until final output which allows you to still get proper date order and maximum query efficiency.
I am pulling transactions that happen on an attribute (attribute ID 4205 in table 1235) by the date that a change happened to the attribute (found in the History table) and counting up the number of changes that occurred by month. So far I have
SELECT TOP(100) PERCENT MONTH(H.transactiondate) AS Month, COUNT(*) AS Count
FROM hsi.rmObjectInstance1235 AS O LEFT OUTER JOIN
hsi.rmObjectHistory AS H ON H.objectID = O.objectID
WHERE H.attributeid = 4205) AND Year(H.transaction date) = '2020'
GROUP BY MONTH(H.transactiondate)
And I get
Month Count
---------------
1 9
2 4
3 11
4 14
5 1
I need to display a zero for months June - December instead of excluding those months.
One option uses a recursive query to generate the dates, and then brings the original query with a left join:
with all_dates as (
select cast('2020-01-01' as date) dt
union all
select dateadd(month, 1, dt) from all_dates where dt < '2020-12-01'
)
select
month(d.dt) as month,
count(h.objectid) as cnt
from all_dates d
left join hsi.rmobjecthistory as h
on h.attributeid = 4205
and h.transaction_date >= d.dt
and h.transaction_date < dateadd(month, 1, d.dt)
and exists (select 1 from hsi.rmObjectInstance1235 o where o.objectID = h.objectID)
group by month(d.dt)
I am quite unclear about the intent of the table hsi.rmObjectInstance1235 in the query, as none of its column are used in the select and group by clauses; it it is meant to filter hsi.rmobjecthistory by objectID, then you can rewrite this as an exists condition, as shown in the above solution. Possibly, you might as well be able to just remove that part of the query.
Also, note that
top without order by does not really make sense
top (100) percent is a no op
As a consequence, I removed that row-limiting clause.
SQL Server 2005
I have an SQL Function (ftn_GetExampleTable) which returns a table with multiple result rows
EXAMPLE
ID MemberID MemberGroupID Result1 Result2 Result3 Year Week
1 1 1 High Risk 2 xx 2011 22
2 11 4 Low Risk 1 yy 2011 21
3 12 5 Med Risk 3 zz 2011 25
etc.
Now I do a count and group by on a table above this for Result 2 for instance so I get
SELECT MemberGroupID, Result2, Count(*) AS ExampleCount, Year, Week
FROM ftn_GetExampleTable
GROUP BY MemberGroupID, Result2, Year, Week
MemberGroupID Result2 ExampleCount Year Week
1 2 4 2011 22
4 1 2 2011 21
5 3 1 2011 25
Now imagine when I go to graph this new table between Weeks 20 and 23 of Year 2011, you'll see that it won't graph 20 or 23 or certain groups or even certain results in this example as they are not in the included data, so I need "false data" inserted into this table which has all the possibilities so they at least show on a graph even if the count is 0, does this make sense?
I am wondering on the easiest and kind of most dynamic way as it could be Result1 or Result3 I want to Graph on (different column types).
Thanks in advance
It looks like your dimensions are: MemberGroupID,Result2, and week (Year,Week).
One approach to solving this is to generate a list of all values you want for all the dimensions, and produce a cartesian product of them. As an example,
SELECT m.MemberGroupID, n.Result2, w.Year, w.Week
FROM (SELECT MemberGroupID FROM ftn_GetExampleTable GROUP BY MemberGroupID) m
CROSS
JOIN (SELECT Result2 FROM ftn_GetExampleTable GROUP BY Result2 ) n
CROSS
JOIN (SELECT Year, Week FROM myCalendar WHERE ... ) w
You don't necessarily need a table named myCalendar. (That approach does seem to be the popular one.) You just need a row source from which you can derive a list of (Year, Week) tuples. (There are answers to the question elsewhere in Stackoverflow, how to generate a list of dates.)
And the list of MemberGroupID and Result2 values doesn't have to come from the ftn_GetExampleTable rowsource, you could substitute another query.
With a cartesian product of those dimensions, you've got a complete "grid". Now you can LEFT JOIN your original result set to that.
Any place you don't have a matching row from the "gappy" result query, you'll get a NULL returned. You can leave the NULL, or replace it with a 0, which is probably what you want if it's a "count" you are returning.
SELECT d.MemberGroupID
, d.Result2
, d.Year
, d.Week
, IFNULL(r.ExampleCount,0) as ExampleCount
FROM ( <dimension query from above> ) d
LEFT
JOIN ( <original ExampleCount query> ) r
ON r.MemberGroupID = d.MemberGroupID
AND r.Result2 = d.Result2
AND r.Year = d.Year
AND r.Week = d.Week
That query can be refactored to make use of Common Table Expressions, which makes the query a little easier to read, especially if you are including multiple measures.
; WITH d AS ( /* <dimension query with no gaps (example above)> */
)
, r AS ( /* <original query with gaps> */
SELECT MemberGroupID, Result2, Count(*) AS ExampleCount, Year, Week
FROM ftn_GetExampleTable
GROUP BY MemberGroupID, Result2, Year, Week
)
SELECT d.*
, IFNULL(r.ExampleCount,0)
FROM d
LEFT
JOIN r
ON r.Year=d.Year AND r.Week=d.Week AND r.MemberGroupID = d.MemberGroupID
AND r.Result2 = d.Result2
This isn't a complete working solution to your problem, but it outlines an approach you can use.
Whenever I need to generate a sequence within SQL-Server I use the sys.all_objects table along with the ROW_NUMBER function, then maninpulate it as required:
SELECT ROW_NUMBER() OVER(ORDER BY Object_ID) AS Sequence
FROM Sys.All_Objects
So for the list of year and week numbers I would use:
DECLARE #StartDate DATETIME,
#EndDate DATETIME
SET #StartDate = '20110101'
SET #EndDate = '20120601'
SELECT DATEPART(YEAR, Date) AS YEAR,
DATEPART(WEEK, Date) AS WeekNum
FROM ( SELECT DATEADD(WEEK, ROW_NUMBER() OVER(ORDER BY Object_ID) - 1, #StartDate) AS Date
FROM Sys.All_Objects
) Dates
WHERE Date < #endDate
Where the dates subquery provides a list of dates at one week intervals between your start and end dates.
So in your example the end result would be something like:
DECLARE #StartDate DATETIME,
#EndDate DATETIME
SET #StartDate = '20110101'
SET #EndDate = '20120601'
;WITH Data AS
( SELECT MemberGroupID,
Result2,
Count(*) AS ExampleCount,
Year,
Week
FROM ftn_GetExampleTable
GROUP BY MemberGroupID, Result2, Year, Week
), Dates AS
( SELECT DATEPART(YEAR, Date) AS YEAR,
DATEPART(WEEK, Date) AS WeekNum
FROM ( SELECT DATEADD(WEEK, ROW_NUMBER() OVER(ORDER BY Object_ID) - 1, #StartDate) AS Date
FROM Sys.All_Objects
) Dates
WHERE Date < #endDate
)
SELECT YearNum,
WeeNum,
MemberID,
Result2,
COALESCE(ExampleCount, 0) AS ExampleCount
FROM Dates
LEFT JOIN Data
ON YearNum = Data.Year
AND WeekNum = Data.Week