This follows on from my previous question, but since I tried to simplify, I appear to have missed something Daily snapshot table using cte loop
I am trying to set up the below cross join between dates and an employee table. I need a daily count according to division and department, but the dates won't link easily since the dates are stored as varchar (not my choice, I can't change it).
I now have a date table that includes a style112 (yyyymmdd) key that I can link to the table, but there seems to be a failure somewhere along the joins.
I'm so close, but really am lost! I have never had to work with string dates and wouldn't wish it upon anyone.
DECLARE #DATESTART AS Date = '20180928';
DECLARE #DATEEND AS Date = '20181031';
WITH Dates AS (
SELECT #DATESTART AS Dte
UNION ALL
SELECT DTE + 1
FROM Dates
WHERE Dte <= #DATEEND )
SELECT
Dt.Dte
,CAST(DTC.Style112 AS VARCHAR)
,Emp.Division_Description
,Emp.Department_Description
,(SELECT
COUNT(*)
FROM ASS_D_EmpMaster_Live E
WHERE
E.[Start_Date] <= CAST(DTC.Style112 AS VARCHAR)
AND (E.Leaving_Date > CAST(DTC.Style112 AS VARCHAR)
OR E.Leaving_Date = '00000000')
) Counts
FROM Dates Dt
LEFT JOIN ASS_C_DateConversions DTC
ON DTC.[Date] = Dt.DtE
CROSS JOIN
(
SELECT DISTINCT
Division_Description
,Department_Description
FROM
ASS_D_EmpMaster_Live e
) Emp
OPTION (MAXRECURSION 1000)
Desired output:
Date
Dept1
Dept2
Dept3
20180901
25
231
154
20180902
23
232
154
I don't think you need the conversion table at all and I would remove it. And I believe the subquery should look like this:
SELECT COUNT(*)
FROM ASS_D_EmpMaster_Live E
WHERE
CAST(E.Start_Date AS DATE) <= Dt.Dte
AND (CAST(E.Leaving_Date AS DATE) > Dt.Dte OR E.Leaving_Date = '00000000')
Related
I'm having trouble writing a recursive function that would count the number of active clients on any given day.
Say I have a table like this:
Client
Start Date
End Date
1
1-Jan-22
2
1-Jan-22
3-Jan-22
3
3-Jan-22
4
4-Jan-22
5-Jan-22
5
4-Jan-22
6-Jan-22
6
7-Jan-22
9-Jan-22
I want to return a table that would look like this:
Date
NumActive
1-Jan-22
2
2-Jan-22
2
3-Jan-22
3
4-Jan-22
4
5-Jan-22
4
6-Jan-22
3
7-Jan-22
3
8-Jan-22
3
9-Jan-22
4
Is there a way to do this? Ideally, I'd have a fixed start date and go to today's date.
Some pieces I have tried:
Creating a recursive date table
Truncated to Feb 1, 2022 for simplicity:
WITH DateDiffs AS (
SELECT DATEDIFF(DAY, '2022-02-02', GETDATE()) AS NumDays
)
, Numbers(Numbers) AS (
SELECT MAX(NumDays) FROM DateDiffs
UNION ALL
SELECT Numbers-1 FROM Numbers WHERE Numbers > 0
)
, Dates AS (
SELECT
Numbers
, DATEADD(DAY, -Numbers, CAST(GETDATE() -1 AS DATE)) AS [Date]
FROM Numbers
)
I would like to be able to loop over the dates in that table, such as by modifying the query below for each date, such as by #loopdate. Then UNION ALL it to a larger final query.
I'm now stuck as to how I can run the query to count the number of active users:
SELECT
COUNT(Client)
FROM clients
WHERE [Start Date] >= #loopdate AND ([End Date] <= #loopdate OR [End Date] IS NULL)
Thank you!
You don't need anything recursive in this particular case, you need as a minimum a list of dates in the range you want to report on, ideally a permanent calendar table.
for purposes of demonstration you can create something on the fly, and use it like so, with the list of dates something you outer join to:
with dates as (
select top(9)
Convert(date,DateAdd(day, -1 + Row_Number() over(order by (select null)), '20220101')) dt
from master.dbo.spt_values
)
select d.dt [Date], c.NumActive
from dates d
outer apply (
select Count(*) NumActive
from t
where d.dt >= t.StartDate and (d.dt <= t.EndDate or t.EndDate is null)
)c
See this Demo Fiddle
I am querying the counts of logs that appear on particular days. However on some days, no log records I'm searching for are recorded. How can I set count to 0 for these days and return a result with the full set of dates in a date range?
SELECT r.LogCreateDate, r.Docs
FROM(
SELECT SUM(TO_NUMBER(REPLACE(ld.log_detail, 'Total Documents:' , ''))) AS Docs, to_char(l.log_create_date,'YYYY-MM-DD') AS LogCreateDate
FROM iwbe_log l
LEFT JOIN iwbe_log_detail ld ON ld.log_id = l.log_id
HAVING to_char(l.log_create_date , 'YYYY-MM-DD') BETWEEN '2020-01-01' AND '2020-01-07'
GROUP BY to_char(l.log_create_date,'YYYY-MM-DD')
ORDER BY to_char(l.log_create_date,'YYYY-MM-DD') DESC
) r
ORDER BY r.logcreatedate
Current Result - Id like to include the 01, 04, 05 with 0 docs.
LOGCREATEDATE
Docs
2020-01-02
7
2020-01-03
3
2020-01-06
6
2020-01-07
1
You need a full list of dates first, then outer join the log data to that. There are several ways to generate the list of dates but now common table expressions (cte) are an ANSI standard way to do this, like so:
with cte (dt) as (
select to_date('2020-01-01','yyyy-mm-dd') as dt from dual -- start date here
union all
select dt + 1 from cte
where dt + 1 < to_date('2020-02-01','yyyy-mm-dd') -- finish (the day before) date here
)
select to_char(cte.dt,'yyyy-mm-dd') as LogCreateDate
, r.Docs
from cte
left join (
SELECT SUM(TO_NUMBER(REPLACE(ld.log_detail, 'Total Documents:' , ''))) AS Docs
, trunc(l.log_create_date) AS LogCreateDate
FROM iwbe_log l
LEFT JOIN iwbe_log_detail ld ON ld.log_id = l.log_id
HAVING trunc(l.log_create_date) BETWEEN to_date('2020-01-01','yyyy-mm-dd' AND to_date('2020-01-07','yyyy-mm-dd')
GROUP BY trunc(l.log_create_date)
) r on cte.dt = r.log_create_date
order by cte.dt
also, when dealing with dates I prefer to not convert them to strings until final output which allows you to still get proper date order and maximum query efficiency.
I'm designing a report that returns PurchaseOrder due in future week.
Query that I've added below returns PurchaseOrder due for a particular Commodity, AmountDue and its DeliveryDate.
Obviously it only returns PO_Dates that are in the table. What I want is to also include dates where no PO is expected, i.e. null for those cell.
To me one possibility is to LEFT JOIN the dataset with set of dates of future week on Date column, that will eventually make the result null where no Purchase Order is expected.
In Firebird I don't know how to select list of week long dates and then use it in the join.
SELECT
PURCHASE_ORDER_DET.COMMODITYID AS COM_ID,
PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('29.12.1899' AS DATE) as DLV_DATE,
SUM(PURCHASE_ORDER_DET.REQQUANTITY) as DLV_DUE
FROM
PURCHASE_ORDER_DET
LEFT JOIN PURCHASE_ORDER_HDR on PURCHASE_ORDER_HDR.POH_ID =
PURCHASE_ORDER_DET.POH_ID
WHERE
PURCHASE_ORDER_DET.COMMODITYID = 1
AND PURCHASE_ORDER_HDR.STATUS in (0,1,2)
AND PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('30.12.1899' AS TIMESTAMP) >= '3.01.2019'
AND PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('30.12.1899' AS TIMESTAMP) <= '9.01.2019'
AND PURCHASE_ORDER_DET.DELETED is NULL
Group by
PURCHASE_ORDER_DET.COMMODITYID,
PURCHASE_ORDER_DET.DELIVERYDATE
DataSet
COM_ID DLV_DATE DLV_DUE
1 3.01.2019 50.000000
1 5.01.2019 10.000000
Expected
COM_ID DLV_DATE DLV_DUE
1 3.01.2019 50.000000
1 4.01.2019 null
1 5.01.2019 10.000000
1 6.01.2019 null
1 7.01.2019 null
1 8.01.2019 null
1 9.01.2019 null
Ignoring your odd use of datatypes*, there are several possible solutions:
Use a 'calendar' table that contains dates, and right join to that table (or left join from that table). The downside of course is having to populate this table (but that is a one-off cost).
Use a selectable stored procedure to generate a date range and join on that.
Generate the range in a recursive common table expression in the query itself
Option 1 is pretty self-explanatory.
Option 2 would look something like:
CREATE OR ALTER PROCEDURE date_range(startdate date, enddate date)
RETURNS (dateval date)
AS
BEGIN
dateval = startdate;
while (dateval <= enddate) do
BEGIN
suspend;
dateval = dateval + 1;
END
END
And then use this in your query like:
select date_range.dateval, ...
from date_range(date'2019-01-03', date'2019-01-09') -- use date_range(?, ?) for parameters
left join ...
on date_range.dateval = ...
Option 3 would look something like:
WITH RECURSIVE date_range AS (
SELECT date'2019-01-03' dateval -- start date, use cast(? as date) if you need a parameter
FROM rdb$database
UNION ALL
SELECT dateval + 1
FROM date_range
WHERE dateval < date'2019-01-09' -- end date use ? if you need a parameter
)
SELECT *
FROM date_range
LEFT JOIN ...
ON date_range.dateval = ...
Recursive common table expressions have a maximum recursion depth of 1024, which means that it isn't suitable if you need a span wider than 1024 days.
*: I'd suggest that you start using DATE instead of what looks like the number of days since 30-12-1899. That avoids having to do awkward calculations like you do now. If you do need those number of days, then you can for example use datediff(DAY FROM date'1899-12-30' TO somedatevalue) or somedatevalue - date'1899-12-30' to convert from date to that numeric value.
I have two different tables FirewallLog and ProxyLog. There is no relation between these two tables. They have four common fields :
LogTime ClientIP BytesSent BytesRec
I need to Calculate the total usage of a particular ClientIP for each day over a period of time (like last month) and display it like below:
Date TotalUsage
2/12 125
2/13 145
2/14 0
. .
. .
3/11 150
3/12 125
TotalUsage is SUM(FirewallLog.BytesSent + FirewallLog.BytesRec) + SUM(ProxyLog.BytesSent + ProxyLog.BytesRec) for that IP. I have to show Zero if there is no usage (no record) for that day.
I need to find the fastest solution to this problem. Any Ideas?
First, create a Calendar table. One that has, at the very least, an id column and a calendar_date column, and fill it with dates covering every day of every year you can ever be interested in . (You'll find that you'll add flags for weekends, bankholidays and all sorts of other useful meta-data about dates.)
Then you can LEFT JOIN on to that table, after combining your two tables with a UNION.
SELECT
CALENDAR.calendar_date,
JOINT_LOG.ClientIP,
ISNULL(SUM(JOINT_LOG.BytesSent + JOINT_LOG.BytesRec), 0) AS TotalBytes
FROM
CALENDAR
LEFT JOIN
(
SELECT LogTime, ClientIP, BytesSent, BytesRec FROM FirewallLog
UNION ALL
SELECT LogTime, ClientIP, BytesSent, BytesRec FROM ProxyLog
)
AS JOINT_LOG
ON JOINT_LOG.LogTime >= CALENDAR.calendar_date
AND JOINT_LOG.LogTime < CALENDAR.calendar_date+1
WHERE
CALENDAR.calendar_date >= #start_date
AND CALENDAR.calendar_date < #cease_date
GROUP BY
CALENDAR.calendar_date,
JOINT_LOG.ClientIP
SQL Server is very good at optimising this type of UNION ALL query. Assuming that you have appropriate indexes.
If you don't have a calendar table, you can create one using a recursive CTE:
declare #startdate date = '2013-02-01';
declare #enddate date = '2013-03-01';
with dates as (
select #startdate as thedate
union all
select dateadd(day, 1, thedate)
from dates
where thedate < #enddate
)
select driver.thedate, driver.ClientIP,
coalesce(fwl.FWBytes, 0) + coalesce(pl.PLBytes, 0) as TotalBytes
from (select d.thedate, fwl.ClientIP
from dates d cross join
(select distinct ClientIP from FirewallLog) fwl
) driver left outer join
(select cast(fwl.logtime as date) as thedate,
SUM(fwl.BytesSent + fwl.BytesRec) as FWBytes
from FirewallLog fwl
group by cast(fwl.logtime as date)
) fwl
on driver.thedate = fwl.thedate and driver.clientIP = fwl.ClientIP left outer join
(select cast(pl.logtime as date) as thedate,
SUM(pl.BytesSent + pl.BytesRec) as PLBytes
from ProxyLog pl
group by cast(pl.logtime as date)
) pl
on driver.thedate = pl.thedate and driver.ClientIP = pl.ClientIP
This uses a driver table that generates all the combinations of IP and date, which it then uses for joining to the summarized table. This formulation assumes that the "FirewallLog" contains all the "ClientIp"s of interest.
This also breaks out the two values, in case you also want to include them (to see which is contributing more bytes to the total, for instance).
I would recommend creating a Dates Lookup table if that is an option. Create the table once and then you can use it as often as needed. If not, you'll need to look into creating a Recursive CTE to act as the Dates table (easy enough -- look on stackoverflow for examples).
Select d.date,
results.ClientIp
Sum(results.bytes)
From YourDateLookupTable d
Left Join (
Select ClientIp, logtime, BytesSent + BytesRec bytes From FirewallLog
Union All
Select ClientIp, logtime, BytesSent + BytesRec bytes From ProxyLog
) results On d.date = results.logtime
Group By d.date,
results.ClientIp
This assumes the logtime and date data types are the same. If logtime is a date time, you'll need to convert it to a date.
I have a database table with the following structure -
Week_End Sales
2009-11-01 43223.43
2009-11-08 4324.23
2009-11-15 64343.23
...
Week_End is a datetime column, and the date increments by 7 days with each new entry.
What I want is a SQL statement that will identify if there is a week missing in the sequence. So, if the table contained the following data -
Week_End Sales
2009-11-01 43223.43
2009-11-08 4324.23
2009-11-22 64343.73
...
The query would return 2009-11-15.
Is this possible? I am using SQL Server 2008, btw.
You've already accepted an answer so I guess you don't need this, but I was almost finished with it anyway and it has one advantage that the selected solution doesn't have: it doesn't require updating every year. Here it is:
SELECT T1.*
FROM Table1 T1
LEFT JOIN Table1 T2
ON T2.Week_End = DATEADD(week, 1, T1.Week_End)
WHERE T2.Week_End IS NULL
AND T1.Week_End <> (SELECT MAX(Week_End) FROM Table1)
It is based on Andemar's solution, but handles the changing year too, and doesn't require the existence of the Sales column.
Join the table on itself to search for consecutive rows:
select a.*
from YourTable a
left join YourTable b
on datepart(wk,b.Week_End) = datepart(wk,a.Week_End) + 1
-- No next week
where b.sales is null
-- Not the last week
and datepart(wk,a.Week_End) <> (
select datepart(wk,max(Week_End)) from YourTable
)
This should return any weeks without a next week.
Assuming your "week_end" dates are always going to be the Sundays of the week, you could try a CTE - a common table expression that lists out all the Sundays for 2009, and then do an outer join against your table.
All those rows missing from your table will have a NULL value for their "week_end" in the select:
;WITH Sundays2009 AS
(
SELECT CAST('20090104' AS DATETIME) AS Sunday
UNION ALL
SELECT
DATEADD(DAY, 7, cte.Sunday)
FROM
Sundays2009 cte
WHERE
DATEADD(DAY, 7, cte.Sunday) < '20100101'
)
SELECT
sun.Sunday 'Missing week end date'
FROM
Sundays2009 sun
LEFT OUTER JOIN
dbo.YourTable tbl ON sun.Sunday = tbl.week_end
WHERE
tbl.week_end IS NULL
I know this has already been answered, but can I suggest something really simple?
/* First make a list of weeks using a table of numbers (mine is dbo.nums(num), starting with 1) */
WITH AllWeeks AS (
SELECT DATEADD(week,num-1,w.FirstWeek) AS eachWeek
FROM
dbo.nums
JOIN
(SELECT MIN(week_end) AS FirstWeek, MAX(week_end) as LastWeek FROM yourTable) w
ON num <= DATEDIFF(week,FirstWeek,LastWeek)
)
/* Now just look for ones that don't exist in your table */
SELECT w.eachWeek AS MissingWeek
FROM AllWeeks w
WHERE NOT EXISTS (SELECT * FROM yourTable t WHERE t.week_end = w.eachWeek)
;
If you know the range you want to look over, you don't need to use the MIN/MAX subquery in the CTE.