Related
I have a table, with types int, datetime, datetime:
id start date end date
-- ---------- ----------
1 2019-04-02 2020-09-17
2 2019-08-10 2020-08-10
Here is create/insert:
CREATE TABLE dbo.something
(
id int,
[start date] datetime,
[end date] datetime
);
INSERT dbo.something(id,[start date],[end date])
VALUES(1,'20190402','20200917'),(2,'20190810','20200810');
What is a SQL query that can produce these results:
id Year Quarter
-- ---- ----------
1 2019 2
1 2019 3
1 2019 4
1 2020 1
1 2020 2
1 2020 3
2 2019 3
2 2019 4
2 2020 1
2 2020 2
2 2020 3
Just use a recursive CTE. This version switches to counting quarters from year 0:
with cte as (
select id,
year(start_date) * 4 + datepart(quarter, start_date) - 1 as yyyyq,
year(end_date) * 4 + datepart(quarter, end_date) - 1 as end_yyyyq
from t
union all
select id, yyyyq + 1, end_yyyyq
from cte
where yyyyq < end_yyyyq
)
select id, yyyyq / 4 as year, (yyyyq % 4) + 1 as quarter
from cte;
Here is a db<>fiddle.
If you cannot make another reference table/etc, you can use DATEDIFF (and DATEPART) using quarters, and then some simple date arithmetic.
The logic below is simply to find, for each startdate, the first quarter and then the number of additional quarters to get to the maximum. Then do a SELECT where the additional quarters are added to the startdate, to get each quarter.
The hardest part of the query to understand imo is the WITH numberlist section - all this does is generate a series of integers between 0 and the maximum number of quarters difference. If you already have a numbers table, you can use that instead.
Key code part is below, and here's a full DB_Fiddle with some additional test data.
CREATE TABLE #yourtable (id int, startdate date, enddate date)
INSERT INTO #yourtable (id, startdate, enddate) VALUES
(1, '2019-04-02', '2020-09-17'),
(2, '2019-08-10', '2020-08-20')
; WITH number_list AS
-- list of ints from 0 to maximum number of quarters
(SELECT n
FROM (SELECT ones.n + 10*tens.n AS n
FROM (VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) ones(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tens(n)
) AS a
WHERE n <= (SELECT MAX(DATEDIFF(quarter,startdate,enddate)) FROM #yourtable)
)
SELECT id,
YEAR(DATEADD(quarter, number_list.n, startdate)) AS [Year],
DATEPART(quarter, DATEADD(quarter, number_list.n, startdate)) AS [Quarter]
FROM (SELECT id, startdate, DATEDIFF(quarter,startdate,enddate) AS num_additional_quarters FROM #yourtable) yt
CROSS JOIN number_list
WHERE number_list.n <= yt.num_additional_quarters
DROP TABLE #yourtable
First create a date dimension table which contains date, corresponding quarter and year. Then use below query to get the result. Tweak column and table name according to your schema.
with q_date as
(
select 1 as id, '2019-04-02' :: date as start_date, '2020-09-17' :: date as end_date
UNION ALL
select 2 as id, '2019-08-10' :: date as start_date, '2020-08-10' :: date as end_date
)
select qd.id, dd.calendar_year, dd.calendar_quarter_number
from dim_date dd, q_date qd
where dd.date_dmk between qd.start_date and qd.end_date
group by qd.id, dd.calendar_year, dd.calendar_quarter_number
order by qd.id, dd.calendar_year, dd.calendar_quarter_number;
I have a table
DATE Val
01-01-2020 1
01-02-2020 3
01-05-2020 2
01-07-2020 8
01-13-2020 3
...
I want to summarize these values by the following Sunday. For example, in the above example:
1-05-2020, 1-12-2020, and 1-19-2020 are Sundays, so I want to summarize these by those dates.
The final result should be something like
DATE SUM
1-05-2020 6 //(01-01-2020 + 01-02-2020 + 01-05-2020)
1-12-2020 8
1-19-2020 3
I wasn't certain if the best place to start would be to create a temp calendar table, and then try to join backwards based on that? Or if there was an easier way involving DATEDIFF. Any help would be appreciated! Thanks!
Here's a solution that uses DATEADD & DATEPART to calculate the closest Sunday.
With a correction for a different setting of ##datefirst.
(Since the datepart weekday values are different depending on the DATEFIRST setting)
Sample data:
create table #TestTable
(
Id int identity(1,1) primary key,
[Date] date,
Val int
);
insert into #TestTable
([Date], Val)
VALUES
('2020-01-01', 1)
, ('2020-01-02', 3)
, ('2020-01-05', 2)
, ('2020-01-07', 8)
, ('2020-01-13', 3)
;
Query:
WITH CTE_DATA AS
(
SELECT [Date], Val
, DATEADD(day,
((7-(##datefirst+datepart(weekday, [Date])-1)%7)%7),
[Date]) AS Sunday
FROM #TestTable
)
SELECT
Sunday AS [Date],
SUM(Val) AS [Sum]
FROM CTE_DATA
GROUP BY Sunday
ORDER BY Sunday;
Date | Sum
:--------- | --:
2020-01-05 | 6
2020-01-12 | 8
2020-01-19 | 3
db<>fiddle here
Extra:
Apparently the trick of adding the difference of weeks from day 0 to day 6 also works independently from the DATEFIRST setting.
So this query will return the same result for the sample data.
WITH CTE_DATA AS
(
SELECT [Date], Val
, CAST(DATEADD(week, DATEDIFF(week, 0, DATEADD(day, -1, [Date])), 6) AS DATE) AS Sunday
FROM #TestTable
)
SELECT
Sunday AS [Date],
SUM(Val) AS [Sum]
FROM CTE_DATA
GROUP BY Sunday
ORDER BY Sunday;
The subtraction of 1 day makes sure that if the date is already a Sunday that it isn't calculated to the next Sunday.
Here is a way to do it:
nb:1-13-2020 wont show cuz its not a sunday
with cte as
(
select cast('01-01-2020'as Date) as Date, 1 as Val
union select '01-02-2020' , 3
union select '01-05-2020' , 2
union select '01-07-2020' , 8
)
select Date, max(dateadd(dd,number,Date)), sum(distinct Val) as SUM
from master..spt_values a inner join cte on Date <= dateadd(dd,number,Date)
where type = 'p'
and year(dateadd(dd,number,Date))=year(Date)
and DATEPART(dw,dateadd(dd,number,Date)) = 7
group by Date
Output:
Date (No column name) SUM
2020-01-01 2020-12-26 1
2020-01-02 2020-12-26 3
2020-01-05 2020-12-26 2
2020-01-07 2020-12-26 8
Here is a simple solution. Putting your values into a temporary table and viewing the results on that table:
DECLARE #dates TABLE
(
mDATE DATE,
Val INT,
Sunday DATE
)
INSERT INTO #dates (mDATE,Val) VALUES
('01-01-2020',1),('01-02-2020',3),('01-05-2020',2),('01-07-2020',8),('01-13-2020',3)
UPDATE #dates
SET Sunday = dateadd(week, datediff(week, 0, mDATE), 6)
SELECT Sunday,SUM(Val) AS Val FROM #dates
GROUP BY Sunday
OUTPUT:
Sunday Val
2020-01-05 4
2020-01-12 10
2020-01-19 3
I have table of products and their sales quantity in months.
Product Month Qty
A 2018-01-01 5
A 2018-02-01 3
A 2018-05-01 5
B 2018-08-01 10
B 2018-10-01 12
...
I'd like to first fill in the data gap between each product's min and max dates like below:
Product Month Qty
A 2018-01-01 5
A 2018-02-01 3
A 2018-03-01 0
A 2018-04-01 0
A 2018-05-01 5
B 2018-08-01 10
B 2018-09-01 0
B 2018-10-01 12
...
Then I would need to perform an accumulation of each product's sales quantity by month.
Product Month total_Qty
A 2018-01-01 5
A 2018-02-01 8
A 2018-03-01 8
A 2018-04-01 8
A 2018-05-01 13
B 2018-08-01 10
B 2018-09-01 10
B 2018-10-01 22
...
I fumbled over the "cross join" clause, however it seems to generate some unexpected results for me. Could someone help to give a hint how I can achieve this in SQL?
Thanks a lot in advance.
I think a recursive CTE is a simple way to do this. The code is just:
with cte as (
select product, min(mon) as mon, max(mon) as end_mon
from t
group by product
union all
select product, dateadd(month, 1, mon), end_mon
from cte
where mon < end_mon
)
select cte.product, cte.mon, coalesce(qty, 0) as qty
from cte left join
t
on t.product = cte.product and t.mon = cte.mon;
Here is a db<>fiddle.
Hi i think this example can help you and perform what you excepted :
CREATE TABLE #MyTable
(Product varchar(10),
ProductMonth DATETIME,
Qty int
);
GO
CREATE TABLE #MyTableTempDate
(
FullMonth DATETIME
);
GO
INSERT INTO #MyTable
SELECT 'A', '2019-01-01', 214
UNION
SELECT 'A', '2019-02-01', 4
UNION
SELECT 'A', '2019-03-01', 50
UNION
SELECT 'B', '2019-01-01', 214
UNION
SELECT 'B', '2019-02-01', 10
UNION
SELECT 'C', '2019-04-01', 150
INSERT INTO #MyTableTempDate
SELECT '2019-01-01'
UNION
SELECT '2019-02-01'
UNION
SELECT '2019-03-01'
UNION
SELECT '2019-04-01'
UNION
SELECT '2019-05-01'
UNION
SELECT '2019-06-01'
UNION
SELECT '2019-07-01';
------------- FOR NEWER SQL SERVER VERSION > 2005
WITH MyCTE AS
(
SELECT T.Product, T.ProductMonth AS 'MMonth', T.Qty
FROM #MyTable T
UNION
SELECT T.Product, TD.FullMonth AS 'MMonth', 0 AS 'Qty'
FROM #MyTable T, #MyTableTempDate TD
WHERE NOT EXISTS (SELECT 1 FROM #MyTable TT WHERE TT.Product = T.Product AND TD.FullMonth = TT.ProductMonth)
)
-- SELECT * FROM MyCTE;
SELECT Product, MMonth, Qty, SUM( Qty) OVER(PARTITION BY Product ORDER BY Product
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) as 'TotalQty'
FROM MyCTE
ORDER BY Product, MMonth ASC;
DROP TABLE #MyTable
DROP TABLE #MyTableTempDate
I have other way to perform this in lower SQL Server Version (like 2005 and lower)
It's a SELECT on SELECT if it's your case let me know and i provide some other example.
You can create the months with a recursive CTE
DECLARE #MyTable TABLE
(
ProductID CHAR(1),
Date DATE,
Amount INT
)
INSERT INTO #MyTable
VALUES
('A','2018-01-01', 5),
('A','2018-02-01', 3),
('A','2018-05-01', 5),
('B','2018-08-01', 10),
('B','2018-10-01', 12)
DECLARE #StartDate DATE
DECLARE #EndDate DATE
SELECT #StartDate = MIN(Date), #EndDate = MAX(Date) FROM #MyTable
;WITH dates AS (
SELECT #StartDate AS Date
UNION ALL
SELECT DATEADD(Month, 1, Date)
FROM dates
WHERE Date < #EndDate
)
SELECT A.ProductID, d.Date, COALESCE(Amount,0) AS Amount, COALESCE(SUM(Amount) OVER(PARTITION BY A.ProductID ORDER BY A.ProductID, d.Date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW),0) AS Total
FROM
(
SELECT ProductID, MIN(date) as DateStart, MAX(date) as DateEnd
FROM #MyTable
GROUP BY ProductID -- As I read in your comments that you need different min and max dates per product
) A
JOIN dates d ON d.Date >= A.DateStart AND d.Date <= A.DateEnd
LEFT JOIN #MyTable T ON A.ProductID = T.ProductID AND T.Date = d.Date
ORDER BY A.ProductID, d.Date
Try this below
IF OBJECT_ID('tempdb..#Temp') IS NOT NULL
DROP TABLE #Temp
;WITH CTE(Product,[Month],Qty)
AS
(
SELECT 'A','2018-01-01', 5 UNION ALL
SELECT 'A','2018-02-01', 3 UNION ALL
SELECT 'A','2018-05-01', 5 UNION ALL
SELECT 'B','2018-08-01', 10 UNION ALL
SELECT 'D','2018-10-01', 12
)
SELECT ct.Product,[MonthDays],ct.Qty
INTO #Temp
FROM
(
SELECT c.Product,[Month],
ISNULL(Qty,0) AS Qty
FROM CTE c
)ct
RIGHT JOIN
(
SELECT -- This code is to get month data
CONVERT(VARCHAR(10),'2018-'+ RIGHT('00'+CAST(MONTH(DATEADD(MM, s.number, CONVERT(DATETIME, 0)))AS VARCHAR),2) +'-01',120) AS [MonthDays]
FROM master.dbo.spt_values s
WHERE [type] = 'P' AND s.number BETWEEN 0 AND 11
)DT
ON dt.[MonthDays] = ct.[Month]
SELECT
MAX(Product)OVER(ORDER BY [MonthDays])AS Product,
[MonthDays],
ISNULL(Qty,0) Qty,
SUM(ISNULL(Qty,0))OVER(ORDER BY [MonthDays]) As SumQty
FROM #Temp
Result
Product MonthDays Qty SumQty
------------------------------
A 2018-01-01 5 5
A 2018-02-01 3 8
A 2018-03-01 0 8
A 2018-04-01 0 8
A 2018-05-01 5 13
A 2018-06-01 0 13
A 2018-07-01 0 13
B 2018-08-01 10 23
B 2018-09-01 0 23
D 2018-10-01 12 35
D 2018-11-01 0 35
D 2018-12-01 0 35
First of all, i would divide month and year to get easier with statistics.
I will give you an example query, not based on your table but still helpful.
--here i create the table that will be used as calendar
Create Table MA_MonthYears (
Month int not null ,
year int not null
PRIMARY KEY ( month, year) )
--/////////////////
-- here i'm creating a procedure to fill the ma_monthyears table
declare #month as int
declare #year as int
set #month = 1
set #year = 2015
while ( #year != 2099 )
begin
insert into MA_MonthYears(Month, year)
select #month, #year
if #month < 12
set #month=#month+1
else
set #month=1
if #month = 1
set #year = #year + 1
end
--/////////////////
--here you are the possible result you are looking for
select SUM(Ma_saledocdetail.taxableamount) as Sold, MA_MonthYears.month , MA_MonthYears.year , item
from MA_MonthYears left outer join MA_SaleDocDetail on year(MA_SaleDocDetail.DocumentDate) = MA_MonthYears.year
and Month(ma_saledocdetail.documentdate) = MA_MonthYears.Month
group by MA_SaleDocDetail.Item, MA_MonthYears.year , MA_MonthYears.month
order by MA_MonthYears.year , MA_MonthYears.month
I have a projects table with a total_hours column and a startdate, enddate column.
If a project has a date range of 5 weeks, I need a query that returns 5 rows with the incremented work week number in a calculated field for all projects.
Here is my table data with a query showing the range in work week format.
drop table #temp
CREATE TABLE #Temp
(ProjectID int, Total_Hours int, StartDate datetime, EndDate datetime)
;
INSERT INTO #Temp
(ProjectID, Total_Hours, StartDate, EndDate)
VALUES
(645, 555, '2016-01-01 00:00:00', '2016-02-01 00:00:00'),
(700, 234, '2015-01-14 00:00:00', '2016-02-01 00:00:00')
Select datepart(week,startdate),datepart(week,Enddate) from #Temp
I need a query that will return the following values
ProjectID WW
645 1
645 2
645 3
645 4
645 5
645 6
700 3
700 4
700 5
700 6
I feel I should use recursion but don't know how.
You could do it with recursion but a numbers table is generally more efficient:
with n as (
select row_number() over (order by (select null)) - 1 as n
from master.spt_values
)
select t.projectid, dateadd(week, n.n, t.startdate) as ww
from #Temp t join
n
on dateadd(week, n.n, t.startdate) <= t.enddate;
If you prefer a recursive query, use
with t as (
select projectid,datepart(week,startdate) sw,datepart(week,enddate) ew from #Temp
union all
select projectid,sw+1,ew from t where sw < ew
)
select projectid, sw
from t
order by 1,2
Sample Demo
Assume that I have a table (MyTable) as follows:
item_id | date
----------------
1 | 2016-06-08
1 | 2016-06-07
1 | 2016-06-05
1 | 2016-06-04
1 | 2016-05-31
...
2 | 2016-06-08
2 | 2016-06-06
2 | 2016-06-04
2 | 2016-05-31
...
3 | 2016-05-31
...
I would like to build a weekly summary table that reports on a running 7 day window. The window would basically say "How many unique item_ids were reported in the preceding 7 days"?
So, in this case, the output table would look something like:
date | weekly_ids
----------------------
2016-05-31| 3 # All 3 were present on the 31st
2016-06-01| 3 # All 3 were present on the 31st which is < 7 days before the 1st
2016-06-02| 3 # Same
2016-06-03| 3 # Same
2016-06-04| 3 # Same
2016-06-05| 3 # Same
2016-06-06| 3 # Same
2016-06-07| 3 # Same
2016-06-08| 2 # item 3 was not present for the entire last week so it does not add to the count.
I've tried:
SELECT
item_id,
date,
MAX(present) OVER (
PARTITION BY item_id
ORDER BY date
ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS is_present
FROM (
# Inner query
SELECT
item_id,
date,
1 AS present,
FROM MyTable
)
GROUP BY date
ORDER BY date DESC
This feels like it is going in the right direction. But as it is, the window runs over the wrong time-frame when dates aren't present (too many dates) and it also doesn't output records for dates when the item_id wasn't present (even if it was present on the previous date). Is there a simple resolution to this problem?
If it's helpful and necessary
I can hard-code an oldest date
I also can get a table of all of the item_ids in existence.
This query will only be run on BigQuery, so BQ specific functions/syntax are fair game and SQL functions/syntax that doesn't run on BigQuery unfortunately doesn't help me ...
I have created a temp table to hold dates, however, you probably would benefit from adding a permanent table to your database for these joins. Trust me it will cause less headaches.
DECLARE #my_table TABLE
(
item_id int,
date DATETIME
)
INSERT #my_table SELECT 1,'2016-06-08'
INSERT #my_table SELECT 1,'2016-06-07'
INSERT #my_table SELECT 1,'2016-06-05'
INSERT #my_table SELECT 1,'2016-06-04'
INSERT #my_table SELECT 1,'2016-05-31'
INSERT #my_table SELECT 2,'2016-06-08'
INSERT #my_table SELECT 2,'2016-06-06'
INSERT #my_table SELECT 2,'2016-06-04'
INSERT #my_table SELECT 2,'2016-05-31'
INSERT #my_table SELECT 3,'2016-05-31'
DECLARE #TrailingDays INT=7
DECLARE #LowDate DATETIME='01/01/2016'
DECLARE #HighDate DATETIME='12/31/2016'
DECLARE #Calendar TABLE(CalendarDate DATETIME)
DECLARE #LoopDate DATETIME=#LowDate
WHILE(#LoopDate<=#HighDate) BEGIN
INSERT #Calendar SELECT #LoopDate
SET #LoopDate=DATEADD(DAY,1,#LoopDate)
END
SELECT
date=HighDate,
weekly_ids=COUNT(DISTINCT item_id)
FROM
(
SELECT
HighDate=C.CalendarDate,
LowDate=LAG(C.CalendarDate, #TrailingDays,0) OVER (ORDER BY C.CalendarDate)
FROM
#Calendar C
WHERE
CalendarDate BETWEEN #LowDate AND #HighDate
)AS X
LEFT OUTER JOIN #my_table MT ON MT.date BETWEEN LowDate AND HighDate
GROUP BY
LowDate,
HighDate
Try below example. It can give you direction to explore
Purely GBQ - Legacy SQL
SELECT date, items FROM (
SELECT
date, COUNT(DISTINCT item_id) OVER(ORDER BY sec RANGE BETWEEN 60*60*24*2 PRECEDING AND CURRENT ROW) AS items
FROM (
SELECT
item_id, date, timestamp_to_sec(timestamp(date)) AS sec
FROM (
SELECT calendar.day AS date, MyTable.item_id AS item_id
FROM (
SELECT DATE(DATE_ADD(TIMESTAMP('2016-05-28'), pos - 1, "DAY")) AS day
FROM (
SELECT ROW_NUMBER() OVER() AS pos, *
FROM (FLATTEN((
SELECT SPLIT(RPAD('', 1 + DATEDIFF(TIMESTAMP(CURRENT_DATE()), TIMESTAMP('2016-05-28')), '.'),'') AS h
FROM (SELECT NULL)),h
)))
) AS calendar
LEFT JOIN (
SELECT date, item_id
FROM
(SELECT 1 AS item_id, '2016-06-08' AS date),
(SELECT 1 AS item_id, '2016-06-07' AS date),
(SELECT 1 AS item_id, '2016-06-05' AS date),
(SELECT 1 AS item_id, '2016-06-04' AS date),
(SELECT 1 AS item_id, '2016-05-28' AS date),
(SELECT 2 AS item_id, '2016-06-08' AS date),
(SELECT 2 AS item_id, '2016-06-06' AS date),
(SELECT 2 AS item_id, '2016-06-04' AS date),
(SELECT 2 AS item_id, '2016-05-31' AS date),
(SELECT 3 AS item_id, '2016-05-31' AS date),
(SELECT 3 AS item_id, '2016-06-05' AS date)
) AS MyTable
ON calendar.day = MyTable.date
)
)
)
GROUP BY date, items
ORDER BY date
Please note
oldest date - 2016-05-28 - is hardcoded in calendar subquery
window size is controled in RANGE BETWEEN 60*60*24*2 PRECEDING AND CURRENT ROW; if you need 7 days - the expression should be 60*60*24*6
have in mind specifics of COUNT(DISTINCT) in BigQuery Legacy SQL