It is a bit difficult to explain but basically I need to create a temporary table (date datetime, #customers int) where #customers is the number of weekly customers pulled from another table. Here's my code.
declare #date datetime
declare #temptable table (date datetime not null,#customers int)
set #date='2018-02-13'
while #date<getdate()
begin
insert into #temptable values
(#date,
(select count(*) from in_ft_conversion
where u4='cfa' and sales_date between #date and #date-7))
set #date=#date+7
end
The result is a table with all the correct date entries but 0 in the customer column... Does anybody know what I'm doing wrong? Thanks!
Your date range is wrong , swap the date values in the BETWEEN so you have BETWEEN <earlier date> AND <later date>
where u4='cfa' and sales_date between #date-7 and #date))
Why would you use a while loop for this? I think you want something like this:
insert into #temptable (date, num_customers)
select dateadd(day, '2018-02-08', weekno * 7)
count(*)
from in_ft_conversion cross apply
(values (datediff(day, '2018-02-08', sales_date) / 7
) v(weekno)
where u4 = 'cfa' and sales_date >= '2018-02-08'
group by v.weekno;
No loop is necessary.
Your problem is specifically the between comparison:
sales_date between #date and #date-7
The dates are backwards -- the lower bound needs to go first.
But, I also doubt that you want to count weeks with 8 days and have one day overlap on each week. I think the above logic does what you want, but you can adjust the date arithmetic to get the exact dates you want.
Using SQL Server 2016.
I have a stored procedure that produces a list of options against a range of dates. Carriage options against days for clarity but unimportant to the specifics here.
The first step in the stored procedure generates a list of dates to store additional data against, and generating this list is taking substantially longer than the balance of the code. While this process is individual short, the number of calls means that this one piece of code is putting the system under more load than anything else.
With that in mind I have been testing efficiency of several options.
Iterative common table expression:
CREATE FUNCTION [dbo].[udf_DateRange_CTE] (#StartDate DATE,#EndDate DATE)
RETURNS #Return TABLE (Date DATE NOT NULL)
AS
BEGIN
WITH dates(date)
AS (SELECT #StartDate [Date]
UNION ALL
SELECT DATEADD(dd, 1, [Date])
FROM dates
WHERE [Date] < #EndDate
)
INSERT INTO #Return
SELECT date
FROM dates
OPTION (MAXRECURSION 0)
RETURN
END
A while loop:
CREATE FUNCTION [dbo].[udf_DateRange_While] (#StartDate DATE,#EndDate DATE)
RETURNS #Retun TABLE (Date DATE NOT NULL,PRIMARY KEY (Date))
AS
BEGIN
WHILE #StartDate <= #EndDate
BEGIN
INSERT INTO #Retun
VALUES (#StartDate)
SET #StartDate = DATEADD(DAY,1,#StartDate)
END
RETURN
END
A lookup from a pre-populated table of dates:
CREATE FUNCTION [dbo].[udf_DateRange_query] (#StartDate DATE,#EndDate DATE)
RETURNS #Return TABLE (Date DATE NOT NULL)
AS
BEGIN
INSERT INTO #Return
SELECT Date
FROM DateLookup
WHERE Date >= #StartDate
AND Date <= #EndDate
RETURN
END
In terms of efficiency I have test generating a years worth of dates, 1000 times and had the following results:
CTE: 10.0 Seconds
While: 7.7 Seconds
Query: 2.6 Seconds
From this the query is definitely the faster option but does require a permanent table of dates that needs to be created and maintained. This means that the query is no loner "self-contained" and it would be possible to request a date outside of the given date range.
Does anyone know of any more efficient ways of generating dates for a range, or any optimisation I can apply to the above?
Many thanks.
You can try like following. This should be fast compared CTE or WHILE loop.
DECLARE #StartDate DATETIME = Getdate() - 1000
DECLARE #EndTime DATETIME = Getdate()
SELECT *
FROM (SELECT #StartDate + RN AS DATE
FROM (SELECT ROW_NUMBER()
OVER (
ORDER BY (SELECT NULL)) RN
FROM master..[spt_values]) T) T1
WHERE T1.DATE <= #EndTime
ORDER BY DATE
Note: This will work for day difference <= 2537 days
If you want to support more range, you can use CROSS JOIN on master..[spt_values] to generate range between 0 - 6436369 days like following.
DECLARE #StartDate DATETIME = Getdate() - 10000
DECLARE #EndTime DATETIME = Getdate()
SELECT #StartDate + RN AS DATE FROM
(
SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) RN
FROM master..[spt_values] T1
CROSS JOIN master..[spt_values] T2
) T
WHERE RN <= DATEDIFF(DAY,#StartDate,#EndTime)
I need to make a temporary table that holds of range of dates, as well as a couple of columns that hold placeholder values (0) for future use. The dates I need are the first day of each month between $startDate and $endDate where these variables can be several years apart.
My original sql statement looked like this:
select dbo.FirstOfMonth(InsertDate) as Month, 0 as Trials, 0 as Sales
into #dates
from customer
group by dbo.FirstOfMonth(InsertDate)
"FirstOfMonth" is a user-defined function I made that pretty much does what it says, returning the first day of the month for the provided date with the time at exactly midnight.
This produced almost exactly what I needed until I discovered there were occasionally gaps in my dates where I had a few months were there were no records insert dates. Since my result must still have the missing months I need a different approach.
I have added the following declarations to the stored procedure anticipating their need for the range of the dates I need ...
declare $startDate set $startDate = select min(InsertDate) from customer
declare $endDate set $endDate = select max(InsertDate) from customer
... but I have no idea what to do from here.
I know this question is similar to this question but, quite frankly, that answer is over my head (I don't often work with SQL and when I do it tends to be on older versions of SQL Server) and there are a few minor differences that are throwing me off.
I needed something similar, but all DAYS instead of all MONTHS.
Using the code from MatBailie as a starting point, here's the SQL for creating a permanent table with all dates from 2000-01-01 to 2099-12-31:
CREATE TABLE _Dates (
d DATE,
PRIMARY KEY (d)
)
DECLARE #dIncr DATE = '2000-01-01'
DECLARE #dEnd DATE = '2100-01-01'
WHILE ( #dIncr < #dEnd )
BEGIN
INSERT INTO _Dates (d) VALUES( #dIncr )
SELECT #dIncr = DATEADD(DAY, 1, #dIncr )
END
This will quickly populate a table with 170 years worth of dates.
CREATE TABLE CalendarMonths (
date DATETIME,
PRIMARY KEY (date)
)
DECLARE
#basedate DATETIME,
#offset INT
SELECT
#basedate = '01 Jan 2000',
#offset = 1
WHILE (#offset < 2048)
BEGIN
INSERT INTO CalendarMonths SELECT DATEADD(MONTH, #offset, date) FROM CalendarMonths
SELECT #offset = #offset + #offset
END
You can then use it by LEFT joining on to that table, for the range of dates you require.
I would probably use a Calendar table. Create a permanent table in your database and fill it with all of the dates. Even if you covered a 100 year range, the table would still only have ~36,525 rows in it.
CREATE TABLE dbo.Calendar (
calendar_date DATETIME NOT NULL,
is_weekend BIT NOT NULL,
is_holiday BIT NOT NULL,
CONSTRAINT PK_Calendar PRIMARY KEY CLUSTERED (calendar_date)
)
Once the table is created, just populate it once in a loop, so that it's always out there and available to you.
Your query then could be something like this:
SELECT
C.calendar_date,
0 AS trials,
0 AS sales
FROM
dbo.Calendar C
WHERE
C.calendar_date BETWEEN #start_date AND #end_date AND
DAY(C.calendar_date) = 1
You can join in the Customers table however you need to, outer joining on FirstOfMonth(InsertDate) = C.calendar_date if that's what you want.
You can also include a column for day_of_month if you want which would avoid the overhead of calling the DAY() function, but that's fairly trivial, so it probably doesn't matter one way or another.
This of course will not work in SQL-Server 2000 but in a modern database where you don't want to create a permanent table. You can use a table variable instead creating a table so you can left join the data try this. Change the DAY to HOUR etc to change the increment type.
declare #CalendarMonths table (date DATETIME, PRIMARY KEY (date)
)
DECLARE
#basedate DATETIME,
#offset INT
SELECT
#basedate = '01 Jan 2014',
#offset = 1
INSERT INTO #CalendarMonths SELECT #basedate
WHILE ( DATEADD(DAY, #offset, #basedate) < CURRENT_TIMESTAMP)
BEGIN
INSERT INTO #CalendarMonths SELECT DATEADD(HOUR, #offset, date) FROM #CalendarMonths where DATEADD(DAY, #offset, date) < CURRENT_TIMESTAMP
SELECT #offset = #offset + #offset
END
A starting point of a useful kludge to specify a range or specific list of dates:
SELECT *
FROM
(SELECT CONVERT(DateTime,'2017-1-1')+number AS [Date]
FROM master..spt_values WHERE type='P' AND number<370) AS DatesList
WHERE DatesList.Date IN ('2017-1-1','2017-4-14','2017-4-17','2017-12-25','2017-12-26')
You can get 0 to 2047 out of master..spt_values WHERE type='P', so that's five and a half year's worth of dates if you need it!
Tested below and it works, though it's a bit convoluted.
I assigned arbitrary values to the dates for the test.
DECLARE #SD smalldatetime,
#ED smalldatetime,
#FD smalldatetime,
#LD smalldatetime,
#Mct int,
#currct int = 0
SET #SD = '1/15/2011'
SET #ED = '2/02/2012'
SET #FD = (DATEADD(dd, -1*(Datepart(dd, #SD)-1), #sd))
SET #LD = (DATEADD(dd, -1*(Datepart(dd, #ED)-1), #ED))
SET #Mct = DATEDIFF(mm, #FD, #LD)
CREATE TABLE #MyTempTable (FoM smalldatetime, Trials int, Sales money)
WHILE #currct <= #Mct
BEGIN
INSERT INTO #MyTempTable (FoM, Trials, Sales)
VALUES
(DATEADD(MM, #currct, #FD), 0, 0)
SET #currct = #currct + 1
END
SELECT * FROM #MyTempTable
DROP TABLE #MyTempTable
For SQL Server 2000, this stackoverflow post looks promising for a way to temporarily generate dates calculated off of a start and end date. It's not exactly the same but quite similar. This post has a very in-depth answer on truncating dates, if needed.
In case anyone stumbles on this question and is working in PostgreSQL instead of SQL Server 2000, here is how you might do it there...
PostgreSQL has a nifty series generating function. For your example, you could use this series of all days instead of generating an entire calendar table, and then do groupings and matchups from there.
SELECT current_date + s.a AS dates FROM generate_series(0,14,7) AS s(a);
dates
------------
2004-02-05
2004-02-12
2004-02-19
(3 rows)
SELECT * FROM generate_series('2008-03-01 00:00'::timestamp,
'2008-03-04 12:00', '10 hours');
generate_series
---------------------
2008-03-01 00:00:00
2008-03-01 10:00:00
2008-03-01 20:00:00
2008-03-02 06:00:00
2008-03-02 16:00:00
2008-03-03 02:00:00
2008-03-03 12:00:00
2008-03-03 22:00:00
2008-03-04 08:00:00
(9 rows)
I would also look into date_trunc from PostgreSQL using 'month' for the truncator field to maybe refactor your original query to easily match with a date_trunc version of the calendar series.
select top (datediff(D,#start,#end)) dateadd(D,id-1,#start)
from BIG_TABLE_WITH_NO_JUMPS_IN_ID
declare #start datetime
set #start = '2016-09-01'
declare #end datetime
set #end = '2016-09-30'
create table #Date
(
table_id int identity(1,1) NOT NULL,
counterDate datetime NULL
);
insert into #Date select top (datediff(D,#start,#end)) NULL from SOME_TABLE
update #Date set counterDate = dateadd(D,table_id - 1, #start)
The code above should populate the table with all the dates between the start and end. You would then just join on this table to get all of the dates needed. If you only needed a certain day of each month, you could dateadd a month instead.
SELECT P.Id
, DATEADD ( DD, -P.Id, P.Date ) AS Date
FROM (SELECT TOP 1000 ROW_NUMBER () OVER (ORDER BY (SELECT NULL)) AS Id, CAST(GETDATE () AS DATE) AS Date FROM master.dbo.spt_values) AS P
This query returns a table calendar for the last 1000 days or so. It can be put in a temporary or other table.
Create a table variable containing a date for each month in a year:
declare #months table (reportMonth date, PRIMARY KEY (reportMonth));
declare #start date = '2018', #month int = 0; -- base 0 month
while (#month < 12)
begin
insert into #months select dateAdd(month, #month, #start);
select #month = #month + 1;
end
--verify
select * from #months;
This is by far the quickest method I have found (much quicker than inserting rows 1 by 1 in a WHILE loop):
DECLARE #startDate DATE = '1900-01-01'
DECLARE #endDate DATE = '2050-01-01'
SELECT DATEADD(DAY, sequenceNumber, #startDate) AS TheDate
INTO #TheDates
FROM (
SELECT ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n + 10000*tenthousands.n AS sequenceNumber
FROM
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) ones(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tens(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) hundreds(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) thousands(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tenthousands(n)
WHERE ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n + 10000*tenthousands.n <= DATEDIFF(day, #startDate, #endDate)
) theNumbers
SELECT *
FROM #TheDates
ORDER BY TheDate
The recursive answer:
DECLARE #startDate AS date = '20220315';
DECLARE #endDate AS date = '20230316'; -- inclusive
WITH cte_minutes(dt)
AS (
SELECT
DATEFROMPARTS(YEAR(#startDate), MONTH(#startDate), 1)
UNION ALL
SELECT
DATEADD(month, 1, dt)
FROM
cte_minutes
WHERE DATEADD(month, 1, dt) < #endDate
)
SELECT
dt
into #dates
FROM
cte_minutes
WHERE
dt >= #startDate
AND
dt <= #endDate
OPTION (MAXRECURSION 2000);
DROP TABLE dbo.#dates
We're looking to populate a number of tables for testing. The data can all be the same except for a timestamp column. Other columns in the row contain complicated data, like objects in xml format and other messy stuff, so we don't want to have to remake it. What would be an easy way to expand a single entry to thousands, by incrementing a timestamp by a constant interval, through a sql query.
Our current idea (in a C-ish pseudocode) is to:
Get the latest (likely only) row and store it in a variable "thisRow"
While(thisRow->time < endTime)
{
Increment thisRow->time by a constant variable
Insert thisRow
}
We are using postgres
I like to keep a Numbers table in my databases for certain queries. If you had one then you could use the single statement below. If you don't have one, it's easy enough to generate one as a temporary table or permanent table.
INSERT INTO Test_Table
(
col1,
col2,
...,
my_timestamp
)
SELECT
ST.col1,
ST.col2,
...,
DATEADD(mi, N.number, ST.my_timestamp)
FROM
Source_Table ST
INNER JOIN Numbers N ON
N.number BETWEEN 1 AND 1000 -- Change this to what you want
This is for SQL Server. The DATEADD function may be different in your RDBMS. You can also add in a random function if you don't want the intervals to be exactly one minute and you could of course change the intervals to be hours, days, or whatever. You could also use a simple equation if you wanted say two hours per interval.
Using SQL SERVER 2005+ you can make use of CTE statements
Have a look at this
DECLARE #StartTime DATETIME,
#EndTime DATETIME
SELECT #StartTime = '01 Jan 2009',
#EndTime = '31 Jan 2009'
DECLARE #Table TABLE(
DateVal DATETIME,
Col1 INT
)
INSERT INTO #Table SELECT #StartTime, 1
SELECT *
FROM #Table
;WITH CTE AS (
SELECT #StartTime StartTime
UNION ALL
SELECT DATEADD(dd, 1, StartTime) StartTime
FROM CTE
WHERE StartTime < #EndTime
)
SELECT CTE.*,
t.Col1
FROM CTE,
#Table t
If you are using a database that supports recursion, for example Microsoft SQL Server, you can use something like:
WITH T AS (
SELECT 1 AS r
UNION ALL
SELECT r + 1 AS r
FROM T
WHERE r < 100 -- Or however many rows you need
)
INSERT INTO TestTable (Col1, Col2,...)
SELECT s.Col1, s.Col2,...
FROM SourceTable s
CROSS JOIN T
I am using SQL Server 2005 and trying to write a query where I want to retrieve payments for a given month. I currently have:
select sum(p1.paymentamount) as subtotal,
CONVERT(char(10), p1.paymentdate, 103) as paymentdate
from tblpayment p1
where 1=1
and p1.paymentdate >= #fromdate
and p1.paymentdate <= #todate
group by p1.paymentdate
order by p1.paymentdate
Schema:
CREATE TABLE [dbo].[tblPayment]
(
[paymentid] [int] IDENTITY(1,1) NOT NULL,
[userid] [int] NULL ,
[paymentdate] [datetime] NOT NULL,
[paymentamount] [int] NULL,
[paymenttype] [varchar](50) NULL,
[paymentnotes] [varchar](200) NULL,
[paymentcurrency] [nchar](10) NULL
)
This query gives me what I want but it doesnt give me the dates where no payments were made. What I want is a query that gives me all days even if there were no payments made on that day and jut shows the subtotal as 0 for that day.
There is another catch. The currency of payments is different. So how can I have another column in the query that gives me eurototal and sterlingtotal based on #currency parameter passed in ? Assuming there is a column in the table for "paymentcurrency"
You have to work backwards. In order to get rows for dates that don't exist, you need to outer join them to rows that do have those dates. In order to outer join, you need to have a sequence to join to. Since you don't have a sequence, you need to create one.
To create that sequence, you have two options:
Create a static date sequence and store it in a permanent table (Larry's answer); or
Use an existing numeric sequence (such as spt_values) to create one on the fly.
Let's assume you want the flexibility of the second approach. Here's a common snippet I use for things like that:
SELECT DATEADD(DAY, v.number, #fromdate)
FROM master.dbo.spt_values v
WHERE v.type = 'P'
AND v.number <= DATEDIFF(DAY, #fromdate, #todate)
Now just toss that in a CTE and join it:
WITH Dates_CTE (dt) AS
(
-- // Paste the snippet above in here
)
SELECT d.dt AS paymentdate, ISNULL(SUM(p.paymentamount), 0) AS subtotal
FROM Dates_CTE d
LEFT JOIN tblpayment p
ON p.paymentdate = d.dt
GROUP BY d.dt
ORDER BY d.dt
(Update: I left out the WHERE clause in the main query because it's technically handled by the the join, but in some instances you might get better performance by leaving it in)
As for the currency conversion, look up the syntax for PIVOT.
Update on PIVOT: You should be able to just enclose that entire query in parentheses, then go:
SELECT paymentdate, [Euro] AS euroamount, [Pound] as poundamount
FROM
(
-- // Insert the full query from above in here
) p
PIVOT
(
SUM(subtotal)
FOR paymentcurrency IN ([Euro], [Pound])
) AS pvt
Hard to verify without knowing exactly what kind of data is in there, but try that as a starting point.
If there are no dummy records in tblPayment for the dates without any payment, those dates will not appear in a query that selects only from tblPayment.
I handle this by creating a separate table with nothing but dates in it (one row per date), checking to make sure that I have all the dates to cover my query, and then LEFT JOINing my main table (in this case tblPayment) on the date table:
SELECT * FROM tblPayment LEFT OUTER JOIN tblDates
ON tblPayment.PaymentDate = tblDates.PossibleDate
This basic idea can be enhanced with GROUP BY to get the summary figures you want.
Here is one approach
Create the following function:
CREATE FUNCTION [dbo].[DateTable] (#StartDate DATETIME, #endDate DATETIME)
RETURNS #Itms TABLE
(
TheDate DATETIME
)
AS
BEGIN
DECLARE #theDate DATETIME
SET #TheDate = #StartDate
WHILE #TheDate <= #endDate
BEGIN
INSERT #Itms VALUES (#theDate)
SET #TheDate =dateAdd(d,1,#theDate)
END
RETURN
END;
Then here is a query that should do what you want
select sum(p1.paymentamount) as subtotal,
CONVERT(char(10), p1.paymentdate, 103) as paymentdate
from
(select * from tblpayment p1
where 1=1
and p1.paymentdate >= #fromDate
and p1.paymentdate <= #toDate
union
select theDate as paymentDate,0 as paymentAmount
from dbo.dateTable (#fromDate,#toDate)
) p1
group by p1.paymentdate
try something like this perhaps?
select sum(p1.paymentamount) as subtotal,
CASE WHEN (CONVERT(char(10), p1.paymentdate, 103) = 0) THEN 'No Sale'
ELSE
CONVERT(char(10), p1.paymentdate, 103)
END as paymentdate
FROM tblpayment
where paymentdate BETWEEN #fromdate and #todate
As mentioned before you have to use a separate table (temp or permanent). The currency conversion can be done using a CASE statement. Check out the below (I made up the conversion factors ;)
declare #dates table (dateitem datetime)
declare #lower datetime
declare #upper datetime
set #lower = '12/1/9'
set #upper = '12/31/9'
while #lower <= #upper
begin
insert into #dates values (#lower)
set #lower = dateadd(day, 1, #lower)
end
select dateitem, paymentcurrency,
paymentindollars = case paymentcurrency when 'dollars' then total when 'euro' then total * 1.7 else 0 end,
paymentineuros = case paymentcurrency when 'dollars' then total * 0.73 when 'euro' then total else 0 end
from
(select dateitem, paymentcurrency, sum(paymentamount) as total
from #dates DT left join tblpayment on DT.dateitem = tblpayment.paymentdate group by dateitem, paymentcurrency
) IQ order by dateitem
Caveats to watch out for:
your payementdate might have times in
it that you will have to remove
(through casting) for the join/grouping to work properly
for the conversions to work right you have to separate the differnt currency types, you could always wrap them in another sql to get a grand total for the day
currency conversion is usually only good for the day so applying a general conversion against a period of time is not going to give you good financial results, only decent ballpark figures (ie don't try and file it on your taxes ;)
Hope that helps a bit.