Improve SQL query: Cumulative amounts over time - sql

Suppose I have a SQL table of Awards, with fields for Date and Amount. I need to generate a table with a sequence of consecutive dates, the amount awarded in each day, and the running (cumulative) total.
Date Amount_Total Amount_RunningTotal
---------- ------------ -------------------
1/1/2010 100 100
1/2/2010 300 400
1/3/2010 0 400
1/4/2010 0 400
1/5/2010 400 800
1/6/2010 100 900
1/7/2010 500 1400
1/8/2010 300 1700
This SQL works, but isn't as quick as I'd like:
Declare #StartDate datetime, #EndDate datetime
Select #StartDate=Min(Date), #EndDate=Max(Date) from Awards
; With
/* Returns consecutive from numbers 1 through the
number of days for which we have data */
Nbrs(n) as (
Select 1 Union All
Select 1+n
From Nbrs
Where n<=DateDiff(d,#StartDate,#EndDate)),
/* Returns all dates #StartDate to #EndDate */
AllDays as (
Select Date=DateAdd(d, n, #StartDate)
From Nbrs )
/* Returns totals for each day */
Select
d.Date,
Amount_Total = (
Select Sum(a.Amount)
From Awards a
Where a.Date=d.Date),
Amount_RunningTotal = (
Select Sum(a.Amount)
From Awards a
Where a.Date<=d.Date)
From AllDays d
Order by d.Date
Option(MAXRECURSION 1000)
I tried adding an index to Awards.Date, but it made a very minimal difference.
Before I resort to other strategies like caching, is there a more efficient way to code the running total calculation?

I generally use a temporary table for this:
DECLARE #Temp TABLE
(
[Date] date PRIMARY KEY,
Amount int NOT NULL,
RunningTotal int NULL
)
INSERT #Temp ([Date], Amount)
SELECT [Date], Amount
FROM ...
DECLARE #RunningTotal int
UPDATE #Temp
SET #RunningTotal = RunningTotal = #RunningTotal + Amount
SELECT * FROM #Temp
If you can't make the date column a primary key then you need to include an ORDER BY [Date] in the INSERT statement.
Also, this question's been asked a few times before. See here or search for "sql running total". The solution I posted is, as far as I know, still the one with the best performance, and also easy to write.

I don't have a database setup in front of me so I hope the below works first shot. A pattern like this should result in a much speedier query...you're just joining twice, similar amount of aggregation:
Declare #StartDate datetime, #EndDate datetime
Select #StartDate=Min(Date), #EndDate=Max(Date) from Awards
;
WITH AllDays(Date) AS (SELECT #StartDate UNION ALL SELECT DATEADD(d, 1, Date)
FROM AllDays
WHERE Date < #EndDate)
SELECT d.Date, sum(day.Amount) Amount_Total, sum(running.Amount) Amount_RunningTotal
FROM AllDays d
LEFT JOIN (SELECT date, SUM(Amount) As Amount
FROM Awards
GROUP BY Date) day
ON d.Date = day.Date
LEFT JOIN (SELECT date, SUM(Amount) As Amount
FROM Awards
GROUP BY Date) running
ON (d.Date >= running.Date)
Group by d.Date
Order by d.Date
Note: I changed your table expression up top, it was leaving out the first day before...if this is intentional just slap a where clause on this to exclude it. Let me know in the comments if this doesn't work or doesn't fit and I'll make whatever adjustments.

Here's a working solution based on #Aaronaught's answer. The only gotcha I had to overcome in T-SQL was that #RunningTotal etc. can't be null (need to be converted to zero).
Declare #StartDate datetime, #EndDate datetime
Select #StartDate=Min(StartDate),#EndDate=Max(StartDate) from Awards
/* #AllDays: Contains one row per date from #StartDate to #EndDate */
Declare #AllDays Table (
Date datetime Primary Key)
; With
Nbrs(n) as (
Select 0 Union All
Select 1+n from Nbrs
Where n<=DateDiff(d,#StartDate,#EndDate)
)
Insert into #AllDays
Select Date=DateAdd(d, n, #StartDate)
From Nbrs
Option(MAXRECURSION 10000) /* Will explode if working with more than 10000 days (~27 years) */
/* #AmountsByDate: Contains one row per date for which we have an Award, along with the totals for that date */
Declare #AmountsByDate Table (
Date datetime Primary Key,
Amount money)
Insert into #AmountsByDate
Select
StartDate,
Amount=Sum(Amount)
from Awards a
Group by StartDate
/* #Result: Joins #AllDays and #AmountsByDate etc. to provide totals and running totals for every day of the award */
Declare #Result Table (
Date datetime Primary Key,
Amount money,
RunningTotal money)
Insert into #Result
Select
d.Date,
IsNull(bt.Amount,0),
RunningTotal=0
from #AllDays d
Left Join #AmountsByDate bt on d.Date=bt.Date
Order by d.Date
Declare #RunningTotal money Set #RunningTotal=0
Update #Result Set #RunningTotal = RunningTotal = #RunningTotal + Amount
Select * from #Result

Related

How to calculate MTD given daily account balance in SQL Server?

I have a table with columns [accountid], [DateEnding], and [AccountBalance].
I need to calculate MTD using the balance of the current month and subtracting the account balance from the last day of the previous month for each accountid.
So far I have this:
SELECT [accountid]
,[DateEnding]
,[AccountBalance]
,[AccountBalance MTD Last] = AccountBalance - FIRST_VALUE(AccountBalance) OVER (PARTITION BY accountid, YEAR(DATEADD(mm,-1,[DateEnding])), MONTH(DATEADD(mm,-1,[DateEnding])) ORDER BY [DateEnding] DESC)
FROM [test]
ORDER BY accountid, DateEnding;
Here, for each distinct account, we find the latest record available according to DateEnding
we then find the last day of the last month by taking a number of days away equal to the current day number. e.g 23rd April 2019 we subtract 23 days to get 1st March 2019
we can then find the balance on that day.
Then put the calculation together in the SELECT
SELECT Q1.accountid,
Q2.DateEnding ,
Q3.EOMbalance,
Q2.LatestBalance,
Q2.LatestBalance - Q3.EOMbalance EOM
FROM (
SELECT Distinct t1.accountid FROM test t1
) Q1
CROSS APPLY (
SELECT TOP 1 t2.AccountBalance LatestBalance, t2.[DateEnding]
FROM test t2
WHERE t2.[accountid] = Q1.accountid
ORDER BY t2.[DateEnding] DESC
) Q2
CROSS APPLY (
SELECT Top 1 t3.AccountBalance EOMbalance
FROM test t3
WHERE t3.[accountid] = Q1.accountid
AND t3.[DateEnding]
= dateadd(day,0 - DAY(q2.dateending), q2.dateending)
ORDER BY t3.[DateEnding] DESC
) Q3
The first answer seems a little complicated for this problem (Cross Apply isn't necessary here).
The following may be easier for you:
I first look at the current day's account balances in subquery 'a'.
Then I look at the account balances from the last day of last month's data, in subquery 'b'.
Then it's just a matter of subtracting the two to show the MTD delta:
select a.accountid,
a.DateEnding,
a.AccountBalance as [Current AccountBalance],
b.AccountBalance as [EOM prior AccountBalance], --added for clarity
a.AccountBalance-b.AccountBalance as [AccountBalance MTD Last]
from
(select accountid, DateEnding, AccountBalance
from #test
where DateEnding = cast(getdate() as date)
/* getdate() returns today's date, so this query will also be with respect to today */
) a
left join
(select *
from #test
where DateEnding = DATEADD(MONTH, DATEDIFF(MONTH, -1, GETDATE())-1, -1)
/*this returns the last day of last month, always*/
) b
on a.accountid = b.accountid
Here is the SQL that makes this sample data and #test table. Simply execute it to have your own '#test' table to run against:
/*drop table #test
drop table #dates */
create table #test ([accountid] varchar(255),[DateEnding] date, [AccountBalance] decimal(16,2))
create table #dates (rnk int,dt date)
insert into #dates (dt)
values (cast('20180101' as date))
DECLARE
#basedate DATE,
#d INT
SELECT
#basedate = '20180101',
#d = 1
WHILE #d < (select datediff(day,cast('20180101' as date),getdate())+2) --select datediff(day,getdate(),cast('20180101' as datetime))
BEGIN
INSERT INTO #dates (dt)
values (DATEADD(day, 1, (select max(dt) from #dates)))
set #d = #d+1
END
update a
set a.rnk = b.rnk
from #dates a
left join (select rank() over (order by dt) rnk,dt from #dates) b on a.dt = b.dt
declare #a int
set #a = 1
declare #i int
set #i = 1
while #a <20
begin
while #i < (select max(rnk) from #dates)
begin
insert into #test
values (#a,(select dt from #dates where rnk = #i),cast(rand()*1000.0+#i as decimal(16,2)))
set #i=#i+1
end
set #a=#a+1
set #i = 1
end

Best way to pairing & finding anomalies in SQL data

The problem is that it takes way to long in SQL and there must be a better way. I’ve picked out the slow part for the scenario bellow.
Scenario:
Two (temp) tables with event times for start and end for vehicles that have to be paired up to figure idle durations. The issue is that some of the event data is missing. I figured out a rudimentary way of going through and determining when the last end time is after the next start time and removing the invalid start. Again not elegant + very slow.
Tables :
create table #start(VehicleIp int null, CurrentDate datetime null,
EventId int null,
StartId int null)
create table #end(VehicleIp int null,
CurrentDate datetime null,
EventId int null,
EndId int null)
--//Note: StartId and EndId are both pre-filled with something like:
ROW_NUMBER() Over(Partition by VehicleIp order by VehicleIp, CurrentDate)
--//Slow SQL
while exists(
select top 1 tOn.EventId
from #start as tOn
left JOIN #end tOff
on tOn.VehicleIp = tOff.VehicleIp and
tOn.StartID = tOff.EndID +1
)
begin
declare #badEntry int
select top 1 #badEntry = tOn.EventId
from #s as tOn
left JOIN #se tOff
on tOn.VehicleIp = tOff.VehicleIp and
tOn.StartID = tOff.EndID +1
order by tOn.CurrentDate
delete from #s where EventId = #badEntry
;with _s as ( select VehicleIp, CurrentDate, EventId,
ROW_NUMBER() Over(Partition by VehicleIp
order by VehicleIp, CurrentDate) StartID
from #start)
update #start
set StartId = _s.StartId
from #s join _s on #s.EventId = _s.EventId
end
Assuming you start with a table containing Vehicle and interval in which it was used, this query will identify gaps.
select b.VehicleID, b.IdleStart, b.IdleEnd
from
(
select VehicleID,
-- If EndDate is not inclusive, remove +1
EndDate + 1 IdleStart,
-- First date after current for this vehicle
-- If you don't want to show unused vehicles to current date remove isnull part
isnull((select top 1 StartDate
from TableA a
where a.VehicleID = b.VehicleID
and a.StartDate > b.StartDate
order by StartDate
), getdate()) IdleEnd
from TableA b
) b
where b.IdleStart < b.IdleEnd
If dates have time portion they should be truncated to required precision, here is for day:
dateadd(dd, datediff(dd,0, getDate()), 0)
Replace dd with hh, mm or whatever precision is needed.
And here is Sql Fiddle with test

How can I generate a temporary table filled with dates in SQL Server 2000?

I need to make a temporary table that holds of range of dates, as well as a couple of columns that hold placeholder values (0) for future use. The dates I need are the first day of each month between $startDate and $endDate where these variables can be several years apart.
My original sql statement looked like this:
select dbo.FirstOfMonth(InsertDate) as Month, 0 as Trials, 0 as Sales
into #dates
from customer
group by dbo.FirstOfMonth(InsertDate)
"FirstOfMonth" is a user-defined function I made that pretty much does what it says, returning the first day of the month for the provided date with the time at exactly midnight.
This produced almost exactly what I needed until I discovered there were occasionally gaps in my dates where I had a few months were there were no records insert dates. Since my result must still have the missing months I need a different approach.
I have added the following declarations to the stored procedure anticipating their need for the range of the dates I need ...
declare $startDate set $startDate = select min(InsertDate) from customer
declare $endDate set $endDate = select max(InsertDate) from customer
... but I have no idea what to do from here.
I know this question is similar to this question but, quite frankly, that answer is over my head (I don't often work with SQL and when I do it tends to be on older versions of SQL Server) and there are a few minor differences that are throwing me off.
I needed something similar, but all DAYS instead of all MONTHS.
Using the code from MatBailie as a starting point, here's the SQL for creating a permanent table with all dates from 2000-01-01 to 2099-12-31:
CREATE TABLE _Dates (
d DATE,
PRIMARY KEY (d)
)
DECLARE #dIncr DATE = '2000-01-01'
DECLARE #dEnd DATE = '2100-01-01'
WHILE ( #dIncr < #dEnd )
BEGIN
INSERT INTO _Dates (d) VALUES( #dIncr )
SELECT #dIncr = DATEADD(DAY, 1, #dIncr )
END
This will quickly populate a table with 170 years worth of dates.
CREATE TABLE CalendarMonths (
date DATETIME,
PRIMARY KEY (date)
)
DECLARE
#basedate DATETIME,
#offset INT
SELECT
#basedate = '01 Jan 2000',
#offset = 1
WHILE (#offset < 2048)
BEGIN
INSERT INTO CalendarMonths SELECT DATEADD(MONTH, #offset, date) FROM CalendarMonths
SELECT #offset = #offset + #offset
END
You can then use it by LEFT joining on to that table, for the range of dates you require.
I would probably use a Calendar table. Create a permanent table in your database and fill it with all of the dates. Even if you covered a 100 year range, the table would still only have ~36,525 rows in it.
CREATE TABLE dbo.Calendar (
calendar_date DATETIME NOT NULL,
is_weekend BIT NOT NULL,
is_holiday BIT NOT NULL,
CONSTRAINT PK_Calendar PRIMARY KEY CLUSTERED (calendar_date)
)
Once the table is created, just populate it once in a loop, so that it's always out there and available to you.
Your query then could be something like this:
SELECT
C.calendar_date,
0 AS trials,
0 AS sales
FROM
dbo.Calendar C
WHERE
C.calendar_date BETWEEN #start_date AND #end_date AND
DAY(C.calendar_date) = 1
You can join in the Customers table however you need to, outer joining on FirstOfMonth(InsertDate) = C.calendar_date if that's what you want.
You can also include a column for day_of_month if you want which would avoid the overhead of calling the DAY() function, but that's fairly trivial, so it probably doesn't matter one way or another.
This of course will not work in SQL-Server 2000 but in a modern database where you don't want to create a permanent table. You can use a table variable instead creating a table so you can left join the data try this. Change the DAY to HOUR etc to change the increment type.
declare #CalendarMonths table (date DATETIME, PRIMARY KEY (date)
)
DECLARE
#basedate DATETIME,
#offset INT
SELECT
#basedate = '01 Jan 2014',
#offset = 1
INSERT INTO #CalendarMonths SELECT #basedate
WHILE ( DATEADD(DAY, #offset, #basedate) < CURRENT_TIMESTAMP)
BEGIN
INSERT INTO #CalendarMonths SELECT DATEADD(HOUR, #offset, date) FROM #CalendarMonths where DATEADD(DAY, #offset, date) < CURRENT_TIMESTAMP
SELECT #offset = #offset + #offset
END
A starting point of a useful kludge to specify a range or specific list of dates:
SELECT *
FROM
(SELECT CONVERT(DateTime,'2017-1-1')+number AS [Date]
FROM master..spt_values WHERE type='P' AND number<370) AS DatesList
WHERE DatesList.Date IN ('2017-1-1','2017-4-14','2017-4-17','2017-12-25','2017-12-26')
You can get 0 to 2047 out of master..spt_values WHERE type='P', so that's five and a half year's worth of dates if you need it!
Tested below and it works, though it's a bit convoluted.
I assigned arbitrary values to the dates for the test.
DECLARE #SD smalldatetime,
#ED smalldatetime,
#FD smalldatetime,
#LD smalldatetime,
#Mct int,
#currct int = 0
SET #SD = '1/15/2011'
SET #ED = '2/02/2012'
SET #FD = (DATEADD(dd, -1*(Datepart(dd, #SD)-1), #sd))
SET #LD = (DATEADD(dd, -1*(Datepart(dd, #ED)-1), #ED))
SET #Mct = DATEDIFF(mm, #FD, #LD)
CREATE TABLE #MyTempTable (FoM smalldatetime, Trials int, Sales money)
WHILE #currct <= #Mct
BEGIN
INSERT INTO #MyTempTable (FoM, Trials, Sales)
VALUES
(DATEADD(MM, #currct, #FD), 0, 0)
SET #currct = #currct + 1
END
SELECT * FROM #MyTempTable
DROP TABLE #MyTempTable
For SQL Server 2000, this stackoverflow post looks promising for a way to temporarily generate dates calculated off of a start and end date. It's not exactly the same but quite similar. This post has a very in-depth answer on truncating dates, if needed.
In case anyone stumbles on this question and is working in PostgreSQL instead of SQL Server 2000, here is how you might do it there...
PostgreSQL has a nifty series generating function. For your example, you could use this series of all days instead of generating an entire calendar table, and then do groupings and matchups from there.
SELECT current_date + s.a AS dates FROM generate_series(0,14,7) AS s(a);
dates
------------
2004-02-05
2004-02-12
2004-02-19
(3 rows)
SELECT * FROM generate_series('2008-03-01 00:00'::timestamp,
'2008-03-04 12:00', '10 hours');
generate_series
---------------------
2008-03-01 00:00:00
2008-03-01 10:00:00
2008-03-01 20:00:00
2008-03-02 06:00:00
2008-03-02 16:00:00
2008-03-03 02:00:00
2008-03-03 12:00:00
2008-03-03 22:00:00
2008-03-04 08:00:00
(9 rows)
I would also look into date_trunc from PostgreSQL using 'month' for the truncator field to maybe refactor your original query to easily match with a date_trunc version of the calendar series.
select top (datediff(D,#start,#end)) dateadd(D,id-1,#start)
from BIG_TABLE_WITH_NO_JUMPS_IN_ID
declare #start datetime
set #start = '2016-09-01'
declare #end datetime
set #end = '2016-09-30'
create table #Date
(
table_id int identity(1,1) NOT NULL,
counterDate datetime NULL
);
insert into #Date select top (datediff(D,#start,#end)) NULL from SOME_TABLE
update #Date set counterDate = dateadd(D,table_id - 1, #start)
The code above should populate the table with all the dates between the start and end. You would then just join on this table to get all of the dates needed. If you only needed a certain day of each month, you could dateadd a month instead.
SELECT P.Id
, DATEADD ( DD, -P.Id, P.Date ) AS Date
FROM (SELECT TOP 1000 ROW_NUMBER () OVER (ORDER BY (SELECT NULL)) AS Id, CAST(GETDATE () AS DATE) AS Date FROM master.dbo.spt_values) AS P
This query returns a table calendar for the last 1000 days or so. It can be put in a temporary or other table.
Create a table variable containing a date for each month in a year:
declare #months table (reportMonth date, PRIMARY KEY (reportMonth));
declare #start date = '2018', #month int = 0; -- base 0 month
while (#month < 12)
begin
insert into #months select dateAdd(month, #month, #start);
select #month = #month + 1;
end
--verify
select * from #months;
This is by far the quickest method I have found (much quicker than inserting rows 1 by 1 in a WHILE loop):
DECLARE #startDate DATE = '1900-01-01'
DECLARE #endDate DATE = '2050-01-01'
SELECT DATEADD(DAY, sequenceNumber, #startDate) AS TheDate
INTO #TheDates
FROM (
SELECT ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n + 10000*tenthousands.n AS sequenceNumber
FROM
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) ones(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tens(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) hundreds(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) thousands(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tenthousands(n)
WHERE ones.n + 10*tens.n + 100*hundreds.n + 1000*thousands.n + 10000*tenthousands.n <= DATEDIFF(day, #startDate, #endDate)
) theNumbers
SELECT *
FROM #TheDates
ORDER BY TheDate
The recursive answer:
DECLARE #startDate AS date = '20220315';
DECLARE #endDate AS date = '20230316'; -- inclusive
WITH cte_minutes(dt)
AS (
SELECT
DATEFROMPARTS(YEAR(#startDate), MONTH(#startDate), 1)
UNION ALL
SELECT
DATEADD(month, 1, dt)
FROM
cte_minutes
WHERE DATEADD(month, 1, dt) < #endDate
)
SELECT
dt
into #dates
FROM
cte_minutes
WHERE
dt >= #startDate
AND
dt <= #endDate
OPTION (MAXRECURSION 2000);
DROP TABLE dbo.#dates

sql query to get daily payments for a month even if no payments on a given day

I am using SQL Server 2005 and trying to write a query where I want to retrieve payments for a given month. I currently have:
select sum(p1.paymentamount) as subtotal,
CONVERT(char(10), p1.paymentdate, 103) as paymentdate
from tblpayment p1
where 1=1
and p1.paymentdate >= #fromdate
and p1.paymentdate <= #todate
group by p1.paymentdate
order by p1.paymentdate
Schema:
CREATE TABLE [dbo].[tblPayment]
(
[paymentid] [int] IDENTITY(1,1) NOT NULL,
[userid] [int] NULL ,
[paymentdate] [datetime] NOT NULL,
[paymentamount] [int] NULL,
[paymenttype] [varchar](50) NULL,
[paymentnotes] [varchar](200) NULL,
[paymentcurrency] [nchar](10) NULL
)
This query gives me what I want but it doesnt give me the dates where no payments were made. What I want is a query that gives me all days even if there were no payments made on that day and jut shows the subtotal as 0 for that day.
There is another catch. The currency of payments is different. So how can I have another column in the query that gives me eurototal and sterlingtotal based on #currency parameter passed in ? Assuming there is a column in the table for "paymentcurrency"
You have to work backwards. In order to get rows for dates that don't exist, you need to outer join them to rows that do have those dates. In order to outer join, you need to have a sequence to join to. Since you don't have a sequence, you need to create one.
To create that sequence, you have two options:
Create a static date sequence and store it in a permanent table (Larry's answer); or
Use an existing numeric sequence (such as spt_values) to create one on the fly.
Let's assume you want the flexibility of the second approach. Here's a common snippet I use for things like that:
SELECT DATEADD(DAY, v.number, #fromdate)
FROM master.dbo.spt_values v
WHERE v.type = 'P'
AND v.number <= DATEDIFF(DAY, #fromdate, #todate)
Now just toss that in a CTE and join it:
WITH Dates_CTE (dt) AS
(
-- // Paste the snippet above in here
)
SELECT d.dt AS paymentdate, ISNULL(SUM(p.paymentamount), 0) AS subtotal
FROM Dates_CTE d
LEFT JOIN tblpayment p
ON p.paymentdate = d.dt
GROUP BY d.dt
ORDER BY d.dt
(Update: I left out the WHERE clause in the main query because it's technically handled by the the join, but in some instances you might get better performance by leaving it in)
As for the currency conversion, look up the syntax for PIVOT.
Update on PIVOT: You should be able to just enclose that entire query in parentheses, then go:
SELECT paymentdate, [Euro] AS euroamount, [Pound] as poundamount
FROM
(
-- // Insert the full query from above in here
) p
PIVOT
(
SUM(subtotal)
FOR paymentcurrency IN ([Euro], [Pound])
) AS pvt
Hard to verify without knowing exactly what kind of data is in there, but try that as a starting point.
If there are no dummy records in tblPayment for the dates without any payment, those dates will not appear in a query that selects only from tblPayment.
I handle this by creating a separate table with nothing but dates in it (one row per date), checking to make sure that I have all the dates to cover my query, and then LEFT JOINing my main table (in this case tblPayment) on the date table:
SELECT * FROM tblPayment LEFT OUTER JOIN tblDates
ON tblPayment.PaymentDate = tblDates.PossibleDate
This basic idea can be enhanced with GROUP BY to get the summary figures you want.
Here is one approach
Create the following function:
CREATE FUNCTION [dbo].[DateTable] (#StartDate DATETIME, #endDate DATETIME)
RETURNS #Itms TABLE
(
TheDate DATETIME
)
AS
BEGIN
DECLARE #theDate DATETIME
SET #TheDate = #StartDate
WHILE #TheDate <= #endDate
BEGIN
INSERT #Itms VALUES (#theDate)
SET #TheDate =dateAdd(d,1,#theDate)
END
RETURN
END;
Then here is a query that should do what you want
select sum(p1.paymentamount) as subtotal,
CONVERT(char(10), p1.paymentdate, 103) as paymentdate
from
(select * from tblpayment p1
where 1=1
and p1.paymentdate >= #fromDate
and p1.paymentdate <= #toDate
union
select theDate as paymentDate,0 as paymentAmount
from dbo.dateTable (#fromDate,#toDate)
) p1
group by p1.paymentdate
try something like this perhaps?
select sum(p1.paymentamount) as subtotal,
CASE WHEN (CONVERT(char(10), p1.paymentdate, 103) = 0) THEN 'No Sale'
ELSE
CONVERT(char(10), p1.paymentdate, 103)
END as paymentdate
FROM tblpayment
where paymentdate BETWEEN #fromdate and #todate
As mentioned before you have to use a separate table (temp or permanent). The currency conversion can be done using a CASE statement. Check out the below (I made up the conversion factors ;)
declare #dates table (dateitem datetime)
declare #lower datetime
declare #upper datetime
set #lower = '12/1/9'
set #upper = '12/31/9'
while #lower <= #upper
begin
insert into #dates values (#lower)
set #lower = dateadd(day, 1, #lower)
end
select dateitem, paymentcurrency,
paymentindollars = case paymentcurrency when 'dollars' then total when 'euro' then total * 1.7 else 0 end,
paymentineuros = case paymentcurrency when 'dollars' then total * 0.73 when 'euro' then total else 0 end
from
(select dateitem, paymentcurrency, sum(paymentamount) as total
from #dates DT left join tblpayment on DT.dateitem = tblpayment.paymentdate group by dateitem, paymentcurrency
) IQ order by dateitem
Caveats to watch out for:
your payementdate might have times in
it that you will have to remove
(through casting) for the join/grouping to work properly
for the conversions to work right you have to separate the differnt currency types, you could always wrap them in another sql to get a grand total for the day
currency conversion is usually only good for the day so applying a general conversion against a period of time is not going to give you good financial results, only decent ballpark figures (ie don't try and file it on your taxes ;)
Hope that helps a bit.

How to Determine Values for Missing Months based on Data of Previous Months in T-SQL

I have a set of transactions occurring at specific points in time:
CREATE TABLE Transactions (
TransactionDate Date NOT NULL,
TransactionValue Integer NOT NULL
)
The data might be:
INSERT INTO Transactions (TransactionDate, TransactionValue)
VALUES ('1/1/2009', 1)
INSERT INTO Transactions (TransactionDate, TransactionValue)
VALUES ('3/1/2009', 2)
INSERT INTO Transactions (TransactionDate, TransactionValue)
VALUES ('6/1/2009', 3)
Assuming that the TransactionValue sets some kind of level, I need to know what the level was between the transactions. I need this in the context of a set of T-SQL queries, so it would be best if I could get a result set like this:
Month Value
1/2009 1
2/2009 1
3/2009 2
4/2009 2
5/2009 2
6/2009 3
Note how, for each month, we either get the value specified in the transaction, or we get the most recent non-null value.
My problem is that I have little idea how to do this! I'm only an "intermediate" level SQL Developer, and I don't remember ever seeing anything like this before. Naturally, I could create the data I want in a program, or using cursors, but I'd like to know if there's a better, set-oriented way to do this.
I'm using SQL Server 2008, so if any of the new features will help, I'd like to hear about it.
P.S. If anyone can think of a better way to state this question, or even a better subject line, I'd greatly appreciate it. It took me quite a while to decide that "spread", while lame, was the best I could come up with. "Smear" sounded worse.
I'd start by building a Numbers table holding sequential integers from 1 to a million or so. They come in really handy once you get the hang of it.
For example, here is how to get the 1st of every month in 2008:
select firstOfMonth = dateadd( month, n - 1, '1/1/2008')
from Numbers
where n <= 12;
Now, you can put that together using OUTER APPLY to find the most recent transaction for each date like so:
with Dates as (
select firstOfMonth = dateadd( month, n - 1, '1/1/2008')
from Numbers
where n <= 12
)
select d.firstOfMonth, t.TransactionValue
from Dates d
outer apply (
select top 1 TransactionValue
from Transactions
where TransactionDate <= d.firstOfMonth
order by TransactionDate desc
) t;
This should give you what you're looking for, but you might have to Google around a little to find the best way to create the Numbers table.
here's what i came up with
declare #Transactions table (TransactionDate datetime, TransactionValue int)
declare #MinDate datetime
declare #MaxDate datetime
declare #iDate datetime
declare #Month int
declare #count int
declare #i int
declare #PrevLvl int
insert into #Transactions (TransactionDate, TransactionValue)
select '1/1/09',1
insert into #Transactions (TransactionDate, TransactionValue)
select '3/1/09',2
insert into #Transactions (TransactionDate, TransactionValue)
select '5/1/09',3
select #MinDate = min(TransactionDate) from #Transactions
select #MaxDate = max(TransactionDate) from #Transactions
set #count=datediff(mm,#MinDate,#MaxDate)
set #i=1
set #iDate=#MinDate
while (#i<=#count)
begin
set #iDate=dateadd(mm,1,#iDate)
if (select count(*) from #Transactions where TransactionDate=#iDate) < 1
begin
select #PrevLvl = TransactionValue from #Transactions where TransactionDate=dateadd(mm,-1,#iDate)
insert into #Transactions (TransactionDate, TransactionValue)
select #iDate, #prevLvl
end
set #i=#i+1
end
select *
from #Transactions
order by TransactionDate
To do it in a set-based way, you need sets for all of your data or information. In this case there's the overlooked data of "What months are there?" It's very useful to have a "Calendar" table as well as a "Number" table in databases as utility tables.
Here's a solution using one of these methods. The first bit of code sets up your calendar table. You can fill it using a cursor or manually or whatever and you can limit it to whatever date range is needed for your business (back to 1900-01-01 or just back to 1970-01-01 and as far into the future as you want). You can also add any other columns that are useful for your business.
CREATE TABLE dbo.Calendar
(
date DATETIME NOT NULL,
is_holiday BIT NOT NULL,
CONSTRAINT PK_Calendar PRIMARY KEY CLUSTERED (date)
)
INSERT INTO dbo.Calendar (date, is_holiday) VALUES ('2009-01-01', 1) -- New Year
INSERT INTO dbo.Calendar (date, is_holiday) VALUES ('2009-01-02', 1)
...
Now, using this table your question becomes trivial:
SELECT
CAST(MONTH(date) AS VARCHAR) + '/' + CAST(YEAR(date) AS VARCHAR) AS [Month],
T1.TransactionValue AS [Value]
FROM
dbo.Calendar C
LEFT OUTER JOIN dbo.Transactions T1 ON
T1.TransactionDate <= C.date
LEFT OUTER JOIN dbo.Transactions T2 ON
T2.TransactionDate > T1.TransactionDate AND
T2.TransactionDate <= C.date
WHERE
DAY(C.date) = 1 AND
T2.TransactionDate IS NULL AND
C.date BETWEEN '2009-01-01' AND '2009-12-31' -- You can use whatever range you want
John Gibb posted a fine answer, already accepted, but I wanted to expand on it a bit to:
eliminate the one year limitation,
expose the date range in a more
explicit manner, and
eliminate the need for a separate
numbers table.
This slight variation uses a recursive common table expression to establish the set of Dates representing the first of each month on or after from and to dates defined in DateRange. Note the use of the MAXRECURSION option to prevent a stack overflow (!); adjust as necessary to accommodate the maximum number of months expected. Also, consider adding alternative Dates assembly logic to support weeks, quarters, even day-to-day.
with
DateRange(FromDate, ToDate) as (
select
Cast('11/1/2008' as DateTime),
Cast('2/15/2010' as DateTime)
),
Dates(Date) as (
select
Case Day(FromDate)
When 1 Then FromDate
Else DateAdd(month, 1, DateAdd(month, ((Year(FromDate)-1900)*12)+Month(FromDate)-1, 0))
End
from DateRange
union all
select DateAdd(month, 1, Date)
from Dates
where Date < (select ToDate from DateRange)
)
select
d.Date, t.TransactionValue
from Dates d
outer apply (
select top 1 TransactionValue
from Transactions
where TransactionDate <= d.Date
order by TransactionDate desc
) t
option (maxrecursion 120);
If you do this type of analysis often, you might be interested in this SQL Server function I put together for exactly this purpose:
if exists (select * from dbo.sysobjects where name = 'fn_daterange') drop function fn_daterange;
go
create function fn_daterange
(
#MinDate as datetime,
#MaxDate as datetime,
#intval as datetime
)
returns table
--**************************************************************************
-- Procedure: fn_daterange()
-- Author: Ron Savage
-- Date: 12/16/2008
--
-- Description:
-- This function takes a starting and ending date and an interval, then
-- returns a table of all the dates in that range at the specified interval.
--
-- Change History:
-- Date Init. Description
-- 12/16/2008 RS Created.
-- **************************************************************************
as
return
WITH times (startdate, enddate, intervl) AS
(
SELECT #MinDate as startdate, #MinDate + #intval - .0000001 as enddate, #intval as intervl
UNION ALL
SELECT startdate + intervl as startdate, enddate + intervl as enddate, intervl as intervl
FROM times
WHERE startdate + intervl <= #MaxDate
)
select startdate, enddate from times;
go
it was an answer to this question, which also has some sample output from it.
I don't have access to BOL from my phone so this is a rough guide...
First, you need to generate the missing rows for the months you have no data. You can either use a OUTER join to a fixed table or temp table with the timespan you want or from a programmatically created dataset (stored proc or suchlike)
Second, you should look at the new SQL 2008 'analytic' functions, like MAX(value) OVER ( partition clause ) to get the previous value.
(I KNOW Oracle can do this 'cause I needed it to calculate compounded interest calcs between transaction dates - same problem really)
Hope this points you in the right direction...
(Avoid throwing it into a temp table and cursoring over it. Too crude!!!)
-----Alternative way------
select
d.firstOfMonth,
MONTH(d.firstOfMonth) as Mon,
YEAR(d.firstOfMonth) as Yr,
t.TransactionValue
from (
select
dateadd( month, inMonths - 1, '1/1/2009') as firstOfMonth
from (
values (1), (2), (3), (4), (5), (7), (8), (9), (10), (11), (12)
) Dates(inMonths)
) d
outer apply (
select top 1 TransactionValue
from Transactions
where TransactionDate <= d.firstOfMonth
order by TransactionDate desc
) t