Select list of dates from given range in Firebird - sql

I'm designing a report that returns PurchaseOrder due in future week.
Query that I've added below returns PurchaseOrder due for a particular Commodity, AmountDue and its DeliveryDate.
Obviously it only returns PO_Dates that are in the table. What I want is to also include dates where no PO is expected, i.e. null for those cell.
To me one possibility is to LEFT JOIN the dataset with set of dates of future week on Date column, that will eventually make the result null where no Purchase Order is expected.
In Firebird I don't know how to select list of week long dates and then use it in the join.
SELECT
PURCHASE_ORDER_DET.COMMODITYID AS COM_ID,
PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('29.12.1899' AS DATE) as DLV_DATE,
SUM(PURCHASE_ORDER_DET.REQQUANTITY) as DLV_DUE
FROM
PURCHASE_ORDER_DET
LEFT JOIN PURCHASE_ORDER_HDR on PURCHASE_ORDER_HDR.POH_ID =
PURCHASE_ORDER_DET.POH_ID
WHERE
PURCHASE_ORDER_DET.COMMODITYID = 1
AND PURCHASE_ORDER_HDR.STATUS in (0,1,2)
AND PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('30.12.1899' AS TIMESTAMP) >= '3.01.2019'
AND PURCHASE_ORDER_DET.DELIVERYDATE + CAST ('30.12.1899' AS TIMESTAMP) <= '9.01.2019'
AND PURCHASE_ORDER_DET.DELETED is NULL
Group by
PURCHASE_ORDER_DET.COMMODITYID,
PURCHASE_ORDER_DET.DELIVERYDATE
DataSet
COM_ID DLV_DATE DLV_DUE
1 3.01.2019 50.000000
1 5.01.2019 10.000000
Expected
COM_ID DLV_DATE DLV_DUE
1 3.01.2019 50.000000
1 4.01.2019 null
1 5.01.2019 10.000000
1 6.01.2019 null
1 7.01.2019 null
1 8.01.2019 null
1 9.01.2019 null

Ignoring your odd use of datatypes*, there are several possible solutions:
Use a 'calendar' table that contains dates, and right join to that table (or left join from that table). The downside of course is having to populate this table (but that is a one-off cost).
Use a selectable stored procedure to generate a date range and join on that.
Generate the range in a recursive common table expression in the query itself
Option 1 is pretty self-explanatory.
Option 2 would look something like:
CREATE OR ALTER PROCEDURE date_range(startdate date, enddate date)
RETURNS (dateval date)
AS
BEGIN
dateval = startdate;
while (dateval <= enddate) do
BEGIN
suspend;
dateval = dateval + 1;
END
END
And then use this in your query like:
select date_range.dateval, ...
from date_range(date'2019-01-03', date'2019-01-09') -- use date_range(?, ?) for parameters
left join ...
on date_range.dateval = ...
Option 3 would look something like:
WITH RECURSIVE date_range AS (
SELECT date'2019-01-03' dateval -- start date, use cast(? as date) if you need a parameter
FROM rdb$database
UNION ALL
SELECT dateval + 1
FROM date_range
WHERE dateval < date'2019-01-09' -- end date use ? if you need a parameter
)
SELECT *
FROM date_range
LEFT JOIN ...
ON date_range.dateval = ...
Recursive common table expressions have a maximum recursion depth of 1024, which means that it isn't suitable if you need a span wider than 1024 days.
*: I'd suggest that you start using DATE instead of what looks like the number of days since 30-12-1899. That avoids having to do awkward calculations like you do now. If you do need those number of days, then you can for example use datediff(DAY FROM date'1899-12-30' TO somedatevalue) or somedatevalue - date'1899-12-30' to convert from date to that numeric value.

Related

iSeries SQL - Use Previous Record Value in JOIN criteria

In my example, I am trying to retrieve records from a transaction file based on the previous calendar work date. The work dates are in a file called CALNDR, the transactions in a file called TRNHST. The logic would be to use the CURRENT_DATE to set the cursor on CALNDR, then retrieve the previous work date which will be used to join to TRNHST. This would be simple enough if done in RPG but I am at a loss with SQL. Thanks, for any input you can provide.
CALNDR
TRNHST
This is even more simple with SQL probably.
You may run these statements as is to check.
LAG function use
WITH
CALNDR (CALDT) AS
(
VALUES
CAST ('2022-07-21' AS DATE)
, CAST ('2022-07-22' AS DATE)
, CAST ('2022-07-25' AS DATE)
)
SELECT C.*
FROM
(
SELECT CALDT, LAG (CALDT) OVER (ORDER BY CALDT) AS LAG_CALDT
FROM CALNDR
) C
-- JOIN TRNHST ...
-- You must keep this expression in WHERE >>OUTSIDE<< of the subselect
-- to get the desired result
WHERE C.CALDT =
CAST ('2022-07-25' AS DATE)
--CAST ('2022-07-24' AS DATE)
;
The problem with LAG is that you can't use it, if you have non-continuous calendar as in your case, and CURRENT_DATE for some day (as 2022-07-24 in the example) doesn't exist there.
BTW, I don't know if it's some problem namely with my IBM i v7.5 or not, but if I move the WHERE clause to the subselect, I get wrong result of the query at all - it doesn't achieve the goal desired here and doesn't return the expected result.
Aggregation function use
WITH
CALNDR (CALDT) AS
(
VALUES
CAST ('2022-07-21' AS DATE)
, CAST ('2022-07-22' AS DATE)
, CAST ('2022-07-25' AS DATE)
)
SELECT C.*
FROM
(
SELECT MAX (CALDT) AS LAG_CALDT
FROM CALNDR
WHERE CALDT <
CAST ('2022-07-25' AS DATE)
--CAST ('2022-07-24' AS DATE)
) C
-- JOIN TRNHST
;
Works disregarding of the presence of CURRENT_DATE in the calendar.
Honestly, if "prior working day" for a given date is something you need. You should consider adding such a column to your calendar table.
While you're at it, might as well add a "next working day".
One of the biggest benefits to a calendar table is the ability to pre-calculate such columns. Thus, greatly simplifying your statements.
Not real sure what LAG does as I have never used it. So, here's another solution using a cross join:
WITH calendar (workdt) AS (
VALUES (CAST('2022-01-01' AS date)),
(CAST('2022-02-01' AS date)),
(CAST('2022-03-01' AS date)),
(CAST('2022-04-01' AS date))
),
orders (orderno, orderdt) AS (
VALUES (1, CAST('2022-01-15' AS date)),
(2, CAST('2022-02-15' AS date)),
(3, CAST('2022-03-15' AS date)),
(4, CAST('2022-04-15' AS date))
)
SELECT *
FROM orders a
CROSS JOIN LATERAL (SELECT max(workdt) prevdt FROM calendar WHERE workdt < a.orderdt) b
The cross join only returns a single row for each record in the calendar file, and that row contains the largest work date less than the order date. LATERAL just lets the sub-query access columns from the outer query.
write an SQL function named calndr_prevWorkDate that returns the previous work date.
Then use that function in the WHERE clause of an SQL SELECT statement to select records from the TRNHST table:
select a.*
from trnhst a
where a.trandate = calndr_prevWorkDate( )
Here is the SQL function.
CREATE OR REPLACE FUNCTION calndr_prevWorkDate(
startDate date default current date )
RETURNS date
language sql
specific calndrf1
SET OPTION datfmt = *ISO, DLYPRP = *YES, DBGVIEW = *SOURCE,
USRPRF = *OWNER, DYNUSRPRF = *OWNER, COMMIT = *CHG
BEGIN
declare vCalDate date ;
declare vSqlCode decimal(5,0) default 0 ;
declare sqlCode int default 0 ;
declare sqlState char(5) default ' ' ;
declare vSqlState char(5) default ' ' ;
declare vErrText varchar(256) default '' ;
DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
begin
SET vSqlCode = SQLCODE ;
SET vSqlState = SQLstate ;
get diagnostics exception 1 vErrText = message_text ;
end ;
select a.caldate
into vCalDate
from calndr a
where a.caldate < startDate
and dayofweek(a.caldate) not in ( 1, 7)
order by a.caldate desc
fetch first row only ;
return vCalDate ;
END
if all u need is the previous date from a table
you can use something like that
select *
from TRNHST
where Trans_Date =
(
select calendar_Date
from CALNDR
where calendar_Date < current date
order by calendar_Date desc
limit 1
)

Operand clash with date table and varchar date strings in cross join

This follows on from my previous question, but since I tried to simplify, I appear to have missed something Daily snapshot table using cte loop
I am trying to set up the below cross join between dates and an employee table. I need a daily count according to division and department, but the dates won't link easily since the dates are stored as varchar (not my choice, I can't change it).
I now have a date table that includes a style112 (yyyymmdd) key that I can link to the table, but there seems to be a failure somewhere along the joins.
I'm so close, but really am lost! I have never had to work with string dates and wouldn't wish it upon anyone.
DECLARE #DATESTART AS Date = '20180928';
DECLARE #DATEEND AS Date = '20181031';
WITH Dates AS (
SELECT #DATESTART AS Dte
UNION ALL
SELECT DTE + 1
FROM Dates
WHERE Dte <= #DATEEND )
SELECT
Dt.Dte
,CAST(DTC.Style112 AS VARCHAR)
,Emp.Division_Description
,Emp.Department_Description
,(SELECT
COUNT(*)
FROM ASS_D_EmpMaster_Live E
WHERE
E.[Start_Date] <= CAST(DTC.Style112 AS VARCHAR)
AND (E.Leaving_Date > CAST(DTC.Style112 AS VARCHAR)
OR E.Leaving_Date = '00000000')
) Counts
FROM Dates Dt
LEFT JOIN ASS_C_DateConversions DTC
ON DTC.[Date] = Dt.DtE
CROSS JOIN
(
SELECT DISTINCT
Division_Description
,Department_Description
FROM
ASS_D_EmpMaster_Live e
) Emp
OPTION (MAXRECURSION 1000)
Desired output:
Date
Dept1
Dept2
Dept3
20180901
25
231
154
20180902
23
232
154
I don't think you need the conversion table at all and I would remove it. And I believe the subquery should look like this:
SELECT COUNT(*)
FROM ASS_D_EmpMaster_Live E
WHERE
CAST(E.Start_Date AS DATE) <= Dt.Dte
AND (CAST(E.Leaving_Date AS DATE) > Dt.Dte OR E.Leaving_Date = '00000000')

Calculating business days in Teradata

I need help in business days calculation.
I've two tables
1) One table ACTUAL_TABLE containing order date and contact date with timestamp datatypes.
2) The second table BUSINESS_DATES has each of the calendar dates listed and has a flag to indicate weekend days.
using these two tables, I need to ensure business days and not calendar days (which is the current logic) is calculated between these two fields.
My thought process was to first get a range of dates by comparing ORDER_DATE with TABLE_DATE field and then do a similar comparison of CONTACT_DATE to TABLE_DATE field. This would get me a range from the BUSINESS_DATES table which I can then use to calculate count of days, sum(Holiday_WKND_Flag) fields making the result look like:
Order# | Count(*) As DAYS | SUM(WEEKEND DATES)
100 | 25 | 8
However this only works when I use a specific order number and cant' bring all order numbers in a sub query.
My Query:
SELECT SUM(Holiday_WKND_Flag), COUNT(*) FROM
(
SELECT
* FROM
BUSINESS_DATES
WHERE BUSINESS.Business BETWEEN (SELECT ORDER_DATE FROM ACTUAL_TABLE
WHERE ORDER# = '100'
)
AND
(SELECT CONTACT_DATE FROM ACTUAL_TABLE
WHERE ORDER# = '100'
)
TEMP
Uploading the table structure for your reference.
SELECT ORDER#, SUM(Holiday_WKND_Flag), COUNT(*)
FROM business_dates bd
INNER JOIN actual_table at ON bd.table_date BETWEEN at.order_date AND at.contact_date
GROUP BY ORDER#
Instead of joining on a BETWEEN (which always results in a bad Product Join) followed by a COUNT you better assign a bussines day number to each date (in best case this is calculated only once and added as a column to your calendar table). Then it's two Equi-Joins and no aggregation needed:
WITH cte AS
(
SELECT
Cast(table_date AS DATE) AS table_date,
-- assign a consecutive number to each busines day, i.e. not increased during weekends, etc.
Sum(CASE WHEN Holiday_WKND_Flag = 1 THEN 0 ELSE 1 end)
Over (ORDER BY table_date
ROWS Unbounded Preceding) AS business_day_nbr
FROM business_dates
)
SELECT ORDER#,
Cast(t.contact_date AS DATE) - Cast(t.order_date AS DATE) AS #_of_days
b2.business_day_nbr - b1.business_day_nbr AS #_of_business_days
FROM actual_table AS t
JOIN cte AS b1
ON Cast(t.order_date AS DATE) = b1.table_date
JOIN cte AS b2
ON Cast(t.contact_date AS DATE) = b2.table_date
Btw, why are table_date and order_date timestamp instead of a date?
Porting from Oracle?
You can use this query. Hope it helps
select order#,
order_date,
contact_date,
(select count(1)
from business_dates_table
where table_date between a.order_date and a.contact_date
and holiday_wknd_flag = 0
) business_days
from actual_table a

Query for dates which are not present in a table

Consider a table ABC which has a column of date type.
How can we get all the dates of a range (between start date and end date) which are not present in the table.
This can be done in PLSQL.I am searching a SQL query for it.
You need to generate the arbitrary list of dates that you want to check for:
http://hashfactor.wordpress.com/2009/04/08/sql-generating-series-of-numbers-in-oracle/
e.g.:
-- generate 1..20
SELECT ROWNUM N FROM dual
CONNECT BY LEVEL <= 20
Then left join with your table, or use a where not exists subquery (which will likely be faster) to fetch the dates amongst those you've generated that contains no matching record.
Assuming that your table's dates do not include a time element (ie. they are effectively recorded as at midnight), try:
select check_date
from (select :start_date + level - 1 check_date
from dual
connect by level <= 1 + :end_date - :start_date) d
where not exists
(select null from mytable where mydate = check_date)
Given a date column in order to do this you need to generate a list of all possible dates between the start and end date and then remove those dates that already exist. As Mark has already suggested the obvious way to generate the list of all dates is to use a hierarchical query. You can also do this without knowing the dates in advance though.
with the_dates as (
select date_col
from my_table
)
, date_range as (
select max(date_col) as maxdate, min(date_col) as mindate
from the_dates
)
select mindate + level
from date_range
connect by level <= maxdate - mindate
minus
select date_col
from the_dates
;
Here's a SQL Fiddle
The point of the second layer of the CTE is to have a "table" that has all the information you need but is only one row so that the hierarchical query will work correctly.

Select data from SQL DB per day

I have a table with order information in an E-commerce store. Schema looks like this:
[Orders]
Id|SubTotal|TaxAmount|ShippingAmount|DateCreated
This table does only contain data for every Order. So if a day goes by without any orders, no sales data is there for that day.
I would like to select subtotal-per-day for the last 30 days, including those days with no sales.
The resultset would look like this:
Date | SalesSum
2009-08-01 | 15235
2009-08-02 | 0
2009-08-03 | 340
2009-08-04 | 0
...
Doing this, only gives me data for those days with orders:
select DateCreated as Date, sum(ordersubtotal) as SalesSum
from Orders
group by DateCreated
You could create a table called Dates, and select from that table and join the Orders table. But I really want to avoid that, because it doesn't work good enough when dealing with different time zones and things...
Please don't laugh. SQL is not my kind of thing... :)
Create a function that can generate a date table as follows:
(stolen from http://www.codeproject.com/KB/database/GenerateDateTable.aspx)
Create Function dbo.fnDateTable
(
#StartDate datetime,
#EndDate datetime,
#DayPart char(5) -- support 'day','month','year','hour', default 'day'
)
Returns #Result Table
(
[Date] datetime
)
As
Begin
Declare #CurrentDate datetime
Set #CurrentDate=#StartDate
While #CurrentDate<=#EndDate
Begin
Insert Into #Result Values (#CurrentDate)
Select #CurrentDate=
Case
When #DayPart='year' Then DateAdd(yy,1,#CurrentDate)
When #DayPart='month' Then DateAdd(mm,1,#CurrentDate)
When #DayPart='hour' Then DateAdd(hh,1,#CurrentDate)
Else
DateAdd(dd,1,#CurrentDate)
End
End
Return
End
Then, join against that table
SELECT dates.Date as Date, sum(SubTotal+TaxAmount+ShippingAmount)
FROM [fnDateTable] (dateadd("m",-1,CONVERT(VARCHAR(10),GETDATE(),111)),CONVERT(VARCHAR(10),GETDATE(),111),'day') dates
LEFT JOIN Orders
ON dates.Date = DateCreated
GROUP BY dates.Date
declare #oldest_date datetime
declare #daily_sum numeric(18,2)
declare #temp table(
sales_date datetime,
sales_sum numeric(18,2)
)
select #oldest_date = dateadd(day,-30,getdate())
while #oldest_date <= getdate()
begin
set #daily_sum = (select sum(SubTotal) from SalesTable where DateCreated = #oldest_date)
insert into #temp(sales_date, sales_sum) values(#oldest_date, #daily_sum)
set #oldest_date = dateadd(day,1,#oldest_date)
end
select * from #temp
OK - I missed that 'last 30 days' part. The bit above, while not as clean, IMHO, as the date table, should work. Another variant would be to use the while loop to fill a temp table just with the last 30 days and do a left outer join with the result of my original query.
including those days with no sales.
That's the difficult part. I don't think the first answer will help you with that. I did something similar to this with a separate date table.
You can find the directions on how to do so here:
Date Table
I have a Log table table with LogID an index which i never delete any records. it has index from 1 to ~10000000. Using this table I can write
select
s.ddate, SUM(isnull(o.SubTotal,0))
from
(
select
cast(datediff(d,LogID,getdate()) as datetime) AS ddate
from
Log
where
LogID <31
) s right join orders o on o.orderdate = s.ddate
group by s.ddate
I actually did this today. We also got a e-commerce application. I don't want to fill our database with "useless" dates. I just do the group by and create all the days for the last N days in Java, and peer them with the date/sales results from the database.
Where is this ultimately going to end up? I ask only because it may be easier to fill in the empty days with whatever program is going to deal with the data instead of trying to get it done in SQL.
SQL is a wonderful language, and it is capable of a great many things, but sometimes you're just better off working the finer points of the data in the program instead.
(Revised a bit--I hit enter too soon)
I started poking at this, and as it hits some pretty tricky SQL concepts it quickly grew into the following monster. If feasible, you might be better off adapting THEn's solution; or, like many others advise, using application code to fill in the gaps could be preferrable.
-- A temp table holding the 30 dates that you want to check
DECLARE #Foo Table (Date smalldatetime not null)
-- Populate the table using a common "tally table" methodology (I got this from SQL Server magazine long ago)
;WITH
L0 AS (SELECT 1 AS C UNION ALL SELECT 1), --2 rows
L1 AS (SELECT 1 AS C FROM L0 AS A, L0 AS B),--4 rows
L2 AS (SELECT 1 AS C FROM L1 AS A, L1 AS B),--16 rows
L3 AS (SELECT 1 AS C FROM L2 AS A, L2 AS B),--256 rows
Tally AS (SELECT ROW_NUMBER() OVER(ORDER BY C) AS Number FROM L3)
INSERT #Foo (Date)
select dateadd(dd, datediff(dd, 0, dateadd(dd, -number + 1, getdate())), 0)
from Tally
where Number < 31
Step 1 is to build a temp table containint the 30 dates that you are concerned with. That abstract wierdness is about the fastest way known to build a table of consecutive integers; add a few more subqueries, and you can populate millions or more in mere seconds. I take the first 30, and use dateadd and the current date/time to convert them into dates. If you already have a "fixed" table that has 1-30, you can use that and skip the CTE entirely (by replacing table "Tally" with your table).
The outer two date function calls remove the time portion of the generated string.
(Note that I assume that your order date also has no time portion -- otherwise you've got another common problem to resolve.)
For testing purposes I built table #Orders, and this gets you the rest:
SELECT f.Date, sum(ordersubtotal) as SalesSum
from #Foo f
left outer join #Orders o
on o.DateCreated = f.Date
group by f.Date
I created the Function DateTable as JamesMLV pointed out to me.
And then the SQL looks like this:
SELECT dates.date, ISNULL(SUM(ordersubtotal), 0) as Sales FROM [dbo].[DateTable] ('2009-08-01','2009-08-31','day') dates
LEFT JOIN Orders ON CONVERT(VARCHAR(10),Orders.datecreated, 111) = dates.date
group by dates.date
SELECT DateCreated,
SUM(SubTotal) AS SalesSum
FROM Orders
GROUP BY DateCreated