So this is somewhat of a common question on here but I haven't found an answer that really suits my specific needs. I have 2 tables. One has a list of ProjectClosedDates. The other table is a calendar table that goes through like 2025 which has columns for if the row date is a weekend day and also another column for is the date a holiday.
My end goal is to find out based on the ProjectClosedDate, what date is 5 business days post that date. My idea was that I was going to use the Calendar table and join it to itself so I could then insert a column into the calendar table that was 5 Business days away from the row-date. Then I was going to join the Project table to that table based on ProjectClosedDate = RowDate.
If I was just going to check the actual business-date table for one record, I could use this:
SELECT actual_date from
(
SELECT actual_date, ROW_NUMBER() OVER(ORDER BY actual_date) AS Row
FROM DateTable
WHERE is_holiday= 0 and actual_date > '2013-12-01'
ORDER BY actual_date
) X
WHERE row = 65
from here:
sql working days holidays
However, this is just one date and I need a column of dates based off of each row. Any thoughts of what the best way to do this would be? I'm using SQL-Server Management Studio.
Completely untested and not thought through:
If the concept of "business days" is common and important in your system, you could add a column "Business Day Sequence" to your table. The column would be a simple unique sequence, incremented by one for every business day and null for every day not counting as a business day.
The data would look something like this:
Date BDAY_SEQ
========== ========
2014-03-03 1
2014-03-04 2
2014-03-05 3
2014-03-06 4
2014-03-07 5
2014-03-08
2014-03-09
2014-03-10 6
Now it's a simple task to find the N:th business day from any date.
You simply do a self join with the calendar table, adding the offset in the join condition.
select a.actual_date
,b.actual_date as nth_bussines_day
from DateTable a
join DateTable b on(
b.bday_seq = a.bday_seq + 5
);
Related
I have a sqlite3 database maintained on an AWS exchange that is regularly updated by a Python script. One of the things it tracks is when any team generates a new post for a given topic. The entries look something like this:
id
client
team
date
industry
city
895
acme industries
blueteam
2022-06-30
construction
springfield
I'm trying to create a table that shows me how many entries for construction occur each day. Right now, the entries with data populate, but they exclude dates with no entries. For example, if I search for just
SELECT date, count(id) as num_records
from mytable
WHERE industry = "construction"
group by date
order by date asc
I'll get results that looks like this:
date
num_records
2022-04-01
3
2022-04-04
1
How can I make sqlite output like this:
date
num_records
2022-04-02
3
2022-04-02
0
2022-04-03
0
2022-04-04
1
I'm trying to generate some graphs from this data and need to be able to include all dates for the target timeframe.
EDIT/UPDATE:
The table does not already include every date; it only includes dates relevant to an entry. If no team posts work on a day, the date column will jump from day 1 (e.g. 2022-04-01) to day 3 (2022-04-03).
Given that your "mytable" table contains all dates you need as an assumption, you can first select all of your dates, then apply a LEFT JOIN to your own query, and map all resulting NULL values for the "num_records" field to "0" using the COALESCE function.
WITH cte AS (
SELECT date,
COUNT(id) AS num_records
FROM mytable
WHERE industry = "construction"
GROUP BY date
ORDER BY date
)
SELECT dates.date,
COALESCE(cte.num_records, 0) AS num_records
FROM (SELECT date FROM mytable) dates
LEFT JOIN cte
ON dates.date = cte.date
I am creating a customer activity by day table, which requires 9 CTEs.
The first table I want to cross join all customer unique IDs with the dates of a calendar table. So there will be multiple rows with the same unique ID for each day.
The problem is making sure the days are consecutive, regardless of the dates in the following CTEs.
This is a shortened example of what it would look like this:
GUID DATE CONDITIONS
1 3/13/2015 [NULL]
1 3/14/2015 Y
1 3/15/2015 [NULL]
....
1 9/2/2020 Y
2 4/15/2015 Y
2 4/16/2015 [NULL]
2 4/17.2015 [NULL]
2 4/18/2015 Y
...
2 9/2/2020 [NULL]
And so on - so that each customers has consecutive dates with their GUID, beginning with the creation date of their account (i.e. 3/13/2015) and ending on the current date.
the create date is on Table 1 with the unique ID, and I'm joining it with a date table.
My problem is that I can't get the query to run with a minimum create date per unique ID. Because if I don't create a minimum start date, the query runs forever (it's trying to create every unique ID for every consecutive date, even before the customer account was created.)
This is the code I have now.
Can anyone tell me if I have made the min. create date right? It's still just timing out when I run the query.
with
cte_carrier_guid (carrier_guid, email, date, carrier_id) as
(
SELECT
guid as carrier_guid
,mc.email
,dt2.date as date
,mc.id as carrier_id
FROM ctms_db_public.msd_carrier mc
CROSS JOIN public.dim_calendar dt2
WHERE dt2.date <= CURRENT_DATE
AND mc.created_at >= dt2.date
GROUP BY guid, mc.id, dt2."date", mc.email
ORDER BY guid, dt2.date asc
)
Select top 10 * from cte_carrier_guid
Here:
dt2.date <= CURRENT_DATE AND mc.created_at >= dt2.date
Since you want dates between the creation date of the user and today, you probably want the inequality condition on the creation date the other way around. I find it easier to follow when we put the lower bound first:
dt2.date >= mc.created_at AND dt2.date <= CURRENT_DATE
Other things about the query:
You want an INNER JOIN in essence, so use that instead of CROSS JOIN ... WHERE; it is clearer
ORDER BY in a cte makes no sense to me
Do you really need GROUP BY? The columns in the SELECT clause are the same as in the GROUP BY, so all this does is remove potential duplicates (but why would there be duplicates?)
You could probably phrase the cte as:
SELECT ...
FROM ctms_db_public.msd_carrier mc
INNER JOIN public.dim_calendar dt2 ON dt2.date >= mc.created_at
WHERE dt2.date <= CURRENT_DATE
I am looking for some general advice rather than a solution. My problem is that I have a list of dates per person where due to administrative procedures, a person may have multiple records stored for this one instance, yet the date recorded is when the data was entered in as this person is passed through the paper trail. I understand this is quite difficult to explain so I'll give an example:
Person Date Audit
------ ---- -----
1 2000-01-01 A
1 2000-01-01 B
1 2000-01-02 C
1 2003-04-01 A
1 2003-04-03 A
where I want to know how many valid records a person has by removing annoying audits that have recorded the date as the day the data was entered, rather than the date the person first arrives in the dataset. So for the above person I am only interested in:
Person Date Audit
------ ---- -----
1 2000-01-01 A
1 2003-04-01 A
what makes this problem difficult is that I do not have the luxury of an audit column (the audit column here is just to present how to data is collected). I merely have dates. So one way where I could crudely count real events (and remove repeat audit data) is to look at individual weeks within a persons' history and if a record(s) exists for a given week, add 1 to my counter. This way even though there are multiple records split over a few days, I am only counting the succession of dates as one record (which after all I am counting by date).
So does anyone know of any db2 functions that could help me solve this problem?
If you can live with standard weeks it's pretty simple:
select
person, year(dt), week(dt), min(dt), min(audit)
from
blah
group by
person, year(dt), week(dt)
If you need seven-day ranges starting with the first date you'd need to generate your own week numbers, a calendar of sorts, e.g. like so:
with minmax(mindt, maxdt) as ( -- date range of the "calendar"
select min(dt), max(dt)
from blah
),
cal(dt,i) as ( -- fill the range with every date, count days
select mindt, 0
from minmax
union all
select dt+1 day , i+1
from cal
where dt < (select maxdt from minmax) and i < 100000
)
select
person, year(blah.dt), wk, min(blah.dt), min(audit)
from
(select dt, int(i/7)+1 as wk from cal) t -- generate week numbers
inner join
blah
on t.dt = blah.dt
group by person, year(blah.dt), wk
I have a rather interesting problem which I first thought would be straight-forward, but it turned out to be more complicated.
I have data like this:
Date User ID
2012-10-11 a
2012-10-11 b
2012-10-12 c
2012-10-12 d
2012-10-13 e
2012-10-14 b
2012-10-14 e
... ...
Each row has a Date, User ID couple which indicates that that user was active on that day. A user can appear on multiple dates and a date will have multiple users -- just like in the example. I have millions of rows like this which cover a time range of about 90 days.
Here's the question: For each day, I want to get the number of users who have not been active for the past 10 days. For instance, if the user "a" was active on 2012-05-31 and but hasn't been active on any of the days between 06-01 and 06-10, I want to count this user on 6/10. I wouldn't count him again on the following days though unless he becomes active and disappears again.
Can I do this in SQL or would I need to some kind of script to organize the data the way I want. What would be your recommendations? I use Hive.
Thank you so much!
I think you can do this in Hive-compatible SQL. Here is the idea.
For each user/date get the next date for the user.
Discard the original record if the next is less than 10 days after the current one.
Add 10 to the date
Aggregate and count
I am not sure of all the Hive functions for things like date. Here is an example of how to do it:
select date+10, count(*)
from (select t.userid, t.date,
min(case when tnext.date > t.date then tnext.date end) as nextdate
from t left outer join
t tnext
on t.userid = tnext.userid
group by t.userid, t.date
) t
where nextdate is null or nextdate - date >= 10
group by date+10;
Note that the inner subquery would be better written using:
on t.userid = tnext.userid and t2.date > t.date
However, I don't know if Hive supports such a join (it doesn't support non-equijoins and it not clear about whether one or all clauses have to be equal).
I would like to know what is the best way of creating a report that will be grouped by the last 7 days - but not every day i have data. for example:
08/01/10 | 0
08/02/10 | 5
08/03/10 | 6
08/04/10 | 10
08/05/10 | 0
08/06/10 | 11
08/07/10 | 1
is the only option is to create a dummy table with those days and join them altogether?
thank you
Try something like this
WITH LastDays (calc_date)
AS
(SELECT DATEADD(DAY, DATEDIFF(DAY, 0, CURRENT_TIMESTAMP) - 6, 0)
UNION ALL
SELECT DATEADD(DAY, 1, calc_date)
FROM LastDays
WHERE DATEADD(DAY, 1, calc_date) < CURRENT_TIMESTAMP)
SELECT ...
FROM LastDays l LEFT JOIN (YourQuery) t ON (l.cal_date = t.YourDateColumn);
Many people will suggest methods for dynamically creating a range of dates that you can then join against. This will certainly work but in my experience a calendar table is the way to go. This will make the SQL trivial and generic at the cost of maintaining the calendar table.
At some point in the future someone will come along and ask for another report that excludes weekends. You then have to make your dynamic days generation account for weekends. Then someone will ask for working-days excluding public-holidays at which point you have no choice but to create a calendar table.
I would suggest you bite the bullet and create a calendar table to join against. Pre-populate it with every date and if you want to think ahead then add columns for "Working Day" and maybe even week number if your company uses a non-standard week-number for reporting
You don't mention the specific language (please do for a more detailed answer), but most versions of sql have a function for the current date (GetDate(), for instance). You could take that date, subtract x (7) days and build your WHERE statement like that.
Then you could GROUP BY the day-part of that date.
select the last 7 transactions and left join it with your query and then group by the date column. hope this helps.