Joining date with datetime and values per day - sql

I am writing a query so that I can fill my facttable. I have a table which registers the weather average per day, registered at 23:59:00 of each day (date).
I have another table in which climate control data of different rooms are registered, per minute (datetime).
And I also have a time dimension available in another database.
I want to fill my facttable with all the available timeKeys combined with all the data from my climate control table and my weather table.
I'm sorry for my English, it isn't my mother tongue.
So, to find the matching timeKey for the date values I wrote this query:
SELECT t.timeKey AS WeathertimeKey,
weather.date AS date,
weather.temperature,
weather.rainAmountMM,
weather.windDirection,
weather.windSpeed
FROM StarSchema.dbo.timeDim t, weather
WHERE DATEPART(mm, t.DATE) = DATEPART(mm, weather.date)
AND DATEPART(dd, t.DATE) = DATEPART(dd, weather.date)
AND DATEPART(Hour, t.DATE) = '23'
AND DATEPART(MINUTE, t.DATE) = '59'
RESULT: Result
My time dimension has a timeKey for every minute in 2015: timeDimension
The facttable I am trying to fill: facttable
My solution for filling the facttable was creating a view with the corresponding timeKey per day and then joining that view in my main query.
SELECT
t.timeKey as timeKey,
rt1.roomId AS roomKey,
1 AS roomDataKey,
1 AS usageKey,
1 AS knmiKey,
rt1.temperature AS temperature,
rt1.locWindow AS locWindow,
rt1.locDoor AS locDoor,
rh1.turnedOn AS turnedOn,
rh1.temperature AS temperatureHeater,
s.storyTemp AS storyTemp,
s.storyHumidity AS storyHumidity,
vw.temperature AS temperatureOutside,
vw.rainAmountMM AS rainAmountMM,
vw.windSpeed AS windSpeed,
vw.windDirection AS windDirection,
vu.gasM3 AS gasM3,
vu.electricityKWH AS electricityKWH
FROM StarSchema.dbo.timeDim t
INNER JOIN roomTemperature1 rt1 ON rt1.date = t.DATE
INNER JOIN roomHeating1 rh1 ON rt1.date = rh1.date
INNER JOIN story s ON s.date = rt1.date
INNER JOIN vw_timeKeyWeatherDay vw ON t.timeKey = vw.WeathertimeKey
INNER JOIN vw_timeKeyUsageDay vu ON t.timeKey = vu.UsagetimeKey
The result is as follows: result2
So now it only uses the timeKey of 23:59 of everyday.
I want the complete days in there, but how do I do this?
Can someone help me out?
And my apologies for my use of the English language, again.
I did my best :-)

If I understand your question property, you want to match two date columns which have a different level of precision: one is per minute the other one is per day. What I suggest is a query which stores the yyyy.mm.dd only for both. Then when you join you get a matching record every time.
You can do that by adding the number of days that distance each of your dates from the date 0 in SQL server
DECLARE #DVal DATE
DECLARE #DTVal DATETIME
SET #DVal = '2018-01-18'
SET #DTVal = '2018-01-18 12:02:01.003'
SELECT #DVal
SELECT #DTVal
SELECT DATEDIFF(D,0,#DTVal)
SELECT DATEDIFF(D,0,#DVal)
SELECT DATEADD(D,(DATEDIFF(D,0,#DTVal) ),0)
SELECT DATEADD(D,(DATEDIFF(D,0,#DVal) ),0)
Comments about the code above:
First I declare the variables, one DATE and one DATETIME and I give them slightly different values which are less than a day.
Then I select them so that we can see they are different
2018-01-18
2018-01-18 12:02:01.003
Then I select the difference in days between each date and 0, and we have the same number of days
43116
43116
Then I add this difference to the date 0, and we end up with two datetime values which are identical
2018-01-18 00:00:00.000
2018-01-18 00:00:00.000
I hope I have answered your question. Please comment on my answer if I have not. At least this is a starting point. If your goal is to get the complete range of minutes per day you can create two calculated columns, one based on the current date and the other one based on the current date +1 day, and join against the time table with a BETWEEEN ON clause, etc.

Related

SQL Server : average count of alerts per day, not including days with no alerts

I have a table that acts as a message log, with the two key tables being TIMESTAMP and TEXT. I'm working on a query that grabs all alerts (from TEXT) for the past 30 days (based on TIMESTAMP) and gives a daily average for those alerts.
Here is the query so far:
--goback 30 days start at midnight
declare #olderdate as datetime
set #olderdate = DATEADD(Day, -30, DATEDIFF(Day, 0, GetDate()))
--today at 11:59pm
declare #today as datetime
set #today = dateadd(ms, -3, (dateadd(day, +1, convert(varchar, GETDATE(), 101))))
print #today
--Grab average alerts per day over 30 days
select
avg(x.Alerts * 1.0 / 30)
from
(select count(*) as Alerts
from MESSAGE_LOG
where text like 'The process%'
and text like '%has alerted%'
and TIMESTAMP between #olderdate and #today) X
However, I want to add something that checks whether there were any alerts for a day and, if there are no alerts for that day, doesn't include it in the average. For example, if there are 90 alerts for a month but they're all in one day, I wouldn't want the average to be 3 alerts per day since that's clearly misleading.
Is there a way I can incorporate this into my query? I've searched for other solutions to this but haven't been able to get any to work.
This isn't written for your query, as I don't have any DDL or sample data, thus I'm going to provide a very simple example instead of how you would do this.
USE Sandbox;
GO
CREATE TABLE dbo.AlertMessage (ID int IDENTITY(1,1),
AlertDate date);
INSERT INTO dbo.AlertMessage (AlertDate)
VALUES('20190101'),('20190101'),('20190105'),('20190110'),('20190115'),('20190115'),('20190115');
GO
--Use a CTE to count per day:
WITH Tots AS (
SELECT AlertDate,
COUNT(ID) AS Alerts
FROM dbo.AlertMessage
GROUP BY AlertDate)
--Now the average
SELECT AVG(Alerts*1.0) AS DayAverage
FROM Tots;
GO
--Clean up
DROP TABLE dbo.AlertMessage;
You're trying to compute a double-aggregate: The average of daily totals.
Without using a CTE, you can try this as well, which is generalized a bit more to work for multiple months.
--get a list of events per day
DECLARE #Event TABLE
(
ID INT NOT NULL IDENTITY(1, 1)
,DateLocalTz DATE NOT NULL--make sure to handle time zones
,YearLocalTz AS DATEPART(YEAR, DateLocalTz) PERSISTED
,MonthLocalTz AS DATEPART(MONTH, DateLocalTz) PERSISTED
)
/*
INSERT INTO #Event(EntryDateLocalTz)
SELECT DISTINCT CONVERT(DATE, TIMESTAMP)--presumed to be in your local time zone because you did not specify
FROM dbo.MESSAGE_LOG
WHERE UPPER([TEXT]) LIKE 'THE PROCESS%' AND UPPER([TEXT]) LIKE '%HAS ALERTED%'--case insenitive
*/
INSERT INTO #Event(DateLocalTz)
VALUES ('2018-12-31'), ('2019-01-01'), ('2019-01-01'), ('2019-01-01'), ('2019-01-12'), ('2019-01-13')
--get average number of alerts per alerting day each month
-- (this will not return months with no alerts,
-- use a LEFT OUTER JOIN against a month list table if you need to include uneventful months)
SELECT
YearLocalTz
,MonthLocalTz
,AvgAlertsOfAlertingDays = AVG(CONVERT(REAL, NumDailyAlerts))
FROM
(
SELECT
YearLocalTz
,MonthLocalTz
,DateLocalTz
,NumDailyAlerts = COUNT(*)
FROM #Event
GROUP BY YearLocalTz, MonthLocalTz, DateLocalTz
) AS X
GROUP BY YearLocalTz, MonthLocalTz
ORDER BY YearLocalTz ASC, MonthLocalTz ASC
Some things to note in my code:
I use PERSISTED columns to get the month and year date parts (because I'm lazy when populating tables)
Use explicit CONVERT to escape integer math that rounds down decimals. Multiplying by 1.0 is a less-readable hack.
Use CONVERT(DATE, ...) to round down to midnight instead of converting back and forth between strings
Do case-insensitive string searching by making everything uppercase (or lowercase, your preference)
Don't subtract 3 milliseconds to get the very last moment before midnight. Change your semantics to interpret the end of a time range as exclusive, instead of dealing with the precision of your datatypes. The only difference is using explicit comparators (i.e. use < instead of <=). Also, DATETIME resolution is 1/300th of a second, not 3 milliseconds.
Avoid using built-in keywords as column names (i.e. "TEXT"). If you do, wrap them in square brackets to avoid ambiguity.
Instead of dividing by 30 to get the average, divide by the count of distinct days in your results.
select
avg(x.Alerts * 1.0 / x.dd)
from
(select count(*) as Alerts, count(distinct CAST([TIMESTAMP] AS date)) AS dd
...

SQL - Get data between two dates grouped by week

This question might have been solved many times or even asked many times. But as I am not a savvy in SQL, I am not able to figure out things found on the internet. Like I am not able to tweak Queries from the Internet to my needs.
And here comes my need
I have a Table named Orders Containing Fields like OrderId, OrderDate etc.
I need to generate an Excel Sheet. The sheet will have the count of orders grouped by week.
(Like how many orders placed within that week)
The user can choose the year in which he/she needs the report for.
So if the user chooses the current year then I need to generate an excel report containing data from Jan 1 to today grouped by week.
If the user chooses any other year(maybe previous years) then I need to generate a report containing all the data for that year grouped by week.
Currently, I am looking for an SQL query that returns data like this(expected output)
Week Date Range Total No of Orders
-----+--------------------------+-------------------
week#1 2018-01-01 - 2018-01-07 10
week#2 2018-01-08 - 2018-01-14 0
week#3 2018-01-15 - 2018-01-21 1
How can I write a query to achieve the same?
Looking for expert advice...
You need to use CTE recursive write calendar by week number,then Orders LEFT JOIN on CTE calendar table get COUNT.
Note:
variable #Dt mock which year you want to start.
Query look like this.
DECLARE #Dt date = '2018-01-01'
;WITH CTE(Dt,maxD) AS (
SELECT DATEPART(ww,#Dt) Dt, DATEPART(ww,MAX(OrderDate)) maxD
FROM Orders
UNION ALL
SELECT (Dt +1) Dt,maxD
FROM CTE
WHERE (Dt +1) <= maxD
)
SELECT CONCAT('week#',c.Dt) 'week',
CONCAT(
CONVERT(char(10),dateadd(week,c.Dt-1, DATEADD(wk, DATEDIFF(wk,-1,DATEADD(yy, DATEDIFF(yy,0,getdate()), 0)), 0)),126)
,'-'
, CONVERT(char(10),dateadd(week,c.Dt, DATEADD(wk, DATEDIFF(wk,-1,DATEADD(yy, DATEDIFF(yy,0,getdate()), 0)), 0)),126)) 'Date Range',
COUNT(o.OrderDate) 'Total No of Orders'
FROM CTE c
LEFT JOIN Orders o on c.Dt = DATEPART(ww,o.OrderDate)
GROUP BY c.Dt
sqlfiddle:http://sqlfiddle.com/#!18/8f089/40

Display a rolling 12 weeks chart in SSRS report

I am calling the data query in ssrs like this:
SELECT * FROM [DATABASE].[dbo].[mytable]
So, the current week is the last week from the query (e.g. 3/31 - 4/4) and each number represents the week before until we have reached the 12 weeks prior to this week and display in a point chart.
How can I accomplish grouping all the visits for all locations by weeks and adding it to the chart?
I suggest updating your SQL query to Group by a descending Dense_Rank of DatePart(Week,ARRIVED_DATE). In this example, I have one column for Visits because I couldn't tell which columns you were using to get your Visit count:
-- load some test data
if object_id('tempdb..#MyTable') is not null
drop table #MyTable
create table #MyTable(ARRIVED_DATE datetime,Visits int)
while (select count(*) from #MyTable) < 1000
begin
insert into #MyTable values
(dateadd(day,round(rand()*100,0),'2014-01-01'),round(rand()*1000,0))
end
-- Sum Visits by WeekNumber relative to today's WeekNumber
select
dense_rank() over(order by datepart(week,ARRIVED_DATE) desc) [Week],
sum(Visits) Visits
from #MyTable
where datepart(week,ARRIVED_DATE) >= datepart(week,getdate()) - 11
group by datepart(week,ARRIVED_DATE)
order by datepart(week,ARRIVED_DATE)
Let me know if I can provide any more detail to help you out.
You are going to want to do the grouping of the visits within SQL. You should be able to add a calculated column to your table which is something like WorkWeek and it should be calculated on the days difference from a certain day such as Sunday. This column will then by your X value rather than the date field you were using.
Here is a good article that goes into first day of week: First Day of Week

Window moving average in sql server

I am trying to create a function that computes a windowed moving average in SQLServer 2008. I am quite new to SQL so I am having a fair bit of difficulty. The data that I am trying to perform the moving average on needs to be grouped by day (it is all timestamped data) and then a variable moving average window needs to be applied to it.
I already have a function that groups the data by day (and #id) which is shown at the bottom. I have a few questions:
Would it be better to call the grouping function inside the moving average function or should I do it all at once?
Is it possible to get the moving average for the dates input into the function, but go back n days to begin the moving average so that the first n days of the returned data will not have 0 for their average? (ie. if they want a 7 day moving average from 01-08-2011 to 02-08-2011 that I start the moving average calculation on 01-01-2011 so that the first day they defined has a value?)
I am in the process of looking into how to do the moving average, and know that a moving window seems to be the best option (currentSum = prevSum + todayCount - nthDayAgoCount) / nDays but I am still working on figuring out the SQL implementation of this.
I have a grouping function that looks like this (some variables removed for visibility purposes):
SELECT
'ALL' as GeogType,
CAST(v.AdmissionOn as date) as dtAdmission,
CASE WHEN #id IS NULL THEN 99 ELSE v.ID END,
COUNT(*) as nVisits
FROM dbo.Table1 v INNER JOIN dbo.Table2 t ON v.FSLDU = t.FSLDU5
WHERE v.AdmissionOn >= '01-01-2010' AND v.AdmissionOn < DATEADD(day,1,'02-01-2010')
AND v.ID = Coalesce(#id,ID)
GROUP BY
CAST(v.AdmissionOn as date),
CASE WHEN #id IS NULL THEN 99 ELSE v.ID END
ORDER BY 2,3,4
Which returns a table like so:
ALL 2010-01-01 1 103
ALL 2010-01-02 1 114
ALL 2010-01-03 1 86
ALL 2010-01-04 1 88
ALL 2010-01-05 1 84
ALL 2010-01-06 1 87
ALL 2010-01-07 1 82
EDIT: To answer the first question I asked:
I ended up creating a function which declared a temporary table and inserted the results from the count function into it, then used the example from user662852 to compute the moving average.
Take the hardcoded date range out of your query. Write the output (like your sample at the end) to a temp table (I called it #visits below).
Try this self join to the temp table:
Select list.dtadmission
, AVG(data.nvisits) as Avg
, SUM(data.nvisits) as sum
, COUNT(data.nvisits) as RollingDayCount
, MIN(data.dtadmission) as Verifymindate
, MAX(data.dtadmission) as Verifymaxdate
from #visits as list
inner join #visits as data
on list.dtadmission between data.dtadmission and DATEADD(DD,6,data.dtadmission) group by list.dtadmission
EDIT: I didn't have enough room in Comments to say this in response to your question:
My join is "kinda cartesian" because it uses a between in the join constraint. Each record in list is going up against every other record, and then I want the ones where the date I report is between a lower bound of (-7) days and today. Every data date is available to list date, this is the key to your question. I could have written the join condition as
list.dtadmission between DATEADD(DD,-6,data.dtadmission) and data.dtadmission
But what really happened was I tested it as
list.dtadmission between DATEADD(DD,6,data.dtadmission) and data.dtadmission
Which returns no records because the syntax is "Between LOW and HIGH". I facepalmed on 0 records and swapped the arguments, that's all.
Try the following, see what I mean: This is the cartesian join for just one listdate:
SELECT
list.[dtAdmission] as listdate
,data.[dtAdmission] as datadate
,data.nVisits as datadata
,DATEADD(dd,6,list.dtadmission) as listplus6
,DATEADD(dd,6,data.dtAdmission ) as datapplus6
from [sandbox].[dbo].[admAvg] as list inner join [sandbox].[dbo].[admAvg] as data
on
1=1
where list.dtAdmission = '5-Jan-2011'
Compare this to the actual join condition
SELECT
list.[dtAdmission] as listdate
,data.[dtAdmission] as datadate
,data.nVisits as datadata
,DATEADD(dd,6,list.dtadmission) as listplus6
,DATEADD(dd,6,data.dtAdmission ) as datapplus6
from [sandbox].[dbo].[admAvg] as list inner join [sandbox].[dbo].[admAvg] as data
on
list.dtadmission between data.dtadmission and DATEADD(DD,6,data.dtadmission)
where list.dtAdmission = '5-Jan-2011'
See how list date is between datadate and dataplus6 in all the records?

MySQL to get the count of rows that fall on a date for each day of a month

I have a table that contains a list of community events with columns for the days the event starts and ends. If the end date is 0 then the event occurs only on the start day. I have a query that returns the number of events happening on any given day:
SELECT COUNT(*) FROM p_community e WHERE
(TO_DAYS(e.date_ends)=0 AND DATE(e.date_starts)=DATE('2009-05-13')) OR
(DATE('2009-05-13')>=DATE(e.date_starts) AND DATE('2009-05-13')<=DATE(e.date_ends))
I just sub in any date I want to test for "2009-05-13".
I need to be be able to fetch this data for every day in an entire month. I could just run the query against each day one at a time, but I'd rather run one query that can give me the entire month at once. Does anyone have any suggestions on how I might do that?
And no, I can't use a stored procedure.
Try:
SELECT COUNT(*), DATE(date) FROM table WHERE DATE(dtCreatedAt) >= DATE('2009-03-01') AND DATE(dtCreatedAt) <= DATE('2009-03-10') GROUP BY DATE(date);
This would get the amount for each day in may 2009.
UPDATED: Now works on a range of dates spanning months/years.
Unfortunately, MySQL lacks a way to generate a rowset of given number of rows.
You can create a helper table:
CREATE TABLE t_day (day INT NOT NULL PRIMARY KEY)
INSERT
INTO t_day (day)
VALUES (1),
(2),
…,
(31)
and use it in a JOIN:
SELECT day, COUNT(*)
FROM t_day
JOIN p_community e
ON day BETWEEN DATE(e.start) AND IF(DATE(e.end), DATE(e.end), DATE(e.start))
GROUP BY
day
Or you may use an ugly subquery:
SELECT day, COUNT(*)
FROM (
SELECT 1 AS day
UNION ALL
SELECT 2 AS day
…
UNION ALL
SELECT 31 AS day
) t_day
JOIN p_community e
ON day BETWEEN DATE(e.start) AND IF(DATE(e.end), DATE(e.end), DATE(e.start))
GROUP BY
day