I've got a .Net application with an attendance table which has fields for a Start and End date. I'm struggling to show a graph of attendance for a given period. I can easily find how many rows are applicable on any given day using between but I can't get my head around pivoting results so that I can graph a count of rows per day. I could run a SQL query for every day individually and then graph the results but is there any way of doing this with T-SQL that I could then use to graph with?
Edit:-
Apologies as this is the first time I've asked a question here, but as huMpty duMpty has stated the question probably needs more clarification. I've got both a startdate and enddate column in the sql db and I need to count per day if the range between these dates falls between the range of the selection criteria. e.g if I've got a start date of 2013-01-01 and end date 2013-01-10 and I report on a period of 2013-01-09 to 2013-01-11 then i'm looking at getting a result for 1 for 2013-01-09 and 1 for 2013-01-10... Hope this make more sense and thanks for your assistance
I think you have a table with start and end dates; for a given date range, you would like to know the given number of records that fall on each date.
I believe this problem may be solved with a numbers table. I created a numbers table on the fly in a stored procedure, but I recommend creating a permanent numbers table in your production code. Here's the SQL Fiddle.
Create Table Attendance (
id int primary key identity(1,1) not null
,start_date date not null
,end_date date not null
);
Go
Insert Attendance(start_date, end_date)
Values ('1/1/2013', '1/10/2013')
,('1/10/2013', '1/15/2013')
,('2/20/2013', '3/1/2013');
Go
-- Create numbers table. See: Method 3 of http://stackoverflow.com/a/1407488/772086
With Numbers(Number) As
(
Select 1 As Number
Union All
Select Number + 1 From Numbers Where Number < 10000
)
Select
AttendanceDate = Convert(date, DateAdd(day,Numbers.number, '1/1/2000'))
,AttendanceCount = Count(*)
From dbo.Attendance
Join Numbers
On Numbers.Number
Between DateDiff(day, '1/1/2000', Attendance.start_date)
And DateDiff(day, '1/1/2000', Attendance.end_date)
-- Reporting range between 1/9 and 1/11
Where DateAdd(day,Numbers.number, '1/1/2000') Between '1/9/2013'
And '1/11/2013'
Group By Convert(date, DateAdd(day,Numbers.number, '1/1/2000'))
Option(MaxRecursion 10000);
All dates are in US format (m/d/yy) - you may want to switch those to the internationalized standard (yyyy-mm-dd) in your production code.
You said you wanted a count by day in a date range. That can be done with a COUNT with a GROUP BY clause.
I don't know your schema, but a solution might look like this:
declare #MyTable table
(
ID int identity(1,1) primary key clustered,
MyDate smalldatetime
)
insert into #MyTable (MyDate)
values
('2012-12-31'), -- before the date range, so not included in results
('2013-01-10'),
('2013-01-10'), -- appears twice
('2013-01-11'), -- appears once
('2013-01-12') -- after the date range, so not included in results
select * from #MyTable
select
MyDate,
count(*)
from #MyTable
where MyDate between '2013-01-09' and '2013-01-11'
group by MyDate
Related
I am working in SQLServer database. I have the following scenario,
My application runs every weekday of the month, when it runs it puts an entry into the table 'CONTROL' into the 'businessDate' column.
Is there a way to find out what are the days the application did not run for a given month. Is it possible to achieve it in a single query ?
You need a calendar table. Then select from that table left joined to your CONTROL table on date, where the month is what you're looking for, and where the control table is null (to show just exclusions).
DECLARE #Month int = 1 //For January
SELECT BaseDate
FROM CalendarTable cal
LEFT OUTER JOIN ControlTable con
ON cal.BaseDate = con.businessDate //make sure to cast as a date if it's a datetime
WHERE MONTH(cal.BaseDate) = #Month
AND con.businessDate IS NULL
I'm looking for input on getting a COUNT of records that were 'active' in a certain date range.
CREATE TABLE member {
id int identity,
name varchar,
active bit
}
The scenario is one where "members" number fluctuate over time. So I could have linear growth where I have 10 members at the beginning of the month and 20 at the end. Currently We go off the number of CURRENTLY ACTIVE (as marked by an 'active' flag in the DB) AT THE TIME OF REPORT. - this is hardly accurate and worse, 6 months from now, my "members" figure may be substantially different than now. and Since I'm doing averages per user, if I run a report now, and 6 months from now - the figures will probably be different.
I don't think a simple "dateActive" and "dateInactive" will do the trick... due to members coming and going and coming back etc. so:
JOE may be active 12-1 and deactivated 12-8 and activated 12-20
so JOE counts as being a 'member' for 8 days and then 11 days for a total of 19 days
but the revolving door status of members means keeping a separate table (presumably) of UserId, status, date
CREATE TABLE memberstatus {
member_id int,
status bit, -- 0 for in-active, 1 for active
date date
} (adding this table would make the 'active' field in members obsolete).
In order to get a "good" Average members per month (or date range) - it seems I'd need to get a daily average, and do an average of averages over 'x' days. OR is there some way in SQL to do this already.
This extra "status" table would allow an accurate count going back in time. So in a case where you have a revenue or cost figure, that DOESN'T change or is not aggregate, it's fixed, that when you want cost/members for last June, you certainly don't want to use your current members count, you want last Junes.
Is this how it's done? I know it's one way, but it the 'better' way...
#gordon - I got ya, but I guess I was looking at records like this:
Members
1 Joe
2 Tom
3 Sue
MemberStatus
1 1 '12-01-2014'
1 0 '12-08-2014'
1 1 '12-20-2014'
In this way I only need the last record for a user to get their current status, but I can track back and "know" their status on any give day.
IF I'm understanding your method it might look like this
CREATE TABLE memberstatus {
member_id int,
active_date,
inactive_date
}
so on the 1-7th the record would look like this
1 '12-01-2014' null
and on the 8th it would change to
1 '12-01-2014' '12-08-2014'
the on the 20th
1 '12-01-2014' '12-08-2014'
1 '12-20-2014' null
Although I can get the same data out, it seems more difficult without any benefit - am i missing something?
You could also use a 2 table method to have a one-to-many relationship for working periods. For example you have a User table
User
UserID int, UserName varchar
and an Activity table that holds ranges
Activity
ActivityID int, UserID int, startDate date, (duration int or endDate date)
Then whenever you wanted information you could do something like (for example)...
SELECT User.UserName, count(*) from Activity
LEFT OUTER JOIN User ON User.UserID = Activity.UserID
WHERE startDate >= '2014-01-01' AND startDate < '2015-01-01'
GROUP BY User.UserID, User.UserName
...to get a count grouped by user (and labeled by username) of the times they were became active in 2014
I have used two main ways to accomplish what you want. First would be something like this:
CREATE TABLE [MemberStatus](
[MemberID] [int] NOT NULL,
[ActiveBeginDate] [date] NOT NULL,
[ActiveEndDate] [date] NULL,
CONSTRAINT [PK_MemberStatus] PRIMARY KEY CLUSTERED
(
[MemberID] ASC,
[ActiveBeginDate] ASC
)
Every time a member becomes active, you add an entry, and when they become inactive you update their ActiveEndDate to the current date.
This is easy to maintain, but can be hard to query. Another option is to do basically what you are suggesting. You can create a scheduled job to run at the end of each day to add entries to the table .
I recommend setting up your tables so that you store more data, but in exchange the structure supports much simpler queries to achieve the reporting you require.
-- whenever a user's status changes, we update this table with the new "active"
-- bit, and we set "activeLastModified" to today.
CREATE TABLE member {
id int identity,
name varchar,
active bit,
activeLastModified date
}
-- whenever a user's status changes, we insert a new record here
-- with "startDate" set to the current "activeLastModified" field in member,
-- and "endDate" set to today (date of status change).
CREATE TABLE memberStatusHistory {
member_id int,
status bit, -- 0 for in-active, 1 for active
startDate date,
endDate date,
days int
}
As for the report you're trying to create (average # of actives in a given month), I think you need yet another table. Pure SQL can't calculate that based on these table definitions. Pulling that data from these tables is possible, but it requires programming.
If you ran something like this once-per-day and stored it in a table, then it would be easy to calculate weekly, monthly and yearly averages:
INSERT INTO myStatsTable (date, activeSum, inactiveSum)
SELECT
GETDATE(), -- based on DBMS, eg., "current_date" for Postgres
active.count,
inactive.count
FROM
(SELECT COUNT(id) FROM member WHERE active = true) active
CROSS JOIN
(SELECT COUNT(id) FROM member WHERE active = true) inactive
I have two SQL tables as follows:
As you may note, the first table has a monthly frequency (date column), while the second table has a quarterly frequency. Here is what I would like to do:
For each issueid from table 1, I would like to look at the date, determine what is the previous end of quarter, and go fetch data from table 2 corresponding to that issue for that end of quarter, and insert it in the first table in the last two columns.
For example: take issueid 123456 and date 1/31/2014. The previous end of quarter is 12/31/2013. I would like to go to table 1, copy q_exp and q_act that correspond to that issueid and 12/31/2013, and paste it into the first table.
Of course, I would like to fill the entire first table and minimize manual inserts.
Any help would be appreciated! Thanks!
Try the following query
UPDATE issues
SET q_exp=(SELECT TOP 1 q.q_exp
FROM quarterlyTable q
WHERE q.issueid=i.issueid
AND q.[date]<=i.[date]
ORDER BY q.[date] DESC)
,q_act= (SELECT TOP 1 q.q_act
FROM quarterlyTable q
WHERE q.issueid=i.issueid
AND q.[date]<=i.[date]
ORDER BY q.[date] DESC)
FROM issues i
Here's the query:
SELECT
count(id) AS count
FROM `numbers`
GROUP BY
MONTH(created_at),
YEAR(created_at)
ORDER BY
YEAR(created_at),
MONTH(created_at)
That query throws a 'Using temporary' and 'Using filesort' when doing EXPLAIN.
Ultimately what I'm doing is looking at a table of user-submitted tracking numbers and counting the number of submitted rows a grouping the counts by month/year.
ie. In November 2008 there were 11,312 submitted rows.
UPDATE, here's the DESCRIBE for the numbers table.
id int(11) NO PRI NULL auto_increment
tracking varchar(255) YES NULL
service varchar(255) YES NULL
notes text YES NULL
user_id int(11) YES NULL
active tinyint(1) YES 1
deleted tinyint(1) YES 0
feed text YES NULL
status varchar(255) YES NULL
created_at datetime YES NULL
updated_at datetime YES NULL
scheduled_delivery date YES NULL
carrier_service varchar(255) YES NULL
Give this a shot:
SELECT COUNT(x.id)
FROM (SELECT t.id,
MONTH(t.created_at) 'created_month',
YEAR(t.created_at) 'created_year'
FROM NUMBERS t) x
GROUP BY x.created_month, x.created_year
ORDER BY x.created_month, x.created_year
It's not a good habit to use functions in the WHERE, GROUP BY and ORDER BY clauses because indexes can't be used.
...query throws a 'Using temporary' and 'Using filesort' when doing EXPLAIN.
From what I found, that's to be expected when using DISTINCT/GROUP BY.
Make sure you have a covering index over YEAR and MONTH (that is, both fields within the same index) so that the ORDER BY component of your query can use an index. This should remove the need for a filesort, although a temporary table may still be needed to handle the grouping.
SELECT
count(`id`) AS count, MONTH(`created_at`) as month, YEAR(`created_at`) as year
FROM `numbers`
GROUP BY month, year
ORDER BY created_at
This will be the best you can get, as far as I can tell. I created a table with an id and a datetime column and filled it with 10000 rows. The query above uses a sub select, but it really doesn't do you any different and has the overhead of a sub select. The resulting time for mine was 0.015s and his was 0.016s.
Make sure that you have an index on created_at, this will help your initial query out. It is pretty rare to not end up with a file sort when the group by comes about, but it may be possible in other situations. MySql's docs have an article about this if you feel so inclined. I do not see how those methods can be applied here, with the information you have provided.
Whenever MySQL has to do work in memory, and that work exceeds the available amount (innodb_buffer_pool_size), it starts having to use the disk to store temporary work. You could increase the variable I mentioned, but setting it too high could cause performance problems in other areas.
If you're running a dedicated server, set it to ~50-75%.
The best method would be creating a helper column that would contain numberic values of YEAR and MONTH concatenated together:
YEAR(created_at) * 100 + MONTH(created_at)
Grouping on this column would use INDEX FOR GROUP BY.
However, you can create two helper tables, the first one containing reasonable number of years (say, from 1900 to 2100), the second one containing months (from 0 to 11), and use these tables to generate the sets:
SELECT (
SELECT COUNT(*)
FROM numbers
WHERE created_at >= '1900-01-01' + INTERVAL y YEAR + INTERVAL m MONTH
AND created_at < '1900-01-01' + INTERVAL y YEAR + INTERVAL m + 1 MONTH
)
FROM year_table
CROSS JOIN
month_table
WHERE y BETWEEN 2008 AND 2010
I'm sorry, but I have to disagree with the other answers.
I think what you need is to add an index to your table, preferably a covering index.
If you add an index on the columns you are searching on (created_at) and also on the columns you want to get a result from (id) then it will be dramatically faster then before.
The reason why you are using a temp table is because you use a group by.
To speed up the group by, you can change the MySQL server settings to increase the size of the tmp table and the max heap table size so that the temp table will be in memory.
I have a table with records that look like this:
CREATE TABLE sample (
ix int unsigned auto_increment primary key,
start_active datetime,
last_active datetime
);
I need to know how many records were active on each of the last 30 days. The days should also be sorted incrementing so they are returned oldest to newest.
I'm using MySQL and the query will be run from PHP but I don't really need the PHP code, just the query.
Here's my start:
SELECT COUNT(1) cnt, DATE(?each of last 30 days?) adate
FROM sample
WHERE adate BETWEEN start_active AND last_active
GROUP BY adate;
Do an outer join.
No table? Make a table. I always keep a dummy table around just for this.
create table artificial_range(
id int not null primary key auto_increment,
name varchar( 20 ) null ) ;
-- or whatever your database requires for an auto increment column
insert into artificial_range( name ) values ( null )
-- create one row.
insert into artificial_range( name ) select name from artificial_range;
-- you now have two rows
insert into artificial_range( name ) select name from artificial_range;
-- you now have four rows
insert into artificial_range( name ) select name from artificial_range;
-- you now have eight rows
--etc.
insert into artificial_range( name ) select name from artificial_range;
-- you now have 1024 rows, with ids 1-1024
Now make it convenient to use, and limit it to 30 days, with a view:
Edit: JR Lawhorne notes:
You need to change "date_add" to "date_sub" to get the previous 30 days in the created view.
Thanks JR!
create view each_of_the_last_30_days as
select date_sub( now(), interval (id - 1) day ) as adate
from artificial_range where id < 32;
Now use this in your query (I haven't actually tested your query, I'm just assuming it works correctly):
Edit: I should be joining the other way:
SELECT COUNT(*) cnt, b.adate
FROM each_of_the_last_30_days b
left outer join sample a
on ( b.adate BETWEEN a.start_active AND a.last_active)
GROUP BY b.adate;
SQL is great at matching sets of values that are stored in the database, but it isn't so great at matching sets of values that aren't in the database. So one easy workaround is to create a temp table containing the values you need:
CREATE TEMPORARY TABLE days_ago (d SMALLINT);
INSERT INTO days_ago (d) VALUES
(0), (1), (2), ... (29), (30);
Now you can compare a date that is d days ago to the span between start_active and last_active of each row. Count how many matching rows in the group per value of d and you've got your count.
SELECT CURRENT_DATE - d DAYS, COUNT(*) cnt,
FROM days_ago
LEFT JOIN sample ON (CURRENT_DATE - d DAYS BETWEEN start_active AND last_active)
GROUP BY d
ORDER BY d DESC; -- oldest to newest
Another note: you can't use column aliases defined in your select-list in expressions until you get to the GROUP BY clause. Actually, in standard SQL you can't use them until the ORDER BY clause, but MySQL supports using aliases in GROUP BY and HAVING clauses as well.
Turn the date into a unix timestamp, which is seconds, in your query and then just look for the difference to be <= the number of seconds in a month.
You can find more information here:
http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_unix-timestamp
If you need help with the query please let me know, but MySQL has nice functions for dealing with datetime.
[Edit] Since I was confused as to the real question, I need to finish the lawn but before I forget I want to write this down.
To get a count of the number by day you will want your where clause to be as I described above, to limit to the past 30 days, but you will need to group by day, and so select by converting each start to a day of the month and then do a count of those.
This assumes that each use will be limited to one day, if the start and end dates can span several days then it will be trickier.