This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
SQL Query Group By Datetime problem?
I am working on an application with 2 steps.
Scan logs and persist data from them in a database.
Read data from database and visualize the data.
The first step is more or less finished. I try to explain the background and my reguirement with the second step.
Each row in the database consists of some info like logdate, logfilename, LogType, logMessage etc. So I want for example write SQL that summarize a given LogType per day.
This is the columns:
[LogDate] [datetime] NOT NULL,
[Computer] [varchar](50) NOT NULL,
[Type] [varchar](50) NOT NULL,
[FileName] [varchar](100) NOT NULL,
[LineNo] [int] NOT NULL,
[UserName] [varchar](50) NOT NULL,
[Message] [varchar](max) NOT NULL,
I imagine the output could be like this if I want to show all rows with Type=TDBError:
Date Sum
2012-10-01 3
2012-10-02 12
2012-10-03 40
2012-10-05 24
2012-10-06 18
So at date 2012-10-01 there was 3 rows in DB where Type=TDBError. At date 2012-10-02 there was 12 etc.
How should I write the SQL for this ?
Assuming SQL Server 2008 or newer:
SELECT
[Date] = CONVERT(DATE, LogDate),
[Sum] = COUNT(*)
FROM dbo.Log_Table_Name
WHERE [Type] = 'DBError'
GROUP BY CONVERT(DATE, LogDate)
ORDER BY [Date];
GROUP BY DATEPART(day, date), DATEPART(month, date), DATEPART(year, date)
You can do a group by the parts of the time
GROUP BY date(log_date), month(log_date), day(log_date)
Select Cast(FLOOR(CAST(DATE as float)) as DateTime) as Date,COUNT(*) as [SUM]
from Log_Table_Name
Group by Cast(FLOOR(CAST(DATE as float)) as DateTime)
order by Cast(FLOOR(CAST(DATE as float)) as DateTime)
Related
I have a float datatype that is actually a date. I need to use it in the condition to obtain the data for the last 10 minutes.
I used CAST to convert from float to datetime. and then DATEADD to collect the last 10 minutes data but its not working.
select CAST(StartTime AS DATETIME) as StartTime
,CAST(endtime AS DATETIME) as EndTime
from BIORADFM_TASK_AUDIT
where CAST(StartTime AS DATETIME) >= DATEADD(minute, 10, GETDATE());
Result:
StartTime EndTime
----------------------- -----------------------
2017-10-12 16:57:06.997 2017-10-12 16:57:06.997
2017-10-12 06:06:59.997 2017-10-12 06:06:59.997
2017-10-12 06:06:47.997 2017-10-12 06:06:47.997
2017-10-11 16:04:53.000 2017-10-11 16:04:53.000
It´s showing data for the next two days, instead of the last 10 minutes starting from getdate.
Table Structure:
CREATE TABLE [dbo].[BIORADFM_TASK_AUDIT](
[STRGUID] [nvarchar](32) NOT NULL,
[ACTIVITYUSERID] [int] NOT NULL,
[ACTIVITYSESSIONID] [int] NOT NULL,
[ACTIVITYCODE] [int] NOT NULL,
[SERVERNAME] [nvarchar](256) NOT NULL,
[APPNAME] [nvarchar](20) NOT NULL,
[STARTTIME] [float] NOT NULL,
[ENDTIME] [float] NOT NULL,
[STRDESCRIPTION] [nvarchar](1000) NULL,
[STRMODULENAME] [nvarchar](300) NULL,
CONSTRAINT [PK_BIORADFM_TASK_AUDIT] PRIMARY KEY CLUSTERED
Sample Data:
StartTime Endtime
43020.2549421296 43020.2549421296
43020.2550810185 43020.2550810185
43020.6342939815 43020.6342939815
43020.2548032407 43020.2548032407
43020.2548263889 43020.2548263889
43020.2549421296 43020.2549421296
43020.2549305556 43020.2549305556
43020.2549421296 43020.2549421296
43019.2549189815 43019.2549189815
Don't use datediff(). It counts the number of "time boundaries" between two date/times.
Instead, use date arithmetic:
WHERE A.ActivityUserID = B.lUserID and
CAST(A.StartTime AS DATETIME) >= DATEADD(minute, -10, GETDATE())
If you want rows with an start time situated only in the last 10 minutes you have to filter by range, for example with BETWEEN:
SELECT CAST(StartTime AS DATETIME) AS StartTime, CAST(endtime AS DATETIME) AS EndTime
FROM BIORADFM_TASK_AUDIT
WHERE CAST(StartTime AS DATETIME) BETWEEN DATEADD(minute,-10,GETDATE()) AND GETDATE()
Just a wild guess since we have nothing to work with. It would be a good idea to store datetime data as datetime instead of a strings so you don't have to constantly convert your data to the right datatype.
SELECT A.strGUID
, u.sUserName
, A.ServerName
, A.AppName
, A.ActivityCode
, CAST(A.StartTime AS DATETIME) as StartTime
, CAST(A.endtime AS DATETIME) as EndTime
, A.strModuleName
, A.strDescription
, GETDATE()
FROM INTRAWPROD_TASK_AUDIT A
join HSV_ACTIVITY_USERS u on A.ActivityUserID = u.lUserID
where convert(datetime, a.StartTime) >= convert(datetime, dateadd(minute, -10, GETDATE()))
I have a repository table which has around 18.7 million rows and every month around 500 thousand to 100 thousand rows are added. The table structure is as follows
CREATE TABLE [dbo].[my_table](
[id] [bigint] NULL,
[a_timestamp] [datetime] NULL,
[eventId] [bigint] NULL,
[userId] [varchar](255) NULL,
[customerid] [varchar](128) NULL,
[messageType] [varchar](100) NULL,
[message] [varbinary](max) NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
I have written the following query to get various counts for each month. The query takes around 10 minutes to execute now. I need help to optimize this query and if possible to bring the time to a couple of mins.
SELECT DATEADD(month, DATEDIFF(month, 0,a_timestamp), 0) AS MonthYear,
COUNT(*) AS [Count],
COUNT(DISTINCT customerid) AS [Unique Customers],
COUNT(DISTINCT userId) AS [Unique Users]
FROM [my_table]
WHERE messageType = 'Outbound'
AND userId NOT IN ('master', 'admin')
AND CHARINDEX('Retrieve Document',CONVERT(VARCHAR(MAX),[message])) > 1
GROUP BY DATEADD(month, DATEDIFF(month, 0,a_timestamp), 0)
ORDER BY MonthYear
I think the key reasons for the long execution time are as follows
CHARINDEX('Retrieve Document',CONVERT(VARCHAR(MAX),[message])) > 1 converting from VARBINARY to VARCHAR and searching if 'Retrieve Document'
userId NOT IN ('master', 'admin') filtering users other than the users in the list (the actual list is longer than 2 strings around 10 strings)
18.7 million rows in the table
A couple of points to note
I don't create this table and I can't change it
I don't have SHOWPLAN permission
I need to use this query in Excel data connections and have the user run it from excel. The user will have only select privileges.
Given that you cannot change the existing table, it may be better to change your strategy.
Instead of running your query and building a new set of results completely every time. Why don't you insert new results into another table (lets call it AccumulatedResults) on a monthly basis.
That way you are only handling the 500K new recs each time. This will be much faster than rebuilding the entire result set every time. The query will look a little like:
INSERT INTO AccumulatedResults
(
MonthYear,
[COUNT],
UniqueCustomers,
UniqueUsers,
)
SELECT
DATEADD(month, DATEDIFF(month, 0, a_timestamp), 0) AS MonthYear,
COUNT(*) AS [Count],
COUNT(DISTINCT customerid) AS [Unique Customers],
COUNT(DISTINCT userId) AS [Unique Users]
FROM
[my_table]
WHERE
messageType = 'Outbound' AND
userId NOT IN ('master', 'admin') AND
CHARINDEX('Retrieve Document', CONVERT(VARCHAR(MAX), [message])) > 1
-- This is a new condition
AND DATEADD(month, DATEDIFF(month, 0, a_timestamp), 0)
> (SELECT MAX(MonthYear) FROM AccumulatedResults)
GROUP BY
DATEADD(month, DATEDIFF(month, 0, a_timestamp), 0)
I got question to split date, month year, dayname from datetime by using trigger, when I insert a datetime, then in next column will split date, time, month year dayname and count year (to know how old the man in my data) is that possible ?
For example, if I insert
INSERT INTO MAN VALUES ('04/06/1982')
then will be like this
DATETIME DATE MONTH YEAR DAYNAME AGE
04/06/1982 00:00:00 04 06 1982 friday 27
Try this :-
Declare #myDate datetime
set #myDate='19820604' --YYYYMMDD
Select #myDate as DateTime,
datename(day,#myDate) as Date,
month(#myDate) as Month,
datename(year,#myDate) as Year,
Datename(weekday,#myDate) as DayName,
DATEDIFF ( year , #myDate , getdate() ) as Age
Result
╔══════════════════════════════╦══════╦═══════╦══════╦═════════╦══════════╗
║ DateTime ║ DATE ║ MONTH ║ YEAR ║ DAYNAME ║ Age ║
╠══════════════════════════════╬══════╬═══════╬══════╬═════════╬══════════╣
║ April, 06 1982 00:00:00+0000 ║ 4 ║ 6 ║ 1982 ║ Friday ║ 31 ║
╚══════════════════════════════╩══════╩═══════╩══════╩═════════╩══════════╝
SQL Fiddle Demo
The code has been slightky altered to give the age correctly.
Select myDate,myDateDate,myDateMonth,myDateYear,myDateDayName, Convert(varchar(50),Age)+ ' Years and '+Convert(varchar(50),nodays) +'days' {Age] from
(
Select #myDate as myDate,
datename(day,#myDate) as myDateDate,
month(#myDate) as myDateMonth,
datename(year,#myDate) as myDateYear,
Datename(weekday,#myDate) as myDateDayName,
DATEDIFF ( year , #myDate , getdate() ) Age ,
DATEDIFF ( dd , #myDate , getdate() ) -365* DATEDIFF ( year , #myDate , getdate() ) as nodays
) As a
Thanks
Arun
Here there are two approaches for solving the issue:
- Approach 1:
you can add some computed columns in to your table, so when you retrieve the table contents the other remaining fields are computed in that time.
-- 1.1) Create the base of table 'MAN'
CREATE TABLE [dbo].[MAN](
[DATETIME] [datetime] NOT NULL
) ON [PRIMARY]
GO
-- 1.2) Insert a record in it
INSERT INTO MAN VALUES ('04/06/1982')
GO
-- 1.3) Add some computed columns
ALTER TABLE MAN
ADD
[DAY] AS DATENAME(DAY, [DATETIME]),
[MONTH] AS MONTH([DATETIME]),
[YEAR] AS DATENAME(YEAR, [DATETIME]),
[DAYNAME] AS DATENAME(WEEKDAY, [DATETIME]),
[AGE] AS DATEDIFF(YEAR, [DATETIME], GETDATE())
GO
-- 1.4) See the result
SELECT * FROM MAN
- Approach 2:
While you creating the table, you add the remaining fields that you need, so that in the next phase you add insert/update triggers in order to calculate the remaining fields and insert/update them.
-- 2.1) Create the table 'MAN' with all needed columns
CREATE TABLE [dbo].[MAN](
[DATETIME] [datetime] NOT NULL,
[DAY] [int] NULL,
[MONTH] [int] NULL,
[YEAR] [int] NULL,
[DAYNAME] [nvarchar](10) NULL,
[AGE] [int] NULL
) ON [PRIMARY]
GO
-- 2.2) Create Insert and update triggers in order to calculate the values of the rest fields while inserting/updating
CREATE TRIGGER [dbo].[trCalculateRemainingDateFields] ON MAN
AFTER INSERT, UPDATE
AS
BEGIN
SET NOCOUNT ON;
UPDATE MAN
SET
[DAY] = DATENAME(DAY, MAN.[DATETIME]),
[MONTH] = MONTH(MAN.[DATETIME]),
[YEAR] = DATENAME(YEAR, MAN.[DATETIME]),
[DAYNAME] = DATENAME(WEEKDAY, MAN.[DATETIME]),
[AGE] = DATEDIFF(YEAR, MAN.[DATETIME], GETDATE())
FROM Inserted i
WHERE i.[DATETIME] = MAN.[DATETIME]
END
GO
-- 2.3) Insert a record in it
INSERT INTO MAN ([DATETIME]) VALUES ('04/06/1985')
GO
-- 2.4) See the result
SELECT * FROM MAN
- Approach 3:
Creating a view of the main table which calculate the remaining fields can be a good option.
-- 3.1) Create the base of table 'MAN'
CREATE TABLE [dbo].[MAN](
[DATETIME] [datetime] NOT NULL
) ON [PRIMARY]
GO
-- 3.2) Insert a record in it
INSERT INTO MAN VALUES ('04/06/1982')
GO
-- 3.3) Create a view which is contained the remaining fields
CREATE VIEW dbo.vMAN
AS
SELECT [DATETIME],
DATENAME(DAY, [DATETIME]) AS [DAY],
MONTH([DATETIME]) AS [MONTH],
DATENAME(YEAR, [DATETIME]) AS [YEAR],
DATENAME(WEEKDAY, [DATETIME]) AS [DAYNAME],
DATEDIFF(YEAR, [DATETIME], GETDATE()) AS [AGE]
FROM MAN
GO
-- 3.4) See the result
SELECT * FROM vMAN
I'm creating a dataset that will be displayed in an SSRS report.
I have a query in a job that puts a count into a table [dbo].[CountMetersDue] on a rolling basis on the 1st of every month; the value changes throughout the month so need to take a snapshot at beginning.
I have the report set up which uses a custom expression to produce a cumulative trend graph. Basically takes one value, divides by another to work out a percentage. Therefore I have two queries that need combining... Took me ages to get my head round all this!
I just need help with the last bit.
SELECT (SELECT [Count]
FROM [MXPTransferDev].[dbo].[CountMetersDue]
WHERE [MXPTransferDev].[dbo].[CountMetersDue].[DateTime] =
[MXPTransferDev].[dbo].[Readings].[dateRead]) AS [MetersDue],
COUNT(readingid) AS [TotalReadings],
CONVERT(DATE, dateread) AS [dateRead]
FROM [MXPTransferDev].[dbo].[Readings]
WHERE ( [MXPTransferDev].[dbo].[Readings].[dateRead] BETWEEN
'01-may-11' AND '31-may-11' )
AND ( webcontactid IS NOT NULL )
AND ( meter = 1 )
GROUP BY CONVERT(DATE, [MXPTransferDev].[dbo].[Readings].[dateRead])
CREATE TABLE [dbo].[CountMetersDue](
[Count] [int] NULL,
[DateTime] [datetime] NULL
) ON [USER]
GO
ALTER TABLE [dbo].[CountMetersDue]
ADD CONSTRAINT [DF_CountMetersDue_DateTime] DEFAULT (getdate()) FOR [DateTime]
GO
CREATE TABLE [dbo].[Readings](
[readingId] [bigint] IDENTITY(1,1) NOT FOR REPLICATION NOT NULL,
[dateRead] [datetime] NOT NULL,
[meter] [int] NOT NULL,
[webcontactid] [bigint] NULL,
Readings
readingId meter reading dateRead webcontactid
583089 4 3662 2011-05-25 15:00:33.040 479
583207 3 682 2011-05-25 15:00:33.027 479
583088 2 98064 2011-05-25 15:00:33.007 479
CountMetersDue
Count DateTime
2793 2011-12-01 00:00:00.000
1057 2011-05-01 14:08:20.437
610 2011-03-01 00:00:00.000
Second stab at answering your question (will probably need some clarification from yourself before the answer is correct):
/* DDL: 2 tables [CountMetersDue] & [Readings]
[CountMetersDue]
([DateTime] datetime,
[Count] int)
[Readings]
([ReadingId] bigint,
[dateRead] datetime,
[webcontactid] bigint,
[meter] int)
[CountMetersDue] - contains 1 record on the first of every month, with count of the number of readings at that date
[Readings] - contains all the individual readings
ie:
[CountMetersDue]
01-Jan-2011 1000
01-Feb-2011 2357
01-Mar-2011 3000
[Readings]
1 01-Jan-2011 11 1
2 02-Jan-2011 12 1
3 03-Jan-2011 13 1
...
*/
SELECT
CONVERT(DATE, [dbo].[Readings].[dateRead]) AS dateRead,
COUNT([dbo].[Readings].[readingId]) AS TotalReadings,
[dbo].[CountMetersDue].[Count] AS MetersDue
FROM
[CountMetersDue] /* get all count meters due */
left join [Readings] /* get any corresponding Reading records
where the dateRead in the same month as
the CountMetersDue */
on DATEPART(year, Readings.dateRead) = DATEPART(year, [CountMetersDue].[DateTime]) /* reading in same year as CountMetersDue */
and DATEPART(month, Readings.dateRead) = DATEPART(month, [CountMetersDue].[DateTime]) /* reading in same month as CountMetersDue */
WHERE ([MXPTransferDev].[dbo].[Readings].[dateRead]) BETWEEN
#StartDate AND #EndDate
AND ( webcontactid IS NOT NULL )
AND ( meter = 1 )
GROUP BY
[dbo].[CountMetersDue].[Count],CONVERT(DATE, [dbo].[Readings].[dateRead])
This would be the query you are looking for then?
Subqueries, as they are called, can be included by enclosing them in parentheses '()'.
SELECT (SELECT [Count] FROM [xxxxx].[dbo].[CountMetersDue] AS tabA WHERE tabA.[datefield] = tabB.dateRead) AS [MetersDue], COUNT(readingId) AS [TotalReadings], CONVERT(DATE, dateRead) AS [dateRead]
FROM [xxxxx] AS tabB
WHERE (dateRead BETWEEN #StartDate AND #EndDate) AND (webcontactid IS NOT NULL) AND (meter = 1)
GROUP BY CONVERT(DATE, dateRead)
I've got a log table in SQL Server that looks like this:
CREATE TABLE [dbo].[RefundProcessLog](
[LogId] [bigint] IDENTITY(1,1) NOT NULL,
[LogDate] [datetime] NOT NULL,
[LogType] [varchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[RefundId] [int] NULL,
[RefundTypeId] [smallint] NULL,
[LogMessage] [varchar](1000) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[LoggedBy] [varchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
CONSTRAINT [PK_RefundProcessLog] PRIMARY KEY CLUSTERED
(
[LogId] ASC
) ON [PRIMARY]
) ON [PRIMARY]
GO
What I want is a list of results that represents how many different refundids were processed each day, throwing out any NULLs.
What SQL would I need to write to produce these results?
I like this approach in (MS SQL):
SELECT
Convert(char(8), LogDate, 112),
count(distinct RefundId)
FROM RefundProcessing
GROUP BY Convert(char(8), LogDate, 112)
select cast(LogDate as date) as LogDate, count(refundId) as refundCount
from yourTable
group by cast(LogDate as date)
Depending on the dialect of SQL you're using, you may have to change the CAST to something else. The expression should convert the LogDate to a date-only value.
Also, if you say "different refundId" because there could be repeated values of refundId that you only want to count once, use count(DISTINCT refundId)
What database vendor are you using? Whichever it is, replace the "DateOnly(LogDate)" in the following with the appropriate construict to extract the date portion (strip off the time) from the logdate column value and then try this:
Select [DateOnly(LogDate)], Count Distinct RefundId
From RefundProcessLog
Group By [DateOnly(LogDate)]
In Sql server, for e.g., the appropriate construct would be:
Select DateAdd(day, 0, DateDiff(day, 0, LogDate)), Count(Distinct RefundId)
From RefundProcessLog
Group By DateAdd(day, 0, DateDiff(day, 0, LogDate))
SELECT COUNT(RefundId), DateOnly(LogDate) LoggingDate
FROM RefundProcessLog
GROUP BY DateOnly(LogDate)
"DateOnly" is specific to your SQL database, which you haven't specified.
For SQL Server you could use DateAdd(dd,0, DateDiff(dd,0,LogDate)) for "DateOnly"
SQL Server 2008 introduced the date datatype which makes the following possible:
select convert(date, LogDate),
,count(refundid) AS 'refunds'
from RefundProcessing
group by convert(date,LogDate)
order by convert(date,LogDate)
In SqlServer, it would be something like:
select datepart(YEAR, [LogDate]), datepart(MONTH, [LogDate]), datepart(DAY, [LogDate]), count(refundid) as [Count]
from [RefundProcessing]
group by datepart(YEAR, [LogDate]), datepart(MONTH, [LogDate]), datepart(DAY, [LogDate])
Select count(*), LogDate, refundid from RefundProcessLog
where refundid is not null
group by LogDate, refundid
Edit:
Or drop RefundID if you don't want it broken down by refunds