I want to be able to count number of rows inserted in a table per second using SQL database. The count has to be for all the rows in the table. Sometimes there could be 100 rows and others 10 etc so this is just for stats. I managed to count rows per day but need more details. Any advise or any scripts would be appreciated
Thanks
If you truncate the datetime column to the second.
Then you can aggregate on it, to get totals per second.
For example:
SELECT
CAST(dt AS DATE) as [Date],
MIN(Total) as MinRecordsPerSec,
MAX(Total) as MaxRecordsPerSec,
AVG(Total) as AverageRecordsPerSec
FROM
(
SELECT
CONVERT(datetime, CONVERT(char(19), YourDatetimeColumn, 120), 120) as dt,
COUNT(*) AS Total
FROM YourTable
GROUP BY CONVERT(char(19), YourDatetimeColumn, 120)
) q
GROUP BY CAST(dt AS DATE)
ORDER BY 1;
Well it depends on language you are using, the way to do this would be to fetch your DB and change date column to timestamp, then group them by each stamp as you would know each timestamp is per second.
OR
Alternatively, you can store timestamps in DB instead of actual date the it will be easy to query from DB.
OR
Use this function 'UNIX_TIMESTAMP()' in mysql to get timestamp of column then you can do whatever and whichever comparison you want to do on it
https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_unix-timestamp
Hope this gives you an idea.
Related
I using date_trunc in order to count events per day. I have a subquery that I use the date_trunc on. The problem is that the query returns multiple rows per one date. Any ideas?
select
date_trunc('day',date_) date_,
count(download),
count(subscribe)
from
(select
min(users.redshifted_at) date_,
users.id_for_vendor download,
subs.id_for_vendor subscribe
from Facetune2_device_info_log users
left join Facetune2_usage_store_user_subscribed subs
on users.id_for_vendor=subs.id_for_vendor
group by users.id_for_vendor,subs.id_for_vendor) b
group by date_
order by date_
date_ is confusing, because it is both a column and an alias. Columns get resolved first. So this should fix your problem:
group by date_trunc('day', date_)
You can also fix it by using a different alias name, one not already used for a column.
I have a simple, but large, database that I need to write a SQL statement for. The statements needs to do the following:
Get the 15 most popular values for a field.
From those 15, get the count that value has appeared within a particular time period.
My table contains both a Date and a Value field. I am able to extract the 15 most popular values, or get the count for a particular value in a given time period. I do not know how to put the two together.
This is my current SQL:
SELECT
Count( Value ) AS Total,
Value AS Value
FROM
Database
GROUP BY
Value
ORDER BY
Total DESC
LIMIT 15
That will get my most popular 15. But from that, I want to display the COUNT() that each Value is between two dates.
Would this require a HAVING clause?
I simplified the previous solution (which would also do a job) a little bit:
SELECT
Value,
Count(*) as TotalInPeriod
FROM Database
WHERE Value in (SELECT Value FROM Database GROUP BY Value
ORDER BY count(*) DESC LIMIT 15)
AND date_field BETWEEN your_start_date and your_end_date
GROUP BY Value
Try something like this. Make an inner query that finds the top 15 values overall, and join it to the main set to limit it to those values.
SELECT
Count( Value) as TotalInPeriod,
Value as Value
FROM
Database a
JOIN (SELECT
Count( Value ) AS Total,
Value AS Value
FROM
Database
GROUP BY
Value
ORDER BY
Total DESC
LIMIT 15) as topValues
ON
a.Value = topValues.Value
WHERE
a.date_field BETWEEN your_start_date and your_end_date
GROUP BY
a.Value
I normally work with MySQL databases, and I am currently encountering some issues on a query towards a SQL Server database.
I'm trying to get the average of a column, grouped by day. This takes anywhere from 20-30 seconds, even if its just returning a few hundred rows.
The table however contains a couple million entries. I'm sure this has got something to do with the indexing properties, but I just can't seem to figure out the correct solution here.
So the query goes like:
select
[unit_id],
avg(weight) AS avg,
max(timestamp) AS dateDay
from
[measurements]
where
timestamp BETWEEN '2017-06-01' AND '2017-10-04'
group by
[unit_id], CAST(timestamp AS DATE)
order by
[unit_id] asc, [dateDay] asc
I have set up a nonclustered index containing the unit_id, weight and timestamp fields.
This is your query:
select unit_id, avg(weight) AS avg, max(timestamp) AS dateDay
from measurements m
where timestamp BETWEEN '2017-06-01' AND '2017-10-04'
group by unit_id, CAST(timestamp AS DATE)
order by unit_id asc, dateDay asc;
Under reasonable assumptions about your data, it is going to have similar performance in either MySQL or SQL Server. Your WHERE is not highly selective. Because of the inequality, SQL Server cannot make use of an index for the GROUP BY.
An index on measurements(timestamp, unit_id, weight) might benefit the query on either database. There might be some fancy ways to get SQL Server to improve the performance. But both it and MySQL will need to take the rows matching the WHERE clause and aggregate them (using a hash-based algorithm in all likelihood in SQL Server and using a filesort in MySQL).
The problem is likely the CAST in the group by. Though you don't say it explicitly, I'm assuming Timestamp is a DateTime value, which is why you CAST to Date in the group by clause. The problem is that the calculated value produced by CAST isn't indexed.
If it's your system, and this query is something done frequently, I'd add a new column of type Date to store just the day, and index that. If you can't, select out the values in the date range you're interested in, with the date casted to Date, into a temp table or CTE, then group by the date.
Or, even try this, just to pull the CAST out of the Group By clause:
select
[unit_id],
avg(weight) AS avg,
dateDay
from (
select [unit_id],
CAST(timestamp as Date) [dateDay],
weight
from [measurements]
where
timestamp BETWEEN '2017-06-01' AND '2017-10-04'
) x
group by
x.[unit_id], x.[dateDay]
order by
x.[unit_id] asc, x.[dateDay] asc
I am storing the date as "14-02-2013" in date column of my table. Now when I get the date using "ORDER BY" then it should display the output as
14-02-2013
15-03-2013
24-05-2013
How to write the query for this. i.e getting order by day and month.
Any suggestion will be helpful.
Use DATE (or DATETIME) type in your column. Otherwise you will have to perform operations on string representation of date, which is not cool and will cost some extra time to perform
Try one of following:
Select * from Table1 order by date(dtcolumn) Asc
Select * from Table1 order by strftime('%d-%m-%Y', dtcolumn)
Try this query
Select * from Table1 Order By mydatecol Asc
Given you store your date as a temporal data type you can use
select * from your_date_table order by date(your_date_column) ASC
If you store it as a string(which you should not do), you can try
select * from your_date_table order by your_date_column ASC
Here is the doc for the sqlite date and time functions
I have a scenario and not quite sure how to query it. As a sample, I have following table structure and want to get the history of the action for bus:
ID-----TIME---------BUSID----OPID----MOVING----STOPPED----PARKED----COUNT
1------10:10:10-----101------1101-----1---------0----------0---------15
2------10:10:11-----102------1102-----0---------1----------0---------5
3------10:11:10-----101------1101-----1---------0----------0---------15
4------10:12:10-----101------1101-----0---------1----------0---------15
5------10:13:10-----101------1101-----1---------0----------0---------19
6------10:14:10-----101------1101-----1---------0----------0---------19
7------10:15:10-----101------1101-----0---------1----------0---------19
8------10:16:10-----101------1101-----0---------0----------1---------0
9------10:17:10-----101------1101-----0---------0----------1---------0
I want to write a query to get the status of a bus like:
BUSID----OPID----STATUS-----TIME---------DURATION---COUNT
101------1101----MOVING-----10:10:10-----2-----------15
101------1101----STOPPED----10:12:10-----1-----------15
101------1101----MOVING-----10:13:10-----2-----------19
101------1101----STOPPED----10:15:10-----1-----------19
101------1101----PARKED-----10:16:10-----2-----------0
I am using SQL Server 2008.
Thanks for your help.
You can use Common Table Expressions to calculate the duration between the different rows.
WITH cte_log AS
(
SELECT
Row_Number()
OVER
(
ORDER BY time DESC
)
AS
id, time, busid, opid, moving, stopped, parked, count
FROM
log_table
WHERE
busid = 101
)
SELECT
current_rows.busid,
current_rows.opid,
current_rows.time,
DATEDIFF(second, current_rows.time, previous_rows.time) AS duration
current_rows.count
FROM
cte_log_position AS current_rows
LEFT OUTER JOIN
log_table AS previous_rows ON ((current_rows.row_id + 1) = previous_rows.row_id)
WHERE
current_rows.busid = 101
ORDER BY
current_rows.time DESC;
The WITH statement creates a temporary result set that is defined within the execution scope of this query. We are using it to fetch the previous records of each row and to calculate the time difference between the the current and the previous record.
This example was not tested, and it may not work perfectly, but I hope it gets you going in the correct direction. Feel free to leave feedback.
You may also want to check the following external links on how to use Common Table Expressions:
SQL Select Next Row and SQL Select Previous Row with Current Row using T-SQL CTE
Calculate Difference between current and previous rows... CTE and Row_Number() rocks!
4 Guys From Rolla: Common Table Expressions (CTE) in SQL Server 2005
MSDN: Using Common Table Expressions
personally i would denormalize the data so you have start_time and end_time in the one row. this will make the query much more efficient.
I don't have access to SQL Server at the moment, so there may be syntax errors in the following:
SELECT
BUSID,
OPID,
IF (MOVING = 1) 'MOVING' ELSE IF (STOPPED = 1) 'STOPPED' ELSE 'PARKED' AS STATUS
TIME,
COUNT
FROM BUS_DATA_TABLE
GROUP BY BUSID
ORDER BY TIME
You'll note that this does not include duration. Until you order your data, you don't know which is the previous entry. Once the data is ordered you can calculate the duration as the difference between the times in consecutive records. You could do this by SELECTing into a new table and then running a second query.
Grouping by BUSID, should give you your report for all buses.
Making certain assumptions about column type, etc:
SELECT
BUSID,
OPID,
STATUS,
TIME,
DURATION,
COUNT
FROM
TABLENAME
WHERE
BUSID = 1O1
ORDER BY
TIME
;