Efficient way to query separate days of data? - sql

I want to query statistics using SQL from 3 different days (in a row). The display would be something like:
15 users created today, 10 yesterday, 12 two days ago
The SQL would be something like (for today):
SELECT Count(*) FROM Users WHERE created_date >= '2012-05-11'
And then I would do 2 more queries for yesterday and the day before.
So in total I'm doing 3 queries against the entire database. The format for created_date is 2012-05-11 05:24:11 (date & time).
Is there a more efficient SQL way to do this, say in one query?
For specifics, I'm using PHP and SQLite (so the PDO extension).
The result should be 3 different numbers (one for each day).
Any chance someone could show some performance numbers in comparison?

You can use GROUP BY:
SELECT Count(*), created_date FROM Users GROUP BY created_date
That will give you a list of dates with the number of records found on that date. You can add criteria for created_date using a normal WHERE clause.
Edit: based on your edit:
SELECT Count(*), created_date FROM Users WHERE created_date>='2012-05-09' GROUP BY date(created_date)

The best solution is to use GROUP BY DAY(created_date). Here is your query:
SELECT DATE(created_date), count(*)
FROM users
WHERE created_date > CURRENT_DATE - INTERVAL 3 DAY
GROUP BY DAY(created_date)

This would work I believe though I have no way to test it:
SELECT
(SELECT Count(*) FROM Users WHERE created_date >= '2012-05-11') as today,
(SELECT Count(*) FROM Users WHERE created_date >= '2012-05-10') as yesterday,
(SELECT Count(*) FROM Users WHERE created_date >= '2012-05-11') as day_before
;

Use GROUP BY like jeroen suggested, but if you're planning for other periods you can also set ranges like this:
SELECT SUM(IF(created_date BETWEEN '2012-05-01' AND NOW(), 1, 0)) AS `this_month`,
SUM(IF(created_date = '2012-05-09', 1, 0)) AS `2_days_ago`
FROM ...
As noted below, SQLite doesn't have IF function but there is CASE instead. So this way it should work:
SELECT SUM(CASE WHEN created_date BETWEEN '2012-05-01' AND NOW() THEN 1 ELSE 0 END) AS `this_month`,
SUM(CASE created_date WHEN '2012-05-09' THEN 1 ELSE 0 END) AS `2_days_ago`
FROM ...

Related

PL-SQL query to calculate customers per period from start and stop dates

I have a PL-SQL table with a structure as shown in the example below:
I have customers (customer_number) with insurance cover start and stop dates (cover_start_date and cover_stop_date). I also have dates of accidents for those customers (accident_date). These customers may have more than one row in the table if they have had more than one accident. They may also have no accidents. And they may also have a blank entry for the cover stop date if their cover is ongoing. Sorry I did not design the data format, but I am stuck with it.
I am looking to calculate the number of accidents (num_accidents) and number of customers (num_customers) in a given time period (period_start), and from that the number of accidents-per-customer (which will be easy once I've got those two pieces of information).
Any ideas on how to design a PL-SQL function to do this in a simple way? Ideally with the time periods not being fixed to monthly (for example, weekly or fortnightly too)? Ideally I will end up with a table like this shown below:
Many thanks for any pointers...
You seem to need a list of dates. You can generate one in the query and then use correlated subqueries to calculate the columns you want:
select d.*,
(select count(distinct customer_id)
from t
where t.cover_start_date <= d.dte and
(t.cover_end_date > d.date + interval '1' month or t.cover_end_date is null)
) as num_customers,
(select count(*)
from t
where t.accident_date >= d.dte and
t.accident_date < d.date + interval '1' month
) as accidents,
(select count(distinct customer_id)
from t
where t.accident_date >= d.dte and
t.accident_date < d.date + interval '1' month
) as num_customers_with_accident
from (select date '2020-01-01' as dte from dual union all
select date '2020-02-01' as dte from dual union all
. . .
) d;
If you want to do arithmetic on the columns, you can use this as a subquery or CTE.

SQL count distinct # of calls 6 months prior to create date

Am trying to figure out the SQL to:
count # of distinct calls
made on an account 6 months prior to the account being created
I also need to CAST the date field.
I'm thinking something like:
case when (call_date as date format 'MM/DD/YYYY')
between (create_date as date format 'MM/DD/YYYY') and
(ADD_MONTHS, (create_date as date format 'MM/DD/YYYY), -6)
then COUNT (DISTINCT call_nbr) as calls
Here's a snippet of the data i am working with. The answer I require 3 Calls.
Note: both dates are flagged in the db table as DATE format.
Call_Nbr.....Call Date......Create Date
12345........03/14/2020....07/23/2020.....include in result set
12345........03/14/2020....07/23/2020.....exclude in result set
45678........02/14/2020....07/23/2020.....include in result set
91011........01/20/2020....07/23/2020.....include in result set
91211........01/24/2020....07/23/2020.....exclude in result set
12345........11/14/2019....07/23/2020.....exclude in result set
I think you want:
select count(distinct call_nbr) no_calls
from mytable
where call_date >= add_months(create_date, -6)
If you have a column that represnets the account_id, then you can use a group by clause to get the count of calls per account:
select account_id, count(distinct call_nbr) no_calls
from mytable
where call_date >= add_months(create_date, -6)
group by account_id
Edit: it seems like you want conditional aggregation instead:
select
account_id,
count(distinct case when call_date >= add_months(create_date, -6) then call_nbr end) no_calls
from mytable
group by account_id

Selecting 2 date sets from the same dataset?

Say I have a query that selects all the sales from the past 90 days. I want to be able to isolate certain rows on a case/when basis, and can't quite figure out how to do this. The case statement is depending on dates, so: If the date falls between 3/1 and 5/31, then I want to select the sales from any month ends (3/31, 4/30, 5/31 and TODAY) otherwise, if the date is not between 3/1 and 5/31, then I just want to select the past 3 month-ends.
What I tried so far is inserting a Case/When statement in the WHERE clause, but that doesn't seem kosher. Is there another way to go about this?
For reference, the #monthends table contains the following single column:
monthends
2019-03-31
2019-02-28
2019-01-31
and the #insideRule table contains similarly:
insiderRule
2019-03-31
2019-04-22
The query:
SELECT *
FROM mytable
WHERE asofdate IN
CASE WHEN asofdate BETWEEN '3-1-2019' AND '5-31-2019' THEN
(SELECT * FROM #insideRule)
ELSE
(SELECT * FROM #monthends)
END
When I execute the above, I get syntax errors around "IN"
You want exists not case expression :
IF EXISTS (SELECT 1 FROM mytable WHERE aasofdate BETWEEN '2019-03-01' AND '2019-05-31')
SELECT *
FROM #insideRule
ELSE
SELECT *
FROM #monthends
I am thinking you want something like this:
SELECT ir.*
FROM #insideRule ir
WHERE getdate() >= '2019-03-01' AND
getdate() < '2019-06-01'
UNION ALL
SELECT me.*
FROM #monthends me
WHERE getdate() < '2019-03-01' OR
getdate() >= '2019-06-01';
This assumes that the two tables have the same columns in the same order with compatible types.

Need to count unique transactions by month but ignore records that occur 3 days after 1st entry for that ID

I have a table with just two columns: User_ID and fail_date. Each time somebody's card is rejected they are logged in the table, their card is automatically tried again 3 days later, and if they fail again, another entry is added to the table. I am trying to write a query that counts unique failures by month so I only want to count the first entry, not the 3 day retries, if they exist. My data set looks like this
user_id fail_date
222 01/01
222 01/04
555 02/15
777 03/31
777 04/02
222 10/11
so my desired output would be something like this:
month unique_fails
jan 1
feb 1
march 1
april 0
oct 1
I'll be running this in Vertica, but I'm not so much looking for perfect syntax in replies. Just help around how to approach this problem as I can't really think of a way to make it work. Thanks!
You could use lag() to get the previous timestamp per user. If the current and the previous timestamp are less than or exactly three days apart, it's a follow up. Mark the row as such. Then you can filter to exclude the follow ups.
It might look something like:
SELECT month,
count(*) unique_fails
FROM (SELECT month(fail_date) month,
CASE
WHEN datediff(day,
lag(fail_date) OVER (PARTITION BY user_id,
ORDER BY fail_date),
fail_date) <= 3 THEN
1
ELSE
0
END follow_up
FROM elbat) x
WHERE follow_up = 0
GROUP BY month;
I'm not so sure about the exact syntax in Vertica, so it might need some adaptions. I also don't know, if fail_date actually is some date/time type variant or just a string. If it's just a string the date/time specific functions may not work on it and have to be replaced or the string has to be converted prior passing it to the functions.
If the data spans several years you might also want to include the year additionally to the month to keep months from different years apart. In the inner SELECT add a column year(fail_date) year and add year to the list of columns and the GROUP BY of the outer SELECT.
You can add a flag about whether this is a "unique_fail" by doing:
select t.*,
(case when lag(fail_date) over (partition by user_id order by fail_date) > fail_date - 3
then 0 else 1
end) as first_failure_flag
from t;
Then, you want to count this flag by month:
select to_char(fail_date, 'Mon'), -- should aways include the year
sum(first_failure_flag)
from (select t.*,
(case when lag(fail_date) over (partition by user_id order by fail_date) > fail_date - 3
then 0 else 1
end) as first_failure_flag
from t
) t
group by to_char(fail_date, 'Mon')
order by min(fail_date)
In a Derived Table, determine the previous fail_date (prev_fail_date), for a specific user_id and fail_date, using a Correlated subquery.
Using the derived table dt, Count the failure, if the difference of number of days between current fail_date and prev_fail_date is greater than 3.
DateDiff() function alongside with If() function is used to determine the cases, which are not repeated tries.
To Group By this result on Month, you can use MONTH function.
But then, the data can be from multiple years, so you need to separate them out yearwise as well, so you can do a multi-level group by, using YEAR function as well.
Try the following (in MySQL) - you can get idea for other RDBMS as well:
SELECT YEAR(dt.fail_date) AS year_fail_date,
MONTH(dt.fail_date) AS month_fail_date,
COUNT( IF(DATEDIFF(dt.fail_date, dt.prev_fail_date) > 3, user_id, NULL) ) AS unique_fails
FROM (
SELECT
t1.user_id,
t1.fail_date,
(
SELECT t2.fail_date
FROM your_table AS t2
WHERE t2.user_id = t1.user_id
AND t2.fail_date < t1.fail_date
ORDER BY t2.fail_date DESC
LIMIT 1
) AS prev_fail_date
FROM your_table AS t1
) AS dt
GROUP BY
year_fail_date,
month_fail_date
ORDER BY
year_fail_date ASC,
month_fail_date ASC

How to select all the entries from the last month?

I have a table with entries which has a DATE field. Each entry has a distinct date.
I'd like to select all the entries from the last month registred in the database. How?
I tried:
SELECT *
FROM registries
WHERE reg_date = DATE_FORMAT(MAX(reg_date), "%m")`
...without success
If you wanted the last 30 days, this will work
SELECT * FROM `registries`
WHERE `reg_date` > DATE_SUB( NOW(), INTERVAL 30 DAY )
Based on OMG Ponies' query with corrections:
SELECT
r.*
FROM
registries AS r
JOIN (
SELECT
MAX(t.reg_date) AS max_date
FROM
registries AS t) AS t
ON DATE_FORMAT(t.max_date, '%Y-%m') = DATE_FORMAT(r.reg_date, '%Y-%m')
Though the performance of the query wouldn't be excellent, since it would operate the JOIN on two calculated values.
I believe it can still perform decently unless you start hitting millions of records.
On the other hand, you could probably run it faster by querying first for the MAX(reg_date)
SELECT
CONCAT(DATE_FORMAT(MAX(r.reg_date), "%Y-%m"), '-01') AS first_day
FROM
registries AS r
And then injecting the result in a query:
SELECT
r.*
FROM
registries AS r
WHERE
r.reg_date BETWEEN '<first_day>' AND LAST_DAY('<first_day>')
With first_day as a place holder for the previous' query result.
Provided you indexed reg_date, this should run pretty fast.
Note: LAST_DAY is a MySQL only function.
This will give you all records for last month (May):
SELECT [col,] DATEDIFF(TIMESTAMP,
2010-05-01 00:00) dif1,
DATEDIFF(TIMESTAMP, 2010-05-31 00:00)
dif2 FROM tablename HAVING dif1 >= 0
AND dif2 <= 0