I have one table of contact records and I'm trying to get the count of duplicate records that were created on each date. I'm not looking to include the original instance in the count. I'm using SQL Server.
Here's an example table
| email | created_on |
| ------------- | ---------- |
| aaa#email.com | 08-16-22 |
| bbb#email.com | 08-16-22 |
| zzz#email.com | 08-16-22 |
| bbb#email.com | 07-12-22 |
| aaa#email.com | 07-12-22 |
| zzz#email.com | 06-08-22 |
| aaa#email.com | 06-08-22 |
| bbb#email.com | 04-21-22 |
And I'm expecting to return
| created_on | dupe_count |
| ---------- | ---------- |
| 08-16-22 | 3 |
| 07-12-22 | 2 |
| 06-08-22 | 0 |
| 04-21-22 | 0 |
Edited to add error message:
error message
I created a sub table based on email and created date row number. Then, you query that, and ignore the date when the email first was created (row number 1). Works perfectly fine in this case.
Entire code:
Create table #Temp
(
email varchar(50),
dateCreated date
)
insert into #Temp
(email, dateCreated) values
('aaa#email.com', '08-16-22'),
('bbb#email.com', '08-16-22'),
('zzz#email.com', '08-16-22'),
('bbb#email.com', '07-12-22'),
('aaa#email.com', '07-12-22'),
('zzz#email.com', '06-08-22'),
('aaa#email.com', '06-08-22'),
('bbb#email.com', '04-21-22')
select datecreated, sum(case when r = 1 then 0 else 1 end) as duplicates
from
(
Select email, datecreated, ROW_NUMBER() over(partition by email
order by datecreated) as r from #Temp
) b
group by dateCreated
drop table #Temp
Output:
datecreated duplicates
2022-04-21 0
2022-06-08 0
2022-07-12 2
2022-08-16 3
You can calculate the difference between total count of emails for every day and the count of unique emails for the day:
select created_on,
count(email) - count(distinct email) as dupe_count
from cte
group by created_on
It seems I have misunderstood your request, and you wanted to consider previous created_on dates' too:
ct as (
select created_on,
(select case when (select count(*)
from cte t2
where t1.email = t2.email and t1.created_on > t2.created_on
) > 0 then email end) as c
from cte t1)
select created_on,
count(distinct c) as dupe_count
from ct
group by created_on
order by 1
It seems that in oracle it is also possible to aggregate it using one query:
select created_on,
count(distinct case when (select count(*)
from cte t2
where t1.email = t2.email and t1.created_on > t2.created_on
) > 0 then email end) as c
from cte t1
group by created_on
order by 1
Related
I have the following data set:
| EMAIL | SIGNUP_DATE |
| A#ABC.COM | 1/1/2021 |
| B#ABC.COM | 1/2/2021 |
| C#ABC.COM | 1/3/2021 |
In order to find the running total of email signups as of a certain day, I ran the following sql query:
select
signup_date,
count(email) OVER (order by signup_date ASC) as running_total_signups
I got the following results:
| SIGNUP_DATE | RUNNING_TOTAL_SIGNUPS |
| 1/1/21 | 1 |
| 1/2/21 | 2 |
| 1/3/21 | 3 |
However for my next step, I want to be able to see not just the running total signups, but the actual signup names themselves. Therefore I want to run the same window function (count(email) OVER (order by signup_date ASC)) but instead of a count(email) just a select distinct email. This would hopefully result in the following output:
| SIGNUP_DATE | RUNNING_TOTAL_SIGNUPS |
| 1/1/21 | a#abc.com |
| 1/2/21 | a#abc.com |
| 1/2/21 | b#abc.com |
| 1/3/21 | a#abc.com |
| 1/3/21 | b#abc.com |
| 1/3/21 | c#abc.com |
How would I do this? I'm getting an error on this code:
select
signup_date,
distinct email OVER (order by signup_date ASC) as running_total_signups
One way would be to cross-join the results and filter the joined table having a total <= to the running total:
with counts as (
select *,
Count(*) over (order by SIGNUP_DATE asc) as tot
from t
)
select c1.EMAIL, c1.SIGNUP_DATE
from counts c1
cross join counts c2
where c2.tot <= c1.tot
I want to run the same window function (count(email) OVER (order by
signup_date ASC)) but instead of a count(email) just a select distinct
email
Why do you want COUNT() window function?
It has nothing to do with with your reqirement.
All you need is a simple self join:
SELECT t1.SIGNUP_DATE, t2.EMAIL
FROM tablename t1 INNER JOIN tablename t2
ON t2.SIGNUP_DATE <= t1.SIGNUP_DATE
ORDER BY t1.SIGNUP_DATE, t2.EMAIL;
which will work for your sample data, but just in case there are more than 1 rows for each day in your table you should use:
SELECT t1.SIGNUP_DATE, t2.EMAIL
FROM (SELECT DISTINCT SIGNUP_DATE FROM tablename) t1 INNER JOIN tablename t2
ON t2.SIGNUP_DATE <= t1.SIGNUP_DATE
ORDER BY t1.SIGNUP_DATE, t2.EMAIL;
See the demo.
It's actually slightly simpler than Stu proposed:
select
x2.signup_date,
x1.email
from
signups x1
INNER JOIN signups x2 ON x1.signup_date <= x2.signup_date
order by signup_date
If you join the table to itself but for any date that is less than or equal to, it causes a half cartesian explosion. The lowest dated row matches with only itself. The next one matches with itself and the earlier one, so one of the table aliases has its data repeated.. This continues adding more rows to the explosion as the dates increase:
In this resultset we can see we want the emails from x1, and the dates from x2
I got the row_num column from a partition. I want each Type to match with at least one Sent and one Resent. For example, Jon's row is removed below because there is no Resent. Kim's Sheet row is also removed because again, there is no Resent. I tried using a CTE to take all columns for a Code if row_num = 2 but Kim's Sheet row obviously shows up because they're all under one Code. If anyone could help, that'd be great!
Edit: I'm using SSMS 2018. There are multiple Statuses other than Sent and Resent.
What my table looks like:
+-------+--------+--------+---------+---------+
| Code | Name | Type | Status | row_num |
+-------+--------+--------+---------+---------+
| 123 | Jon | Sheet | Sent | 1 |
| 221 | Kim | Sheet | Sent | 1 |
| 221 | Kim | Book | Resent | 1 |
| 221 | Kim | Book | Sent | 2 |
| 221 | Kim | Book | Sent | 3 |
+-------+--------+--------+---------+---------+
What I want it to look like:
+-------+--------+--------+---------+---------+
| Code | Name | Type | Status | row_num |
+-------+--------+--------+---------+---------+
| 221 | Kim | Book | Resent| 1 |
| 221 | Kim | Book | Sent | 2 |
| 221 | Kim | Book | Sent | 3 |
+-------+--------+--------+---------+---------+
Here is my CTE code:
WITH CTE AS
(
SELECT *
FROM #MyTable
)
SELECT *
FROM #MyTable
WHERE Code IN (SELECT Code FROM CTE WHERE row_num = 2)
If sent and resent are the only values for status, then you can use:
select t.*
from t
where exists (select 1
from t t2
where t2.name = t.name and
t2.type = t.type and
t2.status <> t.status
);
You can also phrase this with window functions:
select t.*
from (select t.*,
min(status) over (partition by name, type) as min_status,
max(status) over (partition by name, type) as max_status
from t
) t
where min_status <> max_status;
Both of these can be tweaked if other status values are possible. However, based on your question and sample data, that does not seem necessary.
FIDDLE
CREATE TABLE Table1(ID integer,Name VARCHAR(10),Type VARCHAR(10),Status VARCHAR(10),row_num integer);
INSERT INTO Table1 VALUES
('123','Jon','Sheet','Sent','1'),
('221','Kim','Sheet','Sent','1'),
('221','Kim','Book','Resent','1'),
('221','Kim','Book','Sent','2'),
('221','Kim','Book','Sent','3');
SELECT t1.*
FROM Table1 t1
WHERE EXISTS (
select 1
from Table1 t2
where t2.Name=t1.Name
and t2.Type=t1.TYpe
and t2.Status = case when t1.Status='Sent'
then 'Resent'
else 'Sent' end)
It would be easier if you would provide some scripts to create table and put these test data, but try something like
with a1 as (
select
name, type,
row_number() over (partition by code, Name, type, status) as rn
from #MyTable
), a2 as (
select * from a1 where rn > 1
)
select t.*
from #MyTable as t
inner join a2 on t.name = a2.name and t.type = a2.type;
Here you
calculate another row number using partitions by code, name, type and status,
then fetch these with this new row number > 1
and finally, you use that to join to original table and get interesting you rows
Syntax may vary on MSSQL, but you should give it a try. And please use better names than me ;-)
This solution is quite generic because it doesn't rely on used statuses. They're not hardcoded. And you can easily control what matters by changing partitions.
Fiddle
I'm trying to build a monthly tally of active equipment, grouped by service area from a database log table. I think I'm 90% of the way there; I have a list of months, along with the total number of items that existed, and grouped by region.
However, I also need to know the state of each item as they were on the first of each month, and this is the part I'm stuck on. For instance, Item 1 is in region A in January, but moves to Region B in February. Item 2 is marked as 'inactive' in February, so shouldn't be counted. My existing query will always count item 1 in region A, and item 2 as 'active'.
I can correctly show that Item 3 is deleted in March, and Item 4 doesn't show up until the April count. I realize that I'm getting the first values because my query is specifying the min date, I'm just not sure how I need to change it to get what I want.
I think I'm looking for a way to group by Max(OperationDate) for each Month.
The Table looks like this:
| EQUIPID | EQUIPNAME | EQUIPACTIVE | DISTRICT | REGION | OPERATIONDATE | OPERATION |
|---------|-----------|-------------|----------|--------|----------------------|-----------|
| 1 | Item 1 | 1 | 1 | A | 2015-01-01T00:00:00Z | INS |
| 2 | Item 2 | 1 | 1 | A | 2015-01-01T00:00:00Z | INS |
| 3 | Item 3 | 1 | 1 | A | 2015-01-01T00:00:00Z | INS |
| 2 | Item 2 | 0 | 1 | A | 2015-02-10T00:00:00Z | UPD |
| 1 | Item 1 | 1 | 1 | B | 2015-02-15T00:00:00Z | UPD |
| 3 | (null) | (null) | (null) | (null) | 2015-02-21T00:00:00Z | DEL |
| 1 | Item 1 | 1 | 1 | A | 2015-03-01T00:00:00Z | UPD |
| 4 | Item 4 | 1 | 1 | B | 2015-03-10T00:00:00Z | INS |
There is also a subtable that holds attributes that I care about. It's structure is similar. Unfortunately, due to previous design decisions, there is no correlation to operations between the two tables. Any joins will need to be done using the EquipmentID, and have the overlapping states matched up for each date.
Current query:
--cte to build date list
WITH calendar (dt) AS
(SELECT &fromdate from dual
UNION ALL
SELECT Add_Months(dt,1)
FROM calendar
WHERE dt < &todate)
SELECT dt, a.district, a.region, count(*)
FROM
(SELECT EQUIPID, DISTRICT, REGION, OPERATION, MIN(OPERATIONDATE ) AS FirstOp, deleted.deldate
FROM Equipment_Log
LEFT JOIN
(SELECT EQUIPID,MAX(OPERATIONDATE) as DelDate
FROM Equipment_Log
WHERE OPERATION = 'DEL'
GROUP BY EQUIPID
) Deleted
ON Equipment_Log.EQUIPID = Deleted.EQUIPID
WHERE OPERATION <> 'DEL' --AND additional unimportant filters
GROUP BY EQUIPID,DISTRICT, REGION , OPERATION, deldate
) a
INNER JOIN calendar
ON (calendar.dt >= FirstOp AND calendar.dt < deldate)
OR (calendar.dt >= FirstOp AND deldate is null)
LEFT JOIN
( SELECT EQUIPID, MAX(OPERATIONDATE) as latestop
FROM SpecialEquip_Table_Log
--where SpecialEquip filters
group by EQUIPID
) SpecialEquip
ON a.EQUIPID = SpecialEquip.EQUIPID and calendar.dt >= SpecialEquip.latestop
GROUP BY dt, district, region
ORDER BY dt, district, region
Take only last operation for each id. This is what row_number() and where rn = 1 do.
We have calendar and data. Make partitioned join.
I assumed that you need to fill values for months where entries for id are missing. So nvl(lag() ignore nulls) are needed, because if something appeared in January it still exists in Feb, March and we need district, region values from last not empty row.
Now you have everything to make count. That part where you mentioned SpecialEquip_Table_Log is up to you, because you left-joined this table and not used it later, so what is it for? Join if you need it, you have id.
db<>fiddle
with
calendar(mth) as (
select date '2015-01-01' from dual union all
select add_months(mth, 1) from calendar where mth < date '2015-05-01'),
data as (
select id, dis, reg, dt, op, act
from (
select equipid id, district dis, region reg,
to_char(operationdate, 'yyyy-mm') dt,
row_number()
over (partition by equipid, trunc(operationdate, 'month')
order by operationdate desc) rn,
operation op, nvl(equipactive, 0) act
from t)
where rn = 1 )
select mth, dis, reg, sum(act) cnt
from (
select id, mth,
nvl(dis, lag(dis) ignore nulls over (partition by id order by mth)) dis,
nvl(reg, lag(reg) ignore nulls over (partition by id order by mth)) reg,
nvl(act, lag(act) ignore nulls over (partition by id order by mth)) act
from calendar
left join data partition by (id) on dt = to_char(mth, 'yyyy-mm') )
group by mth, dis, reg
having sum(act) > 0
order by mth, dis, reg
It may seem complicated, so please run subqueries separately at first to see what is going on. And test :) Hope this helps.
I have a table with a huge amount of data with this structure (simplidied) :
+--------+-------------------------+-------+
| id | datetime | type |
+--------+-------------------------+-------+
| 1 | 2015-08-13 17:50:41 | 1 |
| 2 | 2015-08-13 17:50:45 | 0 |
| 3 | 2015-08-14 17:50:56 | 0 |
| 4 | 2015-08-14 17:50:59 | 0 |
+--------+-------------------------+-------+
Row with type=1 are followed by a lots of rows with type=0
I need to do an intelligent clean :
I want to keep rows with type=0 following rows with type=1 only during one hour (After the type 1 row timestamp)
And at least one row with type=0 per hour
I don't know if its possible to do that with a query, or if I will have to loop through all rows with a script.
I use PostgreSQL
I dont have postgres here to test, but this should return all of the data you want to keep:
SELECT ID FROM (
SELECT ID FROM (SELECT
id,
datetime,
type,
LAG(type) OVER (ORDER BY id asc) AS prev_type,
LAG(datetime) OVER (ORDER BY id asc) AS prev_date
FROM employees
WHERE
type=1 AND
prev_type=0 AND
EXTRACT(EPOCH FROM (datetime - prev_date)) < 3601
)
UNION
SELECT MAX(ID) FROM employees GROUP BY TO_CHAR(datetime, 'DDMMYYYHH24'))
I have an SQL table for transfer histories like this:
ID | Date_out | Date_in
---+----------+----------
1 | 01.01.15 |
2 | 01.01.15 | 13.05.15
2 | 16.07.14 | 01.01.15
4 | 28.07.15 |
4 | 16.07.14 | 28.07.15
You can say that item with ID 2 was given away on 16.07.14 and returned on 01.01.15 and after that it went out again on 01.01.15 and returned back on 13.05.15. So it's in my office.
The item with ID 4 was given away on 16.07.14 and returned on 28.07.15 after that it went out again on 28.07.15. So it's not in my office
Now I want a list with every Item which is in my office and a list with items which are not in my office. I have to use Microsoft Access.
result 1: Items in my office
ID | Date_out | Date_in
---+----------+----------
2 | 01.01.15 | 13.05.15
result 2: Items outside my office
ID | Date_out | Date_in
---+----------+----------
1 | 01.01.15 |
4 | 28.07.15 |
The problem is that I have the items multiple times. When I group by ID the empty dates disappear like and I still have multiple items:
SELECT
table.item_ID,
table.Date_out,
table.Date_in
FROM table
WHERE table.date_in<Now()
GROUP BY table.item_ID
Does anyone have an idea?
another solution for your problem
SELECT
id,
CASE
WHEN
SUM(CASE WHEN date_in IS NULL THEN 1 ELSE 0 END) > 0
THEN 'out'
ELSE 'in'
END
FROM
table_name
GROUP BY id;
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(ID INT NOT NULL
,Date_out DATE NOT NULL
,Date_in DATE NULL
,PRIMARY KEY(id,date_out)
);
INSERT INTO my_table VALUES
(1 ,'2015-01-01',NULL),
(2 ,'2015-01-01','2015-05-13'),
(2 ,'2014-07-16','2015-01-01'),
(4 ,'2015-07-28',NULL),
(4 ,'2014-07-16','2015-07-28');
SELECT x.id on_loan
FROM my_table x
JOIN
( SELECT id, MAX(date_out) max_date_out FROM my_table GROUP BY id) y
ON y.id = x.id
AND y.max_date_out = x.date_out
WHERE date_in IS NULL;
+---------+
| on_loan |
+---------+
| 1 |
| 4 |
+---------+
SELECT x.id in_stock
FROM my_table x
JOIN (SELECT id, MAX(date_out) max_date_out FROM my_table GROUP BY id) y
ON y.id = x.id
AND y.max_date_out = x.date_out
WHERE date_in IS NOT NULL;
+----------+
| in_stock |
+----------+
| 2 |
+----------+
or, less useful...
SELECT GROUP_CONCAT(CASE WHEN date_in IS NULL THEN x.id END) on_loan
, GROUP_CONCAT(CASE WHEN date_in IS NOT NULL THEN x.id END) in_stock
FROM my_table x
JOIN
( SELECT id, MAX(date_out) max_date_out FROM my_table GROUP BY id) y
ON y.id = x.id AND y.max_date_out = x.date_out;
+---------+----------+
| on_loan | in_stock |
+---------+----------+
| 1,4 | 2 |
+---------+----------+