Hello I have table name FriendsData that contains duplicate records as shown below
fID UserID FriendsID IsSpecial CreatedBy
-----------------------------------------------------------------
1 10 11 FALSE 1
2 11 5 FALSE 1
3 10 11 FALSE 1
4 5 25 FALSE 1
5 10 11 FALSE 1
6 12 11 FALSE 1
7 11 5 FALSE 1
8 10 11 FALSE 1
9 12 11 FALSE 1
I want to remove duplicate combinations rows using MS SQL?
Remove latest duplicate records from MS SQL FriendsData table.
here I attached image which highlights duplicate column combinations.
How I can removed all duplicate combinations from SQL table?
Try this
DELETE
FROM FriendsData
WHERE fID NOT IN
(
SELECT MIN(fID)
FROM FriendsData
GROUP BY UserID, FriendsID)
See here
Or here is more ways to do what you want
Hope this helps
It seems counter-intuitive, but you can delete from a common table expression (under certain circumstances). So, I'd do it like so:
with cte as (
select *,
row_number() over (partition by userid, friendsid order by fid) as [rn]
from FriendsData
)
delete cte where [rn] <> 1
This will keep the record with the lowest fid. If you want something else, change the order by clause in the over clause.
If it's an option, put a uniqueness constraint on the table so you don't have to keep doing this. It doesn't help to bail out a boat if you still have a leak!
I don't know if the syntax is correct for MS-SQL, but in MySQL, the query would look like:
DELETE FROM FriendsData WHERE fID
NOT IN ( SELECT fID FROM FriendsData
GROUP BY UserID, FriendsUserID, IsSpecial, CreatedBy)
In the GROUP BY clause you put the columns you need to be identical in order to consider two records duplicate
Try this query,
select * from FriendsData f1, FriendsData f2
Where f1.fID=f2.fID and f1.UserID =f2.UserID and f1.FriendsID =f2.FriendsID
If it returns you the duplicate rows, then replace Select * by "Delete"
that will solve your problem
Works in Postgres:
DELETE from "FriendsData" where "fID" in
(SELECT "fID" from
(SELECT *, ROW_NUMBER() OVER(PARTITION BY "UserID", "FriendsID" ORDER BY "fID") as rn
FROM "FriendsData") as inner1
WHERE rn > 1);
Related
How to repeat rows based on column value in snowflake using sql.
I tried a few methods but not working such as dual and connect by.
I have two columns: Id and Quantity.
For each ID, there are different values of Quantity.
So if you have a count, you can use a generator:
with ten_rows as (
select row_number() over (order by null) as rn
from table(generator(ROWCOUNT=>10))
), data(id, count) as (
select * from values
(1,2),
(2,4)
)
SELECT
d.*
,r.rn
from data as d
join ten_rows as r
on d.count >= r.rn
order by 1,3;
ID
COUNT
RN
1
2
1
1
2
2
2
4
1
2
4
2
2
4
3
2
4
4
Ok let's start by generating some data. We will create 10 rows, with a QTY. The QTY will be randomly chosen as 1 or 2.
Next we want to duplicate the rows with a QTY of 2 and leave the QTY =1 as they are.
Obviously you can change all parameters above to suit your needs - this solution works super fast and in my opinion way better than table generation.
Simply stack SPLIT_TO_TABLE(), REPEAT() with a LATERAL() join and voila.
WITH TEN_ROWS AS (SELECT ROW_NUMBER()OVER(ORDER BY NULL)SOME_ID,UNIFORM(1,2,RANDOM())QTY FROM TABLE(GENERATOR(ROWCOUNT=>10)))
SELECT
TEN_ROWS.*
FROM
TEN_ROWS,LATERAL SPLIT_TO_TABLE(REPEAT('hire me $10/hour',QTY-1),'hire me $10/hour')ALTERNATIVE_APPROACH;
I have a table that stores different statuses of each transaction. Each transaction can have multiple statuses (pending, rejected, aproved, etc).
I need to build a query that brings only the last status of each transaction.
The definition for the table that stores the statuses is:
[dbo].[Cuotas_Estado]
ID int (PK)
IdCuota int (references table dbo.Cuotas - FK)
IdEstado int (references table dbo.Estados - FK)
Here's the architecture for the 3 tables:
When running a simple SELECT statement on table dbo.Cuotas_Estado you'll get:
SELECT
*
FROM [dbo].[Cuotas_Estado] [E]
But the result I need is:
IdCuota | IdEstado
2 | 1
3 | 2
9 | 3
10 | 3
11 | 4
I'm running the following select statement:
SELECT
DISTINCT([E].[IdEstado]),
[E].[IdCuota]
FROM [dbo].[Cuotas_Estado] [E]
ORDER BY
[E].[IdCuota] ASC;
This will bring this result:
So, as you can see, it's bringing a double value to entry 9 and entry 11, I need the query to bring only the latest IdEstado column (3 in the entry 9 and 4 in the entry 11).
can you try this?
with cte as (
select IdEstado,IdCuota,
row_number() over(partition by IdCuota order by fecha desc) as RowNum
from [dbo].[Cuotas_Estado]
)
select IdEstado,IdCuota
from cte
where RowNum = 1
You can use a correlated subquery:
SELECT e.*
FROM [dbo].[Cuotas_Estado] e
WHERE e.IdEstado = (SELECT MAX(e2.IdEstado)
FROM [dbo].[Cuotas_Estado] e2
WHERE e2.IdCuota = e.IdCuota
);
With an index on Cuotas_Estado(IdCuota, IdEstado) this is probably the most efficient method.
I have the following table structure in Microsoft SQL:
ID Name Number
1 John
2 John
3 John
4 Mark
5 Mark
6 Anne
7 Anne
8 Luke
9 Rachael
10 Rachael
I am looking to set the 'Number' field to the number of times the 'Name' field has appeared previously, using SQL.
Desired output as follows:
ID Name Number
1 John 1
2 John 2
3 John 3
4 Mark 1
5 Mark 2
6 Anne 1
7 Anne 2
8 Luke 1
9 Rachael 1
10 Rachael 2
The table is ordered by 'Name', so there is no worry of 'John' appearing under ID 11 again, using my example.
Any help would be appreciated. I'm not sure if I can do this with a simple SELECT statement, or whether I will need an UPDATE statement, or something more advanced.
Use ROW_NUMBER:
SELECT ID, Name,
ROW_NUMBER() OVER (PARTITION BY Name
ORDER BY ID) AS Number
FROM mytable
There is no need to add a field for this, as the value can be easily calculated using window functions.
You should be able to use the ROW_NUMBER() function within SQL Server to partition each group (by their Name property) and output the individual row in each partition :
SELECT ID,
Name,
ROW_NUMBER() OVER (PARTITION BY Name ORDER BY ID) AS Number
FROM YourTable
ORDER BY ID
You can see what your data looks like prior to the query :
and then after it is executed :
If your system doesnt support OVER PARTITION, you can use following code:
SELECT
ID,
Name,
(
SELECT
SUM(counterTable.nameCount)
FROM
mytable innerTable
JOIN (SELECT 1 as nameCount) as counterTable
WHERE
innerTable.ID <= outerTable.ID
AND outerTable.Name = innerTable.Name
) AS cumulative_sum
FROM
mytable outerTable
ORDER BY outerTable.ID
Following CREATE TABLE statement I used and then filled in your data:
CREATE TABLE `mytable` (
`ID` INT(11) NULL DEFAULT NULL,
`Name` VARCHAR(50) NULL DEFAULT NULL
);
This should work with DBS not supporting OVER PARTITION like MySQL, Maria, ...
I have the following query to count how many times each process_track_id occurs in a table:
SELECT
a.process_track_id,
COUNT(1) AS 'num'
FROM
transreport.process_name a
GROUP BY
a.process_track_id
This returns the following results:
process_track_id | num
1 14
2 44
3 16
5 8
6 18
7 17
8 14
This is great. Now is the part where I am stuck. I would like to get the following table:
num count
8 1
14 2
16 1
17 1
18 1
44 1
Where num are the distinct counts from the first table, and count is how many times that frequency occurs.
Here is what I have tried (it's a subquery, but I'm not sold on the method) and I haven't been able to get it to work just yet. I'm new to SQL and I think I'm missing out on some some key aspects of the syntax.
SELECT
X.id_count,
count(1) as 'num_count'
FROM
(SELECT
a.process_track_id,
COUNT(1) AS 'id_count'
FROM
transreport.process_name a
GROUP BY
a.process_track_id
--COUNT(1) AS 'id_count'
) X;
Any ideas?
It's probably good to keep in mind that this may have to be run on a database with at least 1 million records, and I don't have the ability to create a new table in the process.
Thanks!
Here's the subquery method you were driving at:
SELECT id_count, COUNT(*) AS 'num_count'
FROM (SELECT a.process_track_id
,COUNT(*) AS 'id_count'
FROM transreport.process_name a
GROUP BY a.process_track_id
)sub
GROUP BY id_count
Not sure there's a better method as the aggregation needs to run once anyway.
Try this
SELECT x.num, COUNT(*) AS COUNT
FROM (
SELECT
a.process_track_id, -- <--- You may removed this column
COUNT(*) AS 'num'
FROM
transreport.process_name a
GROUP BY
a.process_track_id
) X
GROUP BY X.num
This question already has answers here:
Finding duplicate rows in SQL Server
(18 answers)
Closed 9 years ago.
I have to clean a table with duplicate rows:
id: serial id
gid: group id
url: string <- this is the column that I have to cleanup
One gid may have multiple url values:
id gid url
---- ---- ------------
1 12 www.gmail.com
2 12 www.some.com
3 12 www.some.com <-- duplicate
4 13 www.other.com
5 13 www.milfsome.com <-- not a duplicate
I want to execute one query against the entire table and delete all rows where the gid and url are duplicate. In the above sample, after the delete, I want to have only 1, 2, 4 and 5 remaining.
;WITH x AS
(
SELECT id, gid, url, rn = ROW_NUMBER() OVER
(PARTITION BY gid, url ORDER BY id)
FROM dbo.table
)
SELECT id,gid,url FROM x WHERE rn = 1 -- the rows you'll keep
-- SELECT id,gid,url FROM x WHERE rn > 1 -- the rows you'll delete
-- DELETE x WHERE rn > 1; -- do the delete
Once you're happy with the first select, which indicates the rows you'll keep, remove it and un-comment the second select. Once you're happy with that, which indicates the rows you'll delete, remove it and un-comment the delete.
And if you don't want to delete data, just ignore the commented lines under the SELECT...
SELECT
MIN(id) AS id,
gid,
url
FROM yourTable
GROUP BY gid, url