I have the following dataframe
Block_id step name
1 1 Marie
1 2 Bob
1 3 John
1 4 Lola
2 1 Alex
2 2 John
2 3 Kate
2 4 Herald
3 1 Alec
3 2 Paul
3 3 Rex
As you can see data frame is sorted by block_id and then by step. I want to delete only in one block_id everything after the row where I have name John(the row with John as well). So the desired output would be
Block_id step name
1 1 Marie
1 2 Bob
2 1 Alex
3 1 Alec
3 2 Paul
3 3 Rex
An updatable CTE with a cumulative conditional COUNT seems to be what you are after:
CREATE TABLE dbo.YourTable (BlockID int,
Step int,
[Name] varchar(10));
GO
INSERT INTO dbo.YourTable
VALUES(1,1,'Marie'),
(1,2,'Bob'),
(1,3,'John'),
(1,4,'Lola'),
(2,1,'Alex'),
(2,2,'John'),
(2,3,'Kate'),
(2,4,'Herald'),
(3,1,'Alec'),
(3,2,'Paul'),
(3,3,'Rex');
GO
WITH CTE AS(
SELECT COUNT(CASE [Name] WHEN 'John' THEN 1 END) OVER (PARTITION BY BlockID ORDER BY Step) AS Johns
FROM dbo.YourTable)
DELETE FROM CTE
WHERE Johns >= 1;
GO
SELECT *
FROM dbo.YourTable;
GO
DROP TABLE dbo.YourTable;
One method uses an updatable CTE:
with todelete as (
select t.*,
min(case when name = 'John' then step end) over (partition by block_id) as john_id
from t
)
delete from todelete
where id >= john_id;
Or, if you prefer, a correlated subquery:
delete from t
where id >= (select min(t2.id)
from t t2
where t2.blockid = t.blockid and t2.name = 'John'
);
For performance, both of these can take advantage of an index on (blockid, name, id).
Related
I am trying to create a flag column called "Related" to use in reporting to highlight specific rows that are related based on the ID column (1 = related, NULL = not related). The original table "table1" looks like below:
Name ID Related
--------------------------------
Jack 101 NULL
John 101 NULL
Pat 105 NULL
Ben 106 NULL
Jordan 106 NULL
George 300 NULL
Alan 500 NULL
Bill 200 NULL
Bob 200 NULL
I then used this UPDATE statement below:
UPDATE a
SET Related = 1
FROM table1 a
JOIN (SELECT ID FROM table1 GROUP BY ID HAVING COUNT(*) > 1) b
ON a.ID = b.ID
Below is the result of this update statement:
Name ID Related
--------------------------------
Jack 101 1
John 101 1
Pat 105 NULL
Ben 106 1
Jordan 106 1
George 300 NULL
Alan 500 NULL
Bill 200 1
Bob 200 1
This gets me close but I need for it to instead of assigning the number 1 to each related row, to increment the number for each set of related rows based on their different ID column values.
Desired result:
Name ID Related
--------------------------------
Jack 101 1
John 101 1
Pat 105 NULL
Ben 106 2
Jordan 106 2
George 300 NULL
Alan 500 NULL
Bill 200 3
Bob 200 3
This is a possible solution using dense_rank to number your related values and an updateable CTE
with r as (
select id
from t
group by id having Count(*) > 1
),
n as (
select t.id, t.related, Dense_Rank() over (order by r.id) r
from r
join t on t.id = r.id
)
update n set related = r
You can do this without a self-join, just using window functions in a CTE, and updating the CTE directly:
WITH tCounted AS (
SELECT
t.id,
t.related,
c = COUNT(*) OVER (PARTITION BY r.id)
FROM t
),
tWithRelated as (
SELECT
t.id,
t.related,
rn = DENSE_RANK() OVER (ORDER BY r.id)
FROM tCounted
WHERE c > 1
)
UPDATE tWithRelated
SET related = rn;
Use an updateable CTE - comments explain the logic.
with cte1 as (
select [Name], ID, Related
-- Get the count within the id partition, less 1 as specified
, count(*) over (partition by id) - 1 cnt
-- Get the row number within the id partition
, row_number() over (partition by id order by id) rn
from #Test
), cte2 as (
select [Name], ID, Related, cnt, rn
-- Add 1 *only* if the count is > 0 *and* its the first row in the id partition
, case when cnt > 0 then sum(case when cnt > 0 and rn = 1 then 1 else 0 end) over (order by id) else null end NewRelated
from cte1
)
update cte2 set Related = NewRelated;
This doesn't assume Related is already null and works for more than 2 rows for any given ID.
It does assume that one can order by the ID column - even though the data provided doesn't do that.
Here is my code
WITH CTE AS(
SELECT COUNT(CASE name WHEN 'John' THEN 1 END) OVER (PARTITION BY BlockID ORDER BY Step) AS Johns
FROM dbo.YourTable)
DELETE FROM CTE
WHERE Johns >= 1;
SELECT *
FROM dbo.YourTable;
It returns me the following error when I run the code in the notebook
ERROR: syntax error at or near "DELETE"
But I can't seem to find any mistake in the query
When I try to do it in online compiler it returns the error that relation "cte" does not exist
Maybe this errors can be related?...
Here what I'm trying to do with cte:
My first table:
Block_id step name
1 1 Marie
1 2 Bob
1 3 John
1 4 Lola
2 1 Alex
2 2 John
2 3 Kate
2 4 Herald
3 1 Alec
3 2 Paul
3 3 Rex
As you can see data frame is sorted by block_id and then by step. I want to delete only in one block_id everything after the row where I have name John(the row with John as well). So the desired output would be
Block_id step name
1 1 Marie
1 2 Bob
2 1 Alex
3 1 Alec
3 2 Paul
3 3 Rex
Create a CTE that returns for each Block_id the step of the first John.
Then join the table to the CTE:
WITH cte AS (
SELECT Block_id, MIN(step) step
FROM tablename
WHERE name = 'John'
GROUP BY Block_id
)
DELETE FROM tablename t
USING cte c
WHERE c.Block_id = t.Block_id AND c.step <= t.step
See the demo.
Lets say my table contains the following data
id
name
version
1
Rahul
1
1
Rahul
2
2
John
1
3
Mike
1
2
John
2
4
Rubel
1
5
David
1
1
Rahul
3
I need to filter the duplicate records with lower version. How can this be done?
The output essentially should be
id
name
version
1
Rahul
3
2
John
2
3
Mike
1
4
Rubel
1
5
David
1
For this dataset, aggregation seems sufficient:
select id, name, max(version) as max_version
from mytable
group by id, name
You can use not exists as follows:
select id, name, version
from your_table t
Where not exists
(Select 1 from your_table tt
Where tt.id = t.id and tt.version > t.version)
Or you can use analytical function row_number as follows:
Select id, name, version from
(select t.*,
Row_number() over (partition by id order by version desc) as rn
from your_table t) t
Where rn = 1
The scenario I'm after for is :
Result = Nothing
CollectionOfTables = Tbl1, Tbl2, Tbl3
While(True){
CurrentTable = GetHighestPriorityTable(CollectionOfTables)
If(CurrentTable) = Nothing Then Break Loop;
RemoveCurrentTableFrom(CollectionOfTables)
ForEach ID in CurrentTable as TempRow {
If(Result.DoesntContainsId(ID)) Then Result.AddRow(TempRow)
}
}
Assume I have following three tables.
IdNameTable1, Priority 1
1 John
2 Mary
3 Elsa
IdNameTable2, Priority 2
2 Steve
3 Max
4 Peter
IdNameTable3, Priority 3
4 Frank
5 Harry
6 Mona
Here is the final result I need.
IdNameResult
1 John
2 Mary
3 Elsa
4 Peter
5 Harry
6 Mona
A few tips to keep in mind.
Number of actual tables is 10.
Number of rows per table exceeds 1 Million.
It's not necessary to use join in query, but because of amount of data I'm working with the query must be optimized and used set-operations in SQL not a Cursor script.
Here's a way to do it using UNION and ROW_NUMBER():
;With Cte As
(
Select Id, Name, 1 As Prio
From Table1
Union All
Select Id, Name, 2 As Prio
From Table2
Union All
Select Id, Name, 3 As Prio
From Table3
), Ranked As
(
Select Id, Name, Row_Number() Over (Partition By Id Order By Prio) As RN
From Cte
)
Select Id, Name
From Ranked
Where RN = 1
Order By Id Asc;
Lets say you have the following table:
Id Index
1 3
1 1
2 1
3 3
1 5
what I would like to have is the following:
Id Index
1 0
1 1
2 0
3 0
1 2
As you might notice, the goal is for every row where Id is the same, to incrementally update the Index column, starting from zero.
Now, I know this is fairly simple with using cursors, but out of curiosity is there a way to do this with single UPDATE query, somehow combining with temp tables, common table expressions or something similar?
Yes, assuming that the you don't really care about the order of the values for the new index values. SQL Server offers updatable CTEs and window functions that do exactly what you want:
with toupdate as (
select t.*, row_number() over (partition by id order by (select NULL)) as newindex
from table t
)
update toupdate
set index = newindex;
If you want them in a specific order, then you need another column to specify the ordering. The existing index column doesn't work.
With Row_number() -1 and CTE you can write as:
CREATE TABLE #temp1(
Id int,
[Index] int)
INSERT INTO #temp1 VALUES (1,3),(1,1),(2,1),(3,3),(1,5);
--select * from #temp1;
With CTE as
(
select t.*, row_number() over (partition by id order by (select null))-1 as newindex
from #temp1 t
)
Update CTE
set [Index] = newindex;
select * from #temp1;
Demo
I'm not sure why you would want to do this really, but I had fun figuring it out!
This solution relies on your table having a primary key for the self join... but you could always create an auto inc index if none exists and this is a one off job... This will also have the added benefit of getting you to think about the precise ordering of this you want... as currently there is no way of saying which order [ID] will get [Index] in.
UPDATE dbo.Example
SET [Index] = b.newIndex
FROM dbo.Example a
INNER JOIN (
select
z.ID,
z.[Index],
(row_number() over (partition by ID order by (select NULL))) as newIndex
from Example z
) b ON a.ID = b.ID AND a.[Index]=b.[Index] --Is this a unique self join for your table?.. no PK provided. You might need to make an index first.
Probably, this is what you want
SELECT *,RANK() OVER(PARTITION BY Id ORDER BY [Index])-1 AS NewIndex FROM
(
SELECT 1 AS Id,3 [Index]
UNION
SELECT 1,1
UNION
SELECT 2,1
UNION
SELECT 3,3
UNION
SELECT 1,5
) AS T
& the result will come as
Now if you want to update the table then execute this script
UPDATE tblname SET Index=RANK() OVER(PARTITION BY t.Id ORDER BY t.[Index])-1
FROM tblname AS t
In case I am missing something or any further assistance is required please let me know.
CREATE TABLE #temp1(
Id int,
Value int)
INSERT INTO #temp1 VALUES (1,2),(1,3),(2,3),(4,5)
SELECT
Id
,Value
,ROW_NUMBER() OVER (PARTITION BY Id ORDER BY Id) Id
FROM #temp1
Start with this :)
Gave me results like
Id Value Count
1 2 1
1 3 2
1 2 3
1 3 4
1 2 5
1 3 6
1 2 7
1 3 8
2 3 1
2 4 2
2 5 3
2 3 4
2 4 5
2 5 6
2 4 7
2 5 8
2 3 9
2 3 10
3 4 1
4 5 1
4 5 2
4 5 3
4 5 4