I have the following table that I want to group by type. When there are multiple rows with the same type (e.g., A & B type), I want to preserve the 'value' from the row with the highest rank (i.e., primary > secondary > tertiary..)
rowid | type | rank | value
1 | A | primary | 1
2 | A | secondary | 2
3 | B | secondary | 3
4 | B | tertiary | 4
5 | C | primary | 5
So the resulting table should look like
rowid | type | rank | value
1 | A | primary | 1
3 | B | secondary | 3
5 | C | primary | 5
Any suggestions will be highly appreciated!
p.s., I'm working in MS SQL Server.
You can use row_number(). Here is a simple'ish method:
select t.*
from (select t.*,
row_number() over (partition by type
order by charindex(rank, 'primary,secondary,tertiary')
) as seqnum
from t
) t
where seqnum = 1;
This uses charindex() as a simple method of ordering the ranks.
try this,
;WITH CTE
AS (
SELECT *
,row_number() OVER (
PARTITION BY [type] ORDER BY value
) rn
FROM #t
)
SELECT *
FROM cte
WHERE rn = 1
Another way of doing is with Row_Number and an Order By specifying your rule with CASE.
Schema:
CREATE TABLE #TAB(rowid INT, [type] VARCHAR(1), rankS VARCHAR(50) , value INT)
INSERT INTO #TAB
SELECT 1 , 'A' , 'primary' , 1
UNION ALL
SELECT 2 , 'A' , 'secondary', 2
UNION ALL
SELECT 3 , 'B' , 'secondary' , 3
UNION ALL
SELECT 4 , 'B' , 'tertiary' , 4
UNION ALL
SELECT 5 , 'C' , 'primary' , 5
Now apply rank rule with Row_Number
SELECT * FROM (
SELECT ROW_NUMBER() OVER(PARTITION BY [type] ORDER BY (CASE rankS
WHEN 'primary' THEN 1
WHEN 'secondary' THEN 2
WHEN 'tertiary' THEN 3 END )) AS SNO, * FROM #TAB
)A
WHERE SNO =1
Result:
+-----+-------+------+-----------+-------+
| SNO | rowid | type | rankS | value |
+-----+-------+------+-----------+-------+
| 1 | 1 | A | primary | 1 |
| 1 | 3 | B | secondary | 3 |
| 1 | 5 | C | primary | 5 |
+-----+-------+------+-----------+-------+
Related
Assume I have this table:
+----+-------+
| id | value |
+----+-------+
| 1 | 5 |
| 2 | 4 |
| 3 | 1 |
| 4 | NULL |
| 5 | NULL |
| 6 | 14 |
| 7 | NULL |
| 8 | 0 |
| 9 | 3 |
| 10 | NULL |
+----+-------+
I want to write a query that will replace any NULL value with the last value in the table that was not null in that column.
I want this result:
+----+-------+
| id | value |
+----+-------+
| 1 | 5 |
| 2 | 4 |
| 3 | 1 |
| 4 | 1 |
| 5 | 1 |
| 6 | 14 |
| 7 | 14 |
| 8 | 0 |
| 9 | 3 |
| 10 | 3 |
+----+-------+
If no previous value existed, then NULL is OK. Ideally, this should be able to work even with an ORDER BY. So for example, if I ORDER BY [id] DESC:
+----+-------+
| id | value |
+----+-------+
| 10 | NULL |
| 9 | 3 |
| 8 | 0 |
| 7 | 0 |
| 6 | 14 |
| 5 | 14 |
| 4 | 14 |
| 3 | 1 |
| 2 | 4 |
| 1 | 5 |
+----+-------+
Or even better if I ORDER BY [value] DESC:
+----+-------+
| id | value |
+----+-------+
| 6 | 14 |
| 1 | 5 |
| 2 | 4 |
| 9 | 3 |
| 3 | 1 |
| 8 | 0 |
| 4 | 0 |
| 5 | 0 |
| 7 | 0 |
| 10 | 0 |
+----+-------+
I think this might involve some kind of analytic function - somehow partitioning over the value column - but I'm not sure where to look.
You can use a running sum to set groups and use max to fill in the null values.
select id,max(value) over(partition by grp) as value
from (select id,value,sum(case when value is not null then 1 else 0 end) over(order by id) as grp
from tbl
) t
Change the over() clause to order by value desc to get the second result in the question.
The best way has been covered by Itzik Ben-Gan here:The Last non NULL Puzzle
Below is a solution which for 10 million rows and completes around in 20 seconds on my system
SELECT
id,
value1,
CAST(
SUBSTRING(
MAX(CAST(id AS binary(4)) + CAST(value1 AS binary(4)))
OVER (ORDER BY id
ROWS UNBOUNDED PRECEDING),
5, 4)
AS int) AS lastval
FROM dbo.T1;
This solution assumes your id column is indexed
You can also try using correlated subquery
select id,
case when value is not null then value else
(select top 1 value from table
where id < t.id and value is not null order by id desc) end value
from table t
Result :
id value
1 5
2 4
3 1
4 1
5 1
6 14
7 14
8 0
9 3
10 3
If the NULLs are scattered I use a WHILE loop to fill them in
However if the NULLs are in longer consecutive strings there are faster ways to do it.
So here's one approach:
First find a record that we want to update. It has NULL in this record and no NULL in the prior record
SELECT C.VALUE, N.ID
FROM TABLE C
INNER JOIN TABLE N
ON C.ID + 1 = N.ID
WHERE C.VALUE IS NOT NULL
AND N.VALUE IS NULL;
Use that to update: (bit hazy on this syntax but you get the idea)
UPDATE N
SET VALUE = C.Value
FROM TABLE C
INNER JOIN TABLE N
ON C.ID + 1 = N.ID
WHERE C.VALUE IS NOT NULL
AND N.VALUE IS NULL;
.. now just keep doing it till you run out of rows
-- This is needed to set ##ROWCOUNT to non zero
SELECT 1;
WHILE ##ROWCOUNT <> 0
BEGIN
UPDATE N
SET VALUE = C.Value
FROM TABLE C
INNER JOIN TABLE N
ON C.ID + 1 = N.ID
WHERE C.VALUE IS NOT NULL
AND N.VALUE IS NULL;
END
The other way is to use a similiar query to get a range of id's to update. This works much faster if your NULLS are usually against consecutive id's
Here is the one simple approach using OUTER APPLY
CREATE TABLE #table(id INT, value INT)
INSERT INTO #table VALUES
(1,5),
(2,4),
(3,1),
(4,NULL),
(5,NULL),
(6,14),
(7,NULL),
(8,0),
(9,3),
(10,NULL)
SELECT t.id, ISNULL(t.value, t3.value) value
FROM #table t
OUTER APPLY(SELECT id FROM #table WHERE id = t.id AND VALUE IS NULL) t2
OUTER APPLY(SELECT TOP 1 value
FROM #table WHERE id <= t2.id AND VALUE IS NOT NULL ORDER BY id DESC) t3
OUTPUT:
id VALUE
---------
1 5
2 4
3 1
4 1
5 1
6 14
7 14
8 0
9 3
10 3
Using this sample data:
if object_id('tempdb..#t1') is not null drop table #t1;
create table #t1 (id int primary key, [value] int null);
insert #t1 values(1,5),(2,4),(3,1),(4,NULL),(5,NULL),(6,14),(7,NULL),(8,0),(9,3),(10,NULL);
I came up with:
with x(id, [value], grouper) as (
select *, row_number() over (order by id)-sum(iif([value] is null,1,0)) over (order by id)
from #t1)
select id, min([value]) over (partition by grouper)
from x;
I noticed, however, that Vamsi Prabhala beat me to it... My solution is identical to what he posted. (arghhhh!). So I thought I'd try a recursive solution. Here's a pretty efficient use of a recursive cte (provided that ID is indexed):
with sorted as (select *, seqid = row_number() over (order by id) from #t1),
firstRecord as (select top(1) * from #t1 order by id),
prev as
(
select t.id, t.[value], lastid = 1, lastvalue = null
from sorted t
where t.id = 1
union all
select t2.id, t2.[value], lastid+1, isnull(prev.[value],lastvalue)
from sorted t2
join prev on t2.id = prev.lastid+1
)
select id, [value]=isnull([value],lastvalue)--, *
from prev;
Normally I don't like recursive cte's (rCte for short) but in this case it offered an elegant solution and was faster than using the window aggregate function (sum over, min over...). Note the execution plans, the rcte on the bottom. The rCTE get's it done with two index seeks, one of which is for just one row. Unlike the window aggregate solution, the rcte does not require a sort. Running this with statistics io on; the rcte produces much less IO.
All this said, don't use either of these solutions, What the TheGameiswar posted will perform the best by far. His solution on a properly indexed id column would be lightening fast.
Following UPDATE statement can be used, please test it before use
update #table
set value = newvalue
from (
select
s.id, s.value,
(select top 1 t.value from #table t where t.id <= s.id and t.value is not null order by t.id desc) as newvalue
from #table S
) u
where #table.id = u.id and #table.value is null
stop worrying..here's the answer for you :)
SELECT *
INTO #TempIsNOtNull
FROM YourTable
WHERE value IS NOT NULL
SELECT *
INTO #TempIsNull
FROM YourTable
WHERE value IS NULL
UPDATE YourTable
SEt YourTable.value = UpdateDtls.value
FROM YourTable
JOIN (
SELECT OuterTab1.id,
#TempIsNOtNull.value
FROM #TempIsNull OuterTab1
CROSS JOIN #TempIsNOtNull
WHERE OuterTab1.id - #TempIsNOtNull.id > 0
AND (OuterTab1.id - #TempIsNOtNull.id) = ( SELECT TOP 1
OuterTab1.id - #TempIsNOtNull.id
FROM #TempIsNull InnerTab
CROSS JOIN #TempIsNOtNull
WHERE OuterTab1.id - #TempIsNOtNull.id > 0
AND OuterTab1.id = InnerTab.id
ORDER BY (OuterTab1.id - #TempIsNOtNull.id) ASC) ) AS UpdateDtls
ON (YourTable.id = UpdateDtls.id)
I have a SQL Server table that has duplicate entries in one of the columns e.g.:
+----+-----------+------------+
| id | object_id | status_val |
+----+-----------+------------+
| 1 | 1 | 0 |
| 2 | 1 | 0 |
| 3 | 1 | 0 |
| 4 | 2 | 0 |
| 5 | 3 | 0 |
| 6 | 4 | 0 |
| 7 | 4 | 0 |
+----+-----------+------------+
I need the output to be like this:
+----+-----------+------------+
| id | object_id | status_val |
+----+-----------+------------+
| 4 | 2 | 0 |
| 5 | 3 | 0 |
+----+-----------+------------+
How to resolve this?
Is this what you are looking for?
SELECT * FROM <yourTable> t1
WHERE t1.object_id NOT IN
(
SELECT t2.object_id
FROM <yourTable> t2
GROUP BY t2.object_id
HAVING COUNT(object_id) > 1
)
Try this:
select min(id),
object_id,
min(status_val)
from table
group by object_id
having count(*) = 1
Use HAVING and GROUP BY
SELECT MIN(id) id, object_id, MIN(status_val) status_val
FROM yourtable
GROUP BY object_id
HAVING COUNT(object_id) = 1
Output
id object_id status_val
4 2 0
5 3 0
SQL Fiddle: http://sqlfiddle.com/#!6/7f643f/9/0
You can use group by for unique record like below :-
SELECT * from TABLENAME
group by TABLE_COLOUM_NAME
This query give you only unique value from your Table.
Give a row number for each row partitioned and ordered by the columns [object_id], [status_val]. Then from the result set select the rows which having maximum row number 1.
Query
;with cte as(
select [rn] = row_number() over(
partition by [object_id], [status_val]
order by [object_id], [status_val]
), *
from [your_table_name]
)
select min([id]) as [id], [object_id], [status_val]
from cte
group by [object_id], [status_val]
having max([rn]) = 1;
Find a demo here
SELECT COUNT(*)
FROM(
SELECT DISTINCT object_id
FROM object_table ) as row_count, status_val,id,object_id FROM object_table where row_count = 1;
I think you are looking for that
I'm looking for options on how to pick the first record per group, in SQLite, Where the sorting of the group is across a composite key.
Example Table:
Key_1 | Sort1 | Sort2 | Val_1 | Val_2
-------+-------+-------+-------+-------
1 | 1 | 3 | 0 | 2
1 | 1 | 2 | 2 | 4
1 | 1 | 1 | 4 | 6
1 | 2 | 2 | 6 | 8
1 | 2 | 1 | 8 | 1
2 | 1 | 2 | 0 | 5
2 | 1 | 1 | 1 | 6
2 | 2 | 3 | 2 | 7
2 | 2 | 2 | 3 | 8
2 | 2 | 1 | 4 | 9
Objective:
- Sort data by Key_1 ASC, Sort1 ASC, Sort2 DESC
- Select first record per unique Key_1
Key_1 | Sort1 | Sort2 | Val_1 | Val_2
-------+-------+-------+-------+-------
1 | 1 | 3 | 0 | 2
2 | 1 | 2 | 0 | 5
Analytic Function Solution...
SELECT
*
FROM
(
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY Key_1
ORDER BY Sort1,
Sort2 DESC
)
AS group_ordinal
FROM
table
)
sorted
WHERE
group_ordinal = 1
Laborious ANSI-92 approach...
SELECT
table.*
FROM
table
INNER JOIN
(
SELECT
table.Key1, table.Sort1, MAX(table.Sort2) AS Sort2
FROM
table
INNER JOIN
(
SELECT
Key_1, MIN(Sort1)
FROM
table
GROUP BY
Key_1
)
first_Sort1
ON table.Key_1 = first_Sort1.Key_1
AND table.Sort1 = first_Sort1.Sort1
GROUP BY
table.Key1, table.Sort1
)
first_Sort1_last_Sort2
ON table.Key_1 = first_Sort1_last_Sort2.Key_1
AND table.Sort1 = first_Sort1_last_Sort2.Sort1
AND table.Sort2 = first_Sort1_last_Sort2.Sort2
This involves a lot of nesting and self joins. Which is cumbersome enough when it involves just two sort columns.
My actual example has six sort columns.
I also would like to avoid anything like the following, as it is not (to my knowledge) guaranteed / deterministic...
SELECT
table.*
FROM
table
GROUP BY
table.Key_1
ORDER BY
MIN(table.Sort1),
MAX(table.Sort2)
Are there any other options that I'm just not seeing?
I believe this will work in SQLite:
select t.*
from table t
where exists (select 1
from (select t2.*
from table t2
where t2.id = t.id
order by t2.sort1 asc, t2.sort2 desc
limit 1
) t2
where t2.sort1 = t.sort1 and t2.sort2 = t.sort2
);
My concern is whether SQLite allows correlated references in nested subqueries. If not, you can just use = and concatenate the values together:
select t.*
from table t
where (sort1 || ':' || sort2) =
(select (sort1 || ':' || sort2)
from table t2
where t2.id = t.id
order by sort1 asc, sort2 desc
limit 1
);
I have a table with data like the following
key | A | B | C
---------------------------
1 | x | 0 | 1
2 | x | 2 | 0
3 | x | NULL | 4
4 | y | 7 | 1
5 | y | 3 | NULL
6 | z | NULL | 4
And I want to merge the rows together based on column A with largest primary key being the 'tie breaker' between values that are not NULL
Result
key | A | B | C
---------------------------
1 | x | 2 | 4
2 | y | 3 | 1
3 | z | NULL | 4
What would be the best way to achieve this assuming my data is actually 40 columns and 1 million rows with an unknown level of duplications?
Using ROW_NUMBER and conditional aggregation:
SQL Fiddle
WITH cte AS(
SELECT *,
rnB = ROW_NUMBER() OVER(PARTITION BY A ORDER BY CASE WHEN B IS NULL THEN 0 ELSE 1 END DESC, [key] DESC),
rnC = ROW_NUMBER() OVER(PARTITION BY A ORDER BY CASE WHEN C IS NULL THEN 0 ELSE 1 END DESC, [key] DESC)
FROM tbl
)
SELECT
[key] = ROW_NUMBER() OVER(ORDER BY A),
A,
B = MAX(CASE WHEN rnB = 1 THEN B END),
C = MAX(CASE WHEN rnC = 1 THEN C END)
FROM cte
GROUP BY A
I have this table shown below...How do I select only the latest data of the id based on changeno?
+----+--------------+------------+--------+
| id | data | changeno | |
+----+--------------+------------+--------+
| 1 | Yes | 1 | |
| 2 | Yes | 2 | |
| 2 | Maybe | 3 | |
| 3 | Yes | 4 | |
| 3 | Yes | 5 | |
| 3 | No | 6 | |
| 4 | No | 7 | |
| 5 | Maybe | 8 | |
| 5 | Yes | 9 | |
+----+---------+------------+-------------+
I would want this result...
+----+--------------+------------+--------+
| id | data | changeno | |
+----+--------------+------------+--------+
| 1 | Yes | 1 | |
| 2 | Maybe | 3 | |
| 3 | No | 6 | |
| 4 | No | 7 | |
| 5 | Yes | 9 | |
+----+---------+------------+-------------+
I currently have this SQL statement...
SELECT id, data, MAX(changeno) as changeno FROM Table1 GROUP BY id;
and clearly it doesn't return what I want. This should return an error because of the aggrerate function. If I added fields under the GROUP BY clause it works but it doesn't return what I want. The SQL statement is by far the closest I could think of. I'd appreciate it if anybody could help me on this. Thank you in advance :)
This is typically referred to as the "greatest-n-per-group" problem. One way to solve this in SQL Server 2005 and higher is to use a CTE with a calculated ROW_NUMBER() based on the grouping of the id column, and sorting those by largest changeno first:
;WITH cte AS
(
SELECT id, data, changeno,
rn = ROW_NUMBER() OVER (PARTITION BY id ORDER BY changeno DESC)
FROM dbo.Table1
)
SELECT id, data, changeno
FROM cte
WHERE rn = 1
ORDER BY id;
You want to use row_number() for this:
select id, data, changeno
from (SELECT t.*,
row_number() over (partition by id order by changeno desc) as seqnum
FROM Table1 t
) t
where seqnum = 1;
Not a well formed or performance optimized query but for small tasks it works fine.
SELECT * FROM TEST
WHERE changeno IN (SELECT MAX(changeno)
FROM TEST
GROUP BY id)
for other alternatives :
DECLARE #Table1 TABLE
(
id INT, data VARCHAR(5), changeno INT
);
INSERT INTO #Table1
SELECT 1,'Yes',1
UNION ALL
SELECT 2,'Yes',2
UNION ALL
SELECT 2,'Maybe',3
UNION ALL
SELECT 3,'Yes',4
UNION ALL
SELECT 3,'Yes',5
UNION ALL
SELECT 3,'No',6
UNION ALL
SELECT 4,'No',7
UNION ALL
SELECT 5,'Maybe',8
UNION ALL
SELECT 5,'Yes',9
SELECT Y.id, Y.data, Y.changeno
FROM #Table1 Y
INNER JOIN (
SELECT id, changeno = MAX(changeno)
FROM #Table1
GROUP BY id
) X ON X.id = Y.id
WHERE X.changeno = Y.changeno
ORDER BY Y.id