Replace NULL with values - sql

Here is my challenge:
I have a log table which every time a record is changed adds a new record but puts a NULL value for each non-changed value in each record. In other words only the changed value is set, the rest unchanged fields in each row simply has a NULL value.
Now I would like to replace each NULL value with the value above it that is NOT a NULL value like below:
Source table: Task_log
ID Owner Status Flag
1 Bob Registrar T
2 Sue NULL NULL
3 NULL NULL F
4 Frank Admission T
5 NULL NULL F
6 NULL NULL T
Desired output table: Task_log
ID Owner Status Flag
1 Bob Registrar T
2 Sue Registrar T
3 Sue Registrar F
4 Frank Admission T
5 Frank Admission F
6 Frank Admission T
How do I write a query which will generate the desired output table?

One the new windowed function of SQLServer 2012 is FIRST_VALUE, wich have quite a direct name, it can be partitioned through the OVER clause, before using it is necessary to divide every column in data block, a block for a column begin when a value is found.
With Block As (
Select ID
, Owner
, OBlockID = SUM(Case When Owner Is Null Then 0 Else 1 End)
OVER (ORDER BY ID)
, Status
, SBlockID = SUM(Case When Status Is Null Then 0 Else 1 End)
OVER (ORDER BY ID)
, Flag
, FBlockID = SUM(Case When Flag Is Null Then 0 Else 1 End)
OVER (ORDER BY ID)
From Task_log
)
Select ID
, Owner = FIRST_VALUE(Owner) OVER (PARTITION BY OBlockID ORDER BY ID)
, Status = FIRST_VALUE(Status) OVER (PARTITION BY SBlockID ORDER BY ID)
, Flag = FIRST_VALUE(Flag) OVER (PARTITION BY FBlockID ORDER BY ID)
FROM Block
SQLFiddle demo
The UPDATE query is easily derived

As I mentioned in my comment, I would try to fix the process that is creating the records rather than fixing the junk data. If that is not an option, the code below should get you pointed in the right direction.
UPDATE t1
set t1.owner = COALESCE(t1.owner, t2.owner),
t1.Status = COALESCE(t1.status, t2.status),
t1.Flag = COALESCE(t1.flag, t2.flag)
FROM Task_log as t1
INNER JOIN Task_log as t2
ON t1.id = (t1.id + 1)
where t1.owner is null
OR t1.status is null
OR t1.flag is null

I can think of several approaches.
You could use a combination of COALESCE with an array aggregate function. Unfortunately it doesn't look like SQL Server supports array_agg natively (although some nice people have developed some workarounds).
You could also use a subselect for each column.
SELECT id,
(SELECT TOP 1 FROM (SELECT owner FROM ... WHERE id = outer_id AND owner IS NOT NULL order by ID desc )) AS owner,
-- other columns
You could probably do something with window functions, too.

A vanilla solution would be:
select id
, owner
, coalesce(owner, ( select owner from t t2
where id = (select max(id) from t t3
where id < t1.id and owner is not null))
) as new_owner
, flag
, coalesce(flag, ( select flag from t t2
where id = (select max(id) from t t3
where id < t1.id and flag is not null))
) as new_flag
from t t1
Rather inefficient, but should work on most DBMS

Related

max latest null value sql

I am experiencing the following problem. I have a table in this table I same history. Due to an error, I'm interested in finding the following information.
The latest record for a user where column1 value is null and the modifiedon date is the newest for this user. The problem is the table contains more records where the modifiedon is not null for this user and mutated after the date I'm looking for.
Can someone please point me in the right direction?
Sample data:
personid FreeField01 ModifiedOn
1 0004998 15-10-2019 11:48:19
1 NULL 20-10-2019 01:53:39
1 0004998 22-10-2019 14:58:44
1 0004998 22-10-2019 14:58:44
1 NULL 23-10-2019 07:52:46
1 0004998 23-10-2019 17:16:45
So for this user, I'm not interested in any record and should be excluded from the result because the modified on datetime should be before 29-10 and before that date the freefield01 value should be null and modifiedon should be the latest.
Three conditions:
There is no newer entry for the person.
The entry value is NULL.
The date is before 2019-10-29.
The query:
select *
from mytable
where not exists
(
select *
from mytable newer
where newer.personid = mytable.personid
and newer.modifiedon > mytable.modifiedon
)
and freefield01 is null
and modifiedon < date '2019-10-29'
order by personid;
You can use this below script-
WITH CTE
AS(
SELECT personid,MAX(ModifiedOn) MD
FROM your_table
GROUP BY personid
HAVING MAX(ModifiedOn) < '30-10-2019'
)
SELECT * FROM your_table A
INNER JOIN CTE B ON A.personid = B.personid
AND A.ModifiedOn = B.MD
AND A.FreeField01 IS NULL
DEMO HERE
If I understand correctly, you are looking for persons where the FreeField01 has a value of NULL as of a certain date.
Here is one method:
select t.*
from t
where t.ModifiedOn = (select max(t2.ModifiedOn)
from t t2
where t2.personid = t.personid and
t2.ModifiedOn <= '2019-10-29'
) and
t.FreeField01 is null;
EDIT:
Based on your comment, you might just want an aggregation and having:
select personid
from t
where t.ModifiedOn <= '2019-10-29'
group by person_id
having sum(case when t.FreeField01 is null then 1 else 0 end) = 0
The simplest query that I found might be the following if I understand your request well :
SELECT t.personid, t.FreeField01, MAX(ModifiedOn) FROM test t
GROUP BY personid
HAVING MAX(ModifiedOn) < '29-10-2019' AND FreeField01 IS NULL
SEE EXAMPLE HERE
EDIT : Following below suggestions you can use this query instead :
SELECT t1.personid, t1.FreeField01, t1.ModifiedOn
FROM test t1
JOIN (
SELECT t.personid, MAX(ModifiedOn) AS MaxModifiedOn FROM test t
GROUP BY personid
HAVING MAX(ModifiedOn) < STR_TO_DATE('29-10-2019','%d-%m-%Y')
) t2 ON (t1.personid = t2.personid AND t1.ModifiedOn = t2.MaxModifiedOn)
WHERE FreeField01 IS NULL
SEE NEW DEMO HERE

Querying a subset

I want to write an SQL query to find records which contain a particular column and from that subset want to find records which doesn't contain a some other value. How do you write a query for that?
cid id2 attribute
--------------------------------
1 100 delete
1 100 payment
1 100 void
2 100 delete
2 102 payment
2 102 void
3 102 delete
3 103 payment
In above example, I want to list cid for which payment and delete attributes exist but void attribute doesn't exist. So it should list out 3 from above example because it doesn't have void attribute.
Forgot to mention that there could be more attributes. However, I need to list out records for which delete and payment exist regardless of other attributes but void doesn’t.
I call this a "set-within-sets" query, because you are looking for particular sets of attributes within each cid.
I would express this with group by and conditions in the having:
select cid
from t
group by cid
having sum(case when attribute = 'payment' then 1 else 0 end) > 0 and
sum(case when attribute = 'delete' then 1 else 0 end) > 0 and
sum(case when attribute = 'void' then 1 else 0 end) = 0 ;
In some databases, you can simplify this with string aggregation -- assuming there are no duplicate attributes for cids. For instance, using the MySQL function:
select cid
from t
where attribute in ('payment', 'delete' 'void')
group by cid
having group_concat(attribute order by attribute) = 'delete,payment';
You can use conditional aggregation:
select cid
from tablename
where attribute in ('delete', 'payment', 'void')
group by cid
having
count(distinct attribute) = 2
and
sum(
case attribute
when 'void' then 1
else 0
end
) = 0
If there are not more attributes than these 3, then you can omit the WHERE clause.
See the demo.
Results:
| cid |
| --- |
| 3 |
I'm assuming that there are only three attributes, so the logic behind this query is:
First COUNT the number of attributes GROUP BY cid, and then LEFT JOIN the original table ON attribute is void. You should grab cid that has exactly 2 attributes and no void.
The original table is named as temp:
SELECT
subq2.result_cid
FROM (
SELECT
*
FROM (
SELECT
T.cid AS result_cid,
COUNT(T.attribute) AS count
FROM
temp AS T
GROUP BY
T.cid
) AS subq
LEFT OUTER JOIN temp AS T2 ON subq.result_cid = T2.cid AND T2.attribute = 'void'
) AS subq2
WHERE subq2.count = 2 AND subq2.id2 IS NULL
use corelated subquery by using not exists
select t1.* from tablename t1
where not exists( select 1 from tablename t2
where t1.cid=t2.cid and attribute='void'
)
and exists ( select 1 from tablename t2
where t1.cid=t2.cid
having count(distinct attribute)=2
)
and attribute in ('payment','delete')
demo online

How to create a query to extract a value from my table

I have an oracle table and i want to extract a value from column ,this my table
id|document_number|container_id|state|
--|---------------|------------|-----|
1 |CC330589 |356 | 40 |
--------------------------------------
1 |CC330589 |null | 99 |
-------------------------------------
I want to create a query that extract the container_id (value 356 ) whhere the document_number is 'CC330589' and the state is 99.
In my case I want the value 356 (not the null value).
Any idea how I can create the query ? Thanks in advance.
You can try below -
select * from tablename A
inner join
(select document_number, min(container_id ) from tablename group by document_number)B
on A.document_number=B.document_number
where A.document_number='CC330589' and state=99
You can use fid all not-null values for the given document and use EXISTS to ensure that state 99 exists for the given document:
SELECT *
FROM t
WHERE document_number = 'CC330589'
AND container_id IS NOT NULL
AND EXISTS (
SELECT 1
FROM t AS x
WHERE document_number = t.document_number
AND state = 99
)
This will get the result set you specify from the sample data. Whether it's the correct solution depends on your business rules which you have not specified:
select document_number
, max(container_id) as container_id
, max(state) as state
from your_table
group by document_number
Here is another solution
select t1.document_number
, coalesce(t1.container_id, t2.container_id) as container_id
, t1.state as state
from your_table t1
join your_table t2
on t1.document_number = t2.document_number
where t1.state = 99
and t1.document_number = 'CC330589'
and t2.state != 99
Again, this will produce the specified result from the sample data but may be incorrect against your real data set.

identify rows with not null values in sql

How to retrieve all rows having value in a status column (not null) group by ID column.
Id Name Status
1394 Test 1 Y
1394 Test 2 null
1394 Test 3 null
1395 Test 4 Y
1395 Test 5 Y
I wrote like select * from table where status = 'Y'. It brings me 3 records, how to add condition to bring in only last 2? the 1394 ID have other 2 records, which status is null.
If you want to select groups where the status is only y, you can do:
select t.*
from t
where not exists (select 1
from t t2
where t2.id = t.id and
(t2.Status <> 'Y' or t2.status is null)
);
If you only want the ids, I would use group by and having:
select id
from t
group by id
having min(status) = 'Y' and max(status) = 'Y' and count(*) = count(status);
The last condition checks for no NULL values.
You could also write:
having min(status = 'Y' then 1 else 0 end) = 1
A simple way is:
select * from mytable
where status = 'Y'
and id not in (select id from mytable where status is null)
The existing query "where status = 'Y'" will bring you not null by definition.
If you are trying to get grouped results, a "GROUP BY id" clause will achieve this, which will also require putting id in the select explicitly instead of "*".
Example: SELECT id, COUNT(id) from table where status = 'Y'
If I am reading this correctly you want to bring in the ID for a grouping that never has a NULL status value:
I would use a subquery with a not-exist:
SELECT DISTINCT ID FROM mytable WHERE status IS NULL;
Then filter IDs that do not exist in that list:
SELECT * FROM mytable WHERE id NOT IN (SELECT DISTINCT ID FROM mytable WHERE status IS NULL);
Here are some possible solutions, because I am unclear on exactly what you want as output:
Select Id, Name, Status from table where status is not null;
results in 3 rows:
Id Name Status
1394 Test 1 Y
1395 Test 4 Y
1395 Test 5 Y
Select Id, count(*) as anAmt from table where status is not null group by Id;
/* only retrieves counts per Id */
results in 1 row for each Id:
Id anAmt
1394 1
1395 2

Query to find ranges of consecutive rows

I have file that contains a dump of a SQL table with 2 columns: int ID (auto increment identity field) and bit Flag. The flag = 0 means a record is good and the flag = 1 means a record is bad (contains an error). The goal is to find all blocks of consecutive bad records (with flag value of 1) with 1,000 or more rows. The solution shouldn't use cursors or while loops and it should use the set-based queries only (selects, joins etc).
We would like to see the actual queries used and the results in the following format:
StartID – EndID NumberOfErrorsInTheBlock
StartID – EndID NumberOfErrorsInTheBlock
……………………….
StartID – EndID NumberOfErrorsInTheBlock
For example if our data were only 30 records and we were looking for blocks with 5 or more records then the results would look as follows (see the screenshot below, the errors blocks that met the criteria are highlighted) :
[ID Range].....[Number of errors in the block]
11-15..... 5
19-25..... 7
sql file containing sample rows, dropbox
T-SQL Solution for SQL Server 2012 and Above
IF OBJECT_ID('tempdb..#tbl_ranges') IS NOT NULL
DROP TABLE #tbl_ranges;
CREATE TABLE #tbl_ranges
(
row_num INT PRIMARY KEY,
ID INT,
Flag BIT,
Label TINYINT
);
WITH cte_yourTable
AS
(
SELECT Id,
Flag,
CASE
--label min
WHEN Flag != LAG(flag,1) OVER (ORDER BY ID) THEN 1
--inner
WHEN Flag = LAG(flag,1) OVER (ORDER BY ID) AND Flag = LEAD(flag,1) OVER (ORDER BY ID) THEN 2
--end
WHEN Flag = LAG(flag,1) OVER (ORDER BY ID) AND Flag != LEAD(flag,1) OVER (ORDER BY ID) THEN 3
END label
FROM yourTable
)
INSERT INTO #tbl_ranges
SELECT ROW_NUMBER() OVER (ORDER BY ID) row_num,
ID,
Flag,
label
FROM cte_yourTable
WHERE label != 2;
SELECT A.ID ID_start,
B.ID ID_end,
B.ID - A.ID range_cnt
FROM #tbl_ranges A
INNER JOIN #tbl_ranges B
ON A.row_num = B.row_num - 1
AND A.Flag = B.Flag;
IF OBJECT_ID('tempdb..#tbl_ranges') IS NOT NULL
DROP TABLE #tbl_ranges;
Abbreviated Results:
ID_start ID_end range_cnt
----------- ----------- -----------
2 3 1
5 8 3
9 10 1
11 35 24
36 356 320
357 358 1
359 406 47
...
With out using Temp Table, This is the best solution, Here is the Answer and It is perfect example for CTE with in CTE ( Nested CTE )
With Evaluation (ID,Flag,Evaluate)
as
(select ID,Flag,Evaluate = ID-row_number() over (order by Flag,ID)
from [dbo].[SqltestRecordsNew]
where Flag = 1
),
Evaluation_Final (StartingRecordID,EndRecordID,Flag,cnt)
as
(
select min(ID) as StartingRecordID,max(ID) as EndRecordID,
Flag, cnt = count(*)
from Evaluation
group by Evaluate, Flag
)
select Concat(StartingRecordID,' - ', EndRecordID) as 'StartingRecordID - EndRecordId',
cnt as GroupItemCnt from Evaluation_Final
where cnt > 999
order by Concat(StartingRecordID,' - ', EndRecordID)
-- Test results Case 1
Select ID,Flag,
Case when Flag=1 then 'Success'
else 'Defect Data'
End as TestResults
from SqltestRecordsNew
where ID between 1494363 and 1495559
-- Test results Case 2
Select ID,Flag,
Case when Flag=1 then 'Success'
else 'Defect Data'
End as TestResults from SqltestRecordsNew
where ID between 1498409 and 1503899
-- Test results Case 3
Select ID,Flag,
Case when Flag=1 then 'Success'
else 'Defect Data'
End as TestResults from SqltestRecordsNew
where ID between 1548257 and 1550489