how to remove null values in a table sql - sql

I have a table (TestTable) as follows
PK | COL1 | COL2 | COL3 | COL4 | COL5
1 | 1 | NULL | NULL | NULL | NULL
2 | NULL | 43 | 1.5 | 7.8 | NULL
3 | NULL | NULL | NULL | NULL | 1
4 | 1 | NULL | NULL | NULL | NULL
5 | NULL | 48 | 10.5 | 17.8 | NULL
6 | NULL | NULL | NULL | NULL | 1
I would like a result as follows
PK | COL1 | COL2 | COL3 | COL4 | COL5
1 | 1 | 43 | 1.5 | 7.8 | 1
2 | 1 | 48 | 10.5 | 17.8 | 1
I have tried the following
Select
[COL1],
[COL2],
[COL3],
[COL4],
(select top 1
[COL5] from TestTable
where [COL5] is not null and PK <= t1.pk
order by PK DESC) as [COL5]
FROM TestTable as t1
Where
[COL1] IS NOT NULL AND
[COL2] IS NOT NULL AND
[COL3] IS NOT NULL AND
[COL4] IS NOT NULL
The script works however I get empty result set. Any thoughts?

It looks like you have a value in col1 followed by a set of rows that should be combined into one row until the next value in col1.
If so, you can do this by assigning to each row a count -- the number of non-null values in col1 on or before the row. This can then be used for aggregation.
You don't mention the database, so I'll do this using a correlated subquery:
select row_number() over (order by max(pk)) as pk,
max(col1) as col1,
max(col2) as col2,
max(col3) as col3,
max(col4) as col4,
max(col5) as col5
from (select t.*,
(select count(t2.col1)
from testtable t2
where t2.pk <= t.pk
) as grpid
from testtable t
) t
group by grpid;

You don't have any rows in TestTable where
[COL1] IS NOT NULL AND
[COL2] IS NOT NULL AND
[COL3] IS NOT NULL AND
[COL4] IS NOT NULL

Related

Create View with reverse Row values SQL

I have data like
name| col1 | col2 | col3 | col4 | col4 | col5 |
rv | rv1 | rv2 | rv3 | rv4 | | |
sgh | sgh1 | sgh2 | | | | |
vik | vik1 | vik2 | vik3 | vik4 |vik5 |vik6 |
shv | shv1 | shv2 | shv3 | shv4 |shv5 | |
Table Name: emp_data
to create View to get DATA like
name| col1 | col2 | col3 | col4 | col4 | col5 |
rv | rv4 | rv3 | rv2 | rv1 | | |
sgh | sgh2 | sgh1 | | | | |
vik | vik6 | vik5 | vik4 | vik3 |vik2 |vik1 |
shv | shv5 | shv4 | shv3 | shv2 |shv1 | |
MySql 8 supports LATERAL, this way you can sort values by positions and conditionally aggregate them back.
with tbl(name, col1, col2, col3 ,col4 ,col5 , col6) as
(
select 'rv ','rv1 ','rv2 ','rv3 ','rv4 ',null,null union all
select 'sgh','sgh1','sgh2', null,null,null,null union all
select 'vik','vik1','vik2','vik3','vik4','vik5','vik6' union all
select 'shv','shv1','shv2','shv3','shv4','shv5', null
)
select tbl.name, t.*
from tbl
, lateral (
select
max(case n when 1 then val end) col1,
max(case n when 2 then val end) col2,
max(case n when 3 then val end) col3,
max(case n when 4 then val end) col4,
max(case n when 5 then val end) col5,
max(case n when 6 then val end) col6
from (
select row_number() over( order by n) n, val
from (
select case when col1 is null then 99 else 6 end n, col1 val union all
select case when col2 is null then 99 else 5 end n, col2 val union all
select case when col3 is null then 99 else 4 end n, col3 val union all
select case when col4 is null then 99 else 3 end n, col4 val union all
select case when col5 is null then 99 else 2 end n, col5 val union all
select case when col6 is null then 99 else 1 end n, col6 val
) t
) t
) t
db<>fidle

Trying to write a query that will display duplicates results as null

I have a table that looks like the first example.
I'm trying to write a MSSQL2012 statement that that will display results like the second example.
Basically I want null values instead of duplicate values in columns 1 and 2. This is for readability purposes during reporting.
This seems like it should be possible, but I'm drawing a blank. No amount of joins or unions I've written has rendered the results I need.
| Col1 | Col2 | Col3 |
+------+------+------+
| 1 | 2 | 4 |
| 1 | 2 | 5 |
| 1 | 3 | 6 |
| 1 | 3 | 7 |
+------+------+------+
| Col1 | Col2 | Col3 |
+------+------+------+
| 1 | 2 | 4 |
| Null | null | 5 |
| null | 3 | 6 |
| null | null | 7 |
+------+------+------+
I would do this with no subqueries at all:
select (case when row_number() over (partition by col1 order by col2, col3) = 1
then col1
end) as col1,
(case when row_number() over (partition by col2 order by col3) = 1
then col2
end) as col2,
col3
from t
order by t.col1, t.col2, t.col3;
Note that the order by at the end of the query is very important. The result set that you want depends critically on the ordering of the rows. Without the order by, the result set could be in any order. So, the query might look like it works, and then suddenly fail one day or on a slightly different set of data.
Using a common table expression with row_number():
;with cte as (
select *
, rn_1 = row_number() over (partition by col1 order by col2, col3)
, rn_2 = row_number() over (partition by col1, col2 order by col3)
from t
)
select
col1 = case when rn_1 > 1 then null else col1 end
, col2 = case when rn_2 > 1 then null else col2 end
, col3
from cte
without the cte
select
col1 = case when rn_1 > 1 then null else col1 end
, col2 = case when rn_2 > 1 then null else col2 end
, col3
from (
select *
, rn_1 = row_number() over (partition by col1 order by col2, col3)
, rn_2 = row_number() over (partition by col1, col2 order by col3)
from t
) sub
rextester demo: http://rextester.com/UYA17142
returns:
+------+------+------+
| col1 | col2 | col3 |
+------+------+------+
| 1 | 2 | 4 |
| NULL | NULL | 5 |
| NULL | 3 | 6 |
| NULL | NULL | 7 |
+------+------+------+

sql copying duplicates

I'm trying to fill 'Col4' null values with the values above using 'Col1' and 'Col3'. Here is a brief explanation of the columns:
Col1 - User Entry date
Col2 - Calendar date
Col3 - Difference between the dates in Col1
Col4 - Entry amount
What the data currently looks like:
| Col1 | Col2 | Col3 | Col4
| 2016-01-01 | 2016-01-01 | 3 | $10
| null | 2016-01-02 | null | null
| null | 2016-01-03 | null | null
| 2016-01-04 | 2016-01-04 | 1 | $2
| 2016-01-05 | 2016-01-05 | 2 | $7
| null | 2016-01-06 | null | null
| 2016-01-07 | 2016-01-07 | 3 | $5
| null | 2016-01-08 | null | null
| null | 2016-01-09 | null | null
Expected Results:
| Col1 | Col2 | Col3 | Col4
| 2016-01-01 | 2016-01-01 | 3 | $10
| null | 2016-01-02 | null | $10
| null | 2016-01-03 | null | $10
| 2016-01-04 | 2016-01-04 | 1 | $2
| 2016-01-05 | 2016-01-05 | 2 | $7
| null | 2016-01-06 | null | $7
| 2016-01-07 | 2016-01-07 | 3 | $5
| null | 2016-01-08 | null | $5
| null | 2016-01-09 | null | $5
The formula works by counting the number in Col3 and then copying the corresponding amount in Col4 to the rows below. Col3 defines how many rows below (including its own row).
I'm currently using Sybase. I don't have any ideas on how I can do this. Any help would be appreciated thanks.
Note: this is a sample of data and the dates go from 2000 - 2016
In SQL Server By given sample data we can achieve the above result Set by using CTE
Sample Data :
declarE #Table1 TABLE
(Col1 varchar(10), Col2 varchar(10), Col3 varchar(4), Col4 varchar(4))
;
INSERT INTO #Table1
(Col1, Col2, Col3, Col4)
VALUES
('2016-01-01', '2016-01-01', '3', '$10'),
(NULL, '2016-01-02', NULL, NULL),
(NULL, '2016-01-03', NULL, NULL),
('2016-01-04', '2016-01-04', '1', '$2'),
('2016-01-05', '2016-01-05', '2', '$7'),
(NULL, '2016-01-06', NULL, NULL),
('2016-01-07', '2016-01-07', '3', '$5'),
(NULL, '2016-01-08', NULL, NULL),
(NULL, '2016-01-09', NULL, NULL)
;
Script :
WITH cte
AS (SELECT T1.Col1,
T1.Col2,
T1.Col3,
Col4 = COALESCE(T1.Col4, (SELECT TOP 1 Col4
FROM #Table1 T2
WHERE T2.Col2 < T1.Col2
AND Col4 IS NOT NULL
ORDER BY Col2 DESC))
FROM #Table1 T1)
UPDATE T
SET
T.Col4 = C.Col4
FROM #Table1 T
INNER JOIN cte C
ON T.col2 = C.col2
WHERE T.col1 IS NULL
OR T.col2 IS NULL
OR T.col3 IS NULL
Select * from #Table1

Issue in joining 2 datasets

I have two datasets like below:
1:
+---------------------------+
| Id | Col1 | Col2 | Col3 |
+---------------------------+
| 1 | abc | 0 | 01/01/2010 |
| 2 | def | 10 | 10/10/2011 |
+---------------------------+
2:
+-------------------------------------------+
| Id | Col4 | Col5 | Col6 |
+-------------------------------------------+
| 1 | abc | 0 | 01/01/2010 |
| 5 | xyz | 12 | 5/6/2013 |
+-------------------------------------------+
Now I want to combine both these into a single dataset which shows something like this:
+----------------------------------------------------------------------+
| ID | Col1 | Col2 | Col3 | Col4 | Col5 | Col6 |
+----------------------------------------------------------------------+
| 1 | abc | 0 | 01/01/2010 | abc | 0 | 01/01/2010 |
| 2 | def | 10 | 10/10/2011 | null | null | null |
| 5 | null | null | null | xyz | 12 | 5/6/2013 |
+----------------------------------------------------------------------+
The issue is not all ids in dataset 1 are in dataset 2 and vice versa. What i need as all data from datasets1 and 2 and only the common from 1 and 2 with 2 transposed with 1 as shown above. I have used pipe as a separator.
An inputs are highly appreciated. i tried everything like full outer join, inner join , CTE etc - nothing is working.
CREATE TABLE #TEMP1 (ID INT, Col1 VARCHAR(100), Col2 INT, Col3 DATETIME)
CREATE TABLE #TEMP2 (ID INT, Col4 VARCHAR(100), Col5 INT, Col6 DATETIME)
INSERT INTO #TEMP1 VALUES (1,'abc',0,'1/1/2010')
INSERT INTO #TEMP1 VALUES (1,'def',0,'1/1/2010')
INSERT INTO #TEMP2 VALUES (1,'abc',0,'1/1/2010')
INSERT INTO #TEMP2 VALUES (1,'def',0,'1/1/2010')
SELECT DISTINCT A.ID,A.Col1,A.Col2,A.Col3,B.Col4,B.Col5,B.Col6
FROM #TEMP1 A
FULL OUTER JOIN #TEMP2 B ON A.ID = B.ID
Thanks.
Try using below SQL :
select t1.Id , Col1 , Col2 , Col3 , Col4 , Col5 , Col6
from temp1 t1 left join temp2 t2
on t1.Id=t2.Id
union
select t2.Id , Col1 , Col2 , Col3 , Col4 , Col5 , Col6
from temp1 t1 right join temp2 t2
on t1.Id=t2.Id
Also, i tried on fiddle for you :
http://sqlfiddle.com/#!2/d60a1e/5

SQL Check if ungroup column values match

I have a sql table with following values
| col1 | col2| source | values
| 1 | 2 | A | null
| 1 | 2 | B | 1.0
| 1 | 2 | C | null
| 1 | 4 | A | 2.0
| 1 | 4 | B | 2.0
| 1 | 4 | C | 2.0
| 1 | 5 | A | null
| 1 | 5 | B | null
| 1 | 5 | C | null
How can I get an output with a group by of col1 and col2 with a flag:
all values match for a group ( flag = 1)
all values are null ( flag = 2)
some values is null (flag = 3)
Output:
| col1 | col2| flag
| 1 | 2 | 3
| 1 | 4 | 1
| 1 | 5 | 2
Or: based on your updated question:
SELECT
col1,
col2,
SUM(CASE WHEN SomeConditionHere THEN 1 ELSE 0 END) AS Flag
FROM Table1
GROUP BY col1, col2;
SQL Fiddle Demo
This will give you:
| COL1 | COL2 | FLAG |
----------------------
| 1 | 2 | 2 |
| 1 | 4 | 0 |
| 1 | 5 | 3 |
Note that: I assumed that the flag is how many NULL values are in the VALUES column, so I used "Values" IS NULL instead of SomeConditionHere.
I couldn't understand how the flag should be computed in the expected results you posted. You have to use the predicate that define your flag instead of "Values" IS NULL.
Update:
Try this:
WITH Flags
AS
(
SELECT
col1, col2,
COUNT(*) ValuesCount,
SUM(CASE WHEN "Values" IS NULL THEN 1 ELSE 0 END) AS NULLValues
FROM Table1
GROUP BY col1, col2
)
SELECT
col1,
col2,
Flag = CASE WHEN ValuesCount = NULLValues THEN 2
WHEN NULLVALUES = 0
AND ValuesCount = (SELECT COUNT(*)
FROM Table1 t2
WHERE t1.col1 = t2.col1
AND t1.col2 = t2.col2) THEN 1
ELSE 3
END
FROM Flags t1;
Updated SQL Fiddle Demo
This will give you:
| COL1 | COL2 | FLAG |
----------------------
| 1 | 2 | 3 |
| 1 | 4 | 1 |
| 1 | 5 | 2 |
In SQLServer2005+
;WITH cte AS
(
SELECT col1, col2, [values],
COUNT(CASE WHEN [values] IS NULL THEN 1 END) OVER(PARTITION BY col1, col2) AS cntNULL,
COUNT(*) OVER(PARTITION BY col1, col2) AS cntCol
FROM dbo.test5
)
SELECT col1, col2, MAX(CASE WHEN cntNULL = 0 THEN 1
WHEN cntNULL = cntCol THEN 2
ELSE 3 END) AS flag
FROM cte
GROUP BY col1, col2
Demo on SQLFiddle
...And solution without CTE if you want more portable SQL:
select col1,
col2,
case
when DistinctValuesWithoutNulls = 1 and NullCount = 0 then 1
when DistinctValuesWithoutNulls = 0 then 2
when NullCount > 0 then 3
end flag
from
(
select col1,
col2,
count(distinct [values]) DistinctValuesWithoutNulls,
sum(case when [values] is null then 1 else 0 end) NullCount
from Table1
group by col1, col2
) tmp