How do I select few columns in a table that only contain NULL values for all the rows?
Suppose if Table has 100 columns, among this 100 columns 60 columns has null values.
How can I write where condition to check if 60 columns are null.
maybe with a COALESCE
SELECT * FROM table WHERE coalesce(col1, col2, col3, ..., colN) IS NULL
where c1 is null and c2 is null ... and c60 is null
shortcut using string concatenation (Oracle syntax):
where c1||c2||c3 ... c59||c60 is null
First of all, if you have a table that has so many nulls and you use SQL Server 2008 - you might want to define the table using sparse columns (http://msdn.microsoft.com/en-us/library/cc280604.aspx).
Secondly I am not sure if coalesce solves the question asks - it seems like Ammu might actually want to find the list of columns that are null for all rows, but I might have misunderstood. Nevertheless - it is an interesting question, so I wrote a procedure to list null columns for any given table:
IF (OBJECT_ID(N'PrintNullColumns') IS NOT NULL)
DROP PROC dbo.PrintNullColumns;
go
CREATE PROC dbo.PrintNullColumns(#tablename sysname)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #query nvarchar(max);
DECLARE #column sysname;
DECLARE columns_cursor CURSOR FOR
SELECT c.name
FROM sys.tables t JOIN sys.columns c ON t.object_id = c.object_id
WHERE t.name = #tablename AND c.is_nullable = 1;
OPEN columns_cursor;
FETCH NEXT FROM columns_cursor INTO #column;
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #query = N'
DECLARE #c int
SELECT #c = COUNT(*) FROM ' + #tablename + ' WHERE ' + #column + N' IS NOT NULL
IF (#c = 0)
PRINT (''' + #column + N''');'
EXEC (#query);
FETCH NEXT FROM columns_cursor INTO #column;
END
CLOSE columns_cursor;
DEALLOCATE columns_cursor;
SET NOCOUNT OFF;
RETURN;
END;
go
If you don't want to write the columns names, Try can do something like this.
This will show you all the rows when all of the columns values are null except for the columns you specified (IgnoreThisColumn1 & IgnoreThisColumn2).
DECLARE #query NVARCHAR(MAX);
SELECT #query = ISNULL(#query+', ','') + [name]
FROM sys.columns
WHERE object_id = OBJECT_ID('yourTableName')
AND [name] != 'IgnoreThisColumn1'
AND [name] != 'IgnoreThisColumn2';
SET #query = N'SELECT * FROM TmpTable WHERE COALESCE('+ #query +') IS NULL';
EXECUTE(#query)
Result
If you don't want rows when all the columns are null except for the columns you specified, you can simply use IS NOT NULL instead of IS NULL
SET #query = N'SELECT * FROM TmpTable WHERE COALESCE('+ #query +') IS NOT NULL';
Result
[
Are you trying to find out if a specific set of 60 columns are null, or do you just want to find out if any 60 out of the 100 columns are null (not necessarily the same 60 for each row?)
If it is the latter, one way to do it in oracle would be to use the nvl2 function, like so:
select ... where (nvl2(col1,0,1)+nvl2(col2,0,1)+...+nvl2(col100,0,1) > 59)
A quick test of this idea:
select 'dummy' from dual where nvl2('somevalue',0,1) + nvl2(null,0,1) > 1
Returns 0 rows while:
select 'dummy' from dual where nvl2(null,0,1) + nvl2(null,0,1) > 1
Returns 1 row as expected since more than one of the columns are null.
It would help to know which db you are using and perhaps which language or db framework if using one.
This should work though on any database.
Something like this would probably be a good stored procedure, since there are no input parameters for it.
select count(*) from table where col1 is null or col2 is null ...
Here is another method that seems to me to be logical as well (use Netezza or TSQL)
SELECT KeyColumn, MAX(NVL2(TEST_COLUMN,1,0) AS TEST_COLUMN
FROM TABLE1
GROUP BY KeyColumn
So every TEST_COLUMN that has MAX value of 0 is a column that contains all nulls for the record set. The function NVL2 is saying if the column data is not null return a 1, but if it is null then return a 0.
Taking the MAX of that column will reveal if any of the rows are not null. A value of 1 means that there is at least 1 row that has data. Zero (0) means that each row is null.
I use the below query when i have to check for multiple columns NULL. I hope this is helpful . If the SUM comes to a value other than Zero , then you have NULL in that column
select SUM (CASE WHEN col1 is null then 1 else 0 end) as null_col1,
SUM (CASE WHEN col2 is null then 1 else 0 end) as null_col2,
SUM (CASE WHEN col3 is null then 1 else 0 end) as null_col3, ....
.
.
.
from tablename
you can use
select NUM_NULLS , COLUMN_NAME from all_tab_cols where table_name = 'ABC' and COLUMN_NAME in ('PQR','XYZ');
Related
There are too many tables in my SQL Server db. Most of them have an 'id' column, but some do not. I want to know which table(s) doesn't have the 'id' column and to count the rows where id=null if an 'id' column exists. The query results may look like this:
TABLE_NAME | HAS_ID | ID_NULL_COUNT | ID_NOT_NULL_COUNT
table1 | false | 0 | 0
table2 | true | 10 | 100
How do I write this query?
Building query:
WITH cte AS (
SELECT t.*, has_id = CASE WHEN COLUMN_NAME = 'ID' THEN 'true' ELSE 'false' END
FROM INFORMATION_SCHEMA.TABLES t
OUTER APPLY (SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS c
WHERE t.TABLE_NAME = c.TABLE_NAME
AND t.[TABLE_SCHEMA] = c.[TABLE_SCHEMA]
AND c.COLUMN_NAME = 'id') s
WHERE t.TABLE_SCHEMA IN (...)
)
SELECT
query_to_run = REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
'SELECT tab_name = ''<tab_name>'',
has_id = ''<has_id>'',
id_null_count = <id_null_count>,
id_not_null_count = <id_not_null_count>
FROM <schema_name>.<tab_name>'
,'<tab_name>', TABLE_NAME)
,'<schema_name>', TABLE_SCHEMA)
,'<has_id>', has_id)
,'<id_null_count>', CASE WHEN has_id = 'false' THEN '0' ELSE 'SUM(CASE WHEN id IS NULL THEN 1 END)' END)
,'<id_not_null_count>', CASE WHEN has_id = 'false' THEN '0' ELSE 'COUNT(id)' END)
FROM cte;
Copy the output and execute in separate window. UNION ALL could be added to get single resultset.
db<>fiddle demo
This might be useful for you... lists out the row count for all tables that have an "id" column. It filters out tables that start with "sys" because those are mostly internal tables. If you have a table that starts with "sys", you'll probably want to delete that part of the WHERE clause.
SELECT DISTINCT OBJECT_NAME(r.[object_id]) AS [TableName], [row_count] AS [RowCount]
FROM sys.dm_db_partition_stats r
WHERE index_id = 1
AND EXISTS (SELECT 1 FROM sys.columns c WHERE c.[object_id] = r.[object_id] AND c.[name] = N'id')
AND OBJECT_NAME(r.[object_id]) NOT LIKE 'sys%'
ORDER BY [TableName]
Note you can change the "c.[name] = N'id'" to be any column name, or even change the "=" to "<>" to find only tables without an id column
pmbAustin answers how to list all tables without "ID" column.
To know how many rows in each table, SQL Server has a built-in report for you.
Right click the database in SSMS, click "Reports", "Standard Reports" then "Disk Usage by Table"
You now know how many rows in each table, and from pmbAustin's answer you know how which tables do and do not have "ID" columns. with a simple Vlookup in Excel you can combine these two datasets to arrive at any answer you wish.
This will give you the info about which tables have or not have column named "ID":
SELECT Table_Name
, case when column_name not like '%ID%' then 'false'
else 'true'
end as HAS_ID
FROM INFORMATION_SCHEMA.COLUMNS;
Here is a small demo
And here is one way that you can use to select all the tables that have columns named ID and if this columns are null or not:
CREATE TABLE #AllIDSNullable (TABLE_NAME NVARCHAR(256) NOT NULL
, HAS_ID VARCHAR(10)
, ID_NULL_COUNT INT DEFAULT 0
, ID_NOT_NULL_COUNT INT DEFAULT 0);
DECLARE CT CURSOR FOR
SELECT Table_Name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name = 'ID';
DECLARE #name NVARCHAR(MAX), #SQL NVARCHAR(MAX);
OPEN CT; FETCH NEXT FROM CT INTO #name;
WHILE ##FETCH_STATUS=0 BEGIN
SET #SQL = 'INSERT #AllIDSNullable (TABLE_NAME , HAS_ID) SELECT Table_Name, case when column_name not like ''%ID%'' then ''false'' else ''true'' end FROM INFORMATION_SCHEMA.COLUMNS;';
EXEC (#SQL);
SET #SQL = 'UPDATE #AllIDSNullable SET ID_NULL_COUNT = (SELECT COUNT(*) FROM ['+#name+'] WHERE ID IS NULL), ID_NOT_NULL_COUNT = (SELECT COUNT(*) FROM ['+#name+'] WHERE ID IS NOT NULL) WHERE TABLE_NAME='''+#name+''';';
EXEC (#SQL);
FETCH NEXT FROM CT INTO #name;
END;
CLOSE CT;
SELECT *
FROM #AllIDSNullable;
Here is a demo
Result:
I created a temp table #test containing 3 fields: ColumnName, TableName, and Id.
I would like to see which rows in the #test table (columns in their respective tables) are not empty? I.e., for every column name that i have in the ColumnName field, and for the corresponding table found in the TableName field, i would like to see whether the column is empty or not. Tried some things (see below) but didn't get anywhere. Help, please.
declare #LoopCounter INT = 1, #maxloopcounter int, #test varchar(100),
#test2 varchar(100), #check int
set #maxloopcounter = (select count(TableName) from #test)
while #LoopCounter <= #maxloopcounter
begin
DECLARE #PropIDs TABLE (tablename varchar(max), id int )
Insert into #PropIDs (tablename, id)
SELECT [tableName], id FROM #test
where id = #LoopCounter
set #test2 = (select columnname from #test where id = #LoopCounter)
declare #sss varchar(max)
set #sss = (select tablename from #PropIDs where id = #LoopCounter)
set #check = (select count(#test2)
from (select tablename
from #PropIDs
where id = #LoopCounter) A
)
print #test2
print #sss
print #check
set #LoopCounter = #LoopCounter + 1
end
In order to use variables as column names and table names in your #Check= query, you will need to use Dynamic SQL.
There is most likely a better way to do this but I cant think of one off hand. Here is what I would do.
Use the select and declare a cursor rather than a while loop as you have it. That way you dont have to count on sequential id's. The cursor would fetch fields columnname, id and tablename
In the loop build a dynamic sql statement
Set #Sql = 'Select Count(*) Cnt Into #Temp2 From ' + TableName + ' Where ' + #columnname + ' Is not null And ' + #columnname <> '''''
Exec(#Sql)
Then check #Temp2 for a value greater than 0 and if this is what you desire you can use the #id that was fetched to update your #Temp table. Putting the result into a scalar variable rather than a temp table would be preferred but cant remember the best way to do that and using a temp table allows you to use an update join so it would well in my opinion.
https://www.mssqltips.com/sqlservertip/1599/sql-server-cursor-example/
http://www.sommarskog.se/dynamic_sql.html
Found a way to extract all non-empty tables from the schema, then just joined with the initial temp table that I had created.
select A.tablename, B.[row_count]
from (select * from #test) A
left join
(SELECT r.table_name, r.row_count, r.[object_id]
FROM sys.tables t
INNER JOIN (
SELECT OBJECT_NAME(s.[object_id]) table_name, SUM(s.row_count) row_count, s.[object_id]
FROM sys.dm_db_partition_stats s
WHERE s.index_id in (0,1)
GROUP BY s.[object_id]
) r on t.[object_id] = r.[object_id]
WHERE r.row_count > 0 ) B
on A.[TableName] = B.[table_name]
WHERE ROW_COUNT > 0
order by b.row_count desc
How about this one - bitmask computed column checks for NULLability. Value in the bitmask tells you if a column is NULL or not. Counting base 2.
CREATE TABLE FindNullComputedMask
(ID int
,val int
,valstr varchar(3)
,NotEmpty as
CASE WHEN ID IS NULL THEN 0 ELSE 1 END
|
CASE WHEN val IS NULL THEN 0 ELSE 2 END
|
CASE WHEN valstr IS NULL THEN 0 ELSE 4 END
)
INSERT FindNullComputedMask
SELECT 1,1,NULL
INSERT FindNullComputedMask
SELECT NULL,2,NULL
INSERT FindNullComputedMask
SELECT 2,NULL, NULL
INSERT FindNullComputedMask
SELECT 3,3,3
SELECT *
FROM FindNullComputedMask
Is there a way to write a SQL query to omit columns that have all of the same values? For example,
row A B
1 9 0
2 7 0
3 5 0
4 2 0
I'd like to return just
row A
1 9
2 7
3 5
4 2
Although it is possible to use SQL to find if all rows in a column have identical values, there is no way to make a fixed SQL statement not return a column based on the content of the query.
Here is how to find out if all items in a column have identical values:
SELECT COUNT(row)=COUNT(DISTINCT B) FROM my_table
You can run a preliminary query to see if a column needs to be displayed, and then form the query dynamically, including the column only when you need it.
The only way to change what columns are returned is by executing separate queries. So you'd have to do something like:
IF EXISTS(SELECT null FROM myTable WHERE B <> 0)
BEGIN
SELECT row, A, B FROM myTable
END
ELSE
BEGIN
SELECT row, A FROM myTable
END
However it's generally bad practice to return different columns based on the data - otherwise you make the client determine if a particular column is in the result set first before trying to access the data.
This sort of requirement is more commonly done when displaying the data, e.g. in a web page, in a report, etc.
There is no way to write a query statement that will only return columns that have disparate values. You can however use some conditonnal statements to execute different queries based on your needs.
You could also insert your query result into a temporary table, loop over the columns, build a new select statement that includes only the columns that have different values and finally execute the statement.
Note: You should probably just include bit columns to indicate wheter columns are all duplicates or not. The application could then just discard any column that has been indicated as all duplicates.
Anyway, here's an example solution for SQL SERVER
-- insert results into a temp table
SELECT *
INTO #data
FROM (
SELECT 1 AS col1, 1 AS col2, 2 AS col3
UNION ALL
SELECT 2, 1, 3
) d;
DECLARE
#column sysname,
#sql nvarchar(max) = '',
#finalSql nvarchar(500) = 'SELECT ',
#allDuplicates bit;
DECLARE colsCur CURSOR
FOR
SELECT name
FROM tempdb.sys.columns
WHERE object_id = OBJECT_ID('tempdb..#data');
OPEN colsCur;
FETCH NEXT FROM colsCur INTO #column;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = N'SELECT #allDuplicates = CASE COUNT(DISTINCT ' + #column + ') WHEN 1 THEN 1 ELSE 0 END FROM #data';
EXEC sp_executesql #sql, N'#allDuplicates int OUT', #allDuplicates = #allDuplicates OUT;
IF #allDuplicates = 0 SET #finalSql = #finalSql + #column + ',';
FETCH NEXT FROM colsCur INTO #column;
END
CLOSE colsCur;
DEALLOCATE colsCur;
SET #finalSql = LEFT(#finalSql, LEN(#finalSql) - 1) + ' FROM #data';
EXEC sp_executesql #finalSql;
DROP TABLE #data;
I have a select statement which I want to convert into an update statement for all the columns in the table which have the name Variable[N].
For example, I want to do these things:
I want to be able to convert the SQL below into an update statement.
I have n columns with the name variable[N]. The example below only updates column variable63, but I want to dynamically run the update on all columns with names variable1 through variableN without knowing how many variable[N] columns I have in advance. Also, in the example below I get the updated result into NewCol. I actually want to update the respective variable column with the results if possible, variable63 in my example below.
I want to have a wrapper that loops over column variable1 through variableN and perform the same respective update operation on all those columns:
SELECT
projectid
,documentid
,revisionno
,configurationid
,variable63
,ISNULL(Variable63,
(SELECT TOP 1
variable63
FROM table1
WHERE
documentid = t.documentid
and projectid=t.projectid
and configurationid=t.configurationid
and cast(revisionno as int) < cast(t.revisionno as int)
AND Variable63 is NOT NULL
ORDER BY
projectid desc
,documentid desc
,revisionno desc
,configurationid desc
)) as NewCol
FROM table1 t;
There's no general way to loop through variables in SQL, you're supposed to know exactly what you want to modify. In some databases, it will be possible to query system tables to dynamically build an update statement (I know how to do that in InterBase and it's decessor Firebird), but you haven't told us anything which database engine you're using.
Below is a way you could update several fields that are null, COALESCE and CASE are two way of doing the same thing, as is using LEFT JOIN or NOT EXISTS. Use the ones you and your database engine is most comfortable with. Beware that all records will be updated, so this is not a good solution if your database contains millions of records, each record is large and you want this query to be executed lots of times.
UPDATE table1 t
SET t.VARIABLE63 =
COALESCE(t.VARIABLE63,
(SELECT VARIABLE63
FROM table1 t0
LEFT JOIN table1 tNot
ON tNot.documentid = t.documentid
AND tNot.projectid=t.projectid
AND tNot.configurationid=t.configurationid
AND cast(tNot.revisionno as int) > cast(t0.revisionno as int)
AND cast(tNot.revisionno as int) < cast(t.revisionno as int)
AND tNot.Variable63 is NOT NULL
WHERE t0.documentid = t.documentid
AND t0.projectid=t.projectid
AND t0.configurationid=t.configurationid
AND cast(t0.revisionno as int) < cast(t.revisionno as int)
AND t0.Variable63 is NOT NULL
AND tNot.Variable63 is NULL)),
t.VARIABLE64 = CASE WHEN t.VARIABLE64 IS NOT NULL then t.VARIABLE64
ELSE (SELECT VARIABLE64
FROM table1 t0
WHERE t0.documentid = t.documentid
AND t0.projectid=t.projectid
AND t0.configurationid=t.configurationid
AND cast(t0.revisionno as int) < cast(t.revisionno as int)
AND t0.Variable64 is NOT NULL
AND NOT EXISTS(SELECT 1
FROM table1 tNot
WHERE tNot.documentid = t.documentid
AND tNot.projectid=t.projectid
AND tNot.configurationid=t.configurationid
AND cast(tNot.revisionno as int) > cast(t0.revisionno as int)
AND cast(tNot.revisionno as int) < cast(t.revisionno as int)
AND tNot.Variable64 is NOT NULL)) END
OK I think I got it. Function that loops through columns and runs an update command per column.
DECLARE #sql NVARCHAR(1000),
#cn NVARCHAR(1000)--,
--#r NVARCHAR(1000),
--#start INT
DECLARE col_names CURSOR FOR
SELECT column_name
FROM information_schema.columns
WHERE table_name = 'PIVOT_TABLE'
ORDER BY ordinal_position
--SET #start = 0
DECLARE #op VARCHAR(max)
SET #op=''
OPEN col_names FETCH next FROM col_names INTO #cn
WHILE ##FETCH_STATUS = 0
BEGIN
--print #cn
IF UPPER(#cn)<> 'DOCUMENTID' and UPPER(#cn)<> 'CONFIGURATIONID' and UPPER(#cn)<> 'PROJECTID' and UPPER(#cn)<> 'REVISIONNO'
BEGIN
SET #sql = 'UPdate pt
set pt.' + #cn + ' = ((SELECT TOP 1 t.' + #cn + ' FROM pivot_table t WHERE t.documentid = pt.documentid and t.projectid=pt.projectid
and t.configurationid=pt.configurationid and cast(t.revisionno as int) < cast(pt.revisionno as int) AND t.' + #cn + ' is NOT NULL
ORDER BY revisionno desc)) from PIVOT_TABLE pt where pt.' + #cn + ' is NULL;'
EXEC Sp_executesql
#sql
--print #cn
END
FETCH next FROM col_names INTO #cn
END
CLOSE col_names
DEALLOCATE col_names;
I am having table with 100 columns. here up to 50 to 60 columns contains NULL value in it. Now i need to Replace this NULL value to 0 in all 50 to 60 columns. I tried with the Update query as,
UPDATE [tableName]
SET col1=0, col2 = 0, ... col60 = 0
WHERE col1 IS NULL AND Col2 IS NULL ... Col60 IS NULL
Is there anyother Query to update these all 60 columns without specifying such columns or we have any other approach???
You have to specify all columns, but you can skip the WHERE clause and have one update deal with them all at once:
UPDATE [tableName] SET
col1=COALESCE(col1, 0),
col2=COALESCE(col2, 0),
col3=COALESCE(col3, 0),
col4=COALESCE(col4, 0),
[...]
You could try this workaround if every value in the columns is NULL:
Edit the table definition and set the columns as "Calculated" and use 0 as formula
Save the table
Remove the formula
It is not very elegant but works
I don't think there's an alternative - but the query you posted will only update records where all the columns are null.
If you want to update individual columns, you need to break it up into individual updates:
update table
set col1 = 0
where col 1 is null
update table
set col2 = 0
where col2 is null
To do not write this query by hand, you can generate this by using dynamic SQL:
DECLARE #Table NVARCHAR(255)= 'Your table'
DECLARE #sSQl NVARCHAR(MAX)= 'UPDATE ' + #Table + ' SET ' + CHAR(13) ;
WITH c AS ( SELECT c.name
FROM sys.all_columns c
JOIN sys.tables T ON c.object_id = T.object_id
WHERE t.name = #Table
)
SELECT #sSQl = #sSQl + c.name + '=ISNULL(' + c.name + ',0)' + ','
+ CHAR(13)
FROM c
IF LEN(#sSQl) > 0
SET #ssql = LEFT(#sSQl, LEN(#sSQl) - 2)
PRINT #ssql