Say I have a table called:
TableA
The following columns exist in the table are:
Column1, Column2, Column3
what I am trying to accomplish is to see how many records are not null.
to do this I have the following case statement:
sum(Case when Column1 is not null then 1 else 0 end)
What I want is the above case statement for every table that exists from a list provided and to be run for each columns that exists in the table.
So for the above example the case statment will run for Column1, Column2 and Column3 as there are 3 columns in that particular table etc
But I want to specfiy a list of tables to loop through executing the logic above
create procedure tab_cols (#tab nvarchar(255))
as
begin
declare #col_count nvarchar(max) = ''
,#col nvarchar(max) = ''
select #col_count += case ORDINAL_POSITION when 1 then '' else ',' end + 'count(' + QUOTENAME(COLUMN_NAME,']') + ') as ' + QUOTENAME(COLUMN_NAME,']')
,#col += case ORDINAL_POSITION when 1 then '' else ',' end + QUOTENAME(COLUMN_NAME,']')
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = #tab
order by ORDINAL_POSITION
declare #stmt nvarchar(max) = 'select * from (select ' + #col_count + ' from ' + #tab + ') t unpivot (val for col in (' + #col + ')) u'
exec sp_executesql #stmt
end
Wouldn't it be easy as this?
SELECT AccountID
,SUM(Total) AS SumTotal
,SUM(Profit) AS SumProfit
,SUM(Loss) AS SumLoss
FROM tblAccount
GROUP BY AccountID
If I understand this correctly you want to get the sums, but not for all rows in one go but for each accountID separately. This is what GROUP BY is for...
If ever possible try to avoid loops, cursors and other procedural approaches...
UPDATE: Generic approach for different tables
With different tables you will - probably - need exactly the statement I show above, but you'll have to generate it dynamically and use EXEC to execute it. You can go through INFORMATION_SCHEMA.COLUMNS to get the columns names...
But:
How should this script know generically, which columns should be summed up? You might head for data_type like 'decimal%' or similar...
What about the other columns and their usage in GROUP BY?
How would you want to place aliases
How do you want to continue with a table of unknown structure?
To be honest: I think, there is no real-generic-one-for-all approach for this...
I have a table that looks something like
ID Col1 Col2 Col3 Col4
1 3 5 3 3
What I want to do is COUNT the number of 3s in this particular row.
I have tried the
select COUNT(*)
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = 'TableName' -- but obviously I need WHERE Col1 = 3 OR Col2 = 3...
What would be the best way to achieve this?
Based on what OP asked, this can be done
select
CASE WHEN Col1 = 3 then 1 ELSE 0 END +
CASE WHEN Col2 = 3 then 1 ELSE 0 END +
CASE WHEN Col3 = 3 then 1 ELSE 0 END +
CASE WHEN Col4 = 3 then 1 ELSE 0 END
From TableName
I don't really enjoy working with PIVOT so here a solution using APPLY.
SELECT
T.id
, Val
, COUNT(*)
FROM MyTable AS T
CROSS APPLY (
VALUES
(T.C1)
, (T.C2)
, (T.C3)
, (T.C4)
) AS X(Val)
GROUP BY T.Id, X.Val
ORDER BY T.Id, X.val
Please find the sample code:
DECLARE #Query VARCHAR(MAX) = 'SELECT Count = '
SELECT
#Query += '( CASE WHEN '+ COLUMN_NAME + ' = 3 THEN 1 ELSE 0 END ) + '
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'TEST' AND COLUMN_NAME <> 'ID'
SET #Query = SUBSTRING(#Query, 1, DATALENGTH(#Query) - 2) + ' FROM TEST WHERE ID = 1'
EXEC(#Query)
I'm trying to create a query that will return all the rows that have a null value across all but 1 column. Some rows will have more than one null entry somewhere. There is one column I will want to exclude, because at this moment in time all of the entries are null and it is the only column that is allowed to have null values. I am stuck because I don't know how to include all of the columns in the WHERE.
SELECT *
FROM Analytics
WHERE * IS NULL
Alternatively, I can do a count for one column, but the table has about 67 columns.
SELECT COUNT(*)
FROM Analytics
WHERE P_Id IS NULL
In SQL Server you can borrow the idea from this answer
;WITH XMLNAMESPACES('http://www.w3.org/2001/XMLSchema-instance' as ns)
SELECT *
FROM Analytics
WHERE (SELECT Analytics.*
FOR xml path('row'), elements xsinil, type
).value('count(//*[local-name() != "colToIgnore"]/#ns:nil)', 'int') > 0
SQL Fiddle
Likely constructing a query with 67 columns will be more efficient but it saves some typing or need for dynamic SQL to generate it.
Depending on which RDBMS you're using, I think your only option (rather than explicitly saying WHERE col1 IS NULL and col2 IS NULL and col3 IS NULL ...) would be to use Dynamic SQL.
For example, if you want to get all the column names from a SQL Server database, you could use something like this to return those names:
SELECT
name
FROM
sys.columns
WHERE
object_id = OBJECT_ID('DB.Schema.Table')
You could use FOR XML to create your WHERE clause:
SELECT Name + ' IS NULL AND ' AS [text()]
FROM sys.columns c1
WHERE object_id = OBJECT_ID('DB.Schema.Table')
ORDER BY Name
FOR XML PATH('')
Hope this helps get you started.
Good luck.
I don't have such a table to test, assuming there is no 'x' as data in any field, I think this should work on Sql-Server; (DEMO)
NOTE: I have filtered keyColumn as c.name != 'keyColumn'
DECLARE #S NVARCHAR(max), #Columns VARCHAR(50), #Table VARCHAR(50)
SELECT #Columns = '66', --Number of cols without keyColumn
#Table = 'myTable'
SELECT #S = ISNULL(#S+'+ ','') + 'isnull(convert(nvarchar, ' + c.name + '),''x'')'
FROM sys.all_columns c
WHERE c.object_id = OBJECT_ID(#Table) AND c.name != 'keyColumn'
exec('select * from '+#Table+' where ' + #S + '= replicate(''x'',' + #Columns + ')')
For the SQL beginner user like me, all the query above seem so hard to digest. I think the quickest way is just to write query for all 67 columns. It's just basically a copy&paste process.
E.g:
select count(*) from user where id is null or
name is null or
review_count is null or
yelping_since is null or
useful is null or
funny is null;
This is for SQLite, so slightly different, but I had the same problem and I ended up writing this small Python script findnulls.py to find all null values in a database:
import sqlite3
import sys
def main():
dbfile = sys.argv[1]
conn = sqlite3.connect(dbfile)
c = conn.cursor()
tables = c.execute("SELECT name FROM sqlite_master WHERE type='table';").fetchall()
for table in tables:
tablename = table[0]
cols = c.execute("PRAGMA table_info({0})".format(tablename)).fetchall()
for col in cols:
colname = col[1]
nullvals = c.execute("SELECT COUNT(*) FROM {0} WHERE {1} IS NULL;".format(tablename, colname)).fetchall()
if nullvals[0][0] != 0:
print("found {0} nulls in table: {1} column {2}".format(nullvals[0][0], tablename, colname))
main()
Where you would run it like this:
findnulls.py somedatabase.db
And the output looks like this:
found 1 nulls in table: Channels column MessageId
found 1 nulls in table: Channels column MessageChannel
found 1 nulls in table: Channels column SignalType
found 7 nulls in table: Channels column MinVal
found 7 nulls in table: Channels column MaxVal
found 9 nulls in table: Channels column AvgVal
found 9 nulls in table: Channels column Median
found 9 nulls in table: Channels column StdDev
I was wondering about the possibility to count the null columns of row in SQL, I have a table Customer that has nullable values, simply I want a query that return an int of the number of null columns for certain row(certain customer).
This method assigns a 1 or 0 for null columns, and adds them all together. Hopefully you don't have too many nullable columns to add up here...
SELECT
((CASE WHEN col1 IS NULL THEN 1 ELSE 0 END)
+ (CASE WHEN col2 IS NULL THEN 1 ELSE 0 END)
+ (CASE WHEN col3 IS NULL THEN 1 ELSE 0 END)
...
...
+ (CASE WHEN col10 IS NULL THEN 1 ELSE 0 END)) AS sum_of_nulls
FROM table
WHERE Customer=some_cust_id
Note, you can also do this perhaps a little more syntactically cleanly with IF() if your RDBMS supports it.
SELECT
(IF(col1 IS NULL, 1, 0)
+ IF(col2 IS NULL, 1, 0)
+ IF(col3 IS NULL, 1, 0)
...
...
+ IF(col10 IS NULL, 1, 0)) AS sum_of_nulls
FROM table
WHERE Customer=some_cust_id
I tested this pattern against a table and it appears to work properly.
My answer builds on Michael Berkowski's answer, but to avoid having to type out hundreds of column names, what I did was this:
Step 1: Get a list of all of the columns in your table
SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'myTable';
Step 2: Paste the list in Notepad++ (any editor that supports regular expression replacement will work). Then use this replacement pattern
Search:
^(.*)$
Replace:
\(CASE WHEN \1 IS NULL THEN 1 ELSE 0 END\) +
Step 3: Prepend SELECT identityColumnName, and change the very last + to AS NullCount FROM myTable and optionally add an ORDER BY...
SELECT
identityColumnName,
(CASE WHEN column001 IS NULL THEN 1 ELSE 0 END) +
-- ...
(CASE WHEN column200 IS NULL THEN 1 ELSE 0 END) AS NullCount
FROM
myTable
ORDER BY
NullCount DESC
For ORACLE-DBMS only.
You can use the NVL2 function:
NVL2( string1, value_if_not_null, value_if_null )
Here is a select with a similiar approach as Michael Berkowski suggested:
SELECT (NVL2(col1, 0, 1)
+ NVL2(col2, 0, 1)
+ NVL2(col3, 0, 1)
...
...
+ NVL2(col10, 0, 1)
) AS sum_of_nulls
FROM table
WHERE Customer=some_cust_id
A more generic approach would be to write a PL/SQL-block and use dynamic SQL. You have to build a SELECT string with the NVL2 method from above for every column in the all_tab_columns of a specific table.
Unfortunately, in a standard SQL statement you will have to enter each column you want to test, to test all programatically you could use T-SQL. A word of warning though, ensure you are working with genuine NULLS, you can have blank stored values that the database will not recognise as a true NULL (I know this sounds strange).
You can avoid this by capturing the blank values and the NULLS in a statement like this:
CASE WHEN col1 & '' = '' THEN 1 ELSE 0 END
Or in some databases such as Oracle (not sure if there are any others) you would use:
CASE WHEN col1 || '' = '' THEN 1 ELSE 0 END
You don't state RDBMS. For SQL Server 2008...
SELECT CustomerId,
(SELECT COUNT(*) - COUNT(C)
FROM (VALUES(CAST(Col1 AS SQL_VARIANT)),
(Col2),
/*....*/
(Col9),
(Col10)) T(C)) AS NumberOfNulls
FROM Customer
Depending on what you want to do, and if you ignore mavens, and if you use SQL Server 2012, you could to it another way. .
The total number of candidate columns ("slots") must be known.
1. Select all the known "slots" column by column (they're known).
2. Unpivot that result to get a
table with one row per original column. This works because the null columns don't
unpivot, and you know all the column names.
3. Count(*) the result to get the number of non-nulls;
subtract from that to get your answer.
Like this, for 4 "seats" in a car
select 'empty seats' = 4 - count(*)
from
(
select carId, seat1,seat2,seat3,seat4 from cars where carId = #carId
) carSpec
unpivot (FieldValue FOR seat in ([seat1],[seat2],[seat3],[seat4])) AS results
This is useful if you may need to do more later than just count the number of non-null columns, as it gives you a way to manipulate the columns as a set too.
This will give you the number of columns which are not null. you can apply this appropriately
SELECT ISNULL(COUNT(col1),'') + ISNULL(COUNT(col2),'') +ISNULL(COUNT(col3),'')
FROM TABLENAME
WHERE ID=1
The below script gives you the NULL value count within a row i.e. how many columns do not have values.
{SELECT
*,
(SELECT COUNT(*)
FROM (VALUES (Tab.Col1)
,(Tab.Col2)
,(Tab.Col3)
,(Tab.Col4)) InnerTab(Col)
WHERE Col IS NULL) NullColumnCount
FROM (VALUES(1,2,3,4)
,(NULL,2,NULL,4)
,(1,NULL,NULL,NULL)) Tab(Col1,Col2,Col3,Col4) }
Just to demonstrate I am using an inline table in my example.
Try to cast or convert all column values to a common type it will help you to compare the column of different type.
I haven't tested it yet, but I'd try to do it using a PL\SQL function
CREATE OR REPLACE TYPE ANYARRAY AS TABLE OF ANYDATA
;
CREATE OR REPLACE Function COUNT_NULL
( ARR IN ANYARRAY )
RETURN number
IS
cnumber number ;
BEGIN
for i in 1 .. ARR.count loop
if ARR(i).column_value is null then
cnumber := cnumber + 1;
end if;
end loop;
RETURN cnumber;
EXCEPTION
WHEN OTHERS THEN
raise_application_error
(-20001,'An error was encountered - '
||SQLCODE||' -ERROR- '||SQLERRM);
END
;
Then use it in a select query like this
CREATE TABLE TEST (A NUMBER, B NUMBER, C NUMBER);
INSERT INTO TEST (NULL,NULL,NULL);
INSERT INTO TEST (1 ,NULL,NULL);
INSERT INTO TEST (1 ,2 ,NULL);
INSERT INTO TEST (1 ,2 ,3 );
SELECT ROWNUM,COUNT_NULL(A,B,C) AS NULL_COUNT FROM TEST;
Expected output
ROWNUM | NULL_COUNT
-------+-----------
1 | 3
2 | 2
3 | 1
4 | 0
This is how i tried
CREATE TABLE #temptablelocal (id int NOT NULL, column1 varchar(10) NULL, column2 varchar(10) NULL, column3 varchar(10) NULL, column4 varchar(10) NULL, column5 varchar(10) NULL, column6 varchar(10) NULL);
INSERT INTO #temptablelocal
VALUES (1,
NULL,
'a',
NULL,
'b',
NULL,
'c')
SELECT *
FROM #temptablelocal
WHERE id =1
SELECT count(1) countnull
FROM
(SELECT a.ID,
b.column_title,
column_val = CASE b.column_title
WHEN 'column1' THEN a.column1
WHEN 'column2' THEN a.column2
WHEN 'column3' THEN a.column3
WHEN 'column4' THEN a.column4
WHEN 'column5' THEN a.column5
WHEN 'column6' THEN a.column6
END
FROM
( SELECT id,
column1,
column2,
column3,
column4,
column5,
column6
FROM #temptablelocal
WHERE id =1 ) a
CROSS JOIN
( SELECT 'column1'
UNION ALL SELECT 'column2'
UNION ALL SELECT 'column3'
UNION ALL SELECT 'column4'
UNION ALL SELECT 'column5'
UNION ALL SELECT 'column6' ) b (column_title) ) AS pop WHERE column_val IS NULL
DROP TABLE #temptablelocal
Similary, but dynamically:
drop table if exists myschema.table_with_nulls;
create table myschema.table_with_nulls as
select
n1::integer,
n2::integer,
n3::integer,
n4::integer,
c1::character varying,
c2::character varying,
c3::character varying,
c4::character varying
from
(
values
(1,2,3,4,'a','b','c','d'),
(1,2,3,null,'a','b','c',null),
(1,2,null,null,'a','b',null,null),
(1,null,null,null,'a',null,null,null)
) as test_records(n1, n2, n3, n4, c1, c2, c3, c4);
drop function if exists myschema.count_nulls(varchar,varchar);
create function myschema.count_nulls(schemaname varchar, tablename varchar) returns void as
$BODY$
declare
calc varchar;
sqlstring varchar;
begin
select
array_to_string(array_agg('(' || trim(column_name) || ' is null)::integer'),' + ')
into
calc
from
information_schema.columns
where
table_schema in ('myschema')
and table_name in ('table_with_nulls');
sqlstring = 'create temp view count_nulls as select *, ' || calc || '::integer as count_nulls from myschema.table_with_nulls';
execute sqlstring;
return;
end;
$BODY$ LANGUAGE plpgsql STRICT;
select * from myschema.count_nulls('myschema'::varchar,'table_with_nulls'::varchar);
select
*
from
count_nulls;
Though I see that I didn't finish parametising the function.
My answer builds on Drew Chapin's answer, but with changes to get the result using a single script:
use <add_database_here>;
Declare #val Varchar(MAX);
Select #val = COALESCE(#val + str, str) From
(SELECT
'(CASE WHEN '+COLUMN_NAME+' IS NULL THEN 1 ELSE 0 END) +' str
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<add table name here>'
) t1 -- getting column names and adding the case when to replace NULLs for zeros or ones
Select #val = SUBSTRING(#val,1,LEN(#val) - 1) -- removing trailling add sign
Select #val = 'SELECT <add_identity_column_here>, ' + #val + ' AS NullCount FROM <add table name here>' -- adding the 'select' for the column identity, the 'alias' for the null count column, and the 'from'
EXEC (#val) --executing the resulting sql
With ORACLE:
Number_of_columns - json_value( json_array( comma separated list of columns ), '$.size()' ) from your_table
json_array will build an array with only the non null columns and the json_query expression will give you the size of the array
There isn't a straightforward way of doing so like there would be with counting rows. Basically, you have to enumerate all the columns that might be null in one expression.
So for a table with possibly null columns a, b, c, you could do this:
SELECT key_column, COALESCE(a,0) + COALESCE(b,0) + COALESCE(c,0) null_col_count
FROM my_table