I'm trying to find a way to grab the count of nulls and blanks of every column in a table and put the results into a table. I found this but don't know how to push it to a table? Would also like to add in a Count(distinct fieldname) as well. Any help would be appreciated!
DECLARE #t nvarchar(max)
SET #t = N'SELECT '
SELECT #t = #t + 'sum(case when ' + c.name + ' is null or ' + c.name + ' = '''' then 1 else 0 end) "' + c.name + '",
sum(case when ' + c.name + ' is null then 0 else 1 end) "Non-Null Values for ' + c.name + '",'
FROM sys.columns c
WHERE c.object_id = object_id('TableName');
SET #t = SUBSTRING(#t, 1, LEN(#t) - 1) + ' FROM TableName;'
EXEC sp_executesql #t
**Edit
This provides the results in one row. Is there a way to do it so each field gets it's own row in a table?
FieldName CountofNullsBlanks
FieldA 0
FieldB 100
Here:
DECLARE #t nvarchar(max)
SET #t = N'SELECT '+convert(nvarchar(max),count(*))+' as [Distinct fieldnames],' -- added the distinct
SELECT #t = #t + 'sum(case when ' + c.name + ' is null or ' + c.name + ' = '''' then 1 else 0 end) "' + c.name + '",
sum(case when ' + c.name + ' is null then 0 else 1 end) "Non-Null Values for ' + c.name + '",'
FROM sys.columns c
WHERE c.object_id = object_id('YOURTABLE');
SET #t = SUBSTRING(#t, 1, LEN(#t) - 1) + 'into ##a FROM YOURTABLE;' --added the into ##a
EXEC sp_executesql #t
Notice the "into ##a" towards the end; This saves the result "table" to a global temp, which will be available until all sessions that accessed it finish. Careful with the name or it might cause conflicts in the database.
Related
I have a table which has more than 30 columns(all are varchar). I need to list out all the columns which contains blank i.e.' ' values.
I tried using 'coalesce' but it is only for NULL.
The following query will give you all the columns in a table that might have null or '' values.
It is written so that you can run it for all tables in your database but you can limit it to a single table, as I have done for this specific example, checking a table called testingNulls:
--two variables needed for table name and column name, when looping through all tables
declare #table varchar(255), #col varchar(255), #sql varchar(max)
--this will be used to store the result, to have one result set instead of one row per each cursor cycle
if object_id('tempdb..#nullcolumns') is not null drop table #nullcolumns
create table #nullcolumns (tablename varchar(255), columnname varchar(255))
declare getinfo cursor for
select t.name tablename, c.name
from sys.tables t join sys.columns c on t.object_id = c.object_id
where t.name = 'testingnulls' --here the condition for the table name
open getinfo
fetch next from getinfo into #table, #col
while ##fetch_status = 0
begin
select #sql = 'if exists (select top 1 * from [' + #table + '] where [' + #col + '] is null or [' + #col + '] like '''' ) begin insert into #nullcolumns select ''' + #table + ''' as tablename, ''' + #col + ''' as all_nulls end'
print(#sql)
exec(#sql)
fetch next from getinfo into #table, #col
end
close getinfo
deallocate getinfo
--this should be the result you need:
select * from #nullcolumns
You can see a working example here. I hope this is what you need.
List all columns that contain a blank in some record? You'd use a query per column and collect the results with UNION ALL:
select 'COL1' where exists (select * from mytable where col1 like '% %')
union all
select 'COL2' where exists (select * from mytable where col2 like '% %')
union all
...
union all
select 'COL30' where exists (select * from mytable where col30 like '% %');
If you want like select * from [your_table_name] where [col1] = '' and [col2] = ''....., then use dynamic sql query like below.
Query
declare #sql as varchar(max);
select #sql = 'select * from [your_table_name] where '
+ stuff((
select ' and [' + [column_name] + '] = ' + char(39) + char(39)
from information_schema.columns
where table_name = 'your_table_name'
for xml path('')
)
, 1, 5, ''
);
exec(#sql);
Update
Or else if you want to list the column names which have a blank value, then you can use the below dynamic sql query.
Query
declare #sql as varchar(max);
select #sql = stuff((
select ' union all select ' + [column_name] + ' as [col1], '
+ char(39) + [column_name] + char(39) + ' as [col2]'
+ ' from your_table_name'
from information_schema.columns
where table_name = 'your_table_name'
for xml path('')
)
, 1, 11, ''
);
set #sql = 'select distinct t.col2 as [blank_cols] from(' + #sql
+ ')t
where coalesce(ltrim(rtrim(t.col1)), ' + char(39) + char(39) + ') = '
+ char(39) + char(39) + ';';
exec(#sql);
Find a demo here
But still I'm not sure that this is what you are looking out for.
you have not many choices but to specify all the columns in your where clause
WHERE COL1 = '' AND COL2 = '' AND COL3 = '' AND . . .
or you can use Dynamic SQL to form your query, but that is not an easy path to go
If you want to count number of columns having '' value in a table (not for each row) then use the following
SELECT max(CASE WHEN col1 = '' THEN 1 ELSE 0 END) +
max(CASE WHEN col2 = '' THEN 1 ELSE 0 END) +
max(CASE WHEN col3 = '' THEN 1 ELSE 0 END) +
...
FROM t
demo
I created a dynamic SQL script that you can use by providing the table name only
Here it is
declare #sql nvarchar(max)
declare #table sysname = 'ProductAttributes'
select #sql =
'select * from ' + #table + ' where ' +
string_agg('[' + name + '] = '' '' ', ' and ')
from sys.columns
where object_id = OBJECT_ID(#table)
select #sql
exec sp_executesql #sql
Unfortunately, for SQL string concatenation String_Agg function is new with SQL Server 2017
But it is also possible to use SQL XML Path to concatenate WHERE clause fragments
SELECT #sql = 'select * from ' + #table + ' where ' +
STUFF(
(
SELECT
' and ' + '[' + [name] + '] = '' '' '
from sys.columns
where object_id = OBJECT_ID(#table)
FOR XML PATH(''),TYPE
).value('.','VARCHAR(MAX)'
), 1, 5, ''
)
select #sql as sqlscript
exec sp_executesql #sql
I am writing a dynamic insert query in a stored procedure. I am receiving the column names as parameter to my stored procedure.
For example, I have an Employee table with EmployeeId and EmployeeName columns. I need to append EMP_ before each employee name while inserting the data into Department table from Employee table.
Non-dynamic query looks like this.
INSERT INTO Department(EmployeeId, EmployeeName)
SELECT
EmployeeId, 'EMP_' + EmployeeName
FROM
Employee
If I write a dynamic insert
SET #SqlCommand =
'INSERT INTO ' + #DepartmentTable + '(' + #EmployeeIdColumn + ',' + #EmployeeNameColumn + ')' +
'SELECT ' + #EmployeeIdColumn + ',''EMP_''' + #EmployeeNameColumn + '''' +
'FROM ' +
#EmployeeTable + ' WTB '
EXEC sp_executesql
#stmt = #SqlCommand
The issue is, for the EmployeeName column, it is inserting "EMP_EmployeeName" instead of actual employee name. I tried putting quotes before and after EmployeeNameColumn, but it didn't work. How can I fix it?
You need just one + after EMP_'', and remove it after ' WTB '. Also i fixed some spaces:
SET #SqlCommand =
'INSERT INTO ' + #DepartmentTable + ' (' + #EmployeeIdColumn + ',' + #EmployeeNameColumn + ')' +
' SELECT ' + #EmployeeIdColumn + ',''EMP_''+' + #EmployeeNameColumn +
' FROM ' + #EmployeeTable + ' WTB '
If you PRINT #SqlCommand you will get something like this:
INSERT INTO Department (EmpId,EmpName) SELECT EmpId,'EMP_'+EmpName FROM Employee WTB
And one note: better use QUOTENAME with table/column names. It will help to avoid situations where column has spaces in the name like 'Employee Name':
SET #SqlCommand =
'INSERT INTO ' + QUOTENAME(#DepartmentTable) + '(' + QUOTENAME(#EmployeeIdColumn) + ',' + QUOTENAME(#EmployeeNameColumn) + ')'+
' SELECT ' + QUOTENAME(#EmployeeIdColumn) + ',''EMP_''+' + QUOTENAME(#EmployeeNameColumn) +
' FROM ' + QUOTENAME(#EmployeeTable) + ' WTB '
To get:
INSERT INTO [Department]([EmpId],[EmpName]) SELECT [EmpId],'EMP_'+[EmpName] FROM [Employee] WTB
Try using the CONCAT function.
-- Syntax for SQL Server, Azure SQL Database, Azure SQL Data Warehouse, Parallel Data Warehouse
CONCAT ( string_value1, string_value2 [, string_valueN ] )
Your query would then look like this :
INSERT INTO deptTable(empId,empName)
SELECT empId,concat('EMP_', empName) as empName FROM empTable
PS: Looks like I misunderstood your question.
INSERT INTO deptTable(empId,empName)
SELECT empId,'SUFFIX'+empName as empName FROM empTable
will also work. I guess you wanted it for the stored proc.
In any case the concat should still do it.
Here is the dynamic query to create stored procedure with insert statement with columns and parameters
> DECLARE #table_name varchar(255) = 'table_name'
> DECLARE #v_col varchar(MAX)
> DECLARE #v_param varchar(MAX)
> DECLARE #SP_param varchar(MAX)
> DECLARE #sp_type varchar(10) = 'Insert'
>
> SET #v_col = STUFF(
> (SELECT ',' + '['+c.name+']'
> FROM sys.tables t JOIN sys.columns c ON t.object_id = c.object_id
> WHERE t.name = #table_name
> AND c.is_identity = 0
> FOR XML PATH ('')), 1, 1, ''
> )
> SET #v_param = STUFF(
> (SELECT ',' + '#'+c.name
> FROM sys.tables t JOIN sys.columns c ON t.object_id = c.object_id
> WHERE t.name = #table_name
> AND c.is_identity = 0
> FOR XML PATH ('')), 1, 1, ''
> )
>
> SET #SP_param = STUFF(
> (SELECT ',' + '#'+c.name+' '+y.name+' '+(CASE WHEN y.name = 'varchar' THEN '('+CAST(c.max_length as varchar)+')'
> ELSE '' END)
> FROM sys.tables t JOIN sys.columns c ON t.object_id = c.object_id
> JOIN sys.types y ON y.user_type_id = c.user_type_id
> WHERE t.name = #table_name
> AND c.is_identity = 0
> FOR XML PATH ('')), 1, 1, ''
> )
> Declare #Query VARCHAR(MAX)
> DECLARE #SPQuery VARCHAR(MAX)
> SET #SPQuery = 'CREATE PROCEDURE '+#sp_type+''+#table_name+' ('+#SP_param+')
> AS BEGIN'
> SET #Query='Insert Into '+#table_name+' ('+#v_col+')
> Values (
> '+#v_param+')
> END'
>
>
> PRINT(#SPQuery);
> PRINT(#Query);
I am trying to execute the following query to check the records from the 4 tables and then calling a function,but i get an error near the brackets.It works fine if I use case,but exits when the first condition is met. I need to evaluate for all the tables in the 4 IF's:
select account_id,
(
if exists (select account_id from [dbo].[TEST_R6])
Begin
select dbo.make_indicator(cent,chem,co,dim,lg,pl,strs,vis) + space(1) + 'rr'
End
if exists (select account_id from tbl_noR6)
begin
select dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(2) + 'cc'
end
if exists (select account_id from tbl_acct)
begin
select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(3) + 'pp'
end
if exists (select account_id from test_con)
begin
select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(4) + 'no'
end
)as value from CRS_PRODLINE
code works partially with case statement and gives output only for the first case is met and does not check for the others:
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX);
SET #cols = STUFF((SELECT distinct ',' + QUOTENAME(c.A_NAME)
FROM TEST_DEL c
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'')
SET #query = 'SELECT account_id, ' + #cols + ' from
(select account_id,
(case
when exists (select account_id from [dbo].[TEST_R6])
then
dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(1)
when exists (select account_id from tbl_noR6)
then
dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(2)
when exists (select account_id from tbl_acct)
then
dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(2)
end) as value,
assay_name
from CRS_PRODLINE
) x
pivot
(
MAX(newvalue)
for a_name in (' + #cols + ')
) p '
execute(#query)
Based on your comment that CASE only evaluates once being a problem, this should build a string of all the applicable cases.
select account_id,
(
''
+
case when exists (select account_id from dbo.[TEST_R6]) THEN select dbo.make_indicator(cent,chem,co,dim,lg,pl,strs,vis) + space(1) + 'rr' ELSE '' END
+
case when exists (select account_id from dbo.tbl_noR6) THEN select dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(2) + 'cc' ELSE '' END
+
case when exists (select account_id from dbo.tbl_acct) THEN select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(3) + 'pp' ELSE '' END
+
case when exists (select account_id from dbo.test_con) THEN select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(4) + 'no' ELSE '' END
) as value
from CRS_PRODLINE
First of all, if there is any record in the table your exists clause will return true. I will assume that this is intentional and you are not trying to match on a specific account_id.
If you want to use this structure you will have to use dynamic SQL. Something like this:
DECLARE #Sql VARCHAR(8000)
IF EXISTS (SELECT account_id FROM tbl_noR6)
BEGIN
SET #Sql = 'select dbo.make_indicator(acent,chem,co,dim,lg,pl,str,vis) + space(2) + ''cc'''
SET #Sql = #Sql + ')as value from CRS_PRODLINE'
EXEC #Sql
END
IF EXISTS (SELECT account_id FROM tbl_acct)
BEGIN
SET #Sql = 'select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(3) + ''pp'''
SET #Sql = #Sql + ')as value from CRS_PRODLINE'
EXEC #Sql
END
IF EXISTS (SELECT account_id FROM test_con)
BEGIN
SET #Sql = 'select dbo.make_indicator(cent,chem,co,dim,lg,pl,str,vis) + space(4) + ''no'''
SET #Sql = #Sql + ')as value from CRS_PRODLINE'
EXEC #Sql
END
I'm a little new at SQL so please bear with me. I am attempting to write some a query that will allow me to loop through an entire table and find the number of times null values appear in each column. This is easy to do the hard way by typing the following:
Select
SUM(CASE COL_1 WHEN IS NULL THEN 1 ELSE 0 END) AS COL_1_NULLS
,SUM(CASE COL_2 WHEN IS NULL THEN 1 ELSE 0 END) AS COL_2_NULLS
FROM TABLE1
This is easy but it can become arduous if you want to do this for multiple tables or if a single table has a lot of columns.
I'm looking for a way to write a query that passes a table name into it and then loops through each column in the defined table (possibly pulling the column name by ordinance via a join to a metadata view?) and then sums the number of nulls in the column. Before anyone jumps on the nitpick bandwagon please keep in mind that this basic idea could be used for more than just finding nulls. Any assistance with this issue is greatly appreciated.
You need to use dynamic sql:
declare #custom_sql varchar(max)
set #custom_sql = 'SELECT null as first_row'
select
#custom_sql = #custom_sql + ', ' + 'SUM(CASE WHEN ' + COLUMN_NAME + ' IS NULL THEN 1 ELSE 0 END) as ' + COLUMN_NAME + '_NULLS'
from
INFORMATION_SCHEMA.COLUMNS where table_name = 'MYTABLE'
set #custom_sql = #custom_sql + ' FROM MYTABLE'
exec(#custom_sql)
You can also use the COALESCE term (just for a slightly different approach):
declare #custom_sql varchar(max)
select
#custom_sql = COALESCE(#custom_sql + ', ', '') + 'SUM(CASE WHEN ' + COLUMN_NAME + ' IS NULL THEN 1 ELSE 0 END) as ' + COLUMN_NAME + '_NULLS'
from
INFORMATION_SCHEMA.COLUMNS where table_name = 'users'
set #custom_sql = 'SELECT ' + #custom_sql
set #custom_sql = #custom_sql + ' FROM Users'
print #custom_sql
exec(#custom_sql)
I don't know how to make a generic query, but you can always generate the script like this
declare #sql nvarchar(max) = 'select 1 as dummy'
select #sql = #sql + '
, sum(case when [' + c.name + '] is null then 1 else 0 end) as [' + c.name + '_NULLS]'
from sys.columns c
join sys.tables t on t.object_id = c.object_id
where t.name = 'TABLE1'
set #sql = #sql + ' from TABLE1'
select #sql
Then you can execute the result eg. with exec sp_executesql #sql
For a cooler approach, you can use ISNULL to skip the first comma.
declare #sql nvarchar(max)
declare #tablename nvarchar(255) = 'xxxx'
Select #sql = ISNULL(#SQL + ',','') + ' ' + COLUMN_NAME + '_count = Sum(case when ' + COLUMN_NAME + ' is null then 1 else 0 end)' + char(13)
From information_schema.columns
where table_name = #tablename
set #sql = 'Select' + #sql + ' From ' + #tablename
print #sql
exec sp_executesql #sql
I am trying to write a script that will show the number of non-null values in each column as well as the total number of rows in the table.
I have found a couple ways to do this:
SELECT sum(case my_column when null then 1 else 0) "Null Values",
sum(case my_column when null then 0 else 1) "Non-Null Values"
FROM my_table;
and
SELECT count(*) FROM my_table WHERE my_column IS NULL
UNION ALL
SELECT count(*) FROM my_table WHERE my_column IS NOT NULL
But these require me to type in each column name manually. Is there a way to perform this action for each column without listing them?
You should use execute:
DECLARE #t nvarchar(max)
SET #t = N'SELECT '
SELECT #t = #t + 'sum(case when ' + c.name + ' is null then 1 else 0 end) "Null Values for ' + c.name + '",
sum(case when ' + c.name + ' is null then 0 else 1 end) "Non-Null Values for ' + c.name + '",'
FROM sys.columns c
WHERE c.object_id = object_id('my_table');
SET #t = SUBSTRING(#t, 1, LEN(#t) - 1) + ' FROM my_table;'
EXEC sp_executesql #t
As Paolo said, but here is an example:
DECLARE #TableName VARCHAR(512) = 'invoiceTbl';
DECLARE #SQL VARCHAR(1024);
WITH SQLText AS (
SELECT
ROW_NUMBER() OVER (ORDER BY c.Name) AS RowNum,
'SELECT ''' + c.name + ''', SUM(CASE WHEN ' + c.Name + ' IS NULL THEN 1 ELSE 0 END) AS NullValues FROM ' + #TableName AS SQLRow
FROM
sys.tables t
INNER JOIN sys.columns c ON c.object_id = t.object_id
WHERE
t.name = #TableName),
Recur AS (
SELECT
RowNum,
CONVERT(VARCHAR(MAX), SQLRow) AS SQLRow
FROM
SQLText
WHERE
RowNum = 1
UNION ALL
SELECT
t.RowNum,
CONVERT(VARCHAR(MAX), r.SQLRow + ' UNION ALL ' + t.SQLRow)
FROM
SQLText t
INNER JOIN Recur r ON t.RowNum = r.RowNum + 1
)
SELECT #SQL = SQLRow FROM Recur WHERE RowNum = (SELECT MAX(RowNum) FROM Recur);
EXEC(#SQL);
may be this works
select count(case when Column1 is null then 1 end) as Column1NullCount,
count(case when Column2 is null then 1 end) as Column2NullCount,
count(case when Column3 is null then 1 end) as Column3NullCount
...
from My_Table
you can go with dynamic sql and sys tables.
depending on the version of sql you are using the syntax will change slightly but these are the steps:
- list the columns reading sys.columns and save the list in a temp table or table variable
- make a loop through that temp table and build the sql using the same logic you would apply manually
- execute the dynamic sql built on previous step with sp_executesql