I have a question regarding the creation of a script to dynamically select columns for a select statement. Here is my scenario:
I have some tables that have columns that are called "text1-n", "value1-n", "checkbox1-n" etc. I want to check whether these columns have values stored and if so what data is stored. The Problem is that different tables have a different number of columns and not all tables have all of these columns.
So Table1 may have "text1", "text2", "value1", "value2",... while Table2 has only "text1-5".
To get the column names I use this statement:
DECLARE #table_name varchar(max) = '...';
SELECT
COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE #table_name
AND (COLUMN_NAME LIKE 'text%'
OR COLUMN_NAME LIKE 'combobox%'
OR COLUMN_NAME LIKE 'date%'
OR COLUMN_NAME LIKE 'checkbox%'
OR COLUMN_NAME LIKE 'value%');
The goal is to get a dynamically created statemant like
SELECT DISTINCT
Combobox1
,Combobox2
,Combobox3
,Combobox4
,Date1
,Date2
,Text1
,Text2
FROM ...
I've found a similar thread, but I don't understand the DynamicSQL part so that I can adopt it to my needs.
Hope someone can help me.
DECLARE #table_name varchar(max)
DECLARE #COLUMN_NAME varchar(max)
Declare cur22 Cursor FORWARD_ONLY READ_ONLY FOR
SELECT
Table_Name , COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (COLUMN_NAME LIKE 'text%'
OR COLUMN_NAME LIKE 'combobox%'
OR COLUMN_NAME LIKE 'date%'
OR COLUMN_NAME LIKE 'checkbox%'
OR COLUMN_NAME LIKE 'value%');
OPEN cur22
FETCH NEXT FROM cur22
INTO #table_name , #COLUMN_NAME
while (##FETCH_STATUS = 0)
begin
EXEC('SELECT * FROM ' + #table_name + ' Where LEN(' + #COLUMN_NAME + ') > 0')
FETCH NEXT FROM cur22
INTO #table_name , #COLUMN_NAME
end
Close cur22
Deallocate cur22
A more lowtech solution would be to make a view per table that have all columns
SELECT DISTINCT
null as Combobox1
,Combobox2
,Combobox3
,null as Combobox4
,null as Date1
,null as Date2
,Text1
,Text2
FROM <tablename>
It may be a little annoying to maintain, but you will get code that is a lot easier to understand.
Just to close this thread... I could manage to create the desired statement/Script.
DECLARE #table_name varchar(max) = 'par_companygroup2';
DECLARE #SQL varchar(max);
SET #SQL =
'SELECT DISTINCT ' + (SELECT
p.*
FROM (SELECT
STUFF((SELECT
+',' +
Column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE #table_name
AND (
Column_name LIKE 'text%'
OR Column_name LIKE 'combobox%'
OR Column_name LIKE 'date%'
OR Column_name LIKE 'checkbox%'
OR Column_name LIKE 'value%'
)
ORDER BY 1 ASC
FOR XML PATH (''), TYPE)
.value('.', 'VARCHAR(MAX)'), 1, 1, '') AS Column_name) p)
+ ' FROM ' + #table_name
EXECUTE (#SQL)
Related
Let’s say I’m looking for a specific column in my database so I have something like this
SELECT COLUMN_NAME, TABLE_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME like ‘%employeeid%’
But I also want to know how many rows each table has, I was told I can do this using Dynamic SQL so I have this now
DECLARE
#tableName NVARCHAR(MAX),
#sql NVARCHAR(MAX),
#colName NVARCHAR(MAX);
DECLARE CUR_TABLE CURSOR FOR
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.COLUMNS
OPEN CUR_TABLE
FETCH NEXT FROM CUR_TABLE
INTO #tableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #colName = '%employeeid%'
SET #sql = 'SELECT COLUMN_NAME, TABLE_NAME, (SELECT COUNT(*) FROM ' + #tableName +') AS ROWS FROM INFORMATION_SCHEMA.COLUMNS where column_name like ' + ''' + #colName + ''';
FETCH NEXT FROM CUR_TABLE
INTO #tableName
END;
CLOSE CUR_TABLE
DEALLOCATE CUR_TABLE
EXEC sp_executesql #sql
But this doesn't work, What I'm trying to do is query a table with the column I am looking for, with the table name, and number of rows in the table.
How can I fix this?
You can make use of SQL Server's dynamic management views to quickly obtain the row counts*.
Find all tables with a column named 'MyColumn' and their current rows:
select Schema_Name(t.schema_id) schemaName, t.name TableName, s.row_count
from sys.columns c
join sys.tables t on t.object_id = c.object_id
join sys.dm_db_partition_stats s on s.object_id = c.object_id and s.index_id <= 1
where c.name='MyColumn';
* Accurate except for frequently updated tables where there could be some lag
The following uses INFORMATION_SCHEMA, dynamic SQL, and STRING_AGG() to build a query that will return a single result set.
DECLARE #ColumnName sysname = 'ProductID'
DECLARE #Newline VARCHAR(2) = CHAR(13) + CHAR(10)
DECLARE #SqlTemplate NVARCHAR(MAX) =
+ 'SELECT'
+ ' ColumnName = <ColumnNameString>,'
+ ' TableName = <TableSchemaAndNameString>,'
+ ' Rows = (SELECT COUNT(*) FROM <TableSchemaAndName>)'
+ #Newline
DECLARE #UnionSql NVARCHAR(100) = 'UNION ALL ' + #Newline
DECLARE #Sql NVARCHAR(MAX) = (
SELECT STRING_AGG(
REPLACE(REPLACE(REPLACE(
#SqlTemplate
, '<ColumnNameString>', QUOTENAME(C.COLUMN_NAME, ''''))
, '<TableSchemaAndNameString>', QUOTENAME(C.TABLE_SCHEMA + '.' + C.TABLE_NAME, ''''))
, '<TableSchemaAndName>', QUOTENAME(C.TABLE_SCHEMA) + '.' + QUOTENAME(C.TABLE_NAME))
, #UnionSql)
WITHIN GROUP(ORDER BY C.TABLE_SCHEMA, C.TABLE_NAME)
FROM INFORMATION_SCHEMA.TABLES T
JOIN INFORMATION_SCHEMA.COLUMNS C
ON C.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C.TABLE_NAME = T.TABLE_NAME
WHERE T.TABLE_TYPE = 'BASE TABLE' -- Omit views
AND C.COLUMN_NAME = #ColumnName
)
SET #Sql = #Sql + 'ORDER BY Rows DESC, TableName' + #Newline
--PRINT #Sql
EXEC (#Sql)
I generalized it a bit by adding TABLE_SCHEMA so that it could be used with the AdventureWorks database. See this db<>fiddle for a working demo. Also included is equivalent logic that uses FOR XML instead of STRING_AGG for older SQL Server versions.
Assuming that you are using SQL Server, here is a shorthand way using sp_msforeachtable.
DECLARE #ColumnName NVARCHAR(200) = 'ContactID'
CREATE TABLE #T
(
ColumnName NVARCHAR(200),
TableName NVARCHAR(200),
RecordCount INT
)
INSERT INTO #T (ColumnName, TableName)
SELECT
ColumnName = C.COLUMN_NAME,
TableName = '['+C.TABLE_SCHEMA+'].['+C.TABLE_NAME+']'
FROM
INFORMATION_SCHEMA.COLUMNS C
WHERE
C.COLUMN_NAME LIKE '%' + #ColumnName + '%'
EXEC SP_MSFOREACHTABLE 'IF EXISTS(SELECT * FROM #T WHERE TableName = ''?'') UPDATE #T SET RecordCount = (SELECT COUNT(*) FROM ? ) WHERE TableName = ''?'''
SELECT
ColumnName,TableName,
TableType = CASE
WHEN RecordCount IS NULL
THEN 'View'
ELSE 'Table'
END,
RecordCount
FROM
#T
ORDER BY
CASE WHEN RecordCount IS NULL THEN 'View' ELSE 'Table' END
DROP TABLE #T
Environment: SQL Server 2005/2008,
pubs database
I have inserted into a table variable a set of data as shown below using information_schema tables.
Now I would like to update the flag column based on the result of executing the query in the column dSQL. I was able to update using loops/cursor and then used sp_executeSQL to
update the column and then update flag column later. But is there an alternate set-based way to do this without looping through all individual rows?
use pubs
go
declare #dsql Nvarchar(max)='', #tablename varchar(100), #colname varchar(100)
declare #t table (
TABLE_NAME varchar(100),
COLUMN_NAME varchar(100)
)
insert into #t
select distinct t.TABLE_NAME, c.COLUMN_NAME
from information_Schema.tables t
inner join
information_Schema.columns c
on t.TABLE_CATALOG = c.TABLE_CATALOG
where t.TABLE_SCHEMA = c.TABLE_SCHEMA
and t.TABLE_TYPE = 'BASE TABLE'
and c.DATA_TYPE = 'varchar'
select *, Dsql = 'select ' + COLUMN_NAME + ' from ' + TABLE_NAME + ' WHERE '
+ COLUMN_NAME + ' = ''Menlo Park''', '' as Flag
FROM #t
GO
I had an idea to create a function and call the function for each row to execute individual query statement but calling the function for each record might be a performance hit.
It's a loop or a function as you suggested (which is really a loop anyway).
Not possible, I made a script like it earlier.
declare #searchvalue varchar(100)
set nocount off
set #searchvalue = 'Hello world'
create table #tt (table_name varchar(64), column_name varchar(64), count int)
select * into #t from
(
select 'select ''' + a.table_name + ''' ''table_name'',''' + a.column_name + ''' ''column_name'', count(*) count from [' + a.table_name +'] where [' +a.column_name+']='''+#searchvalue +'''' + ' group by ['+ a.column_name+']' sqlstring
from INFORMATION_SCHEMA.COLUMNS a
join
INFORMATION_SCHEMA.TABLES b
on a.table_name = b.table_name
and b.table_type = 'base table'
where data_type = 'varchar'
) a
--loop cursor
Declare #sqlstring as nvarchar(500)
Declare SqlCursor CURSOR FAST_FORWARD FOR
SELECT sqlstring FROM #t
OPEN SqlCursor
FETCH NEXT FROM SqlCursor
INTO #sqlstring
WHILE ##FETCH_STATUS = 0
BEGIN
insert #tt
exec(#sqlstring)
FETCH NEXT FROM SqlCursor
INTO #sqlstring
END
CLOSE SqlCursor
DEALLOCATE SqlCursor
select * from #tt
drop table #tt
drop table #t
Use what you want
This is an old question, but I'd like to add a different answer all the same.
Try the following script (no cursor, no loop (according to execution plan)): (tested in MS SQL 2012)
-- Setting up test data/code
SELECT N'SELECT * FROM INFORMATION_SCHEMA.COLUMNS AS C' T
INTO #Code
UNION ALL
SELECT N'SELECT * FROM INFORMATION_SCHEMA.TABLES AS T'
-- Variable to hold the selected queries, seperated by CrLf. You can also add a "GO" or ";"
DECLARE #SQL NVARCHAR(MAX) = CHAR(13) + CHAR(10)
-- Concatenate the selected queries together into the variable
SELECT #SQL = #SQL + CHAR(13) + CHAR(10) + C.T
FROM #Code AS C
-- Execute
EXEC sys.sp_executesql #SQL
-- Clean up
DROP TABLE #Code
I have a table with column names a1,a2...,b1.b2....
How can I select all those with column names like a%?
This will get you the list
select * from information_schema.columns
where table_name='table1' and column_name like 'a%'
If you want to use that to construct a query, you could do something like this:
declare #sql nvarchar(max)
set #sql = 'select '
select #sql = #sql + '[' + column_name +'],'
from information_schema.columns
where table_name='table1' and column_name like 'a%'
set #sql = left(#sql,len(#sql)-1) -- remove trailing comma
set #sql = #sql + ' from table1'
exec sp_executesql #sql
Note that the above is written for SQL Server.
You need to use view INFORMATION_SCHEMA.COLUMNS
select COLUMN_NAME from INFORMATION_SCHEMA.COLUMNS where TABLE_NAME = my_table_name AND COLUMN_NAME like 'a%'
TO inline rows you can use PIVOT and for execution EXEC() function.
This will show you the table name and column name
select table_name,column_name from information_schema.columns
where column_name like '%breakfast%'
SELECT * FROM SysColumns WHERE Name like 'a%'
Will get you a list of columns, you will want to filter more to restrict it to your target table
From there you can construct some ad-hoc sql
Here is a nice way to display the information that you want:
SELECT B.table_catalog as 'Database_Name',
B.table_name as 'Table_Name',
stuff((select ', ' + A.column_name
from INFORMATION_SCHEMA.COLUMNS A
where A.Table_name = B.Table_Name
FOR XML PATH(''),TYPE).value('(./text())[1]','NVARCHAR(MAX)')
, 1, 2, '') as 'Columns'
FROM INFORMATION_SCHEMA.COLUMNS B
WHERE B.TABLE_NAME like '%%'
AND B.COLUMN_NAME like '%%'
GROUP BY B.Table_Catalog, B.Table_Name
Order by 1 asc
Add anything between either '%%' in the main select to narrow down what tables and/or column names you want.
You cannot with standard SQL. Column names are not treated like data in SQL.
If you use a SQL engine that has, say, meta-data tables storing column names, types, etc. you may select on that table instead.
Blorgbeard had a great answer for SQL server. If you have a MySQL server like mine then the following will allow you to select the information from columns where the name is like some key phrase. You just have to substitute the table name, database name, and keyword.
SET #columnnames = (SELECT concat("`",GROUP_CONCAT(`COLUMN_NAME` SEPARATOR "`, `"),"`")
FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA`='your_database'
AND `TABLE_NAME`='your_table'
AND COLUMN_NAME LIKE "%keyword%");
SET #burrito = CONCAT("SELECT ",#columnnames," FROM your_table");
PREPARE result FROM #burrito;
EXECUTE result;
Thank you #Blorgbeard for the genious idea.
By the way Blorgbeard's query was not working for me so I edited it:
DECLARE #Table_Name as VARCHAR(50) SET #Table_Name = 'MyTable' -- put here you table name
DECLARE #Column_Like as VARCHAR(20) SET #Column_Like = '%something%' -- put here you element
DECLARE #sql NVARCHAR(MAX) SET #sql = 'select '
SELECT #sql = #sql + '[' + sys.columns.name + '],'
FROM sys.columns
JOIN sys.tables ON sys.columns.object_id = tables.object_id
WHERE sys.columns.name like #Column_Like
and sys.tables.name = #Table_Name
SET #sql = left(#sql,len(#sql)-1) -- remove trailing comma
SET #sql = #sql + ' from ' + #Table_Name
EXEC sp_executesql #sql
I have a table with 600+ columns imported from a CSV file with special chars % _ - in the column names.
Is there a way to change the column names to remove these special chars?
The code can be in T-SQL.
sp_rename?
EXEC sp_rename 'table.[Oh%Dear]', 'OhDear', 'COLUMN';
Worked example (was not sure about [ ] in sp_rename)
CREATE TABLE foo ([Oh%Dear] int)
EXEC sp_rename 'foo.[Oh%Dear]', 'OhDear', 'COLUMN'
EXEC sys.sp_help 'foo'
You can query INFORMATION_SCHEMA.COLUMNS and generate sp_rename scripts to rename the columns.
SELECT 'EXEC sp_rename ''' + TABLE_NAME + '.' + QUOTENAME(COLUMN_NAME) + ''', ''' + REPLACE(COLUMN_NAME, '%', '') + ''', ''COLUMN''; '
FROM INFORMATION_SCHEMA.COLUMNS
WHERE COLUMN_NAME LIKE '%[%]%'
Here's a permalink to an actual runnable example
Can you use this?
sp_RENAME 'Table.ColumnName%', 'NewColumnName' , 'COLUMN'
here is an example that will loop over a table and change underscores and percent signs
create table Test ([col%1] varchar(50),[col_2] varchar(40))
go
select identity(int,1,1) as id ,column_name,table_name into #loop
from information_schema.columns
where table_name = 'Test'
and column_name like '%[%]%'
or column_name like '%[_]%'
declare #maxID int, #loopid int
select #loopid =1
select #maxID = max(id) from #loop
declare #columnName varchar(100), #tableName varchar(100)
declare #TableColumnNAme varchar(100)
while #loopid <= #maxID
begin
select #tableName = table_name , #columnName = column_name
from #loop where id = #loopid
select #TableColumnNAme = #tableName + '.' + #columnName
select #columnName = replace(replace(#columnName,'%',''),'_','')
EXEC sp_rename #TableColumnNAme, #columnName, 'COLUMN';
set #loopid = #loopid + 1
end
drop table #loop
select * from Test
I want to present a schema to my BA team.
Select TABLE_NAME, col.DATA_TYPE
from INFORMATION_SCHEMA.COLUMNS col
order by col.TABLE_CATALOG, TABLE_NAME, TABLE_SCHEMA, col.ORDINAL_POSITION
I want to present 3 rows of sample data as well but I wanted to pivot it out so they get an output that has the following 3 columns:
Table_Name
Data_Type
Sample_Data
How do I pivot that data?
Here is a solution based on cursors and dynamic SQL. I've includes the table schema and column names in the final result as well, though the question just asks fro table name, data type, and sample data.
Also, I wasn't sure if you wanted three rows of sample data per table/column, or if you wanted one row per table/column with three columns of sample data. I went with the former, please let me know if you wanted the later. I did include a "No data" indicator for tables that don't have any sample data.
Tested on SQL Server 2005, but I think it should work with 2000 as well.
Create Table #results
(id Integer Not Null Identity(1, 1)
,table_schema nVarChar(128) Not Null
,table_name nVarChar(128) Not Null
,column_name nVarChar(128) Not Null
,data_type nVarChar(128) Not Null
,sample_data nVarChar(max) Null);
Declare #table_name nVarChar(128)
,#table_schema nVarChar(128)
,#column_name nVarChar(128)
,#data_type nVarChar(128)
,#sql nVarChar(max)
,#inserted Integer;
Declare rs Cursor Local Forward_Only Static Read_Only
For Select
col.table_schema
,col.table_name
,col.column_name
,col.data_type
From INFORMATION_SCHEMA.COLUMNS col
Order By col.TABLE_CATALOG
,col.TABLE_SCHEMA
,col.TABLE_NAME
,col.ORDINAL_POSITION
Open rs;
Fetch Next From rs Into #table_schema, #table_name, #column_name, #data_Type;
While ##Fetch_Status = 0 Begin;
Set #table_schema = QuoteName(#table_schema);
Set #table_name = QuoteName(#table_name);
Set #column_name = QuoteName(#column_name);
Set #sql = N'
Insert Into #results
(table_schema
,table_name
,column_name
,data_type
,sample_data)
Select Top 3 ' + QuoteName(#table_schema, '''') + N'
,' + QuoteName(#table_name, '''') + N'
,' + QuoteName(#column_name, '''') + N'
,' + QuoteName(#data_type, '''') + N'
,' + #column_name + N'
From ' + #table_schema + N'.' + #table_name;
Exec (#sql);
Select #inserted = count(*)
From #results
Where table_schema = #table_schema
And table_name = #table_name
And column_name = #column_name;
If #inserted = 0
Insert Into #results (table_schema, table_name, column_name, data_type, sample_data)
Values (#table_schema, #table_name, #column_name, #data_type, ' -- No Data -- ');
Fetch Next From rs Into #table_schema, #table_name, #column_name, #data_Type;
End;
Close rs;
Deallocate rs;
-- Probably should include the schema and column name too:
Select table_schema, table_name, column_name, data_type, sample_data
From #results
Order by [id];
-- But this is what the question asked for:
-- Select table_name, data_type, sample_data
-- From #results
-- Order by [id];
Drop Table #results;
There are probably more elegant solutions available, but this should get you started, I think. Good luck!
I built this using XML PATH
SET NOCOUNT ON
SELECT
'SELECT ''' + col.TABLE_NAME + ''' AS TableName,' +
'''' + col.COLUMN_NAME + ''' AS ColumnName,'+
' ''' + col.DATA_TYPE + ''' as DataType, ' +
'
(
SELECT top 3 CONVERT (VARCHAR, p2.' + col.COLUMN_NAME + ') + '',''
FROM ' + col.TABLE_SCHEMA + '.' + col.TABLE_NAME + ' p2
ORDER BY p2. ' + col.COLUMN_NAME + '
FOR XML PATH('''')
)
UNION ALL'
FROM INFORMATION_SCHEMA.COLUMNS col
ORDER BY
col.TABLE_CATALOG,
col.TABLE_NAME,
col.TABLE_SCHEMA,
col.ORDINAL_POSITION
I copy paste the result of this query into a new query editor window. I delete the last UNION ALL and execute the query.
It gives me an additional comma in the Returned data, but my B.A.'s were OK with that.
You can dynamically build queries like this for each table:
SELECT TOP 3 *
FROM mytable
FOR XML AUTO
and parse the resulting XML in the presenation layer.