Dynamically search columns for given table - sql

I need to create a search for a java app I'm building where users can search through a SQL database based on the table they're currently viewing and a search term they provide. At first I was going to do something simple like this:
SELECT * FROM <table name> WHERE CAST((SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<table name>')
AS VARCHAR) LIKE '%<search term>%'
but that subquery returns more than one result, so then I tried to make a procedure to loop through all the columns in a given table and put any relevant fields in a results table, like this:
CREATE PROC sp_search
#tblname VARCHAR(4000),
#term VARCHAR(4000)
AS
SET nocount on
SELECT COLUMN_NAME
INTO #tempcolumns
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #tblname
ALTER TABLE #tempcolumns
ADD printed BIT,
num SMALLINT IDENTITY
UPDATE #tempcolumns
SET printed = 0
DECLARE #colname VARCHAR(4000),
#num SMALLINT
WHILE EXISTS(SELECT MIN(num) FROM #tempcolumns WHERE printed = 0)
BEGIN
SELECT #num = MIN(num)
FROM #tempcolumns
WHERE printed = 0
SELECT #colname = COLUMN_NAME
FROM #tempcolumns
WHERE num = #num
SELECT * INTO #results FROM #tblname WHERE CAST(#colname AS VARCHAR)
LIKE '%' + #term + '%' --this is where I'm having trouble
UPDATE #tempcolumns
SET printed = 1
WHERE #num = num
END
SELECT * FROM #results
GO
This has two problems: first is that it gets stuck in an infinite loop somehow, and second I can't select anything from #tblname. I tried using dynamic sql as well, but I don't know how to get results from that or if that's even possible.
This is for an assignment I'm doing at college and I've gotten this far after hours of trying to figure it out. Is there any way to do what I want to do?

You need to only search columns that actually contain strings, not all columns in a table (which may include integers, dates, GUIDs, etc).
You shouldn't need a #temp table (and certainly not a ##temp table) at all.
You need to use dynamic SQL (though I'm not sure if this has been part of your curriculum so far).
I find it beneficial to follow a few simple conventions, all of which you've violated:
use PROCEDURE not PROC - it's not a "prock," it's a "stored procedure."
use dbo. (or alternate schema) prefix when referencing any object.
wrap your procedure body in BEGIN/END.
use vowels liberally. Are you saving that many keystrokes, never mind time, saying #tblname instead of #tablename or #table_name? I'm not fighting for a specific convention but saving characters at the cost of readability lost its charm in the 70s.
don't use the sp_ prefix for stored procedures - this prefix has special meaning in SQL Server. Name the procedure for what it does. It doesn't need a prefix, just like we know they're tables even without a tbl prefix. If you really need a prefix there, use another one like usp_ or proc_ but I personally don't feel that prefix gives you any information you don't already have.
since tables are stored using Unicode (and some of your columns might be too), your parameters should be NVARCHAR, not VARCHAR. And identifiers are capped at 128 characters, so there is no reason to support > 257 characters for #tablename.
terminate statements with semi-colons.
use the catalog views instead of INFORMATION_SCHEMA - though the latter is what your professor may have taught and might expect.
CREATE PROCEDURE dbo.SearchTable
#tablename NVARCHAR(257),
#term NVARCHAR(4000)
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'SELECT * FROM ' + #tablename + ' WHERE 1 = 0';
SELECT #sql = #sql + '
OR ' + c.name + ' LIKE ''%' + REPLACE(#term, '''', '''''') + '%'''
FROM
sys.all_columns AS c
INNER JOIN
sys.types AS t
ON c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE
c.[object_id] = OBJECT_ID(#tablename)
AND t.name IN (N'sysname', N'char', N'nchar',
N'varchar', N'nvarchar', N'text', N'ntext');
PRINT #sql;
-- EXEC sp_executesql #sql;
END
GO
When you're happy that it's outputting the SELECT query you're after, comment out the PRINT and uncomment the EXEC.

You get into an infinite loop because EXISTS(SELECT MIN(num) FROM #tempcolumns WHERE printed = 0) will always return a row even if there are no matches - you need to EXISTS (SELECT * .... instead
To use dynamic SQL, you need to build up a string (varchar) of the SQL statement you want to run, then you call it with EXEC
eg:
declare #s varchar(max)
select #s = 'SELECT * FROM mytable '
Exec (#s)

Related

In One DB I have 100+ tables, but I need staging table (start with STR_) wise column wise(Which is Status ) row count. in Dynamic query [duplicate]

I am trying to write this query to find all tables with specific column with some specific value. This is what I've done so far -
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
END
'
I hope my intensions are clear, I just want to select only those tables where the column EMP_CODE is present and in those tables I want to select those rows where EMP_CODE='HO081'.
Edit -
Now it stands like this. But I'm not able to replace #EMPCODE variable in the query.
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
EXEC sp_MSforeachtable
#command1='
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
',#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME='''+#EMPCODE+''')'
You know how sp_MSforeachtable is undocumented, and may go away at any time/be modified?
Well, if you're happy to ignore that, it has another parameter called #whereand, which is appended to the WHERE clause of the internal query that is being used to find the tables (and should start with an AND).
You also have to know that there's an alias, o against sysobjects, and a second alias syso against sys.all_objects.
Using this knowledge, you might craft your #whereand parameter as:
EXEC sp_MSforeachtable
#command1='...',
#whereand='AND o.id in (select object_id from sys.columns c where c.name=''EMP_CODE'')'
You can now also simplify your command1, since you know it will only be run against tables containing an EMP_CODE column. I'd probably take out the COUNT(*) condition also, since I don't see what value it's adding.
Updated based on your further work, and tested against one table:
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
declare #sql nvarchar(2000)
set #sql = '
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
'
EXEC sp_MSforeachtable
#command1=#sql,#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME=''EMP_CODE'')'
(I've reverted the #whereand to query for EMP_CODE, since you don't want to replace the value there).
The issue is that, you can pass parameters to a stored procedure, or literals, but you can't perform calculations/combining actions between them - so I moved the construction of the sql statement out into a separate action.
I guess you get an error of some kind, perhaps Invalid column name 'EMP_CODE'?
It's because the code is compiled before you check for the column.
You could do like this instead.
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
EXEC(''
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
'')
END
'

How execute a dynamic query in stored procedure?

I want to get from user variables: num for 'Top' clause and Tbl name from northwind DB,
and to get back result set of top 5, my script is down
create procedure sp_getTop5
(
#num int ,
#tbl nvarchar(max)
)
as
declare #res nvarchar(max);
set #res = 'select top '+str(#num)+' * from '+#tbl
exec #res
---- check
exec sp_getTop5 5, employees
Wow... There's nothing else I can say other than that is one of the most terrifying dynamic statements I have ever seen. You're literally giving a malicious person access to 2GB worth of characters to inject with (that's 1,073,741,824 characters with a nvarchar(MAX)) against a statement that can easily suffer injection. They would LITERALLY be able to do anything they wanted with enough time (and permissions). Please take the time to read my article on Dos and Don'ts of Dynamic SQL and have a look at Little Bobby Tables to understand how dangerous what you have is.
As for your SQL, I don't care that you haven't really asked a question, you need to fix that massive hole in your security model now:
CREATE PROC getTop5 #Num int, #schema sysname, #table sysname AS --Removed sp_ prefix, see after the answer
BEGIN
DECLARE #SQL nvarchar(MAX);
SELECT #SQL = N'SELECT TOP (#Num) FROM ' + QUOTENAME(s.[name]) + N'.' + QUOTENAME(t.[name]) + N';'
FROM sys.schemas s
JOIN sys.tables t ON s.schema_id = t.schema_id
WHERE s.[name] = #schema
AND t.[name] = #table;
EXEC sp_executesql #SQL, N'#Num int', #Num;
END;
GO
Then you can execute it as below:
EXEC dbo.getTop5 5, N'dbo', N'YourTable';
This will (as a commenter just reminded me) give N arbitrary rows from the table (not the "Top" rows, and tables don't have an inbuilt order). So a TOP without an ORDER BY means the rows returned could be different every time the query is run.
As for my comment about the prefix: Is the sp_ prefix still a no-no?

SQL Server passing identifiers to stored procedures/dynamic SQL

Background:
SQL Server Management Studio allows to define own query shortcuts (Tools > Options > Environment > Keyboard > Query Shortcuts):
Image from: http://social.technet.microsoft.com/wiki/contents/articles/3178.how-to-create-query-shortcuts-in-sql-server-management-studio.aspx
my_schema.my_table
-- highlight it
-- press CTRL + 3 and you will get the number of rows in table
It works ok, but it concatenates query in basic form (as far as I know only at the end). Query:
SELECT COUNT(*) FROM my_schema.my_table;
Attempt #1
Now I want to write something more specific, for example pass/concatenate table name to following query (this is just example):
SELECT * FROM sys.columns WHERE [object_id] = OBJECT_ID(...)
So when I write in query shortcuts:
SELECT * FROM sys.columns WHERE [object_id] = OBJECT_ID('
I have to use:
my_schema.my_table')
-- highlight it
-- press CTRL + 3
The additional ') is very ugly and inconvenient.
Attempt #2:
The second trial is to use Dynamic-SQL:
EXEC dbo.sp_executesql
N'SELECT * FROM sys.columns WHERE [object_id] = OBJECT_ID(#obj_name)'
,N'#obj_name SYSNAME'
,
Executing:
my_table
-- highligt it
-- and run
LiveDemo
Works also when table name is quoted [my_table]. As long as object is in dbo(default) schema.
The problem is that when table has schema it won't work:
EXEC dbo.sp_executesql
N'SELECT * FROM sys.columns WHERE [object_id] = OBJECT_ID(#obj_name)'
,N'#obj_name SYSNAME'
,
Executing:
my_schema.my_table
[my_schema].[my_table]
LiveDemo2
Incorrect syntax near '.'.
Of course I could write:
EXEC dbo.sp_executesql
N'SELECT * FROM sys.columns WHERE [object_id] = OBJECT_ID(#obj_name)'
,N'#obj_name SYSNAME'
,'
and call it as:
[my_schema].[my_table]'
But additional ' is also ugly and inconvenient.
Questions:
Is it possible to pass value, to query shortcuts window, in the middle (positional or even more than one value)?
Is it possible to pass do stored_procedure/dynamic-sql qualified identifier without wraping it with ', "?
Remarks:
I do not search for plugins to SSMS
I do not want to wrap object_name as "my_schema.my_table"
I know there is sp_helptext (this is just example, I search for method)
First question is tool specific (I am aware of it), but second is about SQL Server.
EDIT:
To clarify passing identifier to SP without ' or ":
CREATE TABLE dbo.my_table(col INT);
GO
CREATE PROCEDURE dbo.my_proc
#a SYSNAME
AS
SELECT *
FROM sys.columns
WHERE [object_id] = OBJECT_ID(#a)
GO
EXEC dbo.my_proc
#a = my_table;
EXEC dbo.my_proc
#a = dbo.my_table;
-- Incorrect syntax near '.'.
LiveDemo3
1. Is it possible to pass value, to query shortcuts window, in the middle?
To my knowledge, there is no workaround to achieve this.
1-b. Is it possible to pass more than one value?
It can be done for string values using a separator character and then splitting the value on the other side. Sadly, there isn't many special character to fulfill this job because they pretty much all raise a syntax error. However '#' could be a wise choice because it's already a special character for SQL for temp table going in tempDB. Just check if you don't already have identifier that are using it because it's permitted by SQL (tough, it's forbidden as first char).
Here is an example of this :
Create a stored procedure to receive the arguments into one single string and split the string to have each arguments.
CREATE PROCEDURE sp_PassingMultipleStringValues
#Param1 NVARCHAR(MAX)
AS
--Here I'm using a XML split, but feel free to use any string split function you already have.
DECLARE #xml AS XML,
#separator AS VARCHAR(1)
SELECT #separator ='#',
#xml = CAST('<X>'+ (REPLACE(#Param1,#separator ,'</X><X>') +'</X>') AS XML)
SELECT N.value('.', 'VARCHAR(200)') AS value
FROM #xml.nodes('X') as T(N)
--Do whatever is needed with them
Then configure your shortcut as seem on this image. (Note the space at the end)
Result :
2. Is it possible to pass to a stored_procedure/dynamic-sql qualified identifier without wraping it with ', "?
Do you have multiple schema with the same identifier?
Because if not, what about retrieve it on the other side using sys.schemas instead of passing it?
Instead of having an inconvenient character to type at the end, you would have fewer things to type.
With the retrieved schema, you can then do dynamic SQL for whatever is needed with it.
SELECT #Param1 = REPLACE(REPLACE(#Param1, '[', ''), ']', '')
SELECT TOP 1 #Param1 = [Schema].name + '.' + #Param1
FROM sys.objects AS obj
JOIN sys.schemas AS [Schema] ON obj.schema_id = [Schema].schema_id
WHERE obj.name = #Param1
SELECT *
FROM sys.columns
WHERE [object_id] = OBJECT_ID(#Param1)
DECLARE #Query NVARCHAR(MAX) = 'SELECT TOP 1 * FROM ' + #Param1
EXEC sp_sqlexec #Query
If you do want to handle two different schema with the same identifier then it's still feasible by passing the schema and the identifier as two arguments using the method explained in answer 1-b.
Everything in one example
Since here we want to pass multiple identifiers and specify their schema, two separators are needed.
CREATE PROCEDURE sp_MultiArgsWithSchema
#Param1 NVARCHAR(MAX)
AS
SELECT #Param1 = REPLACE(REPLACE(#Param1, '[', ''), ']', '')
--Here I'm using a XML split, but feel free to use any string split function you already have.
DECLARE #xml AS XML,
#ArgSeparator AS VARCHAR(2),
#SchemaSeparor AS VARCHAR(1)
SELECT #ArgSeparator = '##',
#SchemaSeparor = '#',
#xml = CAST('<X>'+ (REPLACE(#Param1,#ArgSeparator, '</X><X>') +'</X>') AS XML)
IF OBJECT_ID('tempdb..#QualifiedIdentifiers') IS NOT NULL
DROP TABLE #QualifiedIdentifiers;
--While splitting, we are putting back the dot instead of '#' between schema and name of object
SELECT QualifiedIdentifier = REPLACE(N.value('.', 'VARCHAR(200)'), #SchemaSeparor, '.')
INTO #QualifiedIdentifiers
FROM #xml.nodes('X') as T(N)
SELECT * FROM #QualifiedIdentifiers
--From here, use what is inside #QualifiedIdentifiers and Dynamic SQL if need to achieve what is needed
DECLARE #QualifiedIdentifier NVARCHAR(500)
WHILE EXISTS(SELECT TOP 1 1 FROM #QualifiedIdentifiers)
BEGIN
SELECT TOP 1 #QualifiedIdentifier = QualifiedIdentifier
FROM #QualifiedIdentifiers
SELECT *
FROM sys.columns
WHERE [object_id] = OBJECT_ID(#QualifiedIdentifier)
DELETE TOP (1)
FROM #QualifiedIdentifiers
WHERE QualifiedIdentifier = #QualifiedIdentifier
END
Usage (note that specifying the schema isn't mandatory) :
So, since it is inconvenient to have to double the splitting character, it would be best if schema could be guessed like stated above.
Here is a long shot to pass multi-part identifier without wrapping it with quotes.
Solution:
The query shortcuts is going to create a synonym in the database with a specific name and a DDLTrigger to intercept this specific synonym creation.
Setup up the following shortcut in Query ShortCuts. (Make sure you include last space)
DECLARE #CreateTriggerSQL NVARCHAR(MAX) = 'CREATE TRIGGER DDLTrigger_QueryShortcutX ON DATABASE FOR CREATE_SYNONYM AS BEGIN DECLARE #EventData XML = EVENTDATA(), #SynonymName NVARCHAR(255), #DbName NVARCHAR(255), #SchemaName NVARCHAR(255), #ObjectName NVARCHAR(255), #Alias NVARCHAR(255) SELECT #SynonymName = #EventData.value(''(/EVENT_INSTANCE/ObjectName)[1]'', ''NVARCHAR(255)'') IF(#SynonymName = ''QueryShortcutX'') BEGIN DROP SYNONYM QueryShortcutX DROP TRIGGER DDLTrigger_QueryShortcutX ON DATABASE SELECT #DbName = #EventData.value(''(/EVENT_INSTANCE/DatabaseName)[1]'', ''NVARCHAR(255)''), #SchemaName = #EventData.value(''(/EVENT_INSTANCE/TargetSchemaName)[1]'', ''NVARCHAR(255)''), #ObjectName = #EventData.value(''(/EVENT_INSTANCE/TargetObjectName)[1]'', ''NVARCHAR(255)''), #Alias = (CASE WHEN LEN(#SchemaName) > 0 THEN #SchemaName + ''.'' ELSE '''' END) + #ObjectName /*EXEC yourStoredProcHere #Param = #Alias*/ SELECT DbName = #DbName, SchemaName = #SchemaName, ObjectName = #ObjectName, Alias = #Alias, ObjectId = OBJECT_ID(#Alias) END END' EXEC sp_executeSQL #CreateTriggerSQL CREATE SYNONYM QueryShortcutX FOR
As #Vladimir suggested, here we use "sp_executesql" to be able to create the trigger and the synonym at the same time.
Here is the code of the trigger without being inlined.
CREATE TRIGGER DDLTrigger_QueryShortcutX ON DATABASE FOR CREATE_SYNONYM
AS
BEGIN
DECLARE #EventData XML = EVENTDATA(),
#SynonymName NVARCHAR(255),
#DbName NVARCHAR(255),
#SchemaName NVARCHAR(255),
#ObjectName NVARCHAR(255),
#Alias NVARCHAR(255)
SELECT #SynonymName = #EventData.value('(/EVENT_INSTANCE/ObjectName)[1]', 'NVARCHAR(255)')
--Safety in case someone else really create a synonym meanwhile.
IF(#SynonymName = 'QueryShortcutX')
BEGIN
--2. Clean up what we created
DROP SYNONYM QueryShortcutX
DROP TRIGGER DDLTrigger_QueryShortcutX ON DATABASE
--3. Parsing identifier code here
SELECT #DbName = #EventData.value('(/EVENT_INSTANCE/DatabaseName)[1]', 'NVARCHAR(255)'),
#SchemaName = #EventData.value('(/EVENT_INSTANCE/TargetSchemaName)[1]', 'NVARCHAR(255)'),
#ObjectName = #EventData.value('(/EVENT_INSTANCE/TargetObjectName)[1]', 'NVARCHAR(255)'),
#Alias = (CASE WHEN LEN(#SchemaName) > 0 THEN #SchemaName + '.' ELSE '' END) + #ObjectName
--4. Here, write any print/select statement you want.
--For maintenance, it would be easier to just call a stored procedure from here with parameter and put the desired print/select there.
--Thus avoiding to redo inlining the whole trigger each time.
--EXEC yourStoredProcHere #Param = #Alias
SELECT DbName = #DbName,
SchemaName = #SchemaName,
ObjectName = #ObjectName,
Alias = #Alias,
ObjectId = OBJECT_ID(#Alias)
END
END
Here is the code of the shortcut without being inlined.
DECLARE #CreateTriggerSQL NVARCHAR(MAX) = 'Trigger creation code here...'
IF EXISTS(SELECT TOP 1 1 FROM sys.triggers WHERE name = 'DDLTrigger_QueryShortcutX')
BEGIN
DROP TRIGGER DDLTrigger_QueryShortcutX ON DATABASE
END
EXEC sp_executeSQL #CreateTriggerSQL
IF EXISTS(SELECT TOP 1 1 FROM sys.synonyms WHERE name = 'QueryShortcutX')
BEGIN
DROP SYNONYM QueryShortcutX
END
CREATE SYNONYM QueryShortcutX FOR
The trigger drop itself and the synonym to avoid schema pollution.
The trigger parse the information to retrieve identifier.
Use the identifier for your needs. (use dynamic SQL if needed)
Results for each test item
1.RealColumnName
2.WhatEverText
3.dbo.tests
4.[No selection]
5.dbo.tests.very.much
DbName SchemaName ObjectName Alias ObjectId
1.TEST RealColumnName RealColumnName NULL --FN OBJECT_ID doesn't return value with only column name
2.TEST WhatEverText WhatEverText NULL
3.TEST dbo tests dbo.tests 245575913
4.Incorrect syntax near 'FOR'.
5.TEST very much very.much NULL
The parsing I've made doesn't handle identifier with more than two multipart properly. If you want to improve it. The following XML show you which tag to use.
<TargetServerName>dbo</TargetServerName>
<TargetDatabaseName>tests</TargetDatabaseName>
<TargetSchemaName>very</TargetSchemaName>
<TargetObjectName>much</TargetObjectName>
Note:
If you prefer, you can let the trigger stay permanently within the database.
Also, if you want to pass multiple identifiers, string parsing like I do in my other answer is still a possibility here.
To use this solution, user will have to have "create synonym permission" and either own the schema or have "ALTER SCHEMA permission".

Return multiple columns as single comma separated row in SQL Server 2005

I'm curious to see if this is possible.
I have a table or this could be specific to any old table with data. A simple SELECT will return the columns and rows as a result set. What I'm trying to find out if is possible to return rows but rather than columns, the columns concatenated and are comma separated. So expected amount of rows returned but only one varchar column holding comma separated results of all the columns just like a CSV file.
Thanks.
[UPDATE]
Here is a bit more detail why I'm asking. I don't have the option to do this on the client, this is a task I'm trying to do with SSIS.
Scenario: I have a table that is dynamically created in SSIS but the column names change each time it's built. The original package uses BCP to grab the data and put it into a flat file but due to permissions when run as a job BCP can't create the flat file at the required destination. We can't get this changed either.
The other issue is that with SSIS 2005, using the flat files destination, you have to map the column name from the input source which I can't do because the column names keep changing.
I've written a script task to grab all the data from the original tables and then use stream writer to write to the CSV but I have to loop through each row then through each column to produce the string built up of all the columns. I want to measure performance of this concatenation of columns on sql server against a nasty loop with vb.net.
If I can get sql to produce a single column for each row I can just write a single line to the text file instead of iterating though each column to build the row.
I Think You Should try This
SELECT UserName +','+ Password AS ColumnZ
FROM UserTable
Assuming you know what columns the table has, and you don't want to do something dynamic and crazy, you can do this
SELECT CONCAT(ColumnA, ',', ColumnB) AS ColumnZ
FROM Table
There is a fancy way to this using SQL Server's XML functions, but for starters could you just cast the contents of the columns you care about as varchar and concatenate them with commas?
SELECT cast(colA as varchar)+', '+cast(colB as varchar)+', '+cast(colC as varchar)
FROM table
Note, that this will get tripped up if any of your contents have a comma or double quotes in them, in which case you can also use a replace function on each cast to escape them.
This could stand to be cleaned up some, but you can do this by using the metadata stored in sys.objects and sys.columns along with dynamic SQL. Note that I am NOT a fan of dynamic SQL, but for reporting purposes it shouldn't be too much of a problem.
Some SQL to create test data:
if (object_id('test') is not null)
drop table test;
create table test
(
id uniqueidentifier not null default newId()
,col0 nvarchar(255)
,col1 nvarchar(255)
,col2 nvarchar(255)
,col3 nvarchar(255)
,col4 nvarchar(255)
);
insert into test (col0,col1,col2,col3,col4)
select 'alice','bob','charlie','dave','emily'
union
select 'abby','bill','charlotte','daniel','evan'
A stored proc to build CSV rows:
-- emit the contents of a table as a CSV.
-- #table_name: name of a permanent (in sys.objects) table
-- #debug: set to 1 to print the generated query
create procedure emit_csv(#table_name nvarchar(max), #debug bit = 0)
as
declare #object_id int;
set nocount on;
set #object_id = object_id(#table_name);
declare #name nvarchar(max);
declare db_cursor cursor for
select name
from sys.columns
where object_id = #object_id;
open db_cursor;
fetch next from db_cursor into #name
declare #query nvarchar(max);
set #query = '';
while ##FETCH_STATUS = 0
begin
-- TODO: modify appended clause to escape commas in addition to trimming
set #query = #query + 'rtrim(cast('+#name+' as nvarchar(max)))'
fetch next from db_cursor into #name;
-- add concatenation to the end of the query.
-- TODO: Rearrange #query construction order to make this unnecessary
if (##fetch_status = 0)
set #query = #query + ' + '','' +'
end;
close db_cursor;
deallocate db_cursor;
set #query = 'select rtrim('+#query+') as csvrow from '+#table_name;
if #debug != 0
begin
declare #newline nvarchar(2);
set #newline = char(13) + char(10)
print 'Generated SQL:' + #newline + #query + #newline + #newline;
end
exec (#query);
For my test table, this generates the query:
select
rtrim(rtrim(cast(id as nvarchar(max)))
+ ','
+rtrim(cast(col0 as nvarchar(max)))
+ ','
+rtrim(cast(col1 as nvarchar(max)))
+ ','
+rtrim(cast(col2 as nvarchar(max)))
+ ','
+rtrim(cast(col3 as nvarchar(max)))
+ ','
+rtrim(cast(col4 as nvarchar(max))))
as csvrow
from test
and the result set:
csvrow
-------------------------------------------------------------------------------------------
EEE16C3A-036E-4524-A8B8-7CCD2E575519,alice,bob,charlie,dave,emily
F1EE6C84-D6D9-4621-97E6-AA8716C0643B,abby,bill,charlotte,daniel,evan
Suggestions
Modify the cursor loop to escape commas
Make sure that #table_name refers to a valid table (if object_id(#table_name) is null) in the sproc
Some exception handling would be good
Set permissions on this so that only the account that runs the report can execute it. String concatenation in dynamic SQL can be a big security hole, but I don't see another way to do this.
Some error handling to ensure that the cursor gets closed and deallocated might be nice.
This can be used for any table that is not a #temp table. In that case, you'd have to use sys.objects and sys.columns from tempdb...
select STUFF((select ','+ convert(varchar,l.Subject) from tbl_Student B,tbl_StudentMarks L
where B.Id=L.Id FOR XML PATH('')),1,1,'') Subject FROM tbl_Student A where A.Id=10

Running the same SQL code against a number of tables sequentially

I have a number of tables (around 40) containing snapshot data about 40 million plus vehicles. Each snapshot table is at a specific point in time (the end of the quarter) and is identical in terms of structure.
Whilst most of our analysis is against single snapshots, on occasion we need to run some analysis against all the snapshots at once. For instance, we may need to build a new table containing all the Ford Focus cars from every single snapshot.
To achieve this we currently have two options:
a) write a long, long, long batch file repeating the same code over and over again, just changing the FROM clause
[drawbacks - it takes a long time to write and changing a single line of code in one of blocks requires fiddly changes in all the other blocks]
b) use a view to union all the tables together and query that instead
[drawbacks - our tables are stored in separate database instances and cannot be indexed, plus the resulting view is something like 600 million records long by 125 columns wide, so is incredibly slow]
So, what I would like to find out is whether I can either use dynamic sql or put the SQL into a loop to spool through all tables. This would be something like:
for each *table* in TableList
INSERT INTO output_table
SELECT *table* as OriginTableName, Make, Model
FROM *table*
next *table* in TableList
Is this possible? This would mean that updating the original SQL when our client changes what they need (a very regular occurrence!) would be very simple and we would benefit from all the indexes we already have on the original tables.
Any pointers, suggestions or help will be much appreciated.
If you can identify your tables (e.g. a naming pattern), you could simply say:
DECLARE #sql NVARCHAR(MAX);
SELECT #sql = N'';
SELECT #sql = #sql + 'INSERT output_table SELECT ''' + name + ''', Make, Model
FROM dbo.' + QUOTENAME(name) + ';'
FROM sys.tables
WHERE name LIKE 'pattern%';
-- or WHERE name IN ('t1', 't2', ... , 't40');
EXEC sp_executesql #sql;
This assumes they're all in the dbo schema. If they're not, the adjustment is easy... just replace dbo with ' + QUOTENAME(SCHEMA_NAME([schema_id])) + '...
In the end I used two methods:
Someone on another forum suggested making use of sp_msforeachtable and a table which contains all the table names. Their suggestion was:
create table dbo.OutputTable (OriginTableName nvarchar(500), RecordCount INT)
create table dbo.TableList (Name nvarchar (500))
insert dbo.TableList
select '[dbo].[swap]'
union select '[dbo].[products]'
union select '[dbo].[structures]'
union select '[dbo].[stagingdata]'
exec sp_msforeachtable #command1 = 'INSERT INTO dbo.OutputTable SELECT ''?'', COUNT(*) from ?'
,#whereand = 'and syso.object_id in (select object_id(Name) from dbo.TableList)'
select * from dbo.OutputTable
This works perfectly well for some queries, but seems to suffer from the fact that one cannot use a GROUP BY clause within the query (or, at least, I could not find a way to do this).
The final solution I used was to use Dynamic SQL with a lookup table containing the table names. In a very simple form, this looks like:
DECLARE #TableName varchar(500)
DECLARE #curTable CURSOR
DECLARE #sql NVARCHAR(1000)
SET #curTable = CURSOR FOR
SELECT [Name] FROM Vehicles_LookupTables.dbo.AllStockTableList
OPEN #curTable
FETCH NEXT
FROM #curTable INTO #TableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = 'SELECT ''' +#TableName + ''', Make, sum(1) as Total FROM ' + #TableName + ' GROUP BY Make'
EXEC sp_executesql #sql
FETCH NEXT
FROM #curTable INTO #TableName
END
CLOSE #curTable
DEALLOCATE #curTable