I have successfully written code to dynamically union the results of a simple select statement across multiple databases. I would like to include the database name itself as a field so that I can identify each record. What is the best way to modify the code below to accomplish that?
My current results look like
field2, field3, field4
b,c,d
2,3,4
I can't tell what database the row containing (b,c,d) comes from.
And I would like to make sure I see
field1, field2, field3, field4
First_DatabaseName, b,c,d
Second_DatabaseName,2,3,4
And above, I could then see that the row containing (b,c,d) comes from First_Database
DECLARE #sql varchar(max)
SELECT #sql = ISNULL(#sql + 'union all ',' ') + ' SELECT * FROM ' + name + '.dbo.CombinedProvider '
FROM sys.databases
WHERE name in ('First_DatabaseName', 'Second_DatabaseName')
EXEC (#sql)
Just add the name into the subquery that you are using:
SELECT #sql = COALESCE(#sql + 'union all ',' ') + ' SELECT '''+name+''' as dbname, c.* FROM ' + name + '.dbo.CombinedProvider c'
FROM sys.databases
WHERE name in ('First_DatabaseName', 'Second_DatabaseName')
By the way, you are using an unsupported technique to do aggregate string concatenation. You might want to learn about "for xml path('')" as an alternative technique. Check out http://www.simple-talk.com/sql/t-sql-programming/concatenating-row-values-in-transact-sql/ for more information than you want to know on this subject.
Use 'AS' after each column and include the source db name in the column.
DECLARE #sql varchar(max)
SELECT #sql = ISNULL(#sql + 'union all ',' ') + ' SELECT ColName as [First_DatabaseName_ColName], ColName2 as [Second_DatabaseName_ColName2], FROM ' + name + '.dbo.CombinedProvider '
FROM sys.databases
WHERE name in ('First_DatabaseName', 'Second_DatabaseName')
EXEC (#sql)
Related
Could you please help me with following issue?
Source table:
Columns defined from INFORMATION_SCHEMA.COLUMNS:
In output I'd like to take my source table, but show only values which column name is the same as column name defined in information schema. Meaning:
Is it possible? Many thanks in advance
You need to use dynamic SQL to do that:
declare #sql varchar(1000) 'select ';
select #sql = #sql + '[' + column_name + '] ,' from INFORMATION_SCHEMA.COLUMNS;
-- remove last character in a string which is comma
select #sql = left(#sql, len(#sql) - 1);
-- you need to change talbe name here
select #sql = #sql + ' from MyTable';
-- execute statement
exec(#sql)
I have a bunch of tables, that have the same first few characters in the names, but the tables have random numbers (equal in length) at the end of the names.
They have the same structure.
I want to union them into one table, dynamically.
This is in SQL Server 2008 Express.
I have no real idea how to do this, but I'm guessing I have to loop thru a list of the tables names, maybe using a list in the system tables?
Example (that illustrates my simple minded thinking, as I'm sure this make no real technical sense)
SELECT * FROM TABLE0*
UNION ALL
SELECT * FROM TABLE0*
Note '*' is a a number with 8 digits.
A quick dynamic SQL script should do it:
declare #sql varchar(max)
set #sql = ''
select #sql = #sql + case len(#sql) when 0 then '' else ' UNION ALL ' end + '
SELECT * FROM [' + table_name + ']'
from
information_schema.tables where table_name like 'TABLE0%'
exec (#sql)
You could use a simple query like this to construct your large query:
SELECT 'SELECT * FROM '+name+ ' UNION '
FROM sys.tables
WHERE name LIKE '%yourtable%'
Or you could use dynamic SQL to build it and run it:
DECLARE #sql VARCHAR(MAX)
SET #sql = ''
SELECT #sql = #sql +'
UNION ALL
SELECT * FROM ['+name+']'
FROM sys.tables
WHERE name LIKE '%yourtable%'
SET #sql = STUFF(#sql,1,15,'')
EXEC(#sql)
Doc table contains a lot of columns (even not used ones):
Doc_DdfID Doc_RentDate Doc_ReturnDate etc.
--------- ------------ -------------- ----
1 2012-07-28 2012-07-28
But I want to query just the used ones within Doc's table.
DocDefinitionFields list columsn that are in use by document:
SELECT Dfl_ColName
FROM DocDefinitionFields
WHERE Dfl_DdfID = 1
DocDefinitionFields:
Dfl_ColName
-----------
Doc_RentDate
Doc_ReturnDate
...........
So I want to select all columns (listed by second query) from Doc table.
Example (if 2 columns are added to document definition form I want to select just them):
Doc:
Doc_RentDate Doc_ReturnDate
------------ --------------
2012-07-28 2012-07-28
Tried to do that by subquerying select with concatenation of fields using XML PATH:
SELECT
(SELECT
Dfl_ColName + ', '
FROM DocDefinitionFields
FOR XML PATH('')
)
FROM Doc
It's not that simple tho. What do you suggest?
What you need here is dynamic SQL, something like this:
DECLARE #sql VARCHAR(MAX)
SET #sql = 'SELECT ' + STUFF((SELECT ', ' + Dfl_ColName FROM DocDefinitionFields FOR XML PATH('') ),1,1,'') + ' FROM Doc'
EXEC (#sql)
Also, in order to eliminate additional comma(,) at the end of columns I have added STUFF function along with FOR XML PATH.
To get the column names for a query dynamically and use these in a query you will need to use Dynamic SQL. Below is an example of how to create the string of available columns
DECLARE #Columns VARCHAR(MAX);
SELECT #Columns =
COALESCE(#Columns + ',
[' + CAST(COLUMN_NAME AS VARCHAR) + ']',
'[' + CAST(COLUMN_NAME AS VARCHAR) + ']')
FROM (SELECT DISTINCT COLUMN_NAME
FROM [SomeDatabase].INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'SomeTableName') AS H
ORDER BY COLUMN_NAME;
GO
You can now use the string of available columns in a Dynamic SQL query. Below we have adopted the above in an INSERT query that build the required fields dynamically. The reason why we need to do it in the below is the inclusion of the set field SomeFieldA along with the others.
DECLARE #SQL NVARCHAR(MAX);
SET #SQL =
N'INSERT INTO [' + #DbName + N']..[SomeOtherTable] ([SomeFieldA], ' + #Columns + N')
SELECT SomeFieldA, ' + #Columns + N'
FROM [SomeTableName];';
EXEC sp_executesql #SQL;
GO
You should be able to amend the above to provide what you need.
I hope this helps.
Suppose you have a table like this:
ID FNAME LNAME
1 Bob Smith
2 Sally Jones
A simple SELECT * FROM [Table] will return all rows. But what if you wanted to build a single string out of the results, and the column names are unknown? In other words, this will not work:
SELECT ID + ',' + FNAME + ',' + LNAME FROM [Table]
because you don't know the column names. Additionally, COALESCE won't work because it doesn't accept wildcards. Ideally you want to execute something like this:
SELECT dbo.FunctionThatSplitsResultsToString(*) FROM [Table]
and have it return
1,Bob,Smith
2,Sally,Jones
Is this possible?
This is a corrected version of the answer #Igor gave. In addition to concatenating comma characters between the values, it converts NULL values to an empty string (because concatenating a string to NULL results in a NULL value).
DECLARE #sql NVARCHAR(max)='SELECT '
DECLARE #TableName NVARCHAR(max) = 'Table_Name' -- <-- Set the target table name here
SELECT #sql=#sql+N'ISNULL(CAST(' + name +' as NVARCHAR(max)), '''')+'',''+'
FROM sys.columns
WHERE object_id=OBJECT_ID(#TableName)
SELECT #sql=SUBSTRING(#sql,1,LEN(#sql)-5)+N' FROM ' + #TableName
--SELECT #sql -- uncomment to see the query string
EXEC sp_executesql #sql
As the first Igor noted, the solution is dynamic SQL. You need to construct the underlying SQL statement correctly.
The following code casts all columns to varchar() and then concatenates them together. The final form of the SQL removes the last "+" sign and adds the from statement:
declare #sql varchar(max);
select #sql = (select 'cast('+coalesce(column_name, '') + ' as varchar(255)) +'
from information_schema.columns
where table_name = <whatever>
for xml path ('')
);
select #sql = left(#sql, len(#sql - 2)) + ' from t';
exec(#sql);
I admit to being US-centric and rarely using internationalization. The whole thing also works with nvarchars().
Try the below one
GO
DECLARE #ColumnsList VARCHAR(MAX), #SelectStatement VARCHAR(MAX),#TargetTable VARCHAR(250) ,#FINALSQL NVARCHAR(MAX)
SET #TARGETTABLE ='TempData'
SELECT #ColumnsList = COALESCE( #ColumnsList+' + '','' +' ,'') + 'Cast('+ A.COLUMN_NAME + ' AS Varchar(250))'
FROM (select Column_Name from INFORMATION_SCHEMA.COLUMNS where TABLE_NAME= #TARGETTABLE) A
SELECT #FinalSql = 'Select ' +#ColumnsList + ' FROM ' + #TARGETTABLE
EXEC SP_EXECUTESQL #FINALSQL
GO
I use MS SQL Server.
Ive been handed some large tables with no constrains on them, no keys no nothing.
I know some of the columns have unique values. Is there a smart way for a given table to finde the cols that have unique values ?
Right now I do it manually for each column by counting if there is as many DISTINCT values as there are rows in the table.
SELECT COUNT(DISTINCT col) FROM table
Could prob make a cusor to loop over all the columns but want to hear if someone knows a smarter or build-in function.
Thanks.
Here's an approach that is basically similar to #JNK's but instead of printing the counts it returns a ready answer for every column that tells you whether a column consists of unique values only or not:
DECLARE #table varchar(100), #sql varchar(max);
SET #table = 'some table name';
SELECT
#sql = COALESCE(#sql + ', ', '') + ColumnExpression
FROM (
SELECT
ColumnExpression =
'CASE COUNT(DISTINCT ' + COLUMN_NAME + ') ' +
'WHEN COUNT(*) THEN ''UNIQUE'' ' +
'ELSE '''' ' +
'END AS ' + COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #table
) s
SET #sql = 'SELECT ' + #sql + ' FROM ' + #table;
PRINT #sql; /* in case you want to have a look at the resulting query */
EXEC(#sql);
It simply compares COUNT(DISTINCT column) with COUNT(*) for every column. The result will be a table with a single row, where every column will contain the value UNIQUE for those columns that do not have duplicates, and empty string if duplicates are present.
But the above solution will work correctly only for those columns that do not have NULLs. It should be noted that SQL Server does not ignore NULLs when you want to create a unique constraint/index on a column. If a column contains just one NULL and all other values are unique, you can still create a unique constraint on the column (you cannot make it a primary key, though, which requires both uniquness of values and absence of NULLs).
Therefore you might need a more thorough analysis of the contents, which you could get with the following script:
DECLARE #table varchar(100), #sql varchar(max);
SET #table = 'some table name';
SELECT
#sql = COALESCE(#sql + ', ', '') + ColumnExpression
FROM (
SELECT
ColumnExpression =
'CASE COUNT(DISTINCT ' + COLUMN_NAME + ') ' +
'WHEN COUNT(*) THEN ''UNIQUE'' ' +
'WHEN COUNT(*) - 1 THEN ' +
'CASE COUNT(DISTINCT ' + COLUMN_NAME + ') ' +
'WHEN COUNT(' + COLUMN_NAME + ') THEN ''UNIQUE WITH SINGLE NULL'' ' +
'ELSE '''' ' +
'END ' +
'WHEN COUNT(' + COLUMN_NAME + ') THEN ''UNIQUE with NULLs'' ' +
'ELSE '''' ' +
'END AS ' + COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #table
) s
SET #sql = 'SELECT ' + #sql + ' FROM ' + #table;
PRINT #sql; /* in case you still want to have a look at the resulting query */
EXEC(#sql);
This solution takes NULLs into account by checking three values: COUNT(DISTINCT column), COUNT(column) and COUNT(*). It displays the results similarly to the former solution, but the possible diagnoses for the columns are more diverse:
UNIQUE means no duplicate values and no NULLs (can either be a PK or have a unique constraint/index);
UNIQUE WITH SINGLE NULL – as can be guessed, no duplicates, but there's one NULL (cannot be a PK, but can have a unique constraint/index);
UNIQUE with NULLs – no duplicates, two or more NULLs (in case you are on SQL Server 2008, you could have a conditional unique index for non-NULL values only);
empty string – there are duplicates, possibly NULLs too.
Here is I think probably the cleanest way. Just use dynamic sql and a single select statement to create a query that gives you a total row count and a count of distinct values for each field.
Fill in the DB name and tablename at the top. The DB name part is really important since OBJECT_NAME only works in the current database context.
use DatabaseName
DECLARE #Table varchar(100) = 'TableName'
DECLARE #SQL Varchar(max)
SET #SQL = 'SELECT COUNT(*) as ''Total'''
SELECT #SQL = #SQL + ',COUNT(DISTINCT ' + name + ') as ''' + name + ''''
FROM sys.columns c
WHERE OBJECT_NAME(object_id) = #Table
SET #SQL = #SQL + ' FROM ' + #Table
exec #sql
If you are using 2008, you can use the Data Profiling Task in SSIS to return Candidate Keys for each table.
This blog entry steps through the process, it's fairly simple:
http://consultingblogs.emc.com/jamiethomson/archive/2008/03/04/ssis-data-profiling-task-part-8-candidate-key.aspx
A few words what my code does:
Read's all tables and columns
Creates a temp table to hold table/columns with duplicate keys
For each table/column it runs a query. If it finds a count(*)>1 for at least one value
it makes an insert into the temp table
Select's column and values from the system tables that do not match table/columns that are found to have duplicates
DECLARE #sql VARCHAR(max)
DECLARE #table VARCHAR(100)
DECLARE #column VARCHAR(100)
CREATE TABLE #temp (tname VARCHAR(100),cname VARCHAR(100))
DECLARE mycursor CURSOR FOR
select t.name,c.name
from sys.tables t
join sys.columns c on t.object_id = c.object_id
where system_type_id not in (34,35,99)
OPEN mycursor
FETCH NEXT FROM mycursor INTO #table,#column
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = 'INSERT INTO #temp SELECT DISTINCT '''+#table+''','''+#column+ ''' FROM ' + #table + ' GROUP BY ' + #column +' HAVING COUNT(*)>1 '
EXEC (#sql)
FETCH NEXT FROM mycursor INTO #table,#column
END
select t.name,c.name
from sys.tables t
join sys.columns c on t.object_id = c.object_id
left join #temp on t.name = #temp.tname and c.name = #temp.cname
where system_type_id not in (34,35,99) and #temp.tname IS NULL
DROP TABLE #temp
CLOSE mycursor
DEALLOCATE mycursor
What about simple one line of code:
CREATE UNIQUE INDEX index_name ON table_name (column_name);
If the index is created then your column_name has only unique values. If there are dupes in your column_name, you will get an error message.