I need to create a function which searches all the tables that contain a certain field
select distinct(table_name) from information_schema.columns where column_name='fieldNeeded';
and then, to do a query on each table that is found:
select * from table_name where ... <parameters> ;
Is there a way to do so?
If you use SQL-server...
You can use the result of the first query is input for your cursor (a for-loop in T-SQL) in where you dynamically create and execute the query. Note 'dynamic sql' is creating an sql-command in string (varchar) format and then run that string as sql-commant (something like: EXEC #SqlCommand), although there are other (better) ways to execute dynamic sql.
To finish things up nicely, you can start with creating a temp-table and insert the result in the temp-table. And in each iteration of the cursor you can store the result in the temp-table
In pseudo-code it would look something like:
Create #TempTable with some columns
Create Cursor for:
select distinct(table_name)
from information_schema.columns
where column_name='fieldNeeded'
For each element in the cursor (fetch next #table_name from the cursor)
DECLARE #SqlCommand VARCHAR(250) =
'Insert Into #TempTable select * from ' + #table_name + 'where ... <parameters>'
PRINT #SqlCommand -- can be useful to check the code you created
EXEC #SqlCommand
End of cursor
SELECT * FROM #TempTable
Related
I am trying to write this query to find all tables with specific column with some specific value. This is what I've done so far -
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
END
'
I hope my intensions are clear, I just want to select only those tables where the column EMP_CODE is present and in those tables I want to select those rows where EMP_CODE='HO081'.
Edit -
Now it stands like this. But I'm not able to replace #EMPCODE variable in the query.
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
EXEC sp_MSforeachtable
#command1='
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
',#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME='''+#EMPCODE+''')'
You know how sp_MSforeachtable is undocumented, and may go away at any time/be modified?
Well, if you're happy to ignore that, it has another parameter called #whereand, which is appended to the WHERE clause of the internal query that is being used to find the tables (and should start with an AND).
You also have to know that there's an alias, o against sysobjects, and a second alias syso against sys.all_objects.
Using this knowledge, you might craft your #whereand parameter as:
EXEC sp_MSforeachtable
#command1='...',
#whereand='AND o.id in (select object_id from sys.columns c where c.name=''EMP_CODE'')'
You can now also simplify your command1, since you know it will only be run against tables containing an EMP_CODE column. I'd probably take out the COUNT(*) condition also, since I don't see what value it's adding.
Updated based on your further work, and tested against one table:
DECLARE #EMPCODE AS VARCHAR(20)
SET #EMPCODE='HO081'
declare #sql nvarchar(2000)
set #sql = '
DECLARE #COUNT AS INT
SELECT #COUNT=COUNT(*) FROM ? WHERE EMP_CODE='''+#EMPCODE+'''
IF #COUNT>0
BEGIN
PRINT PARSENAME("?",1)+'' => ''+CONVERT(VARCHAR,#COUNT)+'' ROW(S)''
--PRINT ''DELETE FROM ''+PARSENAME("?",1)+'' WHERE EMP_CODE='''''+#EMPCODE+'''''''
END
'
EXEC sp_MSforeachtable
#command1=#sql,#whereand='AND O.ID IN (SELECT OBJECT_ID FROM SYS.COLUMNS C WHERE C.NAME=''EMP_CODE'')'
(I've reverted the #whereand to query for EMP_CODE, since you don't want to replace the value there).
The issue is that, you can pass parameters to a stored procedure, or literals, but you can't perform calculations/combining actions between them - so I moved the construction of the sql statement out into a separate action.
I guess you get an error of some kind, perhaps Invalid column name 'EMP_CODE'?
It's because the code is compiled before you check for the column.
You could do like this instead.
EXEC sp_MSforeachtable
#command1='
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=PARSENAME("?",2) AND TABLE_NAME=PARSENAME("?",1) AND COLUMN_NAME="EMP_CODE")
BEGIN
EXEC(''
IF (SELECT COUNT(*) FROM ? WHERE EMP_CODE="HO081")>0
BEGIN
SELECT * FROM ? WHERE EMP_CODE="HO081"
END
'')
END
'
I have a procedure in which there are calculations being done and the final result is inserted into a permanent table. I want to remove the permanent table and I cannot use Temp table as well. So i want to use a dynamic table name, which is stored in a variable:
Current scenario:
Insert into xyz_table
Select col1,col2,sum(col3)
from BaseTable
(In reality, there are lot of columns and a lot of calculations)
What I want:
Select col1,col2,sum(col3) into #DynamicTableName
from BaseTable
where the name of the table would be dynamic in nature i.e.,
#DynamicTableName = 'xyz ' + cast(convert(date,getdate()) as nvarchar)+' '+convert(nvarchar(5),getdate(),108)
It will have date and time in its name every time the procedure is run.
I want to use this name in the "Select * into statement"
How can I achieve this?
i tried it with the some short code. But since my procedure has a lot of calculations and UNIONS , I cannot use that code for this. Any help would be appreciated.
declare #tablename nvarchar(30)= 'xyz ' + cast(convert(date,getdate()) as nvarchar)+' '+convert(nvarchar(5),getdate(),108)
declare #SQL_Statement nvarchar(100)
declare #SQL_Statement2 nvarchar(100)
declare #dropstatement nvarchar(100)
SET #SQL_Statement = N'SELECT * Into ' +'['+#tablename +'] '+'FROM '+ 'dimBranch'
print #SQL_Statement
EXECUTE sp_executesql #SQL_Statement
SET #SQL_Statement= N'select * from ' + '['+#tablename + '] '
print #SQL_Statement
EXECUTE sp_executesql #SQL_Statement
set #dropstatement = 'DROP TABLE' + '['+#tablename + '] '
PRINT #dropstatement
exec sp_executesql #dropstatement
Reason why I want this is because I use this procedure in ETL job as well as in SSRS report. And if someone runs the package and the SSRS report at the same time, the incorrect or weird data gets stored in the table. Therefore I need a dynamic name of the table with date and time.
You can't parameterize an identifier in SQL, only a value
--yes
select * from table where column = #value
--no
select * from #tablename where #columnname = #value
The only thin you can do to make these things dynamic is to build an sql string and execute it dynamically, but your code is already doing this with sp_executesql
More telling is your complaint at the bottom of your question, that if the procedure is invoked simultaneously it gives problems. Perhaps you should consider using local table variables for temporary data storage that the report is using rather than pushing data back into the db
DECLARE #temp TABLE(id INT, name varchar100);
INSERT INTO #temp SELECT personid, firstname FROM person;
-- work with temp data
select count(*) from #temp;
--when #temp goes out of scope it is lost,
--no other procedure invoked simultaneously can access this procedure'a #temp
Consider a local temp table, which is automatically session scoped without the need for dynamic SQL. For example:
SELECT *
INTO #YourTempTable
FROM dimBranch;
The local temp table will automatically be dropped when the proc completes so there is no need for an explict drop in the proc code.
I have a stored procedure where I need to query a table that contains another query that I then need to execute, get the results, and store those results in another table. I will not know what, or how many, columns are returned from this query, but I must be able to map the unknown columns to columns in my results table. I do know that the query can contain anywhere from 1 to 20 columns which need to map to my results table as RSLT_1 up to RSLT_20.
For Example, let's say the query returns 5 columns. I need to iterate over the results and map column1 to RSLT_1, column2 to RSLT_2, etc. Then store those results in my result table
I have this logic already written in C# which was trivial since I can loop over columns to determine how many exist. I don't know how to do that in a stored procedure. Any ideas?
I cannot write you a complete answer, because I don't know your queries/tables. Here main lines of what I'm thinking about. You have to complete for your needs. But keep in mind, cursor is slow and generally not recommended in production.
select * into #tmp from <sourceQuery>
declare #columnName varchar(100)
declare #sql NVARCHAR(MAX)
declare c cursor for
select name from tempdb.sys.columns where object_id = object_id('tempdb..#tmp')
open c
fetch next from c into #columnName
while ##FETCH_STATUS = 0
begin
select #columnName
set #sql = N'insert into <targetTable> (valueColumn) select ' + #columnName + 'from #tmp'
exec #sql
fetch next from c into #columnName
end
close c
deallocate c
drop table #tmp
The db being accessed is on Snowflake; not certain on the storage details behind the scenes.
I have a query right now that creates a new view from 41 data tables stored in separate schemas under the same database, looks something like this:
CREATE VIEW all_data AS
SELECT * FROM db.schema1.data UNION ALL
SELECT * FROM db.schema2.data UNION ALL
SELECT * FROM db.schema3.data
This query is run daily. My issue is I get new data tables added every few days and I have to go manually edit the query to include those new tables, as they're stored under separate schemas (and the naming scheme for the schemas isn't consistent either, for reasons outside my control). Is there a way I can select all the schemas inside a database with a subquery that would allow me to run the query daily without needing manual updates when new schemas + tables are added?
I'd like the resulting query to have a structure somewhat like
CREATE VIEW all_data as
SELECT * FROM [SELECT schemas from db].data
but not sure how that would work, and how to union the resulting data correctly.
Unfortunately, in Snowflake you can't dynamically construct SQL statements (yet). You can of course do what you want to achieve via a script in one of the supported languages (e.g. Python, JS), by first finding all the schemas and then constructing a full SQL statement.
Hope this helps.
You can definitely query the table and schema list available. SQL Authority has a good article on it:
http://blog.sqlauthority.com/2009/06/17/sql-server-list-schema-name-and-table-name-for-database/
In short the query winds up being along these lines to pull the list of tables and schema:
SELECT '['+SCHEMA_NAME(schema_id)+'].['+name+']'
AS SchemaTable
FROM sys.tables
Though you will have to add a database name to the where clause to point to the proper DB.
With the release of Snowflake Scripting dynamic recreation of the view inside Snowflake is now very possible.
create database dynamic_views;
create schema dynamic_views.schema_base;
create schema dynamic_views.schema1;
create table dynamic_views.schema1.data(id int) as select * from values (1);
We can use the INFORMATION_SCEMA.TABLES to find all DATA tables:
SELECT table_schema
FROM dynamic_views.information_schema.tables
WHERE table_name = 'DATA';
TABLE_SCHEMA
SCHEMA1
and now push that into a cursor and build up a view creation SQL
This SQL needs to be run in the new Snowsight Console see (Working with Classic Console):
declare
sql text;
add_union boolean := false;
c1 cursor for SELECT TABLE_SCHEMA
FROM dynamic_views.information_schema.TABLES
WHERE TABLE_NAME = 'DATA';
begin
sql := 'CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS ';
for record in c1 do
if (add_union) then
sql := sql || 'UNION ALL ';
end if;
sql := sql || 'SELECT * FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
add_union := true;
end for;
EXECUTE IMMEDIATE sql;
return sql;
end;
;
and we can use it:
select * from dynamic_views.schema_base.all_data;
ID
1
and add more:
create schema dynamic_views.schema2;
create table dynamic_views.schema2.data(id int) as select * from values (2);
rebuild:
anonymous block
CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS SELECT * FROM dynamic_views.SCHEMA1.data UNION ALL SELECT * FROM dynamic_views.SCHEMA2.data
use it again:
select * from dynamic_views.schema_base.all_data;
ID
1
2
Note: You should not use SELECT * in production as the order of the table columns will be dependent of the create orders, and if newer tables have a different shape you view will become invalid.
So the explicit form really should be used:
'SELECT column1, column2, column4 FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
for anyone who want the solution for this question this is my Idea with useing FETCH
Declare #str nvarchar(maX)
Declare #i int
Set #i =(Select max(id ) from Clinics );
set #str='';
declare #Id int
declare cur CURSOR LOCAL for
select [Id] from [dbo].[Clinics]
GROUP BY [Id]
open cur
fetch next from cur into #Id
while ##FETCH_STATUS = 0 BEGIN
if #i>#id
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] UNION ALL ';
end
else
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] ';
end
fetch next from cur into #Id
END
close cur
deallocate cur
print #str;
exec (#str);
I am trying to store the results of an SQL query into a variable.The query simply detects the datatype of a column, hence the returned result is a single varchar.
SET #SQL =
'declare ##x varchar(max) SET ##x = (select DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS
WHERE Table_name = ' +char(39)+#TabName+char(39) +
' AND column_name = ' +char(39)+#colName+char(39) + ')'
EXECUTE (#SQL)
Anything within the 'SET declaration' cannot access any variables outside of it and vice versa, so I am stuck on how to store the results of this query in a varchar variable to be accessed by other parts of the stored procedure.
You dont need a dynamic query to achieve what you want, below query will give the same result as yours.
declare #x varchar(max)
declare #tableName varchar(100), #ColumnName varchar(50)
set #tableName = 'Employee'
set #ColumnName = 'ID'
select #x = DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS
where
Table_Name = #tableName
and column_name = #ColumnName
select #x
All user-defined variables in T-SQL have private local-scope only. They cannot be seen by any other execution context, not even nested ones (unlike #temp tables, which can be seen by nested scopes). Using "##" to try to trick it into making a global-variable doesn't work.
If you want to execute dynamic SQL and return information there are several ways to do it:
Use sp_ExecuteSQL and make one of the parameters an OUTPUT parameter (recommended for single values).
Make a #Temp table before calling the dynamic SQL and then have the Dynamic SQL write to the same #Temp table (recommended for multiple values/rows).
Use the INSERT..EXEC statement to execute your dynamic SQL which returns its information as the output of a SELECT statement. If the INSERT table has the same format as the dynamic SQL's SELECT output, then the data output will be inserted into your table.
If you want to return only an integer value, you can do this through the RETURN statement in dynamic SQL, and receive it via #val = EXEC('...').
Use the Session context-info buffer (not recommended).
However, as others have pointed out, you shouldn't actually need dynamic SQL for what you are showing us here. You can do just this with:
SET #x = ( SELECT DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS
WHERE Table_name = #TabName
AND column_name = #colName )
You may want to consider using the sp_executesql stored procedure for dynamic sql.
The following link provides a good usage example of sp_executesql procedure with output parameters:
http://support.microsoft.com/kb/262499