Creating a user-defined table-valued function which should return select union all dynamic query.
I have table tbl_tablesinfo which contains table names tbl1, tbl2, tbl3, etc. in all around 3000 table names.
I don't want to create view but function which should return select * from all tables by doing union all.
My attempt:
CREATE FUNCTION udf_alldata()
RETURNS TABLE
AS
BEGIN
DECLARE #Var VARCHAR(MAX) = ''
SELECT
#Var = STUFF((SELECT ' SELECT * FROM [' + tbl.TableNames + '] UNION ALL'
FROM [TestDB].SYS.TABLES tb
INNER JOIN [TestDB].dbo.[tbl_tablesinfo] tbl ON tb.name = tbl.TableNames
FOR XML PATH('')), 1, 1, '');
SET #var = LEFT(#var, LEN(#var) - 10);
EXEC #var
RETURN
END
I'm getting an error:
Incorrect syntax near 'BEGIN'.
Reason for doing this is creating view with 3k tables is getting slow and taking around 30 min of time, so I am looking for an alternative by creating function.
In the docs is clear said, that:
User-defined functions cannot make use of dynamic SQL or temp tables.
Table variables are allowed.
which means that you need to use a stored procedure and this is not bad as you can still insert the data in table if you want:
INSERT INTO #Table
EXEC [dbo].[stored_procedured_name]
INSERT INTO #Table
EXEC [dbo].[stored_procedured_name]
So, in your case you will have:
CREATE PROCEDURE udf_alldata
AS
BEGIN
DECLARE #Var VARCHAR(MAX) = ''
SELECT
#Var = STUFF((SELECT ' SELECT * FROM [' + tbl.TableNames + '] UNION ALL'
FROM [TestDB].SYS.TABLES tb
INNER JOIN [TestDB].dbo.[tbl_tablesinfo] tbl ON tb.name = tbl.TableNames
FOR XML PATH('')), 1, 1, '');
SET #var = LEFT(#var, LEN(#var) - 10);
EXEC sp_executesql #var
RETURN
END
Note, actually you can execute dynamic T-SQL in function but this is special case using SQL CLR. I can show you how to do this, but it will be better to stuck with the stored procedure.
You can't use Dynamic Query in SQL Function...
Functions can return only Scalar Values, or Tables...
Instead you can use Stored Procedure...
Related
I've created a datatype and declared a table of this type which I intend to pass as a parameter to an OPENQUERY statement. OPENQUERY returns an error that the object has no columns.
Firstly, the table parameter is defined as follows:
CREATE TYPE LooseTimecardsTableType AS TABLE ([LABORKEY] [float] NULL)
GO
DECLARE #DataTable AS LooseTimecardsTableType
INSERT INTO #DataTable
SELECT DISTINCT WOBase.LABORKEY
FROM Lab_WO_DataWH AS WOBase
left outer JOIN Lab_hrs_DataWH LabHrsWH ON LabHrsWH.WORKORDERLABORKEY = WOBase.LABORKEY
WHERE LabHrsWH.WORKORDERLABORKEY IS NULL AND WOBase.LABORKEY IS NOT NULL AND WOBase.LABORPRICE <> 0
The data table returns a single column of LABORKEY's which I want use to restrict the number of records from the OPENQUERY. This in turn is defined as follows:
DECLARE #SQLString NVARCHAR(500), #TableVariable LooseTimecardsTableType
SET #SQLString = N'SELECT * FROM OPENQUERY(Remoteserver, ''SELECT DISTINCT
DA.USERNAME, DA.WORKORDERLABORKEY, LB.PERFORMEDBY
FROM
REMOTE.WORKORDERDETAILAUDITS DA
JOIN REMOTE.WORKORDERLABORBASE LB ON LB.LABORKEY = DA.WORKORDERLABORKEY
JOIN #TableVariable ON #TableVariable.LABORKEY = LB.LABORKEY
WHERE DA.WORKORDERAUDITCATEGORY = 0'')'
EXECUTE sp_executesql #SQLString, N'#TableVariable LooseTimecardsTableType READONLY', #DataTable
Please assist in getting this to work
Instead of using a table variable you could turn the select statement that populates the table into a view using the where clause and then join the view to the table on the remote server.
The table variable cannot be passed as a parameter to OPENQUERY. As my table had only a single column I was able to convert it to a string and pass the string to OPENQUERY as a parameter. I did this within a procedure that accepts a table variable. After parsing the table to a string, the procedure also runs the OPENQUERY.
CREATE PROCEDURE PRM_LIST ( #TableVariable LooseTimecardsTableType READONLY)
AS
DECLARE
#LBKY_NVAR NVARCHAR(2500),
#POINTER INT,
#SQLString NVARCHAR(max)
SELECT #POINTER = MIN(LABORKEY) FROM #TableVariable
WHILE #POINTER IS NOT NULL
BEGIN
SET #LBKY_NVAR = IIF(#LBKY_NVAR IS NULL,'('+''''+ CONVERT(VARCHAR,#POINTER) + '''',
#LBKY_NVAR + ',' + ''''+ CONVERT(VARCHAR,#POINTER) + '''')
SELECT #POINTER = MIN(LABORKEY) FROM #TableVariable WHERE LABORKEY > #POINTER
END
SET #LBKY_NVAR = #LBKY_NVAR + ')'
SET #LBKY_NVAR = REPLACE(#LBKY_NVAR,'''','''''')
SET #SQLString =
N'SELECT * FROM OPENQUERY(REMOTE, ''SELECT DISTINCT
DA.USERNAME, DA.WORKORDERLABORKEY, LB.PERFORMEDBY
FROM
REMOTE.WORKORDERDETAILAUDITS DA
JOIN REMOTE.WORKORDERLABORBASE LB ON LB.LABORKEY = DA.WORKORDERLABORKEY
WHERE DA.WORKORDERAUDITCATEGORY = 0
AND LB.LABORKEY IN '+CAST(#LBKY_NVAR AS nvarchar(2500))+ N'
'')'
EXEC (#SQLString)
GO
The procedure call is:
EXECUTE PRM_LIST #DATATABLE
I would like to create a function that returns a comma separated list of field name for any given table. The function should accept database, schema and table name as input as return the comma separated list.
I can do this in a stored procedure but I want to do this in a function so I can join it into datasets. However I am problems with dynamic sql is not allowed in function - so how can I do this?
here is the proc which i want to duplicate in a function
alter proc dbo.usp_generate_column_name_string
#database varchar(100),#schema varchar(100), #table varchar(100)
as
declare #str varchar(max) = '
select stuff((select '','' + name as [text()] from
(
select c.name from ' + #database + '.sys.tables a
inner join ' + #database + '.sys.schemas b on a.schema_id = a.schema_id
inner join ' + #database + '.sys.columns c on c.object_id= a.object_id
where b.name = '''+#schema+''' and a.name ='''+#table+''') x
for xml path ('''')),1,1,'''')
'
exec (#str)
go
exec dbo.usp_generate_column_name_string 'test' , 'dbo','jl1_tmp'
There are so many ways to do it, one easier way is to insert the proc result into a temp table and use it in join
create table #coltemp(colList varchar(max))
insert into #coltemp
exec dbo.usp_generate_column_name_string 'test' , 'dbo','jl1_tmp'
select * from #coltemp
check the following question to know about diff ways to insert proc results into temp table Insert results of a stored procedure into a temporary table
Here is the basic idea:
create function usp_generate_column_name_string (
#schema varchar(100),
#table varchar(100)
)
returns varchar(max) as
begin
return (select stuff( (select ',' + column_name
from information_schema.columns
where table_name = #table and table_schema = #schema
for xml path ('')
), 1, 1, ''
)
);
end;
Notes:
This doesn't handle special characters in the column names. I'm not sure how you want to escape those, but the logic is easily adjusted.
Database is left out. That is much harder in SQL Server, because the system tables are organized by database. If that is a requirement, you basically cannot do this (easily).
I have a stored procedure that returns a grid result. I'm having some fundamental issues with transferring the #idCC parameter to the SQL query as shown. I know the SQL works if I remove the parameter and hard code in a number value. How do I allow the query to read #idCC to the query shown?
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[sp_CC_ContactGroupList]
(#idCC int)
AS
BEGIN
DECLARE #ListOfGroups VARCHAR(MAX) =
(STUFF((SELECT DISTINCT ',' + QUOTENAME(GroupName)
FROM dbo.CC_Groups
WHERE IsContactGroup = '1'
FOR XML PATH('')), 1, 1, ''));
DECLARE #sql VARCHAR(MAX) = ('SELECT *
FROM
(SELECT
dbo.CC_Contacts.idCC,
dbo.CC_Groups.GroupName AS gName,
''X'' AS IsInGroup
FROM
dbo.CC_ContactGroups
INNER JOIN
dbo.CC_Groups ON dbo.CC_ContactGroups.idGroup = dbo.CC_Groups.idGroup
INNER JOIN
dbo.CC_Contacts ON dbo.CC_ContactGroups.idCC = dbo.CC_Contacts.idCC
WHERE
dbo.CC_Contacts.idCC = '''+#idCC+''') AS x
PIVOT (MAX(IsInGroup)
FOR gName IN(' + #ListOfGroups + ')) as p');
EXEC(#sql);
END
In your stored procedure [dbo].[sp_CC_ContactGroupList], replace
dbo.CC_Contacts.idCC = '''+#idCC+'''
with
dbo.CC_Contacts.idCC = '''+cast(#idCC as varchar)+'''
Now it will show you #idCC
Suppose I have the following structure to a set of tables in my SQL Server (2012) DB:
StartDate: Col1: Col2: .... Coln:
And, the way the DBA set up the database (have no control over that - I only have query access), all the tables with this structure that I'd want to query have, say, names beginning with MyTbl....
So, I would like to create a query that queries ALL these tables at once to get data for a specific StartDate and I've done it using the following SQL:
declare #t table(tablename varchar(50))
declare #sql varchar(max)
set #sql = ''
insert into #t
SELECT t.name AS table_name FROM sys.tables AS t
WHERE t.name LIKE 'MyTbl%'
select #sql = #sql + 'Select ''' + tablename + ''' as Table_Name, t.* From ' + tablename +
' t where StartDate = ''2015-01-01'' +
' union ' from #t
Select #sql = substring(#sql, 1, len(#sql) - 6)
exec(#sql)
In other words:
Find all tables in my DB with names beginning with MyTbl
Query each table for any data with StartDate = '2015-01-01`
Union all those queries together to get one big dataset result
The SQL works perfectly, but I'm getting quite stuck in creating a stored procedure from this query that can take in a parameter for StartDate and I don't know enough about stored procedures to do this correctly.
How could I convert this into a stored procedure that takes a date in for StartDate (to replace the ''2015-01-01'' in the query)?
Any help / guidance would be GREATLY appreciated!!!
THANKS!!!
I noticed you were not looping through each table .. here is something I had put together
CREATE PROCEDURE get_tabledata (#date DATE)
AS
BEGIN
DECLARE #t TABLE (
id INT IDENTITY(1, 1)
,tablename VARCHAR(50)
)
DECLARE #id INT
DECLARE #tablename VARCHAR(max)
DECLARE #sql VARCHAR(max)
SET #sql = ''
INSERT INTO #t
SELECT t.NAME AS table_name
FROM sys.tables AS t
WHERE t.NAME LIKE 'MyTbl%'
SET #id = ##ROWCOUNT
IF (#id > 0)
BEGIN
WHILE (#id > 0)
BEGIN
SET #tablename = (
SELECT tablename
FROM #t
WHERE id = #id
)
SELECT #sql = #sql + 'Select ' + #tablename + ''' as Table_Name, t.* From ' + #tablename + ' t where StartDate = ' + '' + convert(VARCHAR, #date) + ''
SET #sql = #sql + ' union'
Set #id = #id -1;
END
SELECT #sql = substring(#sql, 1, len(#sql) - 6)
END
EXEC (#sql)
END
While it can be a little dense if you're not used to the styling Microsoft uses on these pages, the best place to start would be the Create Procedure documentation on MSDN
https://msdn.microsoft.com/en-us/library/ms187926.aspx
That said, creating a stored procedure is pretty straight forward. Here's a really simple procedure that takes a #startDate parameter and then just returns it back. This is just to illustrate how and where you define your parameters
create procedure dbo.MyProcedure
-- put your input parameters here
#StartDate date
as
--put the body of your procedure (i.e. everything you've written in your OP) here
select #StartDate
go
YOu'll notice however that if you run this twice in a row, you get an error, because it tries to build the same procedure again. Another thing which can come in handy is adding some code before your procedure which will basically check to see if it already exists, and if it does, alter the procedure rather than just blindly re-create it.
This is a snippet from a template I use quite often which handles all of that logic for you. The simplest way to use this is press CTRL-SHIFT-M, which brings up a dialogue to replace all those tags with values you provide.
use [<Database Name, sysname,>]
go
if not exists (select 1
from sys.procedures with(nolock)
where name = '<Procedure Name, sysname,>'
and [schema_id] = schema_id('<Schema, sysname,dbo>')
and type = 'P'
)
exec ('create procedure [<Schema, sysname,dbo>].[<Procedure Name, sysname,>]
as
select ''Procedure not defined.'' as ErrorDescription
return')
--Executed as dynamic SQL since SQL Server Management Studio considures the straight SQL code a syntax error for some reason on the create procedure statement
GO
alter procedure [<Schema, sysname,dbo>].[<Procedure Name, sysname,>]
<Parm 1 Name, sysname,include [#]> <Parm 1 Datatype, sysname,><Parm 1 Default, sql_variant,include [=] if used>,
<Parm 2 Name, sysname,include [#]> <Parm 2 Datatype, sysname,><Parm 2 Default, sql_variant,include [=] if used>
as
/*******************************************************************************************************
********************************************************************************************************/
---------------------------------------------
-- declare variables
---------------------------------------------
---------------------------------------------
-- create temp tables
---------------------------------------------
---------------------------------------------
-- set session variables
---------------------------------------------
set nocount on
---------------------------------------------
-- body of stored procedure
---------------------------------------------
return
declare #TableName nvarchar(max)
set #TableName='addresses'
DECLARE #sql NVARCHAR(MAX)
set #sql= 'create table #tempadd ( '
SELECT #sql=#sql + STUFF( -- Remove first comma
(
SELECT ', ' + column_name+' '+ case when DATA_TYPE='varchar' then DATA_TYPE +'(500)' else DATA_TYPE end FROM -- create comma separated values
(
SELECT column_name,DATA_TYPE FROM information_schema.columns where table_name = #TableName --Your query here
) AS T FOR XML PATH('')
)
,1,1,'')
set #sql =#sql+' ) '
print #sql
--SET #sql='SELECT * into #tempadd FROM '+#TableName+ ' WHERE 1=2'
EXEC sp_executesql #sql
select * from #tempadd
This results in an error:
Msg 208, Level 16, State 0, Line 25
Invalid object name '#tempadd'.
Your temp table is limited to the scope of your dynamic query since it is defined within.
You could add your select * from #tempadd statement to the end of your #sql query. Alternatively I think you can define #tempadd before your dynamic query and it should be accessible, but I'm not certain on that.
thanks to this blog
The problem here is the scope of the session. When we execute dynamic sql via EXEC or sp_executesql a new scope is created for a child session. Any objects created in that session are dropped as soon as the session is closed.
One solution I have found for this problem is creating the table in the "parent" scope and then just using dynamic sql to modify the table. For this to work a table is created with a minimum set of colums. And then we use the ALTER TABLE statement with dynamic SQL. The Child session has access to the objects created in the parent session so the table can be modified with dynamic sql:
DECLARE #SQL NVARCHAR(4000)
CREATE TABLE #Temp ( id int null)
SELECT #SQL = 'ALTER #Temp ADD Col1 int null'
EXEC (#SQL)
SELECT * FROM #Temp
DROP TABLE #Temp
This table is visible and both columns will show up.