I have a database with multiple schemas. In every schema I got table called [Logs], so my database tables looks like:
[s1].[Logs]
[s2].[Logs]
[s3].[Logs]
...
[sN].[Logs]
Every day I would like to run stored procedure, which will do same operations on every above table. Is there a way to pass schema name into stored procedure? I am using SQL on Azure.
No, it is not - unless the SP Uses then dynamic SQL to execute some SQL String you constructed in the SP.
This happens via the sp_executesql stored procedure
http://technet.microsoft.com/en-us/library/ms188001.aspx
has more information.
Microsoft has a few undocumented procedures that perform "foreach" operations on tables (sp_msforeachtable) and databases (sp_msforeachdb). Both of these rely on another undocumented proc called sp_msforeachworker which you might be able to exploit to create a foreachschema type of routine. Theres an article (reg required) here that demonstrates this approach.
That said, its unlikely Azure supports anything of these, so you might have to fashion your own using a crude loop:
declare #schemas table (i int identity(1,1), name sysname);
insert into #schemas
select name from sys.schemas where name like 's[0-9]%';
declare #i int, #name sysname, #cmd nvarchar(max);
select #i = min(i) from #schemas;
while #i is not null
begin
select #name = name from #schemas where i = #i;
set #cmd = replace(N'select count(*) from [{0}].[Logs];', '{0}', #name);
print #cmd;
--exec(#cmd);
select #i = min(i) from #schemas where i > #i;
end
Related
I have a bunch of simple expressions, such as:
c=a+b
c=a*b
...
I would like to pass them as parameter to a stored procedure, which is going to perform an update using them.
CREATE TABLE t(
a int,
b int,
c int
);
INSERT INTO t VALUES (1,2,3),(4,5,6);
CREATE PROCEDURE sp #left_member varchar(50), #right_member
AS
BEGIN
UPDATE t
SET #left_member = #right_member
END
EXEC sp 'c', 'a+b'
EXEC sp 'c', 'a*b'
Is there a way of doing something like that ? I would like to possibly avoid dynamic SQL. In my target design, the expressions will be stored in their own table (editable online).
I generally don't recommend doing this, but dynamic SQL is pretty much the solution:
CREATE PROCEDURE usp_exec_dangerous_update (
#left_member nvarchar(50),
#right_member nvarchar(50)
)
AS
BEGIN
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'
UPDATE t
SET [left_member] = [right_member]
';
SET #sql = REPLACE(REPLACE(#sql, '[left_member]', #left_member), '[right_member]', #right_member);
EXEC sp_executesql #sql;
END;
Although such code can be useful in a thoughtful, well-designed system, in general it is not needed:
It exposes the system to SQL injection attacks. Running "generic" code is just dangerous.
It does not handle errors, which are easy to occur with this method.
I'm currently trying to write a default procedure template for reporting from a T-SQL Datawarehouse.
The idea is to wrap each query in a procedure, so that permissions and logging can be managed easily.
Since this will be done by the DBAs, I would like to have this solution work by only pasting some standard code before and after the main query. I'd prefer if the DBA didn't have to modify any part of the logging-code.
I've solved this for most parts, however, I need to log which parameters the user has submitted to the procedure.
The obvious solution would be hardcode the parameters into the logging. However, the procedures can have a varying amount of parameters, and I'd therefore like a catch-all solution.
My understanding is that there is no easy way iterating through all parameters.
I can however access the parameter-names from the table sys.parameters.
The closest to a solution I've come, is this minimal example:
CREATE TABLE #loggingTable (
[ProcedureID] INT
, [paramName] NVARCHAR(128)
, [paramValue] NVARCHAR(128)
)
;
go
CREATE PROCEDURE dbo.[ThisIsMyTestProc] (
#param1 TINYINT = NULL
, #Param2 NVARCHAR(64) = null
)
AS
BEGIN
-- Do some logging here
DECLARE #query NVARCHAR(128)
DECLARE #paramName NVARCHAR(128)
DECLARE #paramValue nvarchar(128)
DECLARE db_cursor CURSOR FOR
SELECT [name] FROM [sys].[parameters] WHERE object_id = ##PROCID
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #paramName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #query = 'SELECT #paramValue = cast(' + #paramName + ' as nvarchar(128))';
SELECT #query;
-- Following line doesn't work due to scope out of bounds, and is prone to SQL-Injections.
--EXEC SP_EXECUTESQL #query; -- Uncomment for error
insert into #loggingTable(ProcedureID, paramName, paramValue)
values(##PROCID, #paramName, #paramValue)
FETCH NEXT FROM db_cursor INTO #paramName
END
CLOSE db_cursor
DEALLOCATE db_cursor
-- Run the main query here (Dummy statement)
SELECT #param1 AS [column1], #Param2 AS [column2]
-- Do more logging after statement has run
END
GO
-- test
EXEC dbo.[ThisIsMyTestProc] 1, 'val 2';
select * from #loggingTable;
-- Cleanup
DROP PROCEDURE dbo.[ThisIsMyTestProc];
DROP table #loggingTable;
However, this does have to major drawbacks.
It doesn't work due to variable scopes
It is prone to SQL-Injections, which is unacceptable
Is there any way to solve this issue?
The values of the parameters are not availiable in a generic approach. You can either create some code generator, which will use sys.parameters to create a chunk of code you'd have to copy into each of your SPs, or you might read this or this about tracing and XEvents. The SQL-Server-Profiler works this way to show you statements together with the parameter values...
If you don't want to get into tracing or XEvents you might try something along this:
--Create a dummy proc
CREATE PROCEDURE dbo.[ThisIsMyTestProc] (
#param1 TINYINT = NULL
, #Param2 NVARCHAR(64) = null
)
AS
BEGIN
SELECT ##PROCID;
END
GO
--call it to see the value of ##PROCID
EXEC dbo.ThisIsMyTestProc; --See the proc-id
GO
--Now this is the magic part. It will create a command, which you can copy and paste into your SP:
SELECT CONCAT('INSERT INTO YourLoggingTable(LogType,ObjectName,ObjectId,Parameters) SELECT ''ProcedureCall'', ''',o.[name],''',',o.object_id,','
,'(SELECT'
,STUFF((
SELECT CONCAT(',''',p.[name],''' AS [parameter/#name],',p.[name],' AS [parameter/#value],''''')
FROM sys.parameters p
WHERE p.object_id=o.object_id
FOR XML PATH('')
),1,1,'')
,' FOR XML PATH(''''),ROOT(''parameters''),TYPE)'
)
FROM [sys].[objects] o
WHERE o.object_id = 525244926; --<-- Use the proc-id here
--Now we can copy the string into our procedure
--I out-commented the INSERT part, the SELECT is enough to show the effect
ALTER PROCEDURE dbo.[ThisIsMyTestProc] (
#param1 TINYINT = NULL
, #Param2 NVARCHAR(64) = null
)
AS
BEGIN
--The generated code comes in one single line
--INSERT INTO YourLoggingTable(LogType,ObjectName,ObjectId,Parameters)
SELECT 'ProcedureCall'
,'ThisIsMyTestProc'
,525244926
,(SELECT'#param1' AS [parameter/#name],#param1 AS [parameter/#value],''
,'#Param2' AS [parameter/#name],#Param2 AS [parameter/#value],''
FOR XML PATH(''),ROOT('parameters'),TYPE)
END
GO
Hint: We need the empty element (,'') at the end of each line to allow multiple elements with the same name.
--Now we can call the SP with some param values
EXEC dbo.ThisIsMyTestProc 1,'hello';
GO
As a result, your Log-Table will get an entry like this
ProcedureCall ThisIsMyTestProc 525244926 <parameters>
<parameter name="#param1" value="1" />
<parameter name="#Param2" value="hello" />
</parameters>
Just add typical logging data like UserID, DateTime, whatever you need...
Scope is the killer issue for this approach. I don't think there's a way to reference the values of parameters by anything but their variable names. If there was a way to retrieve variable values from a collection or by declared ordinal position, it could work on the fly.
I understand wanting to keep the overhead for the DBAs low and eliminating opportunities for error, but I think the best solution is to generate the required code and supply it to the DBAs or give them a tool that generates the needed blocks of code. That's about as lightweight as we can make it for the DBA, but I think it has the added benefit of eliminating processing load in the procedure by turning it into a static statement with some conditional checking for validity and concatenation work. Cursors and looping things should be avoided as much as possible.
Write a SQL script that generates your pre- and post- query blocks. Generate them in mass with a comment at the top of each set of blocks with the stored procedure name and hand it to the DBAs to copy/paste into the respective procs. Alternatively, give them the script and let them run it as needed to generate the pre- and post- blocks themselves.
I would include some checks in the generated script to help make sure it works during execution. This will detect mismatches in the generated code due to subsequent modifications to the procedure itself. We could go the extra mile and include the names of the parameters when the code is generated and verify them against sys.parameters to make sure the parameter names hard-coded into the generated code haven't changed since code generation.
-- Log execution details pre-execution
IF object_name(##PROCID) = 'ThisIsMyTestProc' AND (SELECT COUNT(*) FROM [sys].[parameters] WHERE object_id = ##PROCID) = 2
BEGIN
EXEC LogProcPreExecution #Params = CONCAT('parm1: ', #param1, ' parm2: ', #Param2), #ProcName = 'ThisIsMyTestProc', #ExecutionTime = getdate() #ExecutionUser = system_user
END
ELSE
BEGIN
--Do error logging for proc name and parameter mismatch
END
--Log procedure would look like this
CREATE PROCEDURE
LogProcPreExecution
#Parameters varchar(max),
#ProcName nvarchar(128),
#ExecutionTime datetime,
#ExecutionUser nvarchar(50)
AS
BEGIN
--Do the logging
END
We are writing a stored procedure responsible for getting a stored procedure name and returning a result containing the stored procedure columns and their data types.
However, we bumped into a problem executing a dynamic query to return the results of stored procedure, but we can't store it in a temp table!
You can see our query below:
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100),#DataType VARCHAR(20),
#Query NVARCHAR(MAX)='EXEC '+'spGetOraganizationsList '
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
DROP TABLE #Tmp
EXEC sp_executesql #Query
The thing is I can't store the results of it in a temp table,
and OPENROWSET does not accept variables.
I think it comes from sql concept that it doesn't trust in result of stored procedures and because of that we cannot select on it or store it in a table by 'making in query table' method.
Unless you create a table and define it's columns and sql trust to you and you insert result of it into this table for example take below situation
Create table test (name varchar(10),family varchar(20))
Insert into test
Exec sp-testResult
Now if you define wrong column for your table you will receive query runtime error .actually sql doesn't predict result of sp and leaves it to you to define result of your stored procedure.
You can certainly INSERT the results of a stored procedure into a TEMP table:
CREATE PROCEDURE PurgeMe
AS
SELECT convert(int, 1) AS DaData
UNION
SELECT convert(int, 2)
GO
CREATE TABLE #Doodles (AnInteger int)
INSERT #Doodles EXECUTE PurgeMe
SELECT * FROM #Doodles
Questions arise about the SCOPE of TEMP tables, however. You might find that in your calling routine you will not be able to see a TEMP table created within your routine.
The solution to the SCOPE problem is to do the following:
Create a minimal TEMP table (say, with one column)
Use ALTER TABLE on the TEMP table within your routine to make its schema match
your needs (this can be tricky, but it can be done)
Put data into the TEMP table
return from your routine - the calling routine will now be able to access the temp
table
If this is of interest I can make a longer post with a stored procedure to do the above. It was written to facilitate dynamic SQL
Write select query as you want in the stored procedure. You will get the result without creating temp table.
Use global temp table and dynamic OPENROWSET
DROP TABLE ##Tmp;
GO
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100), #DataType VARCHAR(20),
-- Mind to specify database and schema of the SP
#Query NVARCHAR(MAX)=' EXEC [mydb].[dbo].spGetOraganizationsList ';
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName;
-- Build SP exec
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
SET #Query = left(#Query, len(#Query) - 1);
-- Build ad hoc distributed query which creates ##Tmp from SP exec.
SET #Query = 'SELECT * INTO ##Tmp FROM OPENROWSET(''SQLNCLI'', ''Server=localhost;Trusted_Connection=yes;'',''' + #Query + ''')';
EXEC (#Query);
-- Created by dynamic sql `##Tmp` is availabe in the current context.
SELECT *
FROM ##Tmp;
Don't forget to enable ad hoc distributed queries first.
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
EDIT
My answer solves only one problem, storing the result of a dynamic proc call in a temp table. And there are more problems.
First, #p=null just will not compile if the type of #p is user-defined table type. You need kind of declare #t myType;
exec mySp ... ,#p=#t ....
Next is the 'cannot retrieve matadata for sp because contain dynamic query' error you commented on. Looks like you need an application, SqlClr or standalone, which would be capable to read and parse Datasets returned by procs.
Finally, if an SP contains conditional sql which can return a result set of different schema depending on parameter values, the result of all those efforts is still questionable.
In C#, you can use an SqlDataReader or a DataTable to get the results from a stored procedure without knowing the schema beforehand. If you then want to write that data to a temporary table, I think you can do that from C# (though I've never tried to do it).
There are bunch of databases to the SQL server I am connected.
How should I query the sysobjects in order to spot in what database a stored procedure with name 'myStoredProcedure' is located ?
The query should return the database name.
Thanks
I know you are not asking for this, but I'd really download RedGate's Sql Search add-in for SSMS and use that. It allows you to find any object (proc, table, view, column, etc) on any database easily.
And it's free!
I'd give this a try:
CREATE TABLE ##DatabaseList
(
DatabaseName varchar(50)
)
EXECUTE SP_MSForEachDB 'USE [?]; INSERT INTO ##DatabaseList SELECT DB_NAME() FROM [sys].[objects] WHERE name = "MyStoredProcedure" AND type_desc = "SQL_STORED_PROCEDURE"'
SELECT * FROM ##DatabaseList
DROP TABLE ##DatabaseList
That's using the undocumented/ unsupported system stored procedure SP_MSForEachDb and writing any hits to a global temp table, then outputting the contents to the Results window before dropping the table. If you just need to know which database (or databases - there may of course be more than one) has an appropriately named SP, this should do it. If you want to use the output elsewhere as a parameter, it may take a little more work.
By the way, I'm only learning this stuff myself over the last few months so if anyone can critique the above and suggest a better way to go at it I'm happy to receive feedback. Equally, I can answer any further questions posted here to the best of my ability.
Cheers
So out of curiosity I decided to try write this myself, especially since ADG mentioned his solution was using an unsupported, undocumented procedure. This could also be expanded to take a 2nd parameter so where it checks the type = P (stored Proc) you could probably change it to look for other things like views / tables etc.
My solution is a bit long but here goes:
CREATE PROCEDURE spFindProceduresInDatabases
(
#ProcedureName NVARCHAR(99)
)
AS
BEGIN
-- Get all the database names and put them into a table
DECLARE #Db TABLE (DatabaseName Varchar(99))
INSERT INTO #Db SELECT name FROM Sys.databases
-- Declare a table to hold our results
DECLARE #results TABLE (DatabaseName VARCHAR(99))
-- Make a Loop
-- Declare a variable to be incremented
DECLARE #count INT
SET #count = 0
-- Declare the end condition
DECLARE #endCount INT
SELECT #endCount = COUNT(*) FROM #Db
-- Loop through the databases
WHILE (#count < #endCount )
BEGIN
-- Get the database we are going to look into
DECLARE #dbWeAreChecking VARCHAR(99)
SELECT TOP 1 #dbWeAreChecking = DatabaseName FROM #Db
DELETE FROM #Db WHERE DatabaseName = #dbWeAreChecking
-- Create and execute our query
DECLARE #Query NVARCHAR(3000)
SET #Query = N'SELECT #outParam = COUNT(*) FROM '+#dbWeAreChecking+'.sys.sysobjects WHERE type = ''P'' and name = #ProcedureName'
Declare #outParam INT
print (#Query)
DECLARE #ParmDefinition NVARCHAR(500)
DECLARE #IntVariable INT
SET #ParmDefinition = N'#ProcedureName VARCHAR(99),#outParam INT OUTPUT'
SET #IntVariable = 35
EXECUTE sp_executesql
#Query ,
#ParmDefinition,
#ProcedureName,
#outParam = #outParam OUTPUT
-- If we have a result insert it into the results table
If (#outParam > 0)
BEGIN
INSERT INTO #results(DatabaseName) VALUES(#dbWeAreChecking)
END
-- Increment the counter
SET #count = (#count + 1)
END
-- SELECT ALL OF THE THINGS!!!
SELECT * FROM #results
END
I have a stored procedure that alters user data in a certain way. I pass it user_id and it does it's thing. I want to run a query on a table and then for each user_id I find run the stored procedure once on that user_id
How would I write query for this?
use a cursor
ADDENDUM: [MS SQL cursor example]
declare #field1 int
declare #field2 int
declare cur CURSOR LOCAL for
select field1, field2 from sometable where someotherfield is null
open cur
fetch next from cur into #field1, #field2
while ##FETCH_STATUS = 0 BEGIN
--execute your sproc on each row
exec uspYourSproc #field1, #field2
fetch next from cur into #field1, #field2
END
close cur
deallocate cur
in MS SQL, here's an example article
note that cursors are slower than set-based operations, but faster than manual while-loops; more details in this SO question
ADDENDUM 2: if you will be processing more than just a few records, pull them into a temp table first and run the cursor over the temp table; this will prevent SQL from escalating into table-locks and speed up operation
ADDENDUM 3: and of course, if you can inline whatever your stored procedure is doing to each user ID and run the whole thing as a single SQL update statement, that would be optimal
try to change your method if you need to loop!
within the parent stored procedure, create a #temp table that contains the data that you need to process. Call the child stored procedure, the #temp table will be visible and you can process it, hopefully working with the entire set of data and without a cursor or loop.
this really depends on what this child stored procedure is doing. If you are UPDATE-ing, you can "update from" joining in the #temp table and do all the work in one statement without a loop. The same can be done for INSERT and DELETEs. If you need to do multiple updates with IFs you can convert those to multiple UPDATE FROM with the #temp table and use CASE statements or WHERE conditions.
When working in a database try to lose the mindset of looping, it is a real performance drain, will cause locking/blocking and slow down the processing. If you loop everywhere, your system will not scale very well, and will be very hard to speed up when users start complaining about slow refreshes.
Post the content of this procedure you want call in a loop, and I'll bet 9 out of 10 times, you could write it to work on a set of rows.
You can do it with a dynamic query.
declare #cadena varchar(max) = ''
select #cadena = #cadena + 'exec spAPI ' + ltrim(id) + ';'
from sysobjects;
exec(#cadena);
Something like this substitutions will be needed for your tables and field names.
Declare #TableUsers Table (User_ID, MyRowCount Int Identity(1,1)
Declare #i Int, #MaxI Int, #UserID nVarchar(50)
Insert into #TableUser
Select User_ID
From Users
Where (My Criteria)
Select #MaxI = ##RowCount, #i = 1
While #i <= #MaxI
Begin
Select #UserID = UserID from #TableUsers Where MyRowCount = #i
Exec prMyStoredProc #UserID
Select
#i = #i + 1, #UserID = null
End
Use a table variable or a temporary table.
As has been mentioned before, a cursor is a last resort. Mostly because it uses lots of resources, issues locks and might be a sign you're just not understanding how to use SQL properly.
Side note: I once came across a solution that used cursors to update
rows in a table. After some scrutiny, it turned out the whole thing
could be replaced with a single UPDATE command. However, in this case,
where a stored procedure should be executed, a single SQL-command
won't work.
Create a table variable like this (if you're working with lots of data or are short on memory, use a temporary table instead):
DECLARE #menus AS TABLE (
id INT IDENTITY(1,1),
parent NVARCHAR(128),
child NVARCHAR(128));
The id is important.
Replace parent and child with some good data, e.g. relevant identifiers or the whole set of data to be operated on.
Insert data in the table, e.g.:
INSERT INTO #menus (parent, child)
VALUES ('Some name', 'Child name');
...
INSERT INTO #menus (parent,child)
VALUES ('Some other name', 'Some other child name');
Declare some variables:
DECLARE #id INT = 1;
DECLARE #parentName NVARCHAR(128);
DECLARE #childName NVARCHAR(128);
And finally, create a while loop over the data in the table:
WHILE #id IS NOT NULL
BEGIN
SELECT #parentName = parent,
#childName = child
FROM #menus WHERE id = #id;
EXEC myProcedure #parent=#parentName, #child=#childName;
SELECT #id = MIN(id) FROM #menus WHERE id > #id;
END
The first select fetches data from the temporary table. The second select updates the #id. MIN returns null if no rows were selected.
An alternative approach is to loop while the table has rows, SELECT TOP 1 and remove the selected row from the temp table:
WHILE EXISTS(SELECT 1 FROM #menuIDs)
BEGIN
SELECT TOP 1 #menuID = menuID FROM #menuIDs;
EXEC myProcedure #menuID=#menuID;
DELETE FROM #menuIDs WHERE menuID = #menuID;
END;
Can this not be done with a user-defined function to replicate whatever your stored procedure is doing?
SELECT udfMyFunction(user_id), someOtherField, etc FROM MyTable WHERE WhateverCondition
where udfMyFunction is a function you make that takes in the user ID and does whatever you need to do with it.
See http://www.sqlteam.com/article/user-defined-functions for a bit more background
I agree that cursors really ought to be avoided where possible. And it usually is possible!
(of course, my answer presupposes that you're only interested in getting the output from the SP and that you're not changing the actual data. I find "alters user data in a certain way" a little ambiguous from the original question, so thought I'd offer this as a possible solution. Utterly depends on what you're doing!)
I like the dynamic query way of Dave Rincon as it does not use cursors and is small and easy. Thank you Dave for sharing.
But for my needs on Azure SQL and with a "distinct" in the query, i had to modify the code like this:
Declare #SQL nvarchar(max);
-- Set SQL Variable
-- Prepare exec command for each distinctive tenantid found in Machines
SELECT #SQL = (Select distinct 'exec dbo.sp_S2_Laser_to_cache ' +
convert(varchar(8),tenantid) + ';'
from Dim_Machine
where iscurrent = 1
FOR XML PATH(''))
--for debugging print the sql
print #SQL;
--execute the generated sql script
exec sp_executesql #SQL;
I hope this helps someone...