I have a stored procedure that has an XML variable as input parameter.
Each node of the XML variable matches with a SQL column of a certain database table.
The stored procedure simply iterates over the XML nodes and inserts them into the table.
I have been using the "OPENXML" functionality in my stored procedure to do this, but I am having performance issues (query takes up to 40 sec) => SQL process XML performance: Insert into columns in a table.
I want to boost the performance but I'm not sure where to begin. The standard XML function doesn't seem to improve, so I am looking into an alternative.
I have tried to do this functionality using SSIS packages. So from my stored procedure, I call an SSIS package that has a string variable as input parameter.
I pass the XML I receive from the stored procedure into the package as a string.
But I have come accross an issue with the xp_cmdshell functionality.
It only allows a command of max 8000 character.
This is how the code looks like:
--Execution SSIS Package
DECLARE #Command varchar(8000)
, #PackageLocation varchar(1000)
, #PackageName varchar(1000)
SET #PackageLocation = 'C:\SSIS\Package.dtsx'
SET #Command = 'dtexec /f "' + #PackageLocation + '" /set \package.Variables[Xmldata].Value;"' + #datastring + '"'
EXEC #ExitCode = xp_cmdshell #Command
The problem is that the #datastring variable can be longer than 8000 characters, so then the command fails.
Any idea how I can solve this?
Or perhaps other alternatives to accomplish this functionality?
Thanks.
Best regards,
I will store the XML I receive in the stored procedure in a temporarily "buffer" table, so I can retrieve it in the SSIS package and process it further.
Related
I want to insert the results of a stored procedure into a temp table using OPENROWSET. However, the issue I run into is I'm not able to pass parameters to my stored procedure.
This is my stored procedure:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[N_spRetrieveStatement]
#PeopleCodeId nvarchar(10),
#StatementNumber int
AS
SET NOCOUNT ON
DECLARE #PersonId int
SELECT #PersonId = [dbo].[fnGetPersonId](#PeopleCodeId)
SELECT *
INTO #tempSpRetrieveStatement
FROM OPENROWSET('SQLNCLI', 'Server=PCPRODDB01;Trusted_Connection=yes;',
'EXEC Campus.dbo.spRetrieveStatement #StatementNumber, #PersonId');
--2577, 15084
SELECT *
FROM #tempSpRetrieveStatement;
OpenRowSet will not allow you to execute Procedure with input parameters. You have to use INSERT/EXEC.
INTO #tempSpRetrieveStatement(Col1, Col2,...)
EXEC PCPRODDB01.Campus.dbo.spRetrieveStatement #StatementNumber, #PersonId
Create and test a LinkedServer for PCPRODDB01 before running the above command.
The root of your problem is that you don't actually have parameters inside your statement that you're transmitting to the remote server you're connecting to, given the code sample you provided. Even if it was the very same machine you were connecting to, they'd be in different processes, and the other process doesn't have access to your session variables.
LinkedServer was mentioned as an option, and my understanding is that's the preferred option. However in practice that's not always available due to local quirks in tech or organizational constraints. It happens.
But there is a way to do this.
It's hiding in plain sight.
You need to pass literals into the string that will be executed on the other server, right?
So, you start by building the string that will do that.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[N_spRetrieveStatement]
#PeopleCodeId nvarchar(10),
#StatementNumber int
AS
SET NOCOUNT ON
DECLARE
#PersonId INT,
#TempSQL VARCHAR(4000) = '';
SELECT #PersonId = [dbo].[fnGetPersonId](#PeopleCodeId);
SET #TempSQL =
'EXEC Campus.dbo.spRetrieveStatement(''''' +
FORMAT(#StatementNumber,'D') +''''', ''''' +
FORMAT(#PersonId,'D') + ''''')';
--2577, 15084
Note the seemingly excessive number of quotes. That's not a mistake -- that's foreshadowing. Because, yes, OPENROWSET hates taking variables as parameters. It, too, only wants literals. So, how do we give OPENROWSET what it needs?
We create a string that is the entire statement, no variables of any kind. And we execute that.
SET #TempSQL =
'SELECT * INTO #tempSpRetrieveStatement ' +
'FROM OPENROWSET(''SQLNCLI'', ''Server=PCPRODDB01;Trusted_Connection=yes;'', ' + #TempSQL +
'EXEC Campus.dbo.spRetrieveStatement #StatementNumber, #PersonId';
EXEC (#TempSQL);
SELECT *
FROM #tempSpRetrieveStatement;
And that's it! Pretty simple except for counting your escaped quotes, right?
Now... This is almost beyond the scope of the question you asked, but it is a 'gotcha' I've experienced in executing stored procedures in another machine via OPENROWSET. You're obviously used to using temp tables. This will fail if the stored procedure you're calling is creating temp tables or doing a few other things that -- in a nutshell -- inspire the terror of ambiguity into your SQL server. It doesn't like ambiguity. If that's the case, you'll see a message like this:
"Msg 11514, Level 16, State 1, Procedure sp_describe_first_result_set, Line 1
The metadata could not be determined because statement '…your remote EXEC statement here…' in procedure '…name of your local stored procedure here…' contains dynamic SQL. Consider using the WITH RESULT SETS clause to explicitly describe the result set."
So, what's up with that?
You don't just get data back with OPENROWSET. The local and remote servers have a short conversation about what exactly the local server is going to expect from the remote server (so it can optimize receiving and processing it as it comes in -- something that's extremely important for large rowsets). Starting with SQL Server 2012, sp_describe_first_result_set is the newer procedure for this, and normally it executes quickly without you noticing it. It's just that it's powers of divination aren't unlimited. Namely, it doesn't know how to get the type and name information regarding temp tables (and probably a few other things it can't do -- PIVOT in a select statement is probably right out).
I specifically wanted to be sure to point this out because of your reply regarding your hesitation about using LinkedServer. In fact, the very same reasons you're hesitant are likely to render that error message's suggestion completely useless -- you can't even predict what columns you're getting and in what order until you've got them.
I think what you're doing will work if, say, you're just branching upstream based on conditional statements and are executing one of several potential SELECT statements. I think it will work if you're just not confident that you can depend on the upstream component being fixed and are trying to ensure that even if it varies, this procedure doesn't have to because it's very generic.
But on the other hand you're facing a situation in which you literally cannot guarantee that SQL Server can predict the columns, you're likely going to have to force some changes in the stored procedure you're calling to insist that it's stable. You might, for instance work out how to ensure all possible fields are always present by using CASE expressions rather than any PIVOT. You might create a session table that's dedicated to housing what you need to SELECT just long enough to do that then DELETE the contents back out of there. You might change the way in which you transmit your data such that it's basically gone through the equivalent of UNPIVOT. And after all that extra work, maybe it'll be just a matter of preference if you use LinkedServer or OPENROWSET to port the data across.
So that's the answer to the literal question you asked, and one of the limits on what you can do with the answer.
I have two identical stored procedures with dynamic query.
Let's say
Stored procedure A
Stored procedure B
Both are in different databases. But they have the same code (Not Complete Identical.4-5 lines differ).
Is there a way to update any modification done in stored procedure A to stored procedure B automatically?
Otherwise I always need to copy and paste changes manually. It is an error-prone activity. Can anyone help me on this ?
You could do something like that:
In database A:
design your stored procedure in a way, that you have a parameter for
the database in which you want to do the work
In database B:
Create a synonym for the procedure in database A
Example:
--create procedure in database A
create procedure dbo.StoredProc
(
#dbname --or dbid if you want
)
as
begin
--create your sql command here, using dynamic sql maybe
declare #sqlcmd NVARCHAR(MAX)=N''
set #sqlcmd = 'SELECT * FROM ' + #dbname + '.dbo.AnyTable'
exec sp_executesql #sqlcmd
end
--create a synonym for this procedure in database b:
create synonym dbo.StoredProc FOR databaseA.dbo.StoredProc
--then you can call your procedure in Database A and B like this:
declare #dbname NVARCHAR(100) = DB_NAME()
exec dbo.StoredProc #dbname
so you have to maintain your code only once, and in database b you only have kind of a "link" to this procedure.
hope this helps :)
This is basically what SSDT was designed for, the idea is that you write your T-SQL and schema as CREATE statements, you build a "dacpac" and then you use sqlpackage.exe to deploy the dacpac to whatever database you want.
Doing it this way you have an overhead of the SSDT project but it fixes exactly your main problem with the existing method "It is an error-prone activity."
My blog post shows how to get an existing database into SSDT (in this case adventureworks but replace adventureworks with your database):
https://the.agilesql.club/Blog/Ed-Elliott/AdventureWorksCI-Step2-MDF-To-Dot-Sql
Ed
I have an Application Database (A-DB) which imports from a set of Databases (B-set-DBs, (around 50 DBs)).
The user selects 4 DBs of "B-set-DB`s" from which he wannts to import into the A-DB for his application.
I wrote an stored procedure for the import (about 4000 Lines of SQL Code) with 4 hard coded DB`s of B-set-DBs and it works perfect.
So my question is how to make the imported Database names variable?
I can not use dynamic SQL for the whole 4000 lines, because im callin functions and other stored procedures, there are a lot of parameters and strings and definitions of temporary tables and supqueries of subqueries of sub......and a lot of other weird things.
(I know it all sounds ugly but i just have to get it to run, i did not design the whole thing.)
Maybe it is possible to define a synonyme at the first line of the import stored procedure:
ALTER PROCEDURE [dbo].[test_storedProcedure]
(#DBName_1 nvarchar(512), #table_name_1 nvarchar(512))
AS
BEGIN
DECLARE #cmd nvarchar(max);
set #cmd = N'create synonym DB_SYN' +
N' for ' +
#DBName_1 + --[A70_V70_V280]
#table_name_1 --.[dbo].[UNITCHANGEMAP]
exec sp_executesql #cmd
...
SELECT * INTO A-DB
FROM DB_SYN
...
END
And afterwards work only with the Synonyms.
Or does anyone knows a way which will do the same?
BTW I am using SQL Server 2008
So i see all the 50 Databases in the Object Explorer of SQL Server.
I have a stored procedure with an nvarchar parameter. I expect callers to supply the text for a sql command when using this SP.
How do I execute the supplied sql command from within the SP?
Is this even possible?-
I thought it was possible using EXEC but the following:
EXEC #script
errors indicating it can't find a stored procedure by the given name. Since it's a script this is obviously accurate, but leads me to think it's not working as expected.
Use:
BEGIN
EXEC sp_executesql #nvarchar_parameter
END
...assuming the parameter is an entire SQL query. If not:
DECLARE #SQL NVARCHAR(4000)
SET #SQL = 'SELECT ...' + #nvarchar_parameter
BEGIN
EXEC sp_executesql #SQL
END
Be aware of SQL Injection attacks, and I highly recommend reading The curse and blessing of Dynamic SQL.
you can just exec #sqlStatement from within your sp. Though, its not the best thing to do because it opens you up to sql injection. You can see an example here
You use EXECUTE passing it the command as a string. Note this could open your system up to serious vulnerabilities given that it is difficult to verify the non-maliciousness of the SQL statements you are blindly executing.
How do I execute the supplied sql command from within the SP?
Very carefully. That code could do anything, including add or delete records, or even whole tables or databases.
To be safe about this, you need to create a separate user account that only has dbreader permissions on just a small set of allowed tables/views and use the EXECUTE AS command to limit the context to that user.
I have a sql server stored procedure that I use to backup data from our database before doing an upgrade, and I'd really like it to be able to run the stored procedure on multiple databases by passing in the database name as a parameter. Is there an easy way to do this? The best I can figure is to dynamically build the sql in the stored procedure, but that feels like its the wrong way to do it.
build a procedure to back up the current database, whatever it is. Install this procedure on all databases that you want to backup.
Write another procedure that will launch the backups. This will depend on things that you have not mentioned, like if you have a table containing the names of each database to backup or something like that. Basically all you need to do is loop over the database names and build a string like:
SET #ProcessQueryString=
'EXEC '+DatabaseServer+'.'+DatabaseName+'.dbo.'+'BackupProcedureName param1, param2'
and then just:
EXEC (#ProcessQueryString)
to run it remotely.
There isn't any other way to do this. Dynamic SQL is the only way; if you've got strict controls over DB names and who's running it, then you're okay just truncating everything together, but if there's any doubt use QUOTENAME to escape the parameter safely:
CREATE PROCEDURE doStuff
#dbName NVARCHAR(50)
AS
DECLARE #sql NVARCHAR(1000)
SET #sql = 'SELECT stuff FROM ' + QUOTENAME(#dbName) + '..TableName WHERE stuff = otherstuff'
EXEC sp_ExecuteSQL (#sql)
Obviously, if there's anything more being passed through then you'll want to double-check any other input, and potentially use parameterised dynamic SQL, for example:
CREATE PROCEDURE doStuff
#dbName NVARCHAR(50)
#someValue NVARCHAR(10)
AS
DECLARE #sql NVARCHAR(1000)
SET #sql = 'SELECT stuff FROM ' + QUOTENAME(#dbName) + '..TableName WHERE stuff = #pOtherStuff'
EXEC sp_ExecuteSQL (#sql, '#pOtherStuff NVARCHAR(10)', #someValue)
This then makes sure that parameters for the dynamic SQL are passed through safely and the chances for injection attacks are reduced. It also improves the chances that the execution plan associated with the query will get reused.
personally, i just use a batch file and shell to sqlcmd for things like this. otherwise, building the sql in a stored proc (like you said) would work just fine. not sure why it would be "wrong" to do that.
best regards,
don
MSSQL has an OPENQUERY(dbname,statement) function where if the the server is linked, you specify it as the first parameter and it fires the statement against that server.
you could generate this openquery statement in a dynamic proc. and either it could fire the backup proc on each server, or you could execute the statement directly.
Do you use SSIS? If so you could try creating a couple ssis packages and try scheduling them,or executing them remotely.