Using an SSIS package, is it possible to load sql code that contains variables in it, replace those variables and run the code without saving it(similarly to DTS 2000 packages)?
For example, I want to run Code.sql, which contains the following:
SELECT *
INTO XXDBXX.dbo.Table2
FROM XXREADDBXX.dbo.Table1
I was planning on creating a script to replace the "XX" strings with variables I set up using SSIS.
So, here are the steps I want to do:
Run script to replace sql file variables.
Run SQL code
SQL code finishes running, and code still retains its variables.
Please let me know if anything needs clarification.
create a variable in SSIS #SQL (User::SQL)
Script task to load template sql from file & replace variables with required values. Use C# String.Replace();
Execute task to use #SQL (SQLSourceType = Variable)
Create a variable in SSIS
https://www.red-gate.com/simple-talk/sql/ssis/ssis-basics-introducing-variables/
You can also run SQL code and then replace parameters such as:
EXEC PROC XXDBX ?,?
The question marks being your parameter or variable created above.
Related
I'm reaching out to the experts as I have hit a wall with a recent project. I have created an SSIS package (2008R2) that uses a script task to build a SQL statement, where a variable(#month1) is being used within the SQL statement, to specify a month look back in a membership table. I want to also use the #month1 variable as a "counter" for the loop container to specify how many times to execute the query. The SQL query is attached to a data flow task to append these records into a table on a SQL server database. The script task and data flow task work outside of the for loop container with the initial value given for the #month1 variable but I cannot figure out how to make the for loop container update the #month1 "counter" variable so that the for each loop can use it as a "counter" and the SQL statement can use it as a condition with in the created SQL statement. Any one have any ideas or examples on how to do this?
** Update **
The For Loop container is the issue. The script task and data flow task work outside of the For Loop container. It will use the initial variable setting for #month1 and create the dynamic sql script, execute script and transfer data from source database server to the destination source server. The issue is when I place these steps within the For Loop container, the container executes and turns green but does not invoke the steps within it. This is why I'm thinking the container is not reading the variable #month1, even though the variable is set at the package level. Any thoughts?
First of all, try to set the data flow Delay Validation property to True. If it still not working, instead of passing the variable as parameter in the OLEDB Source use expressions:
Create a variable of type string.
Change its EvaluateAsExpression property to True
Set the expression similar to:
"select * from table
Where column =" + (dt_str, 50)#[User::month1]
In the OLEDB Source select the Access mode as SQL Command from variable and select this variable.
Be aware that the month1 variable is not created twice wuth different scopes, click on the data flow task and check the variable panel if it shows additional variables.
I appreciate everyone's responses but it seems I tricked myself on this one. In looking for the most complicated issue I overlooked the most simple and obvious one. The reason my For Loop was not executing the steps inside of the container was because I had the initial value for #month1 set to 3 (intentionally) and wanted to loop until it was resolved to -49. In the EvalExpression setting, it will evaluate until the statement is FALSE...so the evaluation I had in there of #month1 <= -49 was already false. It needed to be #month1 > -49 so as soon as it fell to -49 the statement would be false. I do this to myself more than I should admit, can't see the forest for the trees!
I have simple SQL Query and need to get the input from parameter. I am getting the when i click parameter button. Please let me know how to get the parameter if i have declare statement in my scripts.
Note :OLEDB connection manager used for connection
You can't use parameters in an OLEDB Source if your command contains DECLARE, or anything other than a single SELECT statement.
The way to do what you want is to use build your entire SQL Command into a single SSIS variable, and use the "SQL Command from variable" option in the OLEDB source.
EDIT based on comments:
If you have to handle a SQL Command over 4000 characters, the only way to do it is in a Script Component. The SQL Command in a script component can be any length. Here is an example.
We are using edge-sql to execute T-SQL scripts and also stored procedures via C#. I noticed recently that stored proc support has been added and I'm trying to execute would would be:
exec dbo.sgRouteExportDelete #TripDate='2014-05-06', #RouteId = 1234, #Action='DELETE', #Comment='xxxxxx';
in SQL Server Management Studio, using edge-sql 0.1.2.
I've played around with several variations, but I get one of 2 error messages. Either cannot find stored procedure '' or "cannot find stored procedure 'sgRouteExportDelete #TripDate='2014-05-06', #RouteId = 1234, #Action='DELETE', #Comment='xxxxxx'" The stored proc executes just fine in edge.sql when invoked via C# method.
I did some additional experimentation and found I can execute a stored proc with no parameters: exec dbo.sgVersionGet, but any stored procs with parameters return errors.
So what am I doing wrong? And how could I invoke with parameter values that aren't hard-coded, as above? Both SQL Server and edge use the # character for parsing params.
Any help appreciated ...
-BillyB
In SQL Server when you are referring to a database object without the object full path ([database.schema].object), SQL Server will try to locate the object using the Default/Initial Catalog value which points to the default database if that was not specified within the connection string then chances are that when you try running your statement SQL Server won't be able to find the object throwing the "Cannot find XX" error, You should either specify an initial catalog on your connection string or execute your procedures using the full path, database.schema.procedure E.g. mydatabase.dbo.sp_customerCleanUp. On the other hand there is an internal procedure sp_executesql that you can use to run your procedures without having to hard code the parameters, all you need to do is build a string concatenating the hard coded part of the string (the procedure name) and whatever number of variables you are passing as parameters, see example ( https://technet.microsoft.com/en-us/library/ms175170%28v=sql.105%29.aspx )
The variables are assumed when calling a stored procedure with edge-sql. Any parameter you would preface with an '#' symbol will need to have the same name within the stored procedure.
I've always thought that looping over sql files in directory in SSIS is easy... But I've got a problem today:
Execute SQL Task isn't executing statements that are in the sql file.
In the sql file I've got delete statement and then insert statement.
SSIS Execute SQL Task component is done after about 2 sec, while executing the same script manually takes usually about 2 minutes, and of course in SSIS it doesn't insert anything.
I checked variable value that is coming from Foreach loop (with full filemane path) and it is ok.
I've got parametrized (by Expression) File Connection with this variable.
What am I doing wrong? Thanks for help.
Do you have GO after each call within your SQL file?
Example:
// Your insert TSQL code here
GO
// Your delete TSQL code here
GO
// Etc...
It will not continue if you don't have this.
SSIS 2008. Very simple task. I want to retrieve a System Variable and use it in an SQL INSERT. I want to retrieve the value of System:MachineName and use it in an insert statement.
Using the statement INSERT INTO MYLOG (COL1) SELECT #[System::MachineName] gives the error Error: ..failed to parse. Must declare the scalar variable "#"
Using the statements SELECT #System::MachineName or SELECT ##[System::MachineName] gives the error 'Error Incorrect systax near '::'
I am not trying to pass a parameter to the query. I have searched for a day already but couldn't find how to do this one simple thing!
Here is one way you can do this. The following sample package was created using SSIS 2008 R2 and uses SQL Server 2008 R2 as backend.
Create a sample table in your SQLServer database named dbo.PackageData
Create an SSIS package.
On the SSIS, add an OLE DB connection manager named SQLServer to connect to your database, say to an SQL Server database.
On the Control flow tab, drag and drop an Execute SQL Task
Double-click on the Execute SQL task to bring the Execute SQL Task Editor.
On the General tab of the editor, set the Connection property to your connection manager named SQLServer.
In the property SQLStatement, enter the insert statement INSERT INTO dbo.PackageData (PackageName) VALUES (?)
On the Parameter Mapping tab, click Add button, select the Package variable that you would like to use. Change the data type accordingly. This example is going to insert the PackageName into a table, so the Data Type would be VARCHAR. Set the Parameter Name to 0, which indicates the index value of the parameter. Click OK button.
Execute the package.
You will notice a new record inserted into the table. I retained the package name as Package. That's why the table
Hope that helps.
Per my comment against #ZERO's answer (repeated here as an answer so it isn't overlooked by SSIS newcomers).
The OP's question is pretty much the use case for SSIS property expressions.
To pass SSIS variables into the query string one would concatenate it into an expression set for the SqlStatementSource property:
"INSERT INTO MYLOG (COL1) SELECT " + #[System::MachineName]
This is not to suggest the accepted answer isn't a good pattern, as in general, the parameterised approach is safer (against SQL injection) and faster (on re-use) than direct query string manipulation. But for a system variable (as opposed to a user-entered string) this solution should be safe from SQL injection, and this will be roughly as fast or faster than a parameterised query if re-used (as the machine name isn't changing).
I never use it before but maybe you can check out the use of expression in Execute SQL task for that.
Or just put the whole query into an expression of a variable with evaluateAsExpression set to true. Then use OLE DB to do you insert
Along with #user756519's answer, Depending on your connection string, your variable names and SQLStatementSource Changes