I'm storing large (varchar(max)) SQL scripts in a table. I'd like to execute the scripts in an SSIS package.
Looking at other posts on this site it's easy enough to get the varchar(max) into an object variable. But then what to do? Is there a way for an Execute Sql Task (SQLSourceType of Variable) to specify an Object variable rather than a String variable?
Is there an approach that will work?
Here's how I might approach it:
Add a Data Flow task to your control flow
Add a Source (ADO.NET) that connects to your database
Create a Package level Object variable (for the next step)
Add a Recordset destination that populates your data into the Object variable created in the previous step
Back on the control flow:
Create a package level String variable for the "current" query (see next step)
Add a For Each ADO.NET enumerator
Connect the previous Data Flow task to the For Each task
Configure the For Each to use the Object variable as a source, and to store the column index with the SQL into the String variable
Add an Execute SQL task inside the For Each task
Configure it to execute a SQL Command from Variable, and pick the string variable containing the current query
Basically it will collect the queries from the table, then for each collected query, assign it to a variable, and then the Execute SQL command can pull the command text from that variable.
Related
This is merely a SSIS question for advanced programmers. I have a sql table that holds clientid, clientname, Filename, Ftplocationfolderpath, filelocationfolderpath
This table holds a unique record for each of my clients. As my client list grows I add a new row in my sql table for that client.
My question is this: Can I use the values in my sql table and somehow reference each of them in my SSIS package variables based on client id?
The reason for the sql table is that sometimes we get request to change the delivery or file name of a file we send externally. We would like to be able to change those things dynamically on the fly within the sql table instead of having to export the package each time and manually change then re-import the package. Each client has it's own SSIS package
let me know if this is feasible..I'd appreciate any insight
Yes, it is possible. There are two ways to approach this and it depends on how the job runs. First is if you are running for a single client for a single job run or if you are running for multiple clients for a single job run.
Either way, you will use the Execute SQL Task to retrieve data from the database and assign it to your variables.
You are running for a single client. This is fairly straightforward. In the Result Set, select the option for Single Row and map the single row's result to the package variables and go about your processing.
You are running for multiple clients. In the Result Set, select Full Result Set and assign the result to a single package variable that is of type Object - give it a meaningful name like ObjectRs. You will then add a ForEachLoop Enumerator:
Type: Foreach ADO Enumerator
ADO object source variable: Select the ObjectRs.
Enumerator Mode: Rows in all the tables (ADO.NET dataset only)
In Variable mappings, map all of the columns in their sequential order to the package variables. This effectively transforms the package into a series of single transactions that are looped.
Yes.
I assume that you run your package once per client or use some loop.
At the beginning of the "per client" code read all required values from the database into SSIS varaibles and the use these variables to define what you need. You should not hardcode client specific information in the package.
I have an SSIS package that takes in (through a package parameter) a value, passes it into a script task via a script variable (readonly variable), converts it to another value inside the script task, and finally writes that value out to another script variable (readwrite variable). There are no other SSIS modules in the package aside from the one script task.
What I would like to do, from outside the package (via SQL) is:
Call the SSIS package, passing in a value for my parameter and variable
Get the value of the read/write variable that is determined at the end of the script task execution
I've got step #1 working, just can't figure out #2.
How do I get the value of package variable in an SSIS package after it has executed? Is it accessible? Is it stored anywhere or can I store it somewhere in the SSIS catalog? I've tried to see if it's stored in the SSISDB.[catalog].executions table somewhere, but it doesn't seem to be.
Do I need to write that script variable to a package parameter in order to see it from SQL after execution? Could I then perhaps see it by using EXEC [SSISDB].[catalog].get_parameter_values, or does that only show parameter values before package execution? Am I going about this completely the wrong way?
Thanks in advance!
What I would do is add one last step to the package to write the value of the variable to a table.
Then you can retrieve the value from the table via SQL.
You can either truncate the table every time, or keep a permanent history associated with each time the package runs.
I need to store all the SQL Queries under one folder and refer in the SSIS package to better organize the list of SQL file I am using so we can easily change the SQL file later without having to rebuild the package. This will include all queries that I am using for "Execute SQL task " as well as the queries in "OLE DB Data Source" under Data Flow component.
Under Data Flow task, Instead of putting the SQL query for source data base into the Query Window:
I see four options under Data Access mode for OLE DB Data source-
1. Table or View
2. Table of view name variable
3. SQL Command
4. SQL Command from variable
I understand I could use a variable, store the query in the variable and refer it in "Execute SQL Task" but I am looking for a way to store all the queries in SQL files and it in Data Flow component as well as in "Execute SQL Script Task". I can't seem to find a way to make it dynamic in Data Flow task. Can anyone help with this please?
I don't think storing them as sql files is any good for any type of scenario.
Here is what I would do given what you have described.
You can store the queries as varchar in a table in a database instead of as files. Then you can foreach-loop over the result set and map each row to the variable that you would then use as the query for your oledb data source in the dataflow.
So create a variable and make a for each loop that loops over "select query from dbo.queries" or what ever. Set the output to the variable you created and in the container create your dataflow and set either the source-task with an expression or with "SQL Command from variable".
As for the control flow queries why not just have them be stored procedures that you can change when you need to?
I have a situation where I need to copy a stored procedure from one server to another server every day. I cannot use Transfer SQL object task as those servers are not linked. I'm looking for a solution something like, export procedure to a text file and import from text file to the destination. Can anyone help me with this?
I wanted this to be done through SSIS as I already have a package which is transforming data between these two servers daily.
Log into Server A using SQL Server Management Studio.
Find the stored procedure in the tree on the left and generate a CREATE script by right clicking
Run the script in your target database
I think we can do it as:
Execute SQL Task: using ADO.NET to run "sp_helptext YourSPName" (which returns the definition of your stored procedure as a single column table result). Save the "Full Result Set" to an Object type variable, say "ResultSet".
Script Task: pass in "ResultSet" as readonly. Create another variable, say "SPConent" type of String, pass in as readwrite. Inside the script, using the following code to reassemble the stored procedure definition to a string
public void Main()
{
// TODO: Add your code here
var table = ((DataSet) Dts.Variables["ResultSet"].Value).Tables[0];
string spContent = "";
foreach (DataRow row in table.Rows)
{
spContent += row[0];
}
Dts.Variables["SPContent"].Value = spContent;
Dts.TaskResult = (int)ScriptResults.Success;
}
Execute SQL Task: delete the stored procedure in target database.
Execute SQL Task: run sql command from "SPContent" against the target database.
I have a table with a column type of xml. I also have a directory that can have 0 to n number of xml documents. For each xml document, i need to insert a new row in the table and throw the xml into the xml column.
To fit with our clients needs, I need to perform this operation using an SSIS package. I plan to use a Stored Procedure to insert the xml, passing in the file path.
I've created the stored procedure and tested, it functions as expected.
My question is, how do I execute the stored procedure from an SSIS package for each xml document is a specific directory?
Thanks in advance for any help.
-
Basically you just need to loop through the files and get the full file paths to pass to the stored proc. This can be done easily using a For Each Loop and the ForEach File Enumerator. This page has a good description of how to set that up:
http://www.sqlis.com/post/Looping-over-files-with-the-Foreach-Loop.aspx
Within the loop then you just access the variable that is populated each time the loop executes (an XML file is found) and send it as a parameter into an Execute SQL Task (residing inside your For Eacu Loop container) to call your stored procedure. Here is an example of passing variables as parameters:
http://geekswithblogs.net/stun/archive/2009/03/05/mapping-stored-procedure-parameters-in-ssis-ole-db-source-editor.aspx
You don't need to use a stored procedure for this. You can do all of this within an SSIS package. Here's how:
Have a For-Each Loop task read all available files in the folder. Put the full path of the file into a variable called XMLFileName
Inside the For-Each loop, use a Data-Flow task read the contents.
The OLE_SRC is reading from the same SQL Server and it's statement is SELECT GetDate() as CurrentDateTime
The DerivedColumn component creates a column called XMLFilePath with the full path of the XML file
The ImportColumn component is the one that does the magic. It will take the XMLFilePath as an input column, give it the LineageId of a new output column you create and it will import the full XML for you. Read more on how to set it up here:
http://www.bimonkey.com/2009/09/the-import-column-transformation/
Use the OleDB Destination to write to the table.