I'm reaching out to the experts as I have hit a wall with a recent project. I have created an SSIS package (2008R2) that uses a script task to build a SQL statement, where a variable(#month1) is being used within the SQL statement, to specify a month look back in a membership table. I want to also use the #month1 variable as a "counter" for the loop container to specify how many times to execute the query. The SQL query is attached to a data flow task to append these records into a table on a SQL server database. The script task and data flow task work outside of the for loop container with the initial value given for the #month1 variable but I cannot figure out how to make the for loop container update the #month1 "counter" variable so that the for each loop can use it as a "counter" and the SQL statement can use it as a condition with in the created SQL statement. Any one have any ideas or examples on how to do this?
** Update **
The For Loop container is the issue. The script task and data flow task work outside of the For Loop container. It will use the initial variable setting for #month1 and create the dynamic sql script, execute script and transfer data from source database server to the destination source server. The issue is when I place these steps within the For Loop container, the container executes and turns green but does not invoke the steps within it. This is why I'm thinking the container is not reading the variable #month1, even though the variable is set at the package level. Any thoughts?
First of all, try to set the data flow Delay Validation property to True. If it still not working, instead of passing the variable as parameter in the OLEDB Source use expressions:
Create a variable of type string.
Change its EvaluateAsExpression property to True
Set the expression similar to:
"select * from table
Where column =" + (dt_str, 50)#[User::month1]
In the OLEDB Source select the Access mode as SQL Command from variable and select this variable.
Be aware that the month1 variable is not created twice wuth different scopes, click on the data flow task and check the variable panel if it shows additional variables.
I appreciate everyone's responses but it seems I tricked myself on this one. In looking for the most complicated issue I overlooked the most simple and obvious one. The reason my For Loop was not executing the steps inside of the container was because I had the initial value for #month1 set to 3 (intentionally) and wanted to loop until it was resolved to -49. In the EvalExpression setting, it will evaluate until the statement is FALSE...so the evaluation I had in there of #month1 <= -49 was already false. It needed to be #month1 > -49 so as soon as it fell to -49 the statement would be false. I do this to myself more than I should admit, can't see the forest for the trees!
Related
I'm trying to build a template SSIS package that will pull from our source system and populate our load tables. The package works as follows, since we're taking from Oracle and moving to SQL Server.
Variable of "SourceDelta", String, value is blank
Execute SQL Task that is a select statement against a SQL table holding the delta column name, LastUpdateDT, the result set is mapped to the variable SourceDelta.
I then have a data flow task, where the source is an ado.net source. I have an expression on the data flow task which is:
select *
from VariableSourceLocation
where
SourceDelta >= ##
and SourceDelta < ##
Fine, you'd think, it'd populate the SourceDelta colum name into the expression. However, when opening the data flow task and opening the ado.net source, the sql command text reads as follows:
select *
from VariableSourceLocation (sets correctly showing the value)
where
BLANK>= ##
and BLANK< ##
It just passes a blank, as the variable has no value until the previous step. It's as though it is not inputting the variable prior to the package validation, so validation errors as it can't run the where clause of the query.
Banging my head against the wall with this, any help appreciated. Fully aware of injection issues surrounding this but it's purely for me to create SSIS data flow packages utilising a single sql table only I will have access to for variables. It'll speed up the onerous task a lot.
I am designing an incremental load for my ETL solution in SSIS. For that, I have an Execute SQL task that gets the maximum load time from the data warehouse and then stores it in a package variable, which is already set to evaluate as expression.
Then I have set the ODBC source's sql command property to an expression that has my query and the variable. However, I have looked into the variable during debug and when I run the package it seems that this variable doesn't get used in the sql command, instead it remains null.
I have already tried setting the variable property 'EvaluateAsExpression' to True, I have tried storing the query in a different variable and then setting that as the sql command for my ODBC source.
Bumping due to no suggestions
I have an SSIS package with a declared variable - claimMaxDate. The first step in my package is to populate the variable with the MAX(TIME_STAMP) from a SQL Server table.
I want to use that date to run a different query but it must use the ODBC data source in SSIS.
Since parameters can't be passed to the ODBC data source, I'm trying to use expressions.
This is what I've added to the data flow task:
However, the expression never refreshes with the date that is populated in the variable. I've debugged and confirmed that the variable is being populated. Variable property EvaluateAsExpression is also true.
Am I missing a step here?
I would rather change your package design. Create a string variable like SQL_DFT_Select which evaluates with your expression. Then specify this variable as a query source at ODBC data source. By doing this you can set a breakpoint at your Data Flow Task step and check this variable.
On your original question -- it can be the case that your expression is evaluated at validation time, then your claimMaxTime variable is empty. Later change of this variable does not trigger update of property expression. However, every reading of a variable with an expression re-evaluates this expression; that is why I recommended switching to query from variable design.
Just in case anyone else come across this question. The answer was simple. The expression was evaluating at run time, it just doesn't show the update when debugging.
I have an SSIS package where I need to Execute a usp, check the row count coming back from the usp, then use the results from the usp if the row count is not 0 in a For Each loop. I would like to do this by only calling the DB once, but I can't figure out how. Right now I am using a Data Flow task with a OLE DB Source to EXECUTE the usp. I then have a row count task to get the count and assign it to a variable. In my control flow I then use this variable as an expression into the for each loop, which is working, but then I don't know how to actually loop through the query results as it is not currently assigned to anything. I was thinking to assign the OLE DB Source to a Variable but I am not sure how to do that, so I am open to other options as well.
Here is what I currently have:
Control Flow:
Data Flow:
there probably are better solutions but not knowing your application and following your attempted solution what I can suggest you is the use of recordset destination:
you'll need to create a variable of type object to save the recordset. The you can read that variable in a for loop set to for each ADO enumeration:
How can I access a 'RecordSet' within a 'Execute SQL' task when using SSIS?
I have looked at the parameter mapping options within the Execute SQL Task Editor and cannot find a type of object to allow me to pass the variable holding my record set to the task.
You can't directly. A recordset is an in memory pointer to a dataset, while an ExecuteSql task accesses data from a database. You can use a foreach container with an executeSql task inside it in a technique I have heard called "Shredding" where you point the recordset to the foreach as the enumerator, set the columns in the recordset to variables for each inside the loop, and pass those individual record values (columns) as parameters to an executeSql task.
Andy Leonard does a good job of describing the use of the ForEach.
I think this is possible. You can pass an entire record set as a parameter to some SQL query executed in Execute SQL task if the type of the variable is object. Not sure if there are any glitches in your particular case or if I got your question all wrong.