I have a job that calls two transformations and depending on a situation I need to update the value of a variable that is received as parameter:
I'm using Set Variables component but is not changing the value of variable. Does anybody know how can I achieve? I'm using Pentaho server 8.1.0.0-365 version.
I needed to create a new transformation to assign the value to the variable:
Related
Can I define inside a "select value step" a variable?
like ${column_name}
How can I make it work, I have a parameter that will change this column every time.
That step doesn't allow using variables to select the columns of the stream you want to keep.
In this case you could use Metadata Injection, read Pentaho documentation as well as study the example in the samples folder of your installation on how to use Metadata Injection.
You just need to check the box in the step select values
I am designing an incremental load for my ETL solution in SSIS. For that, I have an Execute SQL task that gets the maximum load time from the data warehouse and then stores it in a package variable, which is already set to evaluate as expression.
Then I have set the ODBC source's sql command property to an expression that has my query and the variable. However, I have looked into the variable during debug and when I run the package it seems that this variable doesn't get used in the sql command, instead it remains null.
I have already tried setting the variable property 'EvaluateAsExpression' to True, I have tried storing the query in a different variable and then setting that as the sql command for my ODBC source.
I'm reaching out to the experts as I have hit a wall with a recent project. I have created an SSIS package (2008R2) that uses a script task to build a SQL statement, where a variable(#month1) is being used within the SQL statement, to specify a month look back in a membership table. I want to also use the #month1 variable as a "counter" for the loop container to specify how many times to execute the query. The SQL query is attached to a data flow task to append these records into a table on a SQL server database. The script task and data flow task work outside of the for loop container with the initial value given for the #month1 variable but I cannot figure out how to make the for loop container update the #month1 "counter" variable so that the for each loop can use it as a "counter" and the SQL statement can use it as a condition with in the created SQL statement. Any one have any ideas or examples on how to do this?
** Update **
The For Loop container is the issue. The script task and data flow task work outside of the For Loop container. It will use the initial variable setting for #month1 and create the dynamic sql script, execute script and transfer data from source database server to the destination source server. The issue is when I place these steps within the For Loop container, the container executes and turns green but does not invoke the steps within it. This is why I'm thinking the container is not reading the variable #month1, even though the variable is set at the package level. Any thoughts?
First of all, try to set the data flow Delay Validation property to True. If it still not working, instead of passing the variable as parameter in the OLEDB Source use expressions:
Create a variable of type string.
Change its EvaluateAsExpression property to True
Set the expression similar to:
"select * from table
Where column =" + (dt_str, 50)#[User::month1]
In the OLEDB Source select the Access mode as SQL Command from variable and select this variable.
Be aware that the month1 variable is not created twice wuth different scopes, click on the data flow task and check the variable panel if it shows additional variables.
I appreciate everyone's responses but it seems I tricked myself on this one. In looking for the most complicated issue I overlooked the most simple and obvious one. The reason my For Loop was not executing the steps inside of the container was because I had the initial value for #month1 set to 3 (intentionally) and wanted to loop until it was resolved to -49. In the EvalExpression setting, it will evaluate until the statement is FALSE...so the evaluation I had in there of #month1 <= -49 was already false. It needed to be #month1 > -49 so as soon as it fell to -49 the statement would be false. I do this to myself more than I should admit, can't see the forest for the trees!
Bumping due to no suggestions
I have an SSIS package with a declared variable - claimMaxDate. The first step in my package is to populate the variable with the MAX(TIME_STAMP) from a SQL Server table.
I want to use that date to run a different query but it must use the ODBC data source in SSIS.
Since parameters can't be passed to the ODBC data source, I'm trying to use expressions.
This is what I've added to the data flow task:
However, the expression never refreshes with the date that is populated in the variable. I've debugged and confirmed that the variable is being populated. Variable property EvaluateAsExpression is also true.
Am I missing a step here?
I would rather change your package design. Create a string variable like SQL_DFT_Select which evaluates with your expression. Then specify this variable as a query source at ODBC data source. By doing this you can set a breakpoint at your Data Flow Task step and check this variable.
On your original question -- it can be the case that your expression is evaluated at validation time, then your claimMaxTime variable is empty. Later change of this variable does not trigger update of property expression. However, every reading of a variable with an expression re-evaluates this expression; that is why I recommended switching to query from variable design.
Just in case anyone else come across this question. The answer was simple. The expression was evaluating at run time, it just doesn't show the update when debugging.
Using Pentaho, I would like to update records of database with the value of some of public parameters of the transformation. I know that I can use the value of parameters in some steps such as Table input or Database Join by checking the Replace variable option. However, I cannot find any way to pass value of public parameters of the transformation in order to use in Update step. Can anyone help me with that?
For clarificattion: I want to use the parameter which is defined here
In order to use public parameter of transformation, Get Variable step can be used.