I am creating pentaho jobs
In first set variable box i am passing value sysdate the first dfp job working perfectly.
In second set variable box i am passing value sysdate+1 ,so sysdate+1 file is picked correctly to process but second dfp job only getting error.
Is this logic is possible in pentaho Jobs?
I have numerous example of that kind that works perfectly every night. And I guess the Set variables have the appropriate level (valid in parent job).
So the bug is probably in the value you give to the variable in Set variable 2. The value sysdate+1 has the literal value (the string "sysdate+1") not the Date of tomorrow.
You must first compute that value. Which is done in a transformation replacing Set variable 2, which would do something like this:
Related
I have an excel with 300 rows. I need to use each of these rows as a field name in a transformation.
I was thinking of creating a job that for each row of a table sets a variable that I use afterwards on my transformation.
I tried defining a variable as the value I have in one row and the transformation works. Now I need a loop that gets value after value and redefines the variable I created then executes the transformation.
I tried to define a Job that has the following:
Start -> Transformation(ExcelFileCopyRowsToResult) -> SetVariables -> Transformation(The transf that executes using whatever the variable name is at the moment).
The problem is that the variable I defined never changes and the transformation result is always the same because of that.
Executing a transformation for each row in a result set is a standard way of doing things in PDI. You have most of it correct, but instead of setting a variable (which only happens once in the job flow), use the result rows directly.
First, configure the second transformation to Execute for each row in the Edit window.
You can then use one of two ways to pass the fields into the transformation, depending on which is easier for you:
Start the transformation with a get rows from result. This should get you one row each time. The fields will be in stream directly and can be used as such.
Pass the fields as parameters, so they can be used like variables. I use this one more often, but it takes a bit more setup.
Inside the second transformation, go to the properties and enter variable names you want in the Parameters tab.
Save the transformation.
In the job, open the transformation edit window and go to Parameters.
Click Get Parameters.
Type the field name from the first transformation under Stream Column Name for each parameter.
When you comment a parameter in a SUBMIT of a report, what would be its value? The initial value of that type or the value that should have in a standard run of the equivalent transaction?
I mean, if you want to run a transaction setting in a defined way several dynprofields by submitting the transaction report, would you have to state every field that does have initial value or set just those that differ from the usual value they have when executing the transaction?
The initial value, so you don't need to pass those values in the SUBMIT (you can comment those out).
I am creating a transformation in pentaho di to extract data from google analytics. I need to set in "Query Definition" the start date and end date as yesterday and today. I understand this can be done by create two varialbes e.g. ${todsy},${yesterday}. However, I don't know how to set these to change values dynamically at every run. ANy idea on how to do this?
Thanks,
I can think of an easy way to do this. The first thing is that you can't declare and use the variables in the same transformation. I would suggest you to approach this problem in the following way:
Create a transformation before this one, say "set variables transformation". In this transformation you will set the variables.
You can use Get System Info step to set today's and yesterday's dates as the variables. Use copy rows to result step to pass these rows to the next transformation.
In the next transformation, which will be the one you have attached the screenshot of, use the Get Variables step and use these variables in your input step. Or you can use Get rows from result step as well.
You don't need to worry about the dates now, because dates will be generated and your variables get the values dynamically.
You can check this article if you want to learn more about how to pass the values from one transformation to another:
https://anotherreeshu.wordpress.com/2014/12/23/using-copy-rows-to-result-in-pentaho-data-integration/
Hope it helps!
for that, you have to use a job, add the first transformation and inside it use
get system info step then add today's and yesterday's date as a variable, and link to the set variable step. Set the scope of variable as parent job,
in second job use **get variables **.
It took me a while to solve this myself and the way I ended up doing it is as follows:
I created a transformation (called 'set formatted_today variable') the transformation contains two objects:
The transformation contained a 'table input' object with a query like:
select to_char(current_timestamp, 'YYYY-MM-DD-HH-MI') as formatted_today
The output of my 'table input' goes to a 'set variables' object, you can use the 'get fields button to wire the fields you've named in your query to the variable you want to set. In this case, my field is called 'formatted_today' and so is my variable.
In my main job, I have a 'set session variables' object that creates my 'formatted_today' variable.
Immediately after it, I call my 'set formatted_today variable' transformation
Anywhere I need this variable I insert ${formatted_today} in the text
Two Issues:
I've set the value of a Date parameter (Start Date) when moving from the home screen (Report 1) to the next screen (Report 2) as the first day of the current year. In report 2, if I try to change the Start Date to another date, it gets reset to the first day of the year. I had the default value set as the first day of the year and even after removing it, the Start Date gets reset.
I've set a multi-value parameter (Area) in Report 1 that collects four integers. I pass Area to Report 2 (through Join (Parameters!Area.value,",")). I also used SPLIT (Join (Parameters!Area.value,","),",") to get the value. Then, in the data set, I call this parameter as "Select A1,A2,A3,B1,B2,B3 FROM TEST_VIEW WHERE ID IN (?)". But this query throws an error that the query could not be executed. I ran the same query on the server side and it runs fine.
Note:
We cannot reference parameter names as #area for example because the Composite Server doesn't allow that.
Split and Join operate on strings but my parameters are of integer type. Is this an issue?
Please help.
Re #2:
Few things to answer and work through here. First of all, JOIN can operate on integers as it returns a string value while SPLIT cannot operate on an integer as it operates on a string (needs a string to "split"). I'd suggest that the JOIN/SPLITS are necessarily complicating what you are trying to do, although they should still work in this scenario.
Try just passing your multivalue parameter to Report2 from the parent report as Parameters!Area.Value (no joins) and then catch it on report2 again with a multivalue integer parameter. SSRS will automatically handle the multi-value passing between reports and then your query should work.
Note that JOIN/SPLIT is necessary if you want to send a multivalue parameter to a SQL stored procedure for data retrieval, but you have to perform the split on the SQL side.
I have a requirement where a particular Kettle transformation (ktr) needs to be run multiple times.
This is the scenario:
A transformation has a table input which pulls user details belonging to a particular country.
I have almost 5 countries and this is saved in a separate table.
Can i make a variable and assign the country name to it and run the same transformation in a loop of five times, where every time the variable gets updated to the next country name.
I need the variable to be used in the table input query and in the column name also.
This is how i mentioned the variable in the table input.
When i am giving the variable as value, in the output i am getting '${COUNTRY
}' instead of the value of the variable
PDI allows you to do multiple iteration using a variable. You need to use "copy rows to result" in kettel step. I have a blog written on this topic.
Blog Link : https://anotherreeshu.wordpress.com/2014/12/23/using-copy-rows-to-result-in-pentaho-data-integration/
Please check if it helps you. :)