I need to passing variable from modified java script value and pass it to table input query
this is my
If i click preview the output
initator_reference | a | '' |
1 | a | null |
2 | a | null |
3 | a | null |
4 | a | null |
It shouldnt be null but "testing" string on that third field
and dont ask me why i put that variable in select, its just for testing before i put it in "where condition=variable"
Table Input supports two different ways of making the SQL dynamic: Variable Substitution and inserting data from an incoming stream.
Variable substitution
This is what you currently have configured in the Table Input: You put ${variable} somewhere and when the step initializes, the value is pasted as text into the SQL.
Because all the steps initialize at the same time in a transformation, your Javascript step has not had time to set the value. In PDI you cannot set and use a variable within the same transformation.
Insert data from step
The second way is used by selecting a source step in the "Insert data from step" option in the Table input. In this mode, the Table Input takes a row from the selected step and inserts fields (in order) into the SQL at question marks (?) you insert. Normally it expects a single row, but you can choose to execute for EACH row.
This way should work for your scenario:
Put a Generate Rows step at the start and set it to generate 1 row.
Connect it to the Javascript step.
In the Javascript step, specify the return variable as an output field in the bottom grid, it will get added to the stream.
In the Table Input, select the Javascript step at "Insert data from step"
In your SQL, insert ? at the position of the variable. You may need single quotes around it if the value is a string: WHERE column = '?'.
Preview the transformation to see the result, the Table Input's preview is disabled because of the dependency.
Notes:
Step 1 and 2 may not be needed, I'm not sure if the JS step generates a row by itself. I like to be explicit about that, often naming the Generate Rows step "Generate 1 row".
If you have an existing stream or multiple fields to insert, you can use a Select Values step to put the fields in the order they need to be inserted into the SQL.
Well i found the solution myself
The First Step is Modified java script value->set variables
and then get variable->table input(check replace variable in script) and you just need to place "?" to your query (this "?" string is your variable, i dont know if there any other way to call variable)
and as reminder javascript on pentaho is a bit different with real javascript
Related
I have a lookup function that returns a list of valid GUID IDs in ADF. I then have a foreach process which runs a stored procedure for each GUID ID and assigns an ID column to it.
What I want to do is then have another lookup run which will run the below query to bring me the GUID and also the newly assigned ID. It is very simple to write in SQL
SELECT GUID, Identifier from DBO.GuidLOAD
WHERE GUID in ('GUIDID','GUIDID','GUIDID')
However I am struggling to translate this in ADF.. I have got as far as the #Concat part and most of the help I find online only refers to dynamic queries with single values as input parameters.. where mine is a list of GUIDs where they may be 1, more or none at all..
Can someone advise the best way of writing this dynamic query?
first 2 run fine, I just need the third lookup to run the query based on the output of the first lookup
You can use string interpolation (#{...}) instead of concat(). I have a sample table with with 2 records in my demo table as shown below:
Now, I have sample look up which returns 3 guid records. The following is debug output of lookup activity.
Now, I have used a for loop to create an array of these guid's returned by lookup activity using append variable activity. The items value for each activity is #activity('get guid').output.value. The following is the configuration of append variable inside for each.
#item().guids
I have used join function on the above array variable to create a string which can be used in the required query.
"#{join(variables('req'),'","')}"
Now, the query accepts guid's wrapped inside single quotes i.e., WHERE GUID in ('GUIDID','GUIDID','GUIDID'). So, I created 2 parameters with following values. I used them in order to replace double quotes from the above final variable with single quotes.
singlequote: '
doublequote: "
Now in the look up where you want to use your query, you can build it using the below dynamic content:
SELECT guid, identifier from dbo.demo WHERE GUID in (#{replace(variables('final'),pipeline().parameters.doublequote,pipeline().parameters.singlequote)})
Now, when I debug the pipeline, the following query would be executed which can be seen in the debug input of the final lookup.
The output would be as below. Only one row should be returned from the sample I have taken and output is as expected:
I have an excel with 300 rows. I need to use each of these rows as a field name in a transformation.
I was thinking of creating a job that for each row of a table sets a variable that I use afterwards on my transformation.
I tried defining a variable as the value I have in one row and the transformation works. Now I need a loop that gets value after value and redefines the variable I created then executes the transformation.
I tried to define a Job that has the following:
Start -> Transformation(ExcelFileCopyRowsToResult) -> SetVariables -> Transformation(The transf that executes using whatever the variable name is at the moment).
The problem is that the variable I defined never changes and the transformation result is always the same because of that.
Executing a transformation for each row in a result set is a standard way of doing things in PDI. You have most of it correct, but instead of setting a variable (which only happens once in the job flow), use the result rows directly.
First, configure the second transformation to Execute for each row in the Edit window.
You can then use one of two ways to pass the fields into the transformation, depending on which is easier for you:
Start the transformation with a get rows from result. This should get you one row each time. The fields will be in stream directly and can be used as such.
Pass the fields as parameters, so they can be used like variables. I use this one more often, but it takes a bit more setup.
Inside the second transformation, go to the properties and enter variable names you want in the Parameters tab.
Save the transformation.
In the job, open the transformation edit window and go to Parameters.
Click Get Parameters.
Type the field name from the first transformation under Stream Column Name for each parameter.
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
I need to make a dimension for a datawarehouse using pentaho.
I need to compare a number in a table with the number I get from a REST call.
If the number is not in the table, I need to set it to a default (999). I was thinking to use table input step with a select statement, and a javascript step that if the result is null to set it to 999. The problem is if there is no result, there is nothing passed through. How can this be done? Another idea was to get all values from that table and somehow convert it to something so I can read id as an array in javascript. I'm very new to pentaho DI but I've did some research but couldn't find what I was looking for. Anyone know how to solve this? If you need information, or want to see my transformation let me know!
Steps something like this:
Load number from api
Get Numbers from table
A) If number not in table -> set number to value 999
B) If number is in table -> do nothing
Continue with transformation with that number
I have this atm:
But the problem is if the number is not in the table, it returns nothing. I was trying to check in javascript if number = null or 0 then set it to 999.
Thanks in advance!
Replace the Input rain-type table by a lookup stream.
You read the main input with a rest, and the dimension table with an Input table, then make a Stream Lookup in which you specify that the lookup step is the dimension input table. In this step you can also specify a default value of 999.
The lookup stream works like this: for each row coming in from the main stream, the steps looks if it exists in the reference step and adds the reference fields to the row. So there is always one and exactly one passing by.
What I may have learned is that AutoNumber data types in Access are Read Only and cannot be edited.
I am having trouble with the the following SQL Code:
SELECT BaseClassID, BaseClassID+1 as BaseClassIncrement
FROM BaseClasses;
This returned the values in a query:
BaseClassID | BaseClassIncrement
1 | 2
Will it work when in form view after I apply this in command button wizard?
It would be after I selected RunQuery after I close the BaseClasses form.
Afterwards, it will it increments in the form after user reopens it.
How could I improve this task?
JohnDB
As the name of the command suggest, SELECTÂ does not modify the records it selects. In this case you are merely returning a row that contains an increment on that row alone. It does not increment anything.