Pentaho: how to read automatically column names and save them in variables? - pentaho

Question

I am new in Pentaho and I Need a help.
I am getting data from database table, and then I need to count the number of columns of these tables and save their names into variables. Does somebody know how to make this? Or how to read the column names in Java Script step? Thank you for your help.

Answer 1

First of all, please note, that you will not be able to see variable value in the same transformation, where you have set/changed it. Basically, variable values for transformation are read on its initalization. So, you can change variable value for parent/grandparent job, but not for same transformation.
Option 1 (Metadata structure of stream step):
One option is to use Metadata structure of stream step http://wiki.pentaho.com/display/EAI/Metadata+Structure+of+Stream
The step will give you following fields:
Position: The position of the field in the inbound data stream
Fieldname: The field name in the inbound data stream
Comments: undocummented
Type: String, Integer, Number etc
Length: The field length
Precision: Applicable for Integer, Number and -1 if not applicable
(e.g. Strings)
Origin: The step where the meta data was derived from.
The output of this step can subsequently used to set variable values.
Option 2 (Javascript):
Java Script step has getInputRowMeta() function
Which returns RowMetaInterface object.
Further you may call getValueMetaList() method, which will return you List of ValueMetaInterface objects. Then you can just iterate through all items in the list and call getName() for each object to get a name of each input field.
Example (most of the code here is actually just to illustrate output):
// get instance of RowMetaInterface, which represents metadata of your input row:
var rowMetaInterface = getInputRowMeta();
// get List of ValueMetaInterface objects, which represent each of your input fields:
var valueMetaList = rowMetaInterface.getValueMetaList();
// get this object to be able to iterate over the list:
var iterator = valueMetaList.iterator();
// I am going to save all results into this JS object, then convert it to JSON string and pass into a variable
var allFields = [];
while (iterator.hasNext()) {
// get current ValueMetaInterface object:
var valueMetaInterface = iterator.next();
// The name of current field:
var fieldName = valueMetaInterface.getName();
// getType() returns just an internal integer number of the type:
var typeIndex = valueMetaInterface.getType();
// typeName will hold the actual String name of the type:
var typeName = org.pentaho.di.core.row.ValueMetaInterface.typeCodes[typeIndex];
// You proabably don't need the following part. It is here just for illustration purpose:
// Prepare data to be converted to JSON:
var field = {};
// wrap Java String into a JavaScript String object to be able to convert them into Json
field.fieldName = String(fieldName);
field.fieldType = String(typeName);
allFields.push(field)
}
var allFieldsJson = JSON.stringify(allFields);
/* scope of the variable:
"s": System scope
"r": root
"p": parent
"g": grandparent*/
var variableScope = "r";
setVariable("inputFields", allFieldsJson, variableScope);
writeToLog("b", "JSON string:");
writeToLog("b", allFieldsJson);
// In other job/transformation you'll be able to parse json object into a javascript object back:
var fields = JSON.parse(allFieldsJson);
for (var i = 0; i < fields.length; i++) {
fieldName = fields[i].fieldName;
fieldType = fields[i].fieldType;
writeToLog("b", "Field number " + i + ":");
writeToLog("b", fieldName);
writeToLog("b", fieldType);
}
In the log you will see following output:
2016/10/13 15:33:09 - Modified Java Script Value.0 - Optimization level set to 9.
2016/10/13 15:33:09 - Modified Java Script Value.0 - JSON string:
2016/10/13 15:33:09 - Modified Java Script Value.0 - [{"fieldName":"test","fieldType":"String"},{"fieldName":"test 2","fieldType":"Integer"},{"fieldName":"another field","fieldType":"BigNumber"}]
2016/10/13 15:33:09 - Modified Java Script Value.0 - Field number 0:
2016/10/13 15:33:09 - Modified Java Script Value.0 - test
2016/10/13 15:33:09 - Modified Java Script Value.0 - String
2016/10/13 15:33:09 - Modified Java Script Value.0 - Field number 1:
2016/10/13 15:33:09 - Modified Java Script Value.0 - test 2
2016/10/13 15:33:09 - Modified Java Script Value.0 - Integer
2016/10/13 15:33:09 - Modified Java Script Value.0 - Field number 2:
2016/10/13 15:33:09 - Modified Java Script Value.0 - another field
2016/10/13 15:33:09 - Modified Java Script Value.0 - BigNumber
2016/10/13 15:33:09 - Modified Java Script Value.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)