anyone know how to set variable for file name in 'Text File Input'?
I want the file name depends on when I execute the transformation, example:
D:\input_file_<variable>.txt
today = D:\input_file_20131128.txt
tomorrow = D:\input_file_20131129.txt
FYI, I'm using Kettle Spoon - 4.2.0
In set form, you can use variable as ${Variable_Name} in file name.
You should notice the system information:
Please remember that the variables you define with this step can't be used in this transformation. This is simply because all steps in a transformation run in parallel without a certain order of execution.
As alternative correct usage, you can set variables you want to use in the first transformation of a job
Related
I need to archive the txt file using Pentaho PDI by giving it a dynamic timestamp and append the variable to the output filename. I used get system info which automatically assigns variable as well as value. So my job was Start__ get system info___zip file. In the zip file component, I tried called the variable while giving the output filename along with ${Variable} but the output filename is not coming properly. It should be off filename__timestamp__variable. Can someone please help me with this?
I am trying to run a Transformation/Job by passing a user variable in command line.
I have tried by passing variable value as below.
sh pan.sh -file='test.ktr' '-param:input_directory=/path/to/directory' -level=basic
where input_directory is variable in transformation and i mentioned it as ${input_directory}
But when I do this, the pan is unable to find the variable value. It is throwing error as below
Could not list the contents of "file:///home/user1/pdi8.1/data-integration8.1/${input_directory}" because it is not a folder.
can someone help me on this. Thank you
To pass named parameters to your job or transformation, the parameters need to be defined in the properties window, shown here for a transformation. The default value is not needed, but works well for testing. Pay attention to capitalization.
So the pieces of the puzzle are:
From the command line, pass the parameter like -param:yourparam=yourvalue
Define this same parameter in the highest-level job or transformation
Use it as you would use any variable, with ${yourparam}
i think the parameter names to be used in job should be ${PARAM_NAME1}
using command line i follow the below convention
call "{Replace with kitchen.bat File Path}" /file:"{Replace with JOB File Path}" "-param:PARAM_NAME1=PARAM_VALUE1" "-param:PARAM_NAME2=PARAM_VALUE2"
New to PDI here. Need to output data from a view in a postgresql database to a file daily. The output file will be like xxxx_20160427.txt, so need to append the dynamic date in the file name. How to do it?
EDIT-----------------
I was not clear here by asking how to add dynamic date, I was trying to add not just date but optional other parts to the file name. E.g adding a serial no (01) at the end: xxxx_2016042701.txt etc. So my real question is how to make a dynamic file name? In other ETL tool e.g. SSIS it will be a simple expression. Not sure how it is done in PDI?
In your Text file output step, simply check "Include date in filename?" under the files tab.
You can create a dynamic filename variable with a Modified Java Script value STEP.
and then in the Text File Output STEP click on "Accept file name from field", and select your variable declared from previous step (filename_var on this example).
When I define a custom variable in the new TFS 2015 team build as follows:
Name: SomeOutput
Value: $(System.DefaultWorkingDirectory)\Some
...it doesn't seems to expand $(System.DefaultWorkingDirectory).
Is there a way around this?
EDIT:
At least it seems it's not expanded everywhere.
For example, in MSBuild-Arguments, /p:OUTPUT="$(SomeOutput)" is expanded to /p:OUTPUT="C:\TfsData\BuildAgents\_work\3\s\Some" but when i add a cmd line build task with tool set to cmd and parameter set to /k set, it prints
SOMEOUTPUT=$(System.DefaultWorkingDirectory)\Some
EDIT 2:
Here are my variables
This is my workflow step
And this is what the build prints
You can use the VSTS Variable Tasks extension from the Visual Studio Marketplace.
When you define a variable in the Variables screen and use other variables as value, they won't be expanded (as you may have expected). Instead the literal text is passed to the tasks in the workflow. Without this little task the following configuration won't work:
Variable Value
Build.DropLocation \\share\drops\$(Build.DefinitionName)\$(Build.BuildNumber)
By adding the Expand variable(s) task to the top of your workflow, it will take care of the expansion, so any task below it will receive the value you're after.
https://github.com/jessehouwing/vsts-variable-tasks/wiki/Expand-Variable
PS: The new agent (version 2.x) auto-expands variables now.
It can be achieved.
You may need use % % instead of $ to call the variables in cmd to print the result. It is also necessary to add call in the front of the command. Here is a simple example:
Note: System.DefaultWorkingDirectory is not available in cmd (not sure why); you need use System_DefaultWorkingDirectory instead. Details can be viewed in the logs.
I had the same problem - wanted to piece together a path made up of several built-in variables and pass it to a PS script.
Workaround:
I ended up combining the variables in the actual script through the corresponding generated environment variables (for example $env:BUILD_SOURCESDIRECTORY).
Not what I had in mind originally, but it works at least. Drawback - if I need to change the path, I always have to change the PS script instead of a build variable.
How can I find out (with Windows a batch command), if, for example, a variable starts with ABC?
I know that I can search for variables if I know the whole content (if "%variable%"=="abc"), but I want that it only looks after the beginning.
I also need it to find out where the batch file is located, so if there is a other command that reveals the file's location, please let me know.
Use the variable substring syntax:
IF "%variable:~0,3%"=="ABC" [...]
If you need the path to the batch file without the batch file name, you can use the variable:
%~dp0
Syntax for this is explained in the help for the for command, although this variable syntax extends beyond just the for command syntax.
to find batch file location use %0 (gives full patch to current batch file) or %CD% variable which gives local directory