I have a CSV input file with commas. I need to change the delimiter to pipe. Which step should I use in Pentaho kettle? Please do suggest.
Thanks!
Do not use big gun when you try to shoot small target. Can use sed or awk. Or when you want to integrate with kettle, can use step to run shell script and within script use sed for example.
If your goal is to output a pipe separated CSV file from data within a transform and you're already running Kettle, just use a Text File output step.
If the goal is to do something unusual with CSV data within the transform itself, you might look into the Concat Fields step.
If the goal is simply to take a CSV file and write out another CSV with different separators, use the solution #martinnovoty suggests.
You can achieve this easy:
Add a javascript step after the load your csv step into a variable "foo" and add this code onto the js step:
var newFoo = replace(foo,",", "|");
now your cvs file is loaded in newFoo var with pipes.
Related
I have a "solution.pig" file which contain all load, join and dump queries. I need to run them by typing "solution.pig" in grunt> and save all the result in other file. How can I do that?
You can run the file directly with pig -f solution.pig. Don't open the grunt REPL
And in the file, you can use as many STORE commands as you want to save results into files, rather than DUMP
I am trying to understand if it's possible to change the model output format to .csv instead of the default .mat file when simulating a model using dymosim.exe.
I can do this in dymola itself by using the function "convertMATtoCSV" in the base Data files library. Something like below,
DataFiles.convertMATtoCSV("output.mat", {"t"}, "output.csv");
Is there a way to do this conversion using dymosim.exe?
Kindly advise.
Thanks.
Note: cmd "dymosim.exe -h" has some options for .csv but I am not sure how to use this.
No, it is currently not possible to have dymosim.exe generated by Dymola write the result as csv-file. The CSV-options used by dymosim.exe are only for running multiple simulations.
You can:
Generate a txt result instead, if that is easier to handle for you. (By setting Simulation Setup>Output>Textual data format, this is stored as last element of settings in dsin.txt).
Perform the conversion using dymola\bin\alist.exe
Have the model write a cvs-file as well
Set up to perform this as a post-processing command in Dymola 2017 FD01.
I am not able to open an .xlsx file. Is this the correct way to output the result to an .xlsx file?
hive -f hiveScript.hql > output.xlsx
hive -S -f hiveScript.hql > output.xls
This will work
There is no easy way to create an Excel (.xlsx) file directly from hive. You could output you queries content to an older version of Excel (.xls) by the answers given above and it would open in Excel properly (with an initial warning in latest versions of Office) but in essence it is just a text file with .xls extension. If you open this file with any text editor you would see the contents of the query output.
Take any .xlsx file on your system and open it with a text editor and see what you get. It will be all junk characters since that is not a simple text file.
Having said that there are many programming languages that allow you to convert/read a text file and create xlsx. Since no information is provided/requested on this I will not go into details. However, you may use Pandas in Python to create excels.
output csv or tsv file, and I used Python to do converting (pandas library)
I am away from my setup right now so really cannot test this. But you can give this a try in your hive shell:
hive -f hiveScript.hql >> output.xls
I'm using JMeter for performance testing and generate an CSV file for data analysis.
How should I add JMeter Variable names to the CSV output? It seems that only pre-defined columns can be added (see screenshot).
This is my log-file:
14/03/2014,Login,OK,982
14/03/2014,Search,OK,2182
14/03/2014,Login,OK,3982
I would like to add a column with the ${username} to generate something like:
14/03/2014,Login,OK,982,user1
14/03/2014,Search,OK,2182,user7
14/03/2014,Login,OK,3982,user9
Try to pass sample_variables property, for example as command-line parameter -Jsample_variables=username,some_other_var
Also see FlexibleFileWriter plugin if you need custom formatting http://jmeter-plugins.org/wiki/FlexibleFileWriter/?utm_source=jmeter&utm_medium=helplink&utm_campaign=FlexibleFileWriter
Is it possible to redirect two or more files to standard input in one command? For example
$ myProgram < file1 < file 2
I tried that command however, it seemed like the OS is only taking the first file and ignoring the other...
If not, how can I achieve that?
NOTE: concatenating the two files will not help in my case.
When you do this from bash, it isn't inputting multiple files to standard input, it is called Process Substitution
The output is sent to an file descriptor under /dev/fd/<n> for each substitution