Anyone know how (if) I can use gdal at the command prompt level to subsample an ascii comma separated dems file to lower density data? I've been trying to use gdal_translate -outsize but haven't had any success. Thanks so much!
try like this:
gdal_translate.exe -outsize 400 200 src.dem trg.dem
start from GDAL>=2.0, support resampling algorithm selection by using:
-r {nearest (default),bilinear,cubic,cubicspline,lanczos,average,mode}
more information in http://www.gdal.org/gdal_translate.html
Related
I'm new to Pentaho and have a little problem with the Text file Input.
Currently I have to have several data records written to a database. In the files, the decimal numbers are separated by a point.
Pentaho is currently transforming the number 123.3659 € to 12.33 €.
Can someone help?
When you read the file, do you read it as a csv, excel or something like that? If that's the case, then you can specify the format of the column to interpret the number correctly (I think, I'm talking from memory now) Or maybe playing with the language of the file might work.
If it's a file containing a string, you can use some step like the string operator to replace the point with a comma.
This problem might come from various reasons.
Although I think that by following the next steps you can solve the issue.
-First, you must get a "Replace in String" step;
-Then search for the dot and replace it with nothing as I show in the following image, or with a coma if the number you show is a float;
Example snip
Hope this helped!
Give feedback if so!
Have a good day!
I have a ksh file which have set of SQL commands (for ex. 35 commands) in it and I used to open it in notepad++. It is executing and returning expected results for the first 5 SQL queries, but for the rest, it is throwing an error.
When I dug into it, in between the SQL commands, there is an extra character like Â. How to remove this? I have tried a lot to get rid of it with no luck.
Thanks.
There's a Hex Editor plugin for NotePad++. Look at your file as a sequence of byte values rather than an encoded text file. Find the byte value that corresponds with the  glyph.
I'm fairly new with apache pig and trying to work with some fixed width text. In pig, I'm reading every line in as a chararray (I know I can use fixedwidthloader, but am not in this instance). One of the fields I'm working with is an email field and one entry has a carriage return that generates extra lines of output in the finished data dump (I show 12 rows instead of the 9 I'm expecting). I know which entry has the error but I'm unable to filter it out using pig.
Thus far I've tried to use pig's REPLACE to replace on \r or \uFFFD and even tried a python UDF which works on the command line but not when I run it as a UDF through PIG. Anyone have any suggestions? Please let me know if more details are required.
My original edit with a solution turned out to only work part of the time. This time I had to clean the data before I ran it through pig. On the raw data file I did a perl -i -pe 's/\r//g' filename to remove the rogue carriage return.
I have a text file that I want to Load onto my Pig Engine,
The text file have names in it in separate rows, and the data but has errors in it.....special characters....Something like this:
Ja##$s000on
J##a%^ke
T!!ina
Mel#ani
I want to remove the special characters from all the names using REGEX ....One way i found to do the job in pig and finally have the output as...
Jason
Jake
Tina
Melani
Can someone please tell me the regex that will do this job in Pig.
Also write the command that will do it as I unable to use the REGEX_EXTRACT and REGEX_EXTRACT_ALL function.
Also can someone explain what is the Significance of the number 1 that we pass to this function as Argument after defining the Regex.
Any help would be highly appreciated.
You can use REPLACE with RegEx to solve this problem.
input.txt
Ja##$s000on
J##a%^ke T!!ina Mel#ani
PigScript:
A = LOAD 'input.txt' as line;
B = FOREACH A GENERATE REPLACE(line,'([^a-zA-Z\\s]+)','');
dump B;
Output:
(Jason)
(Jake Tina Melani)
There is no way to escape these characters when they are part of the values in a tuple, bag, or map, but there is no problem whatsoever in loading these characters in when part of a string. Just specify that field as type chararray
Please Have a look here
Here is my simplified scenario:
I have a table in SQL Server 2005 with single column of type varchar(500). Data in the column is always 350 characters in length.
When I run a select on it in SSMS query editor, copy & paste the result set in to a text file, the line length in the file is 350, which matches the actual data length.
But when I use sqlcmd with the -o parameter, the resulting file has line length 500, which matches the max length of varchar(500).
So question is, without using any string functions in select, is there a way to let sqlcmd know not to treat it like char(500) ?
You can use the sqlcmd formatting option -W to remove trailing spaces from the output file.
Read more at this MSDN article.
-W only works with default size of 256 for variable size columns. If you want more than that you got to use -y modifier which will tell you its mutually exclusive with -W. Basically you are out of luck and as in my case file grows from 0.5M to 172M. You have to use other ways to strip white space post file generation. Some PowerShell command or something.