Having troubles finding a good way to get a string from a text file (separated by line breaks) randomly.
I want to do a setStringValue:#"random string from file here";
pretty much. Thanks in advance.
Reservoir sampling if you want to avoid loading the complete file into memory at once. For a file just a few lines in length I'd just go with vodkhang's answer, though.
How about you load the whole file (if it is not too big) into an array and then you randomize the index and use that index to get the string from the array?
Related
I'm new to Pentaho and have a little problem with the Text file Input.
Currently I have to have several data records written to a database. In the files, the decimal numbers are separated by a point.
Pentaho is currently transforming the number 123.3659 € to 12.33 €.
Can someone help?
When you read the file, do you read it as a csv, excel or something like that? If that's the case, then you can specify the format of the column to interpret the number correctly (I think, I'm talking from memory now) Or maybe playing with the language of the file might work.
If it's a file containing a string, you can use some step like the string operator to replace the point with a comma.
This problem might come from various reasons.
Although I think that by following the next steps you can solve the issue.
-First, you must get a "Replace in String" step;
-Then search for the dot and replace it with nothing as I show in the following image, or with a coma if the number you show is a float;
Example snip
Hope this helped!
Give feedback if so!
Have a good day!
Let's say you only want to parse the start of a large file using Perl 6 grammar. In order to avoid reading the whole file into a string, and then call subparse on the string. Is it possible to do a subparse when reading the file?
I could not find any subparsefile() method in the Grammar class, so I guess this is difficult to implement. But it should be possible in theory, see for example How do I search a file for a multiline pattern without reading the whole file into memory?
Currently you can't. Parsing anything at the moment requires the entire string to exist in memory.
Having said that, if you know the maximum number of lines your pattern may expand over, you could do something like:
my $max = 3; # maximum number of lines
for "textfile".IO.lines(:!chomp).rotor( $max => -$max + 1 ) -> #lines {
#lines.join.subparse( $grammar)
# and whatever you would like to do
}
It wouldn't be the fastest way of doing it, but it would not have to read the whole file in memory.
Please note that I'm not asking how to append texts at the end of the file. I'm asking how to prepend texts to the beginning of file.
let handle = try FileHandle(forWritingTo: someFile)
//handle.seekToEndOfFile() // This is for appending
handle.seek(toFileOffset: 0) // Me trying to seek to the beginning of file
handle.write(content)
handle.closeFile()
It seems like my content is being written at the beginning of the file, but it just replaces the existing consent as well... Thanks!
One reasonable solution is to write the new content to a temporary file, then append the existing contents to the end of the temporary file. Then move the temporary file over the old file.
When you seek to a point in an existing file and then perform a write, the existing contents are overwritten from that point. This is why your current approach fails.
In general, most file systems don't have built-in support for prepending data to files. Likewise, most file I/O APIs don't either.
In order to prepend data, you first have to shift all of the existing data further along the file to make room for the new data at the beginning. You typically do this by starting near the end, reading a chunk of data, writing that data to the original position plus the length of data you hope to eventually prepend, and then repeating with the next chunk closer to the beginning of the file. In this way, you gradually shift everything down. Only after you've done all of that can you safely write the new data at the beginning of the file safely.
Frankly, if there's any way to avoid this, you should try to. The performance is likely to be terrible if the file is large and/or you're doing it frequently.
I'm a newbie to Lucene. In the course of understanding it, I could successfully index the files in a directory and I did a basic lucene search to get the list of files in which a particular word is present.
Now I'm trying to extract the sentence from a file in which the search word is present.
I've searched a lot but couldn't figure out.
Regards.
Thank you all for your response.
I was trying to extract index of sentences in the directory of files but not the "relavent/best text/fragment".
Here is how I solved the problem:
Using "two-level indexing" --> first index the files in a directory & then index sentences in each file. This made my job pretty easier & faster.
Anyways, thanks again for the help :)
You're looking for the method
org.apache.lucene.search.highlight.Highlighter.getBestFragment
Such method gets in input the set of tokens generated analyzing the original text, and returns in output the most relevant text fragments. Please remember to trim the fragments if they are too big.
I am looking for ways to read in a PDF file with SAS. Apparently this is not basic functionality and there is very little to be found on the internet. (Let alone that google is not easy with PDF in you search giving you also links to PDF documents that go about other things.)
The only things that can be found, are people looking for ways to import data into datasets from a PDF. For me, that is not even necesarry. I would like to be able to read the contents of the PDF file in one big character variable. If possible, it would even be better to be able to read in the file's binary data.
Is this possible with SAS and how? (I got it to work in Access VBA, but can't find any similar ways in SAS.)
(In the end, the purpose is to convert this to base64 and put that base64-string into an XML document.)
You probably will not be able to read the entire file into one character variable since the maximum size of a character variable is around 33 KB. A simple way to read in one line at a time, though, is something like the following:
%let pdfFileName = Test.pdf;
%let lineSize = 2000;
data base;
format text_line $&lineSize..;
infile "&pdfFileName" lrecl=&lineSize;
input text_line $;
run;
This requires that you have a general idea of the maximum record length ahead of time, but you could write additional code to determine the maximum record size prior to reading in the file. In this example each line of text is read into one character variable named "text_line." From there, you could use a RETAIN statement or double trailers (##) in the INPUT line to process multiple lines at a time. The SAS web-site has plenty of documentation on how to read and process text from various types of input files.