Make selecting and writing into file faster in Groovy - optimization

Question

I have this groovy code which is pretty simple:
sql.eachRow("""SELECT
LOOP_ID,
FLD_1,
... 20 more fields
FLD_20
FROM MY_TABLE ORDER BY LOOP_ID"""){ res->
if(oldLoopId != res.loop_id){
oldLoopId = res.loop_id
fileToWrite = new File("MYNAME_${type}_${res.loop_id}_${today.format('YYYYmmDDhhMM')}.txt")
fileToWrite.append("20 fields header\n")
}
fileToWrite.append("${res.FLD_1}|${res.FLD_2}| ... |${res.FLD_20}\n");
}
}
it selects things from a table and writes to the database. For each new loop_id it creates a new file. The problem is it takes about 15 minutes to write 50mb file.
How do I make it faster?

Answer 1

Try writing to a BufferedWriter instead of using append directly:
sql.eachRow("""SELECT
LOOP_ID,
FLD_1,
... 20 more fields
FLD_20
FROM MY_TABLE ORDER BY LOOP_ID""") { res ->
def writer
if (oldLoopId != res.loop_id) {
oldLoopId = res.loop_id
def fileToWrite = new File("MYNAME_${type}_${res.loop_id}_${today.format('YYYYmmDDhhMM')}.txt")
if (writer != null) { writer.close() }
writer = fileToWrite.newWriter()
writer.append("20 fields header\n")
}
writer.append("${res.FLD_1}|${res.FLD_2}| ... |${res.FLD_20}\n");
File::withWriter automagically close the resources, but to use it you'd need to do way more trips do DB, getting all the loop_id and fetching the data for each one.
The following script:
f=new File("b.txt")
f.write ""
(10 * 1024 * 1024).times { f.append "b" }
Execution:
$ time groovy Appends.groovy
real 1m9.217s
user 0m45.375s
sys 0m31.902s
And using a BufferedWriter:
w = new File("/tmp/a.txt").newWriter()
(10 * 1024 * 1024).times { w.write "a" }
Execution:
$ time groovy Writes.groovy
real 0m1.774s
user 0m1.688s
sys 0m0.872s