In my Kotlin project in folder src/resources/ I has file pairs_ids.txt.
This is a property file:
key=value
The count of all lines are 1389.
Here code that read content of this file line by line.
open class AppStarter : Application<AppConfig>() {
override fun getName() = "stats"
override fun run(configuration: AppConfig?, environment: Environment?) {
val logger = LoggerFactory.getLogger(this::class.java)
val inputStream = javaClass.getResourceAsStream("/pairs_ids.txt")
val isr = InputStreamReader(inputStream)
val br = BufferedReader(isr)
for (line in br.lines()) {
logger.info("current_line = " + line)
}
br.close()
isr.close()
inputStream.close()
}
}
fun main(args: Array<String>) {
AppStarter().run(*args)
}
The problem is that count of current_line is every time different.
Start project - the count of current_line is 803.
Start again project - the count of current_line is 1140.
Why every time the count is different and not equal to 1389?
Kotlin has some brilliant extension methods to easily deal with streams, reading lines and text and such.
Try this:
open class AppStarter : Application<AppConfig>() {
override fun getName() = "stats"
override fun run(configuration: AppConfig?, environment: Environment?) {
val logger = LoggerFactory.getLogger(this::class.java)
javaClass.getResourceAsStream("/pairs_ids.txt").bufferedReader().use { reader -> reader.readLines() }.forEach { line -> logger.info(line) }
}
}
fun main(args: Array<String>) {
AppStarter().run(*args)
}
When playing with input/output streams, use Kotlin's use extension methods, and do all of your processing inside of the use block.
This will handle all opening and closing of the streams so that there are no leaks, or forgetting to close/flush etc.
Given an InputStream, you can use forEachLine to read each line separately:
inputStream.bufferedReader().use {
it.forEachLine {
println(it)
}
}
Note: You should use use which is a convenient way to make sure the stream is closed once the reading is done (as user8159708 already suggested).
Related
I have a data structure which has members that are not thread safe and the caller needs to lock the resource for reading and writing as appropriate. Here's a minimal code sample:
class ExampleResource : LockableProjectItem {
override val readWriteLock: ReadWriteLock = ReentrantReadWriteLock()
#RequiresReadLock
val nonThreadSafeMember: String = ""
}
interface LockableProjectItem {
val readWriteLock: ReadWriteLock
}
fun <T : LockableProjectItem, Out> T.readLock(block: T.() -> Out): Out {
try {
readWriteLock.readLock().lock()
return block(this)
} finally {
readWriteLock.readLock().unlock()
}
}
fun <T : LockableProjectItem, Out> T.writeLock(block: T.() -> Out): Out {
try {
readWriteLock.writeLock().lock()
return block(this)
} finally {
readWriteLock.writeLock().unlock()
}
}
annotation class RequiresReadLock
A call ExampleResource.nonThreadSafeMember might then look like this:
val resource = ExampleResource()
val readResult = resource.readLock { nonThreadSafeMember }
To make sure that the caller is aware that the resource needs to be locked, I would like the IDE to issue a warning for any members that are annotated with #RequiresReadLock and are not surrounded with a readLock block. Is there any way to do this in IntelliJ without writing a custom plugin for the IDE?
I think this is sort of a hack, but using context receivers might work. I don't think they are intended to be used in this way though.
You can declare a dummy object to act as the context receiver, and add that as a context receiver to the property:
object ReadLock
class ExampleResource : LockableProjectItem {
override val readWriteLock: ReadWriteLock = ReentrantReadWriteLock()
// properties with context receivers cannot have a backing field, so we need to explicitly declare this
private val nonThreadSafeMemberField: String = ""
context(ReadLock)
val nonThreadSafeMember: String
get() = nonThreadSafeMemberField
}
Then in readLock, you pass the object:
fun <T : LockableProjectItem, Out> T.readLock(block: context(ReadLock) T.() -> Out): Out {
try {
readWriteLock.readLock().lock()
return block(ReadLock, this)
} finally {
readWriteLock.readLock().unlock()
}
}
Notes:
This will give you an error if you try to access nonThreadSafeMember without the context receiver:
val resource = ExampleResource()
val readResult = resource.nonThreadSafeMember //error
You can still access nonThreadSafeMember without acquiring a read lock by doing e.g.
with(ReadLock) { // with(ReadLock) doesn't acquire the lock, just gets the context receiver
resource.nonThreadSafeMember // no error
}
But it's way harder to accidentally write something like this, which I think is what you are trying to prevent.
If you call another function inside readLock, and you want to access nonThreadSafeMember inside that function, you should mark that function with context(ReadLock) too. e.g.
fun main() {
val resource = ExampleResource()
val readResult = resource.readLock {
foo(this)
}
}
context(ReadLock)
fun foo(x: ExampleResource) {
x.nonThreadSafeMember
}
The context receiver is propagated through.
I have a SAX parser reading an XML file (specifically, a .xlsx file) and returning the content as a list of Row objects: it's roughly like this
fun readExcelContent(data: InputStream) {
val pkg = OPCPackage.open(file)
val reader = XSSFReader(pkg)
val sst = reader.sharedStringsTable
val parser = XMLHelper.newXMLReader()
val handler = ExcelSheetHandler(sst)
parser.contentHandler = handler
val sheet = reader.sheetsData.next()
val source = InputSource(sheet)
parser.parse(source)
return handler.content
}
Where ExcelSheetHandler is a class that extends DefaultHandler and takes care of filling up a list:
class ExcelSheetHandler(sst: SharedStringsTable): DefaultHandler() {
private val content = mutableListOf<Row>()
#Throws(SAXException::class)
override fun endElement(uri: String?, localName: String?, name: String) {
// If it's the end of a content element, add a row to content
}
}
It's basically a slight modification of the event model example in the Apache POI howto.
I was wondering if there was a way to have readExcelContent return an asynchronous object, such as a flow, and emit the rows to its client as soon as they are read rather than having to wait for the whole file to be processed.
I'd prefer kotlinx.coroutines.Channel over kotlinx.coroutines.Flow for this use case since this is a hot stream of data triggered by the parse() method. Here's what Kotlin Language Guide states.
Flows are cold streams similar to sequences — the code inside a flow
builder does not run until the flow is collected
Here's a quick implementation that you can try.
class ExcelSheetHandler : DefaultHandler() {
private val scope = CoroutineScope(Dispatchers.Default)
private val rows = Channel<Row>()
override fun endDocument() {
// To avoid suspending forever!
rows.close()
}
#Throws(SAXException::class)
override fun endElement(uri: String?, localName: String?, name: String) {
readRow(uri, localName, name)
}
private fun readRow(uri: String?, localName: String?, name: String) = runBlocking {
// If it's the end of a content element, add a row to content
rows.send(row)
}
// Client code - if it needs to be somewhere else
// you can expose a reference to Channel object
private fun processRows() = scope.launch {
for(row in rows) {
// Do something
println(row)
}
}
}
In my Kotlin project in folder src/resources/ I has file pairs_ids.txt.
Here code:
This is a property file:
key=value
The count of all lines are 1389.
Here code that read content of this file line by line.
import org.slf4j.LoggerFactory
private val logger = LoggerFactory.getLogger("Exchange")
private var allSymbolsIDsMap: Map<String, String> = mapOf()
val pairsIDs = getResourceAsText("/pairs_ids.txt")
allSymbolsIDsMap = pairsIDs.split(",").associate {
val (left, right) = it.split("=")
left to right.toString()
}
logger.info("allSymbolsIDsMap_size = " + allSymbolsIDsMap.size)
var countAllSymbolHandler = 0
for ((key, value) in allSymbolsIDsMap) {
countAllSymbolHandler++
logger.info("countAllSymbolHandler = $countAllSymbolHandler")
}
private fun getResourceAsText(path: String): String {
return object {}.javaClass.getResourceAsStream(path).bufferedReader().use { it.readText() }
}
Result:
Start project:
allSymbolsIDsMap_size = 1389
Start project - "countAllSymbolHandler =" print 1113 times
Again start project:
allSymbolsIDsMap_size = 1389
"countAllSymbolHandler =" print 242 times
If replace logger.info by simple println then success work. The count is always 1389.
Why loop (for) not work correct with logger?
Try:
override fun run(configuration: AppConfig?, environment: Environment?) {
val logger = LoggerFactory.getLogger(this::class.java)
javaClass.getResourceAsStream("/pairs_ids.txt").bufferedReader().use { reader -> reader.readLines() }.forEach { line -> logger.info(line) }
}
When playing with input/output streams, use Kotlin's use extension methods, and do all of your processing inside of the use block.
This will handle all opening and closing of the streams so that there are no leaks, or forgetting to close/flush etc.
I want to write a Spek test in Kotlin.
How to read an HTML file from the src/test/resources folder?
class MySpec : Spek(
{
describe("blah blah") {
given("blah blah") {
var fileContent: String = ""
beforeEachTest {
// How to read the file.html in src/test/resources/html/
fileContent = ...
}
it("should blah blah") {
...
}
}
}
}
)
val fileContent = MySpec::class.java.getResource("/html/file.html").readText()
No idea why this is so hard, but the simplest way I've found (without having to refer to a particular class) is:
fun getResourceAsText(path: String): String? =
object {}.javaClass.getResource(path)?.readText()
It returns null if no resource with this name is found (as documented).
And then passing in an absolute URL, e.g.
val html = getResourceAsText("/www/index.html")!!
another slightly different solution:
#Test
fun basicTest() {
"/html/file.html".asResource {
// test on `it` here...
println(it)
}
}
fun String.asResource(work: (String) -> Unit) {
val content = this.javaClass::class.java.getResource(this).readText()
work(content)
}
A slightly different solution:
class MySpec : Spek({
describe("blah blah") {
given("blah blah") {
var fileContent = ""
beforeEachTest {
html = this.javaClass.getResource("/html/file.html").readText()
}
it("should blah blah") {
...
}
}
}
})
Kotlin + Spring way:
#Autowired
private lateinit var resourceLoader: ResourceLoader
fun load() {
val html = resourceLoader.getResource("classpath:html/file.html").file
.readText(charset = Charsets.UTF_8)
}
Using Google Guava library Resources class:
import com.google.common.io.Resources;
val fileContent: String = Resources.getResource("/html/file.html").readText()
private fun loadResource(file: String) = {}::class.java.getResource(file).readText()
val fileContent = javaClass.getResource("/html/file.html").readText()
This is the way that I prefer to do it:
fun getResourceText(path: String): String {
return File(ClassLoader.getSystemResource(path).file).readText()
}
this top-level kotlin function will do the job in any case
fun loadResource(path: String): URL {
return Thread.currentThread().contextClassLoader.getResource(path)
}
or if you want a more robust function
fun loadResource(path: String): URL {
val resource = Thread.currentThread().contextClassLoader.getResource(path)
requireNotNull(resource) { "Resource $path not found" }
return resource
}
FYI: In all the above cases. getResource() is unsafe way of using nullable.
Haven't tried locally but I prefer this way:
fun readFile(resourcePath: String) = String::class.java.getResource(resourcePath)?.readText() ?: "<handle default. or handle custom exception>"
Or even as custom datatype function
private fun String.asResource() = this::class.java.getResource(resourcePath)?.readText() ?: "<handle default. or handle custom exception>"
and then you can call directly on path like:
// For suppose
val path = "/src/test/resources"
val content = path.asResource()
I prefer reading resources in this way:
object {}.javaClass.getResourceAsStream("/html/file.html")?.use { it.reader(Charsets.UTF_8).readText() }
Explenation:
getResourceAsStream instead getResource. The resource on classpath can be basically anywhere. e.g. packed inside another .jar file.
In these situations accessing resource via URL class returned from getResource method will fail. But accessing via method getResourceAsStream works in every situation.
object {} - This is not nice syntax, but it is not dependent on name of your class MyClass and works even in static (compenion object) block.
use to close stream - in most cases it is not necessary, but there can be some special classloaders, which may need it.
reader(Charsets.UTF_8) - UTF_8 is default encoding, but I prefer to be explicit. If you will encode your resource files in other encoding e.g. ISO-8859-2 you will not overlook it.
Another variation that handles null resource in place:
val content = object {}.javaClass
.getResource("/html/file.html")
?.let(URL::readText)
?: error("Cannot open/find the file")
// ?: "default text" // Instead of error()
You might find the File class useful:
import java.io.File
fun main(args: Array<String>) {
val content = File("src/main/resources/input.txt").readText()
print(content)
}
I can't seem to find this question yet, but what is the simplest, most-idiomatic way of opening/creating a file, writing to it, and then closing it? Looking at the kotlin.io reference and the Java documentation I managed to get this:
fun write() {
val writer = PrintWriter("file.txt") // java.io.PrintWriter
for ((member, originalInput) in history) { // history: Map<Member, String>
writer.append("$member, $originalInput\n")
}
writer.close()
}
This works, but I was wondering if there was a "proper" Kotlin way of doing this?
A bit more idiomatic. For PrintWriter, this example:
File("somefile.txt").printWriter().use { out ->
history.forEach {
out.println("${it.key}, ${it.value}")
}
}
The for loop, or forEach depends on your style. No reason to use append(x) since that is basically write(x.toString()) and you already give it a string. And println(x) basically does write(x) after converting a null to "null". And println() does the correct line ending.
If you are using data classes of Kotlin, they can already be output because they have a nice toString() method already.
Also, in this case if you wanted to use BufferedWriter it would produce the same results:
File("somefile.txt").bufferedWriter().use { out ->
history.forEach {
out.write("${it.key}, ${it.value}\n")
}
}
Also you can use out.newLine() instead of \n if you want it to be correct for the current operating system in which it is running. And if you were doing that all the time, you would likely create an extension function:
fun BufferedWriter.writeLn(line: String) {
this.write(line)
this.newLine()
}
And then use that instead:
File("somefile.txt").bufferedWriter().use { out ->
history.forEach {
out.writeLn("${it.key}, ${it.value}")
}
}
And that's how Kotlin rolls. Change things in API's to make them how you want them to be.
Wildly different flavours for this are in another answer: https://stackoverflow.com/a/35462184/3679676
Other fun variations so you can see the power of Kotlin:
A quick version by creating the string to write all at once:
File("somefile.txt").writeText(history.entries.joinToString("\n") { "${it.key}, ${it.value}" })
// or just use the toString() method without transform:
File("somefile.txt").writeText(x.entries.joinToString("\n"))
Or assuming you might do other functional things like filter lines or take only the first 100, etc. You could go this route:
File("somefile.txt").printWriter().use { out ->
history.map { "${it.key}, ${it.value}" }
.filter { ... }
.take(100)
.forEach { out.println(it) }
}
Or given an Iterable, allow writing it to a file using a transform to a string, by creating extension functions (similar to writeText() version above, but streams the content instead of materializing a big string first):
fun <T: Any> Iterable<T>.toFile(output: File, transform: (T)->String = {it.toString()}) {
output.bufferedWriter().use { out ->
this.map(transform).forEach { out.write(it); out.newLine() }
}
}
fun <T: Any> Iterable<T>.toFile(outputFilename: String, transform: (T)->String = {it.toString()}) {
this.toFile(File(outputFilename), transform)
}
used as any of these:
history.entries.toFile(File("somefile.txt")) { "${it.key}, ${it.value}" }
history.entries.toFile("somefile.txt") { "${it.key}, ${it.value}" }
or use default toString() on each item:
history.entries.toFile(File("somefile.txt"))
history.entries.toFile("somefile.txt")
Or given a File, allow filling it from an Iterable, by creating this extension function:
fun <T: Any> File.fillWith(things: Iterable<T>, transform: (T)->String = {it.toString()}) {
this.bufferedWriter().use { out ->
things.map(transform).forEach { out.write(it); out.newLine() }
}
}
with usage of:
File("somefile.txt").fillWith(history.entries) { "${it.key}, ${it.value}" }
or use default toString() on each item:
File("somefile.txt").fillWith(history.entries)
which if you had the other toFile extension already, you could rewrite having one extension call the other:
fun <T: Any> File.fillWith(things: Iterable<T>, transform: (T)->String = {it.toString()}) {
things.toFile(this, transform)
}
It mostly looks ok to me. The only thing different I would do is use the "use" extension defined in ReadWrite to auto close the writer.
PrintWriter("file.txt").use {
for ((member, originalInput) in history) { // history: Map<Member, String>
it.append("$member, $originalInput\n")
}
}
At the very minimum, you could use:
FileWriter(filename).use { it.write(text) }
FileWriter is a convenience class for writing character files (provided by Java, and hence available in Kotlin). It extends Closeable, and hence can be used by Kotlin's ".use" extension method.
The .use extension method automatically closes the calling object once the block exits, thus providing an idiomatic way to close the file after it's written.
Some Kotlin magic allows to omit referencing the stream on each read or write call:
fun <T : Closeable, R> T.useWith(block: T.() -> R): R = use { with(it, block) }
File("a.in").bufferedReader().useWith {
File("a.out").printWriter().useWith {
val (a, b) = readLine()!!.split(' ').map(String::toInt)
println(a + b)
}
}
Scanner(File("b.in")).useWith {
PrintWriter("b.out").useWith {
val a = nextInt()
val b = nextInt()
println(a + b)
}
}
try{
val fileWriter = FileWriter("test.txt", true)
fileWriter.write(string+ "\n")
fileWriter.close()
} catch (exception: Exception){
println(exception.message)
}
Example as easy
val path = context!!.filesDir.absolutePath // => /data/user/0/com.example.test/files
File("$path/filename.txt").writeText("hello")
File(requireContext().filesDir, "TodayTaskListChange.txt").writeText("write your test here...")