I wanted to learn more about Kotlin coroutines and was wondering if mimicking the Go merge function example can be done in a more idiomatic way in Kotlin?
As a newbie I can just translate the merge function directly into Kotlin as follows:
fun <T> CoroutineScope.merge(vararg channels: ReceiveChannel<T>) : Channel<T> {
val outgoing = Channel<T>()
val jobs = channels.map { channel ->
launch {
for ( message in channel) {
outgoing.send(message)
}
}
}
launch {
jobs.joinAll()
println("done merging")
outgoing.close()
}
return outgoing
}
This does however feel like I'm writing Go in Kotlin which is probably wrong.
I'd prefer no experimental API's and functions if possible, only code you can show your boss ;-)
Here's an example of it working https://pl.kotl.in/6ErnutS2X
Related
I'm building a repository to retrieve data from a Room database. The Room dao returns a Flow<List<ObjectDto>>. However, I need to convert this to Flow<List<Object>>. What is the right way to do this?
This is the solution I've come up with. I have a mapper extension ObjectDto.toObject(). However, this solution doesn't seem right to me. I have no experience with flows, but collecting and emitting again can't be correct, right?
override fun getObjects(): Flow<List<Object>> {
return flow {
objectDao.getObjects().collect { objectDtoList ->
val objects = objectDtoList.map { it.toObject() }
emit(objects) }
}
}
I also found several operators to use on flows without collecting them, but while some of them are able to change the type, I'm not sure how to change the type of a list using these operators.
I think Flow.map is what you're looking for
override fun getObjects(): Flow<List<Object>> =
objectDao.getObjects().map { objectDtoList ->
objectDtoList.map { it.toObject() }
}
}
I have a flow :
val myflow = kotlinx.coroutines.flow.flow<Message>{}
and want to emit values with function:
override suspend fun sendMessage(chat: Chat, message: Message) {
myflow.emit(message)
}
But compiler does not allow me to do this, is there any workarounds to solve this problem?
You can use StateFlow for such use case.
Here's a sample code.
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
val chatFlow = MutableStateFlow<String>("")
fun main() = runBlocking {
// Observe values
val job = launch {
chatFlow.collect {
print("$it ")
}
}
// Change values
arrayOf("Hey", "Hi", "Hello").forEach {
delay(100)
sendMessage(it)
}
delay(1000)
// Cancel running job
job.cancel()
job.join()
}
suspend fun sendMessage(message: String) {
chatFlow.value = message
}
You can test this code by running below snippet.
<iframe src="https://pl.kotl.in/DUBDfUnX3" style="width:600px;"></iframe>
The answer of Animesh Sahu is pretty much correct. You can also return a Channel as a flow (see consumeAsFlow or asFlow on a BroadcastChannel).
But there is also a thing called StateFlow currently in development by Kotlin team, which is, in part, meant to implement a similar behavior, although it is unknown when it is going to be ready.
EDIT: StateFlow and SharedFlow have been released as part of a stable API (https://blog.jetbrains.com/kotlin/2020/10/kotlinx-coroutines-1-4-0-introducing-stateflow-and-sharedflow/). These tools can and should be used when state management is required in an async execution context.
Use a SharedStateFlow it has got everything you need.
Initialization of your flow:
val myFlow = MutableSharedFlow<Message>()
and now it should just work as you were trying earlier with:
override suspend fun sendMessage(chat: Chat, message: Message) {
myFlow.emit(message)
}
Flow is self contained, once the block (lambda) inside the flow is executed the flow is over, you've to do operations inside and emit them from there.
Here is the similar github issue, says:
Afaik Flow is designed to be a self contained, replayable, cold stream, so emission from outside of it's own scope wouldn't be part of the contract. I think what you're looking for is a Channel.
And IMHO you're probably looking at the Channels, or specifically a ConflatedBroadcastChannel for multiple receivers. The difference between a normal channel and a broadcast channel is that multiple receivers can listen to a broadcast channel using openSubscription function which returns a ReceiveChannel associated with the BroadcastChannel.
This is what I thought:
When using coroutines you go piling up async ops and once you are done with synchronous op..call them in FIFO order..but that's not always true
In this example you get what I expected:
fun main() = runBlocking {
launch {
println("1")
}
launch {
println("2")
}
println("0")
}
Also here(with nested launch):
fun main() = runBlocking {
launch {
println("1")
}
launch {
launch {
println("3")
}
println("2")
}
println("0")
}
Now in this example with a scope builder and creating another "pile"(not the real term) the order changes but still..you get as expected
fun main() = runBlocking {
launch {
println("2")
}
// replacing launch
coroutineScope {
println("0")
}
println("1")
}
Finally..the reason of this question..example 2 with scope builder:
fun main() = runBlocking {
launch {
println("3")
}
coroutineScope {
launch {
println("1")
}
println("0")
}
println("2")
}
I get this:
0
3
1
2
Why??
Was my assumption wrong and that's not how coroutines work
If so..then how should I determine the correct order when coding
edited: I've tried running the same code on different machines and different platforms but always got the same result..also tried more complicated nesting to prove non-variability of results
And digging the documentation found that coroutines are just code transformation(as I initially thought)
Remember that even if the like to call them 'light-weight' threads they run in a single 'real' thread(note: without newSingleThreadContext)
Thus I chose to believe execution order is pre-established at compile-time and not decided at runtime
After all..I still can't anticipate the order..and that's what I want
Don't assume coroutines will be run in a specific order, the runtime will decide what's best to run when and in what order. What you may be interested in that will help is the kotlinx.coroutines documentation. It does a great job of explaining how they work and also provides some handy abstractions to help managing coroutines make more sense. I personally recommend checking out channels, jobs, and Deferred (async/await).
For example, if I wanted things done in a certain order by number, I'd use channels to ensure things arrived in the order I wanted.
runBlocking {
val channel = Channel<Int>()
launch {
for (x in 0..5) channel.send(x * x)
channel.close()
}
for (msg in channel) {
// Pretend we're doing some work with channel results
println("Message: $msg")
}
}
Hopefully that can give you more context or what coroutines are and what they're good for.
Is there any kotlin idiomatic way to read a file content's asynchronously? I couldn't find anything in documentation.
A least as of Java 7 (which is where Android is stuck), there isn't any API that would tap into the low-level async file IO support (like io_uring). There is a class called AsynchronousFileChannel, but, as its docs state,
An AsynchronousFileChannel is associated with a thread pool to which tasks are submitted to handle I/O events and dispatch to completion handlers that consume the results of I/O operations on the channel.
That makes it no better than the following, bog-standard Kotlin idiom:
launch {
val contents = withContext(Dispatchers.IO) {
FileInputStream("filename.txt").use { it.readBytes() }
}
processContents(contents)
}
go_on_with_other_stuff_while_file_is_loading()
This uses Kotlin's own dedicated IO thread pool and unblocks the UI thread. If you're on Android, that is your actual concern, anyway.
Java NIO Asynchronous Channel is the tool you want.
Check out this AsynchronousFileChannel.aRead extension function from coroutine example:
suspend fun AsynchronousFileChannel.aRead(buf: ByteBuffer): Int =
suspendCoroutine { cont ->
read(buf, 0L, Unit, object : CompletionHandler<Int, Unit> {
override fun completed(bytesRead: Int, attachment: Unit) {
cont.resume(bytesRead)
}
override fun failed(exception: Throwable, attachment: Unit) {
cont.resumeWithException(exception)
}
})
}
You just open an AsynchronousFileChannel then call this aRead() in a coroutine,
val channel = AsynchronousFileChannel.open(Paths.get(fileName))
try {
val buf = ByteBuffer.allocate(4096)
val bytesRead = channel.aRead(buf)
} finally {
channel.close()
}
It's an essential function, don't know why it is not part of coroutine-core lib.
javasync/RxIo uses Java NIO Asynchronous Channel to provide a non-blocking API to read and write a file content's asynchronously, including kotlin idiomatic way. Next you have two examples: one reading/writing in bulk through coroutines, and other iterating lines through an asynchronous Kotlin Flow:
suspend fun copyNio(from: String, to: String) {
val data = Path(from).readText() // suspension point
Path(to).writeText(data) // suspension point
}
fun printLinesFrom(filename: String) {
Path(filename)
.lines() // Flow<String>
.onEach(::println)
.collect() // block if you want to wait for completion
}
Disclaimer I am the author and main contributor of javasync/RxIo
I need to upload many files to S3, it would take hours to complete that job sequentially. That's exactly what Kotlin's new coroutines excels in, so I wanted to give them a first try instead of fiddling around again with some Thread-based execution service.
Here is my (simplified) code:
fun upload(superTiles: Map<Int, Map<Int, SuperTile>>) = runBlocking {
val s3 = AmazonS3ClientBuilder.standard().withRegion("eu-west-1").build()
for ((x, ys) in superTiles) {
val jobs = mutableListOf<Deferred<Any>>()
for ((y, superTile) in ys) {
val job = async(CommonPool) {
uploadTile(s3, x, y, superTile)
}
jobs.add(job)
}
jobs.map { it.await() }
}
}
suspend fun uploadTile(s3: AmazonS3, x: Int, y: Int, superTile: SuperTile) {
val json: String = "{}"
val key = "$s3Prefix/x4/$z/$x/$y.json"
s3.putObject(PutObjectRequest("my_bucket", ByteArrayInputStream(json.toByteArray()), metadata))
}
The problem: the code is still very slow and logging reveals that requests are still executed sequentially: a job is finished before the next one is created. Only in very few cases (1 out of 10) I see jobs running concurrently.
Why does the code not run much faster / concurrently? What can I do about it?
Kotlin coroutines excel when you work with asynchronous API, while AmazonS3.putObject API that you are using is an old-school blocking, synchronous API, so you get only as many concurrent uploads as the number of threads in the CommonPool that you are using. There is no value in marking your uploadTile function with suspend modified, because it does not use any suspending functions in its body.
The first step in getting more throughput in your upload task is to start using asynchronous API for that. I'd suggest to look at Amazon S3 TransferManager for that purse. See if that gets your problem solved first.
Kotlin coroutines are designed to help you to combine your async APIs into a easy-to-use logical workflows. For example, it is straightforward to adapt asynchronous API of TransferManager for use with coroutines by writing the following extension function:
suspend fun Upload.await(): UploadResult = suspendCancellableCoroutine { cont ->
addProgressListener {
if (isDone) {
// we know it should not actually wait when done
try { cont.resume(waitForUploadResult()) }
catch (e: Throwable) { cont.resumeWithException(e) }
}
}
cont.invokeOnCompletion { abort() }
}
This extension enables you to write very fluent code that works with TransferManager and you can rewrite your uploadTile function to work with TransferManager instead of working with blocking AmazonS3 interface:
suspend fun uploadTile(tm: TransferManager, x: Int, y: Int, superTile: SuperTile) {
val json: String = "{}"
val key = "$s3Prefix/x4/$z/$x/$y.json"
tm.upload(PutObjectRequest("my_bucket", ByteArrayInputStream(json.toByteArray()), metadata))
.await()
}
Notice, how this new version of uploadTile uses a suspending function await that was defined above.