I have a Flow<FileChunk> which I need to append into a File. My naive approach would be:
File(fileame).bufferedWriter().use { writer ->
withContext(IO) {
requests.collect {
writer.write(it.fileChunk.content.toStringUtf8())
}
}
}
Unfortuntately, IntelliJ marks the "write" as "blocking". What is the right way of achieving what I want? Thanks!
You actually need a CoroutineScope to launch a suspend function. In this case I'd go with these two options:
Create or use an available coroutine scope. For ex:
File(fileame).bufferedWriter().use { writer ->
CoroutineScope(Dispatchers.IO).launch{
requests.collect {
writer.write(it.fileChunk.content.toStringUtf8())
}
}
}
Or if you're inside a function, you could make it a suspend fun, and handle the coroutine scope on the site of the function calling
Related
How to execute batches all at once in a parallel way.
I have the following code:
val batches = list
.windowed(100, 100, true)
.map { Process(it) }
Is there a way to parallel execute all patches each in a coroutine at once?
Is there a better way than
withContext(Dispatchers.IO) {
batches.map { launch{}}.joinAll()
}
Your code sample is exactly how it should be done - simply use map() or foreach() with async() or launch().
Note that withContext() automatically waits for all children, so if you don't need any result from each batch (it doesn't seem so), you don't need to use joinAll() and map():
withContext(Dispatchers.IO) {
batches.foreach { launch{} }
}
I'm fairly new to kotlin coroutines, and I have what I think is a somewhat esoteric use case related to how runBlocking and coroutine contexts interact.
To start with, a simple example. Let's say I've got a dead simple context element. Nothing fancy.
class ExampleContext(val s: String) : AbstractCoroutineContextElement(Key) {
companion object Key : CoroutineContext.Key<ExampleContext>
}
When I run these examples, they behave exactly the way I'd expect them to:
runBlocking(ExampleContext("foo")) {
println(coroutineContext[ExampleContext.Key]?.s) // prints "foo
}
runBlocking(ExampleContext("foo")) {
launch {
println(coroutineContext[ExampleContext.Key]?.s) // prints "foo"
}
}
runBlocking(ExampleContext("foo")) {
launch(ExampleContext("bar")) {
println(coroutineContext[ExampleContext.Key]?.s) // prints "bar"
}
}
When I do this it prints null (as I would expect it to, because it runBlocking defaults to having EmptyContext in its constructor):
runBlocking(ExampleContext("foo")) {
runBlocking {
println(coroutineContext[ExampleContext.Key]?.s) // prints null
}
}
So here's my conundrum. The docs (and all the guidance I've found on the web) basically say don't do this: runBlocking is supposed to be run at the outermost layer of the coroutine logic and that's it. No nesting. What I'm working on is a library that needs to populate some context for access inside code that I don't own that gets called later (basically, you can think of it like an interceptor). The rough pseudocode looks a little like this:
class MyLibrary(otherPeoplesLogic: OtherPeoplesBusinessLogic) {
fun <IN, OUT> execute(input: IN): OUT {
... do my library's thing, including adding in a custom context element ...
try {
return otherPeoplesLogic.execute(input)
} finally {
... do my library's cleanup ...
}
}
}
To support coroutines in OtherPeoplesBusinessLogic, all I'd really have to do is add runBlocking like this:
class MyLibrary(otherPeoplesLogic: OtherPeoplesBusinessLogic) {
fun <IN, OUT> execute(input: IN): OUT {
... do my library's thing ...
runBlocking(myCustomContext) {
try {
return otherPeoplesLogic.execute(input)
} finally {
... do my library's cleanup ...
}
}
}
}
So long as all OtherPeoplesBusinessLogic::execute does is launch/async/etc, everything is fine: myCustomContext will be accessible. What I'm worried about is what happens if OtherPeoplesBusinessLogic::execute (which I'm not in control of) misbehaves and does its own runBlocking call with no context argument passed at all: what I think will happen is that myCustomContext will just silently get dropped like the example above. Not good, because it needs to be accessible.
Phew. A lot of explanation. Thanks for bearing with me. :)
So my ultimate question here is this: is there anything I can do (outside of scolding the users of my library to not call runBlocking) to prevent an accidental nested runBlocking call from dropping my context? Or am I just out of luck here and should scrap the whole idea?
I tried reading the docs, but it just doesn't make sense to me.
I need to make three calls to an external webserivce and log the result of each after it returns. Each webservice call is indepdentant of the responses of the others. Done synchronously, it looks like this:
fun makeWebserviceCalls(){
callOne()
callTwo()
callThree()
}
fun callOne(){
// make webservice call
// log result
}
fun callTwo(){
// make webservice call
// log result
}
fun callThree(){
// make webservice call
// log result
}
Now I just need to do that in parallel. It shouldn't be that hard, but it's just not making sense to me.
I've tried:
fun makeWebserviceCalls(){
callOne()
callTwo()
callThree()
}
fun callOne(){
launch{
// make webservice call
// log result
}
}
but that doesn't compile.
I've tried:
fun makeWebserviceCalls(){
runBlocking{
callOne()
callTwo()
callThree()
}
}
suspend fun callOne(){
launch{
// make webservice call
// log result
}
}
but that doesn't compile.
I've tried:
fun makeWebserviceCalls(){
runBlocking{
callOne()
callTwo()
callThree()
}
}
suspend fun callOne(){
withContext(Dispatchers.IO){
// make webservice call
// log result
}
}
but this can't be right, because withContext is used when you need a result returned, which I don't.
What's the right way to do what I'm trying to do?
The primary goal of Coroutines is to do efficient asynchronous operations, which is different than doing concurrent operations. Here’s an example from the official docs:
val client = HttpClient()
//Running in the main thread, start a `get` call
client.get<String>("https://example.com/some/rest/call")
//The get call will suspend and let other work happen in the main thread, and resume when the get call completes
All of the above happen on the main thread, no separate threads are spawned. A future versions of Coroutines will provide OOTB support for concurrent Coroutines on multiple threads, but the main branch has no support for that yet.
A suspend function can be executed on a different thread:
suspend fun differentThread() = withContext(Dispatchers.Default){
println("Different thread")
}
Of course, if you call the suspend function from a regular function, you’ll need to do it within runBlocking.
Other alternatives are proposed here: https://kotlinlang.org/docs/mobile/concurrency-and-coroutines.html#alternatives-to-kotlinx-coroutines. For simple use cases, CoroutineWorker is a good option.
For more details, see the official docs: https://kotlinlang.org/docs/mobile/concurrency-and-coroutines.html
If you need your makeWebserviceCalls() to wait until all requests to finish:
runBlocking(Dispatchers.IO) {
launch { callOne() }
launch { callTwo() }
launch { callThree() }
}
If you need to start them in the background and return immediately:
GlobalScope.launch(Dispatchers.IO) {
launch { callOne() }
launch { callTwo() }
launch { callThree() }
}
But you need to understand that this is not the usual way of using coroutines. Normally, your makeWebserviceCalls() function and all call* functions would be suspend functions which makes them more coroutines-friendly.
You start parallel execution from out of the coroutines context, also parallel blocks of code are blocking and in such a case I'm not sure if it makes sense to use coroutines at all. You can just start 3 background threads, it will be effectively almost the same.
This is what I thought:
When using coroutines you go piling up async ops and once you are done with synchronous op..call them in FIFO order..but that's not always true
In this example you get what I expected:
fun main() = runBlocking {
launch {
println("1")
}
launch {
println("2")
}
println("0")
}
Also here(with nested launch):
fun main() = runBlocking {
launch {
println("1")
}
launch {
launch {
println("3")
}
println("2")
}
println("0")
}
Now in this example with a scope builder and creating another "pile"(not the real term) the order changes but still..you get as expected
fun main() = runBlocking {
launch {
println("2")
}
// replacing launch
coroutineScope {
println("0")
}
println("1")
}
Finally..the reason of this question..example 2 with scope builder:
fun main() = runBlocking {
launch {
println("3")
}
coroutineScope {
launch {
println("1")
}
println("0")
}
println("2")
}
I get this:
0
3
1
2
Why??
Was my assumption wrong and that's not how coroutines work
If so..then how should I determine the correct order when coding
edited: I've tried running the same code on different machines and different platforms but always got the same result..also tried more complicated nesting to prove non-variability of results
And digging the documentation found that coroutines are just code transformation(as I initially thought)
Remember that even if the like to call them 'light-weight' threads they run in a single 'real' thread(note: without newSingleThreadContext)
Thus I chose to believe execution order is pre-established at compile-time and not decided at runtime
After all..I still can't anticipate the order..and that's what I want
Don't assume coroutines will be run in a specific order, the runtime will decide what's best to run when and in what order. What you may be interested in that will help is the kotlinx.coroutines documentation. It does a great job of explaining how they work and also provides some handy abstractions to help managing coroutines make more sense. I personally recommend checking out channels, jobs, and Deferred (async/await).
For example, if I wanted things done in a certain order by number, I'd use channels to ensure things arrived in the order I wanted.
runBlocking {
val channel = Channel<Int>()
launch {
for (x in 0..5) channel.send(x * x)
channel.close()
}
for (msg in channel) {
// Pretend we're doing some work with channel results
println("Message: $msg")
}
}
Hopefully that can give you more context or what coroutines are and what they're good for.
I want to iterate over a sequence of objects and return the first non-null of an async call.
The point is to perform some kind of async operation that might fail, and I have a series of fallbacks that I want to try in order, one after the other (i.e. lazily / not in parallel).
I've tried to do something similar to what I'd do if it were a sync call:
// ccs: List<CurrencyConverter>
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asSequence()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
IntelliJ complains:
Suspension functions can only be called within coroutine body
Edit: To clarify, this works as expected if mapping on a List, but I want to see how I'd do this on a sequence.
So I guess this is because the map lambda isn't suspended? But I'm not sure how to actually do that. I tried a bunch of different ways but none seemed to work. I couldn't find any examples.
If I re-write this in a more procedural style using a for loop with an async block, I can get it working:
override suspend fun getExchangeRateAsync(from: String, to: String) {
for (cc in ccs) {
var res: BigDecimal? = async {
cc.getExchangeRateAsync(from, to)
}.await()
if (res != null) {
return res
}
}
throw CurrencyConverterException()
}
You are getting an error, because Sequence is lazy by default and it's map isn't an inline function, so it's scope isn't defined
You can avoid using Sequence by creating a list of lazy coroutines
// ccs: List<CurrencyConverter>
suspend fun getExchangeRateAsync(from: String, to: String) =
ccs
.map { async(start = CoroutineStart.LAZY) { it.getExchangeRateAsync(from, to) } }
.firstOrNull { it.await() != null }
?.getCompleted() ?: throw Exception()
This doesn't give any errors and seems to be working. But I'm not sure it's an idiomatic way
I would suggest replacing Sequence with Flow. Flow api and behavior is pretty much same as for Sequence, but with suspending options.
https://kotlinlang.org/docs/reference/coroutines/flow.html
Code:
override suspend fun getExchangeRateAsync(from: String, to: String) =
ccs.asFlow()
.map { it.getExchangeRateAsync(from, to) }
.firstOrNull { it != null }
?: throw CurrencyConverterException()
FWIW, I found the suggestion in How to asynchronously map over sequence to be very intuitive. The code at https://github.com/Kotlin/kotlin-coroutines-examples/blob/master/examples/suspendingSequence/suspendingSequence.kt defines SuspendingIterator which allows next() to suspend, then builds SuspendingSequence on top of it. Unfortunately, you need to duplicate extension functions like flatMap(), filter(), etc. since SuspendingSequence can't be related to Sequence, but I did this and am much happier with the result than using a Channel.