Update Coroutines 1.3.0-RC
Working version:
#FlowPreview
suspend fun streamTest(): Flow<String> = channelFlow {
listener.onSomeResult { result ->
if (!isClosedForSend) {
offer(result)
}
}
awaitClose {
listener.unsubscribe()
}
}
Also checkout this Medium article by Roman Elizarov: Callbacks and Kotlin Flows
Original Question
I have a Flow emitting multiple Strings:
#FlowPreview
suspend fun streamTest(): Flow<String> = flowViaChannel { channel ->
listener.onSomeResult { result ->
if (!channel.isClosedForSend) {
channel.sendBlocking(result)
}
}
}
After some time I want to unsubscribe from the stream. Currently I do the following:
viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
this.cancel()
}
}
If the coroutine gets canceled, the stream is still executed. There is just no subscriber listening to new values. How can I unsubscribe and stop the stream function?
A solution is not to cancel the flow, but the scope it's launched in.
val job = scope.launch { flow.cancellable().collect { } }
job.cancel()
NOTE: You should call cancellable() before collect if you want your collector stop when Job is canceled.
You could use the takeWhile operator on Flow.
flow.takeWhile { it != "someString" }.collect { emittedValue ->
//Do stuff until predicate is false
}
For those willing to unsubscribe from the Flow within the Coroutine scope itself, this approach worked for me :
viewModelScope.launch {
beaconService.streamTest().collect {
//Do something then
this.coroutineContext.job.cancel()
}
}
With the current version of coroutines / Flows (1.2.x) I don't now a good solution. With onCompletion you will get informed when the flow stops, but you are then outside of the streamTest function and it will be hard to stop listening of new events.
beaconService.streamTest().onCompletion {
}.collect {
...
}
With the next version of coroutines (1.3.x) it will be really easy. The function flowViaChannel is deprecated in favor for channelFlow. This function allows you to wait for closing of the flow and do something in this moment, eg. remove listener:
channelFlow<String> {
println("Subscribe to listener")
awaitClose {
println("Unsubscribe from listener")
}
}
When a flow runs in couroutin scope, you can get a job from it to controls stop subscribe.
// Make member variable if you want.
var jobForCancel : Job? = null
// Begin collecting
jobForCancel = viewModelScope.launch {
beaconService.streamTest().collect {
Timber.i("stream value $it")
if(it == "someString")
// Here the coroutine gets canceled, but streamTest is still executed
// this.cancel() // Don't
}
}
// Call whenever to canceled
jobForCancel?.cancel()
For completeness, there is a newer version of the accepted answer. Instead of explicitly using the launch coroutine builder, we can use the launchIn method directly on the flow:
val job = flow.cancellable().launchIn(scope)
job.cancel()
Based on #Ronald answer this works great for testing when you need to make your Flow emits again.
val flow = MutableStateFlow(initialValue)
flow.take(n).collectIndexed { index, _ ->
if (index == something) {
flow.value = update
}
}
//your assertions
We have to know how many emissions in total we expect n and then we can use the index to know when to update the Flow so we can receive more emissions.
If you want to cancel only the subscription being inside it, you can do it like this:
viewModelScope.launch {
testScope.collect {
return#collect cancel()
}
}
There are two ways to do this that are by design from the Kotlin team:
As #Ronald pointed out in another comment:
Option 1: takeWhile { //predicate }
Cancel collection when the predicate is false. Final value will not be collected.
flow.takeWhile { value ->
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will never hit this
}
Option 2: transformWhile { //predicate }
When predicate is false, collect that value, then cancel
flow.transformWhile { value ->
emit(value)
value != "finalString"
}.collect { value ->
//Do stuff, but "finalString" will be the last value
}
https://github.com/Kotlin/kotlinx.coroutines/issues/2065
Related
I am observing inside a fragment the events of a sharedflow such as this:
myEvent.collectInLifeCycle(viewLifecycleOwner) { event ->
when (state) {
//check the event. The event emited form onStart is never reached here :(
}
}
Whereas in the viewmodel I have
private val _myEvent = MutableSharedFlow<MyEvent>()
val myEvent: SharedFlow<MyEvent> = _myEvent
fun loadData() =
viewModelScope.launch {
getDataUseCase
.safePrepare(onGenericError = { _event.emit(Event.Error(null)) })
.onStart { _event.emit(Event.Loading) }
.onEach { result ->
result.onSuccess { response ->
_event.emit(Event.Something)
}
}
.launchIn(viewModelScope)
}
So the problem is that only the Event.Something is the one being properly collected from the fragment, whereas _event.emit(Event.Loading) is not being collected... If I debug it goes to the onStart, but it is never called in the fragment.
Your SharedFlow needs to have a replay so that collectors always get at least the most recent value. Otherwise, if you emit to the Flow before the collector is registered, it will never see anything emitted. Do this:
private val _myEvent = MutableSharedFlow<MyEvent>(replay = 1)
Personally, unless I'm missing some detail here that would change my mind, I would simplify all your code to avoid having to manually call loadData(). Something like this but I'm guessing a bit because I don't know all your types and functions.
val myEvent: SharedFlow<MyEvent> = flow {
emit(Event.Loading)
emitAll(
getDataUseCase
.transform { result ->
result.onSuccess { response ->
emit(Event.Something)
}
}
.catch { error -> emit(Event.Error(null)) }
)
}.shareIn(viewModelScope, SharingStarted.Lazily, replay = 1)
emphasized textI am trying to use Kotlin Flow to process some data asynchronously and in parallel, and stream the responses to the client as they occur, as opposed to waiting until all the jobs are complete.
After unsuccessfully trying to just send the flow itself to the response, like this: call.respond(HttpStatusCode.OK, flow.toList())
... I tinkered for hours trying to figure it out, and came up with the following. Is this correct? It seems there should be a more idiomatic way of sending a Flow<MyData> as a response, like one can with a Flux<MyData> in Spring Boot.
Also, it seems that using the below method does not cancel the Flow when the HTTP request is cancelled, so how would one cancel it in Ktor?
data class MyData(val number: Int)
class MyService {
fun updateAllJobs(): Flow<MyData> =
flow {
buildList { repeat(10) { add(MyData(Random.nextInt())) } }
// Docs recommend using `onEach` to "delay" elements.
// However, if I delay here instead of in `map`, all elements are held
// and emitted at once at the very end of the cumulative delay.
// .onEach { delay(500) }
.map {
// I want to emit elements in a "stream" as each is computed.
delay(500)
emit(it)
}
}
}
fun Route.jobRouter() {
val service: MyService by inject() // injected with Koin
put("/jobs") {
val flow = service.updateAllJobs()
// Just using the default Jackson mapper for this example.
val mapper = jsonMapper { }
// `respondOutputStream` seems to be the only way to send a Flow as a stream.
call.respondOutputStream(ContentType.Application.Json, HttpStatusCode.OK) {
flow.collect {
println(it)
// The data does not stream without the newline and `flush()` call.
write((mapper.writeValueAsString(it) + "\n").toByteArray())
flush()
}
}
}
}
The best solution I was able to find (although I don't like it) is to use respondBytesWriter to write data to a response body channel. In the handler, a new job to collect the flow is launched to be able to cancel it if the channel is closed for writing (HTTP request is canceled):
fun Route.jobRouter(service: MyService) {
put("/jobs") {
val flow = service.updateAllJobs()
val mapper = jsonMapper {}
call.respondBytesWriter(contentType = ContentType.Application.Json) {
val job = launch {
flow.collect {
println(it)
try {
writeStringUtf8(mapper.writeValueAsString(it))
flush()
} catch (_: ChannelWriteException) {
cancel()
}
}
}
job.join()
}
}
}
I am trying to create a Flow that emits a value after a timeout, without cancelling the underlying coroutine. The idea is that the network call has X time to complete and emit a value and after that timeout has been reached, emit some initial value without cancelling the underlying work (eventually emitting the value from the network call, assuming it succeeds).
Something like this seems like it might work, but it would cancel the underlying coroutine when the timeout is reached. It also doesn't handle emitting some default value on timeout.
val someFlow = MutableStateFlow("someInitialValue")
val deferred = async {
val networkCallValue = someNetworkCall()
someFlow.emit(networkCallValue)
}
withTimeout(SOME_NUMBER_MILLIS) {
deferred.await()
}
I'd like to be able to emit the value returned by the network call at any point, and if the timeout is reached just emit some default value. How would I accomplish this with Flow/Coroutines?
One way to do this is with a simple select clause:
import kotlinx.coroutines.selects.*
val someFlow = MutableStateFlow("someInitialValue")
val deferred = async {
someFlow.value = someNetworkCall()
}
// await the first of the 2 things, without cancelling anything
select<Unit> {
deferred.onAwait {}
onTimeout(SOME_NUMBER_MILLIS) {
someFlow.value = someDefaultValue
}
}
You would have to watch out for race conditions though, if this runs on a multi-threaded dispatcher. If the async finished just after the timeout, there is a chance the default value overwrites the network response.
One way to prevent that, if you know the network can't return the same value as the initial value (and if no other coroutine is changing the state) is with the atomic update method:
val deferred = async {
val networkCallValue = someNetworkCall()
someFlow.update { networkCallValue }
}
// await the first of the 2 things, without cancelling anything
val initialValue = someFlow.value
select<Unit> {
deferred.onAwait {}
onTimeout(300) {
someFlow.update { current ->
if (current == initialValue) {
"someDefaultValue"
} else {
current // don't overwrite the network result
}
}
}
}
If you can't rely on comparisons of the state, you can protect access to the flow with a Mutex and a boolean:
val someFlow = MutableStateFlow("someInitialValue")
val mutex = Mutex()
var networkCallDone = false
val deferred = async {
val networkCallValue = someNetworkCall()
mutex.withLock {
someFlow.value = networkCallValue
networkCallDone = true
}
}
// await the first of the 2 things, without cancelling anything
select<Unit> {
deferred.onAwait {}
onTimeout(300) {
mutex.withLock {
if (!networkCallDone) {
someFlow.value = "someDefaultValue"
}
}
}
}
Probably the easiest way to solve the race condition is to use select() as in #Joffrey's answer. select() guarantees to execute only a single branch.
However, I believe mutating a shared flow concurrently complicates the situation and introduces another race condition that we need to solve. Instead, we can do it really very easily:
flow {
val network = async { someNetworkCall() }
select {
network.onAwait{ emit(it) }
onTimeout(1000) {
emit("initial")
emit(network.await())
}
}
}
There are no race conditions to handle. We have just two simple execution branches, depending on what happened first.
If we need a StateFlow then we can use stateIn() to convert a regular flow. Or we can use a MutableStateFlow as in the question, but mutate it only inside select(), similarly to above:
select {
network.onAwait{ someFlow.value = it }
onTimeout(1000) {
someFlow.value = "initial"
someFlow.value = network.await()
}
}
You can launch two coroutines simultaneously and cancel the Job of the first one, which responsible for emitting default value, in the second one:
val someFlow = MutableStateFlow("someInitialValue")
val firstJob = launch {
delay(SOME_NUMBER_MILLIS)
ensureActive() // Ensures that current Job is active.
someFlow.update {"DefaultValue"}
}
launch {
val networkCallValue = someNetworkCall()
firstJob.cancelAndJoin()
someFlow.update { networkCallValue }
}
You can send the network request and start the timeout delay simultaneously. When the network call succeeds, update the StateFlow with the response. And, when the timeout finishes and we haven't received the response, update the StateFlow with the default value.
val someFlow = MutableStateFlow(initialValue)
suspend fun getData() {
launch {
someFlow.value = someNetworkCall()
}
delay(TIMEOUT_MILLIS)
if(someFlow.value == initialValue)
someFlow.value = defaultValue
}
If the response of the network call can be same as the initialValue, you can create a new Boolean to check the completion of network request. Another option can be to store a reference of the Job returned by launch and check if job.isActive after the timeout.
Edit: In case you want to cancel delay when the network request completes, you can do something like:
val someFlow = MutableStateFlow(initialValue)
suspend fun getData() {
val job = launch {
delay(TIMEOUT_MILLIS)
someFlow.value = defaultValue
}
someFlow.value = someNetworkCall()
job.cancel()
}
And to solve the possible concurrency issue, you can use MutableStateFlow.update for atomic updates.
So I have a flow where I need it to emit a value from cache, but at the end it will make an API call to pull values in case there was nothing in cache (or refresh the value it has). I am trying this
override val data: Flow<List<Data>> = dataDao.getAllCachedData()
.onCompletion {
coroutineScope {
launch {
requestAndCacheDataOrEmitError()
}
}
}
.map { entities ->
entities
.map { it.toData() }
.filter { it !is Data.Unknown }
}
.filterNotNull()
.catch { emitRepositoryError(it) }
So the idea is that we emit the cache, and then make an API call to fetch new data regardless of the original mapping. But I do not want it blocking. For example, if we use this flow, I do not ever want the calling function to be blocked by the onCompletion.
I think the problem is that the onCompletion never runs. I set some breakpoints/logs and it never runs at all, even outside of the coroutineScope.
I don't quite understand the work you are doing but I think when you are collecting flow on a certain scope. You end the scope that flow will be put into onCompletion
var job : Job? = null
fun scan() {
job = viewModelScope.launch {
bigfileManager.bigFile.collect {
if (it is ResultOrProgress.Result) {
_bigFiles.value = it.result ?: emptyList()
} else {
_updateProgress.value = (it as ResultOrProgress.Progress).progress ?: 0
}
}
}
}
fun endScreen(){
job?.cancel()
}
This question is linked to one of my previous questions: Kotlin - Coroutines with loops.
So, this is my current implementation:
fun propagate() = runBlocking {
logger.info("Propagating objectives...")
val variablesWithSetObjectives: List<ObjectivePropagationMapping> =
variables.filter { it.variable.objective != Objective.NONE }
variablesWithSetObjectives.forEach { variableWithSetObjective ->
logger.debug("Propagating objective ${variableWithSetObjective.variable.objective} from variable ${variableWithSetObjective.variable.name}")
val job: Job = launch {
propagate(variableWithSetObjective, variableWithSetObjective.variable.objective, this, variableWithSetObjective)
}
job.join()
traversedVariableNames.clear()
}
logger.info("Done")
}
private tailrec fun propagate(currentVariable: ObjectivePropagationMapping, objectiveToPropagate: Objective, coroutineScope: CoroutineScope, startFromVariable: ObjectivePropagationMapping? = null) {
if (traversedVariableNames.contains(currentVariable.variable.name)) {
logger.debug("Detected loopback condition, stopping propagation to prevent loop")
return
}
traversedVariableNames.add(currentVariable.variable.name)
val objectiveToPropagateNext: Objective =
if (startFromVariable != currentVariable) {
logger.debug("Propagating objective $objectiveToPropagate to variable ${currentVariable.variable.name}")
computeNewObjectiveForVariable(currentVariable, objectiveToPropagate)
}
else startFromVariable.variable.objective
logger.debug("Choosing variable to propagate to next")
val variablesToPropagateToNext: List<ObjectivePropagationMapping> =
causalLinks
.filter { it.toVariable.name == currentVariable.variable.name }
.map { causalLink -> variables.first { it.variable.name == causalLink.fromVariable.name } }
if (variablesToPropagateToNext.isEmpty()) {
logger.debug("Detected end of path, stopping propagation...")
return
}
val variableToPropagateToNext: ObjectivePropagationMapping = variablesToPropagateToNext.random()
logger.debug("Chose variable ${variableToPropagateToNext.variable.name} to propagate to next")
if (variablesToPropagateToNext.size > 1) {
logger.debug("Detected split condition")
variablesToPropagateToNext.filter { it != variableToPropagateToNext }.forEach {
logger.debug("Launching child thread for split variable ${it.variable.name}")
coroutineScope.launch {
propagate(it, objectiveToPropagateNext, this)
}
}
}
propagate(variableToPropagateToNext, objectiveToPropagateNext, coroutineScope)
}
I'm currently running the algorithm on the following variable topology (Note that the algorithm follows arrows coming to a variable, but not arrows leaving from a variable):
Currently I am getting the following debug print result: https://pastebin.com/ya2tmc6s.
As you can see, even though I launch coroutines they don't begin executing until the main propagate recursive function has finished exploring a complete path.
I would want the launched coroutines to start executing immediately instead...
Unless otherwise specified, all the coroutines you start within runBlocking will run on the same thread.
If you want to enable multithreading, you can just change that to runBlocking(Dispatchers.Default). I'm just going to assume that all that code is thread-safe.
If you don't really want to enable multithreading, then you really shouldn't care what order the coroutines run in.