Double subscriptions on inner flow with kotlin-flow - kotlin

I have the following problem with my code snippet below;
When the set of currently connected barcode-scanners changes, a set with the currently connected scanners will be emitted by the scanners: Flow<Set<BarcodeScanner>> flow.
If I have connected a single scanner, called A, and I scan a barcode this works fine and on the result will be shown on the text-view.
If I connect an additional device, and then scan with device A again, I will see the scan result 2 times.
It looks like that on every change of the Set<> an additional flow subscription is created in the collect { scanner.scan() } and the previous flow is still in place.
How can I solve this? I already played the whole day with different operators etc...
I'm using Flow 1.2.2
uiScope.launch { //CoroutineScope(Job() + Dispatchers.Main)
scanners //Flow<Set<BarcodeScanner>> will emit the set of scanner, currently connected scanners, when one connects/disconnects
.onEach {
txtView.text = "Barcode scanners: ${it.size}\n"
}
.flatMapMerge { // Flatten to Flow<BarcodeScanner>
it.asFlow()
}
.onEach { scanner ->
txtView.append("${scanner.name()} [${scanner.hashCode()}]\n")
}
.collect { scanner ->
//if a new Set<BarcodeScanner> is emitted, this Flow<Barcode> returned by scanner.scan() will be subscribed again, without removing the old stream
//resulting in multiple results on a single scan.
scanner.scan() //returns an Flow<Barcode>
.onEach { barcode ->
findViewById<TextView>(R.id.txtViewBarcodes).append("[${scanner.name()}]: ${barcode.value} (${barcode.type.description})\n")
}
.onCompletion {
Timber.i("[${scanner.name()}]: DISCONNECTED") //Never called
}
.launchIn(scanScope)
}
}

I think rather than:
.collect { scanner ->
scanner.scan()
.onEach { barcode ->
...
You should use:
.flatMapLatest { scanner ->
scanner.scan()
}
.onEach { barcode ->
...
Where the flow is launched in the same UI scope. The problem you have is launchIn(scanScope) is creating a completely separate coroutine that the outer coroutine can't cancel when a new set of barcodes is emitted.

Related

Why is the value not entering the list?

At 'urichecking2' log, I can see there is value. But in 'uriChecking' the uriList is null.
why the uriList.add not work??
private fun getPhotoList() {
val fileName = intent.getStringExtra("fileName")
Log.d("fileNameChecking", "$fileName")
val listRef = FirebaseStorage.getInstance().reference.child("image").child(fileName!!)
var tmpUrl:Uri = Uri.parse(fileName)
Log.d("firstTmpUri","$tmpUrl")
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
tmpUrl = task.result
Log.d("secondTmpUri","$tmpUrl")
Log.d("urichecking2","$task.result")
uriList.add(task.result)
} else {
}
}.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
}
Log.d("thirdTmpUri","$tmpUrl")
Log.d("urichecking", "$uriList")
}
If I do this, the log is output in the order of first, third, and second, and the desired value is in second, but when third comes out, it returns to the value of first.
The listAll method (like most cloud APIs these days, including downloadUrl which you also use) is asynchronous, since it needs to make a call to the server - which may take time. This means the code executes in a different order than you may expect, which is easiest to see if you add some logging:
Log.d("Firebase","Before starting listAll")
listRef.listAll()
.addOnSuccessListener { listResult ->
Log.d("Firebase","Got listResult")
}
Log.d("Firebase","After starting listAll")
When you run this code it outputs:
Before starting listAll
After starting listAll
Got listResult
This is probably not the order you expected, but it perfectly explains why you can't see the list result. By the time your Log.d("urichecking", "$uriList") runs, none of the uriList.add(task.result) has been called yet.
The solution for this is always the same: any code that needs the list result, has to be inside the addOnCompleteListener callback, be called from there, or be otherwise synchronized.
So in its simplest way:
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
uriList.add(task.result)
Log.d("urichecking", "$uriList")
}
}
}
}
This is an incredibly common mistake to make if you're new to programming with asynchronous APIs, so I recommend checking out
Asynchronous programming techniques in the Kotlin language guide
How to get URL from Firebase Storage getDownloadURL
Can someone help me with logic of the firebase on success listener
Why does my function that calls an API or launches a coroutine return an empty or null value?

Kotlin Flow only collect every second

I have a Flow which downloads a file. The flow emits the download progress. I want to show the progress in the phone notification center, but only update the value once every second to prevent lagging the device.
My Flow:
return callbackFlow {
someJavaCallback { progress ->
trySend(progress)
}
close()
}
My CoroutineWorker, which displays the notification and downloads the file:
myFlow.collect { // update notification }
Result.Success()
My question is, how can I "throttle" the collection so that I for example collect 1%, but 1 second later it collects 5% and ignores all values between those two points
fun <T> Flow<T>.throttleLatest(delayMillis: Long): Flow<T> = this
.conflate()
.transform {
emit(it)
delay(delayMillis)
}
source: https://github.com/Kotlin/kotlinx.coroutines/issues/1446#issuecomment-1198103541

Send upstream exception in SharedFlow to collectors

I want to achieve the following flow logic in Kotlin (Android):
Collectors listen to a List<Data> across several screens of my app.
The source-of-truth is a database, that exposes data and all changes to it as a flow.
On the first initialization the data should be initialized or updated via a remote API
If any API exception occurs, the collectors must be made aware of it
In my first attempt, the flow was of the type Flow<List<Data>>, with the following logic:
val dataFlow = combine(localDataSource.dataFlow, flow {
emit(emptyList()) //do not wait for API on first combination
emit(remoteDataSource.suspendGetDataMightThrow())
}) { (local, remote) ->
remote.takeUnless { it.isEmpty() }?.let { localDataSource.updateIfChanged(it) }
local
}.shareIn(externalScope, SharingStarted.Lazily, 1)
This worked fine, except when suspendGetDataMightThrow() throws an exception. Because shareIn stops propagating the exception through the flow, and instead breaks execution of the externalScope, my collectors are not notified about the exception.
My solution was to wrap the data with a Result<>, resulting of a flow type of Flow<Result<List<Data>>>, and the code:
val dataFlow = combine(localDataSource.dataFlow, flow {
emit(Result.success(emptyList())) //do not wait for API on first combination
emit(runCatching { remoteDataSource.suspendGetDataMightThrow() })
}) { (local, remote) ->
remote.onSuccess {
data -> data.takeUnless { it.isEmpty() }?.let { localDataSource.updateIfChanged(it) }
}
if (remote.isFailure) remote else local
}.shareIn(externalScope, SharingStarted.Lazily, 1)
I can now collect it as follows, and the exception is passed to the collectors:
dataRepository.dataFlow
.map { it.getOrThrow() }
.catch {
// ...
}
.collect {
// ...
}
Is there a less verbose solution to obtain the exception, than to wrap the whole thing in a Result?
I am aware that there are other issues with the code (1 API failure is emitted forever). This is only a proof-of-concept to get the error-handling working.

Should Flux.then and Mono.then behave differently in error case?

I encountered a case where I have a nested Flux. I don't care about the individual results of the inner flux as it returns Unit (in Kotlin / Void in Java), but I want to know if the Flux aborted due to an error or not. I thought I could use the then function, as the doc states: Error signal is replayed in the resulting Mono<V>
My problem can be reduced to the minimum (Kotlin) unit test:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Flux.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So we have 2 elements. In the inner Flux one of the elements will fail on processing and the other won't. I expect the error to propagate through the pipeline where it is catched and discarded in the onErrorContinue.
Therefore I'd expect 1 element in the resulting list, but I get the original 2. I have no clue why.
Now comes the fun part: In this particular test case, I can replace Flux.just(1) with Mono.just(1) (in my real world case this doesn't work ofc because the flux has more than 1 element) and suddenly my test passes:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Mono.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So obviously there is a difference in Mono.then(Mono<T>) and in Flux.then(Mono<T>), but it shouldn't since the Javadoc is the same right?
Side note: Instead of Flux.then(Mono.just(outer)) I also tried Mono.defer but that is not changing anything.

Kotlin Coroutines - unlimited stream to fan out batches

I'm looking to implement a pipeline for processing an infinite stream of messages. I'm new to coroutines and trying to follow along with the docs but I'm not confident I'm doing the right thing.
My infinite stream is of batches of records and I'd like to fan out the processing of each record to a coroutine, wait for a batch to finish (to log stats and stuff) before continuing to the next batch.
-> process [record] \
source -> [records] -> process [record] -> [log batch stats]
-> process [record] /
|------------------- while(true) -------------------|
What I had planned is to have 2 Channels, one for the infinite stream, and one for the intermediate records that will fill up and empty on each batch.
runBlocking {
val infinite: Channel<List<Record>> = produce { send(source.getBatch()) }
val records = Channel<Record>(Channel.Factory.UNLIMITED)
while(true) {
infinite.receive().forEach { records.send(it) }
while(!records.isEmpty()) {
launch { process(records.receive()) }
}
// ??? Wait for jobs?
logBatchStats()
}
}
From googling, it seems that waiting for jobs is discouraged, plus I wasn't sure if calling .map on a channel will actually receive messages to convert them to jobs:
records.map { record -> launch { process(record) } }
yields a Channel<Job>. It seems I can call .toList() on it to collapse it, but then I need to join the jobs? Again, google suggested to do that by having a parent job, but I'm not really sure how to do that with launch.
Anyway, very much a n00b to this.
Thanks for the help.
I don't see a reason to have two channels. You could directly iterate over the list of records. And you should use async instead of launch. Then you can use await or even better awaitAll for the list of results.
val infinite: ReceiveChannel<List<Record>> = produce { ... }
while(true) {
val resultsDeferred = infinite.receive().map {
async {
process(it)
}
}
val results = resultsDeferred.awaitAll()
logBatchStats()
}