Kotlin - How To Collect X Values From a Flow? - kotlin

Let's say I have a flow that is constantly sending updated like the following:
locationFlow = StateFlow<Location?>(null)
I have a use-case where after a particular event occurs, I want to collect X values from the flow and continue, so something like what I have below. I know that collect is a terminal operator, so I don't think the logic I have below works, but how could I do this in this case? I'd like to collect X items, save them, and then send them to another function for processing/handling.
fun onEventOccurred() {
launch {
val locations = mutableListOf<Location?>()
locationFlow.collect {
//collect only X locations
locations.add(it)
}
saveLocations(locations)
}
}
Is there a pre-existing Kotlin function for something like this? I'd like to collect from the flow X times, save the items to a list, and pass that list to another function.

It doesn't matter that collect is terminal. The upstream StateFlow will keep behaving normally because StateFlows don't care what their collectors are doing. you can use the take function to get a specific number of items, and you can use toList() (another terminal function) to concisely copy them into a list once they're all ready.
fun onEventOccurred() {
launch {
saveLocations(locationFlow.take(5).toList())
}
}

If I understood correctly your use case, you want to:
discard elements until a specific one is sent – actually, after re-reading your question I don't think this is the case.. I'm leaving it in the example just FYI
when that happens, you want to collect X items for further processing
Assuming that's correct, you can use a combination of dropWhile and take, like so:
fun main() = runBlocking {
val messages = flow {
repeat(10) {
println(it)
delay(500)
emit(it)
}
}
messages
.dropWhile { it < 5 }
.take(3)
.collect { println(it) } // prints 5, 6, 7
}
You can even have more complex logic, i.e. discard any number that's less than 5, and then take the first 10 even numbers:
fun main() = runBlocking {
val messages = flow {
repeat(100) {
delay(500)
emit(it)
}
}
messages
.dropWhile { it < 5 }
.filter { it % 2 == 0}
.take(10)
.collect { println(it) } // prints even numbers, 6 to 24
}

Related

Flows in Kotlin not collecting all values

I'm doing some exercises to learn Flows in Kotlin, and I found some issues which I cannot understand.
When using a MutableStateFlow, in the next example it only prints the number 3. I would expect to print 0 to 3 instead. One could say that maybe is going too fast, or I should put a delay, but this seems to me a patch if such is the case, since if it is true that if sending MutableStateFlow data too fast makes it skip some values, then is something to consider every single time when using it.
val flow = MutableStateFlow<Int>(0)
fun main(): Unit = runBlocking {
launch {
flow.collect {
println(it)
}
}
(0..3).forEach {
flow.emit(it)
}
}
// Expected to print 0, 1, 2, 3
// Printing only 3
Next, I tried to use a MutableSharedFlow instead, but it emits nothing at all, not even 3. Same code as above but replacing the flow with:
val flow = MutableSharedFlow<Int>()
MutableStateFlow cannot be used here because its behavior does not allow to get every value
so I used SharedFlow
Example with SharedFlow:
val flow = MutableSharedFlow<Int>()
fun main(): Unit = runBlocking {
val scope = // scope
flow
.onEach {
println(it)
}
.launchIn(scope)
(0..3).forEach {
flow.emit(it)
}
// delay to wait for println
launch {
delay(10000)
}
}

Kotlin - Debounce Only One Specific Value When Emitting from Flow

I have two flows that are being combined to transform the flows into a single flow. One of the flows has a backing data set that emits much faster than the the other.
Flow A - emits every 200 ms
Flow B - emits every ~1s
The problem I am trying to fix is this one:
combine(flowA, flowB) { flowAValue, flowBValue // just booleans
flowAValue && flowBValue
}.collect {
if(it) {
doSomething
}
}
Because Flow A emits extremely quickly, the boolean that's emitted can get cleared rapidly, which means that when flowB emits true, flowA already emitted true and the state is now false.
I've attempted something like:
suspend fun main() {
flowA.debounce {
if (it) {
1250L
} else {
0L
}
}.collect {
println(it)
}
}
But this doesn't work as sometimes the true values aren't emitted - inverting the conditional (so that if(true) = 0L else 1250L) also doesn't work. Basically what I'm looking for is that if flowA is true - hold that value for 1 second before changing values. Is something like that possible?
I made this use conflated on the 2nd flow, that is drastically faster, so that zipping them will always take the latest value from fastFlow, when slowFlow is finally ready, if you don't use conflated on the 2nd flow, it will always be the first time both emit.
fun forFlow() = runTest {
val slowString = listOf("first", "second", "third", "fourth")
val slowFlow = flow {
slowString.forEach {
delay(100)
emit(it)
}
}
val fastFlow = flow {
(1 until 1000).forEach { num ->
delay(5)
emit(num)
}
}.conflate()
suspend fun zip() {
slowFlow.zip(fastFlow) { first, second -> "$first: $second" }
.collect {
println(it)
}
}
runBlocking {
zip()
}
println("Done!")
}
With Conflated on fastFlow:
first: 1
second: 15
third: 32
fourth: 49
Done!
Without Conflated on fastFlow:
first: 1
second: 2
third: 3
fourth: 4
Done!

Combine a Flow and a non Flow api response Kotlin

I currently have a piece of logic as follows:
interface anotherRepository {
fun getThings(): Flow<List<String>>
}
interface repository {
suspend fun getSomeThings(): AsyncResult<SomeThings>
}
when (val result = repository.getSomeThings()) {
is AsyncResult.Success -> {
anotherRepository.getThings().collectLatest {
// update the state
}
else -> { }
}
}
The problem I am having is that, if repository.getSomeThings has been triggered multiple times before, anotherRepository.getThings is getting triggered for the amount of all the pre-loaded values from repository.getSomeThings. I was wondering what is the proper way to use these repositories, one a suspend function, the other a Flow together. The equivalent behaviour that is combineLatest{} in Rx.
Thank you.
There are a couple of ways to solve your problem. One way is just to call
repository.getSomeThings() in the collectLatest block and cache last result:
var lastResult: AsyncResult<SomeThings>? = null
anotherRepository.getThings().collectLatest {
if (lastResult == null) {
lastResult = repository.getSomeThings()
}
// use lastResult and List<String>
}
Another approach is to create a Flow, which will be calling repository.getSomeThings() function, and combine two Flows:
combine(
anotherRepository.getThings(),
flow {emit(repository.getSomeThings())}
) { result1: List<String>, result2: AsyncResult<SomeThings> ->
...
}

Parallelly consuming a long sequence in Kotlin

I have a function generating a very long sequence of work items. Generating these items is fast, but there are too many in total to store a list of them in memory. Processing the items produces no results, just side effects.
I would like to process these items across multiple threads. One solution is to have a thread read from the generator and write to a concurrent bounded queue, and a number of executor threads polling for work from the bounded queue, but this is a lot of things to set up.
Is there anything in the standard library that would help me do that?
I had initially tried
items.map { async(executor) process(it) }.forEach { it.await() }
But, as pointed out in how to implement parallel mapping for sequences in kotlin, this doesn't work for reasons that are obvious in retrospect.
Is there a quick way to do this (possibly with an external library), or is manually setting up a bounded queue in the middle my best option?
You can look at coroutines combined with channels.
If all work items can be emmited on demand with producer channel. Then it's possible to await for each items and process it with a pool of threads.
An example :
sealed class Stream {
object End: Stream()
class Item(val data: Long): Stream()
}
val produceCtx = newSingleThreadContext("producer")
// A dummy producer that send one million Longs on its own thread
val producer = CoroutineScope(produceCtx).produce {
for (i in (0 until 1000000L)) send(Stream.Item(i))
send(Stream.End)
}
val workCtx = newFixedThreadPoolContext(4, "work")
val workers = Channel<Unit>(4)
repeat(4) { workers.offer(Unit) }
for(_nothing in workers) { // launch 4 times then wait for a task to finish
launch(workCtx) {
when (val item = producer.receive()) {
Stream.End -> workers.close()
is Stream.Item -> {
workFunction(item.data) // Actual work here
workers.offer(Unit) // Notify to launch a new task
}
}
}
}
Your magic word would be .asSequence():
items
.asSequence() // Creates lazy executable sequence
.forEach { launch { executor.process(it) } } // If you don't need the value aftrwards, use 'launch', a.k.a. "fire and forget"
but there are too many in total to store a list of them in memory
Then don't map to list and don't collect the values, no matter if you work with Kotlin or Java.
As long as you are on the JVM, you can write yourself an extension function, that works the sequence in chunks and spawns futures for all entries in a chunk. Something like this:
#Suppress("UNCHECKED_CAST")
fun <T, R> Sequence<T>.mapParallel(action: (value: T) -> R?): Sequence<R?> {
val numThreads = Runtime.getRuntime().availableProcessors() - 1
return this
.chunked(numThreads)
.map { chunk ->
val threadPool = Executors.newFixedThreadPool(numThreads)
try {
return#map chunk
.map {
// CAUTION -> needs to be written like this
// otherwise the submit(Runnable) overload is called
// which always returns an empty Future!!!
val callable: () -> R? = { action(it) }
threadPool.submit(callable)
}
} finally {
threadPool.shutdown()
}
}
.flatten()
.map { future -> future.get() }
}
You can then just use it like:
items
.mapParallel { /* process an item */ }
.forEach { /* handle the result */ }
As long as workload per item is similar, this gives a good parallel processing.

Observe many times from same Observable (RxAndroidBle)

I'm using the RxAndroidBle library with RxJava2 to read from a BLE Characteristic. I think this question is just an RxJava question, but including the detail that I'm using RxAndroidBle in case that is useful.
I get connection, and then use it to call readCharacteristic(), which itself returns a Single<ByteArray>. At this point, I don't just want to just get the one ByteArray though. I need to read from this characteristic several times, because the BLE device is set up to let me get a small file back, and characteristics can only send 20 bytes back at a time, hence my need to read repeatedly.
Is it possible to modify this code so that the switchMap() below returns an Observable that will emit many ByteArrays, instead of just the single one?
I'm new to RxJava.
val connection: Observable<RxBleConnection> = selectedDevice.record.bleDevice.establishConnection(false, Timeout(30, TimeUnit.SECONDS))
return connection
.subscribeOn(Schedulers.io())
.switchMap {
// I want to get an Observable that can read multiple times here.
it.readCharacteristic(serverCertCharacteristicUUID).toObservable()
}
.doOnNext {
Timber.e("Got Certificate bytes")
}
.map {
String(it as ByteArray)
}
.doOnNext {
Timber.e("Got certificate: $it")
}
.singleOrError()
To repeat a read multiple times until a specific value is emitted one needs to change this part:
// I want to get an Observable that can read multiple times here.
it.readCharacteristic(serverCertCharacteristicUUID).toObservable()
to something like what was suggested by the RxJava author in the first answer that google gives for phrase rxjava single repeat:
// this will repeat until a `checkRepeatIf` returns false
Observable.defer {
val successValue = AtomicReference<ByteArray>()
connection.readCharacteristic(serverCertCharacteristicUUID)
.doOnSuccess { successValue.lazySet(it) }
.repeatWhen { completes -> completes.takeWhile { checkRepeatIf(successValue.get()) } }
}
I was able to get this working by sending a signal to stop both the connectionObservable, and the read on the Bluetooth characteristic. Of note is that you need to call toObservable() AFTER repeat() or this doesn't work, although I don't know why exactly.
override fun readMultipartCharacteristic(macAddress: String): Single<String> {
val CERTIFICATE_TERMINATOR = 0x30.toByte()
val device = bluetoothService.getBleDevice(macAddress)
if (connectionObservable == null || !device.connectionState.equals(RxBleConnection.RxBleConnectionState.CONNECTED)) {
connectionObservable = device.establishConnection(false, Timeout(30, TimeUnit.SECONDS))
}
val stop: PublishSubject<Unit> = PublishSubject.create()
return connectionObservable!!
.subscribeOn(Schedulers.io())
.takeUntil(stop)
.switchMap {
it.readCharacteristic(UUID("my-uuid"))
.repeat()
.toObservable()
.takeUntil(stop)
}
.collectInto(ByteArrayOutputStream(), { buffer, byteArray ->
// Watch for the signal of the end of the stream
if (byteArray.size == 1 && byteArray.get(0).equals(CERTIFICATE_TERMINATOR)) {
stop.onComplete()
} else {
buffer.write(byteArray)
}
})
.map {
String(it.toByteArray())
}
}
You can use the notification to buffer your data.
device.establishConnection(false)
.flatMap(rxBleConnection -> rxBleConnection.setupNotification(characteristicUuid))
.flatMap(notificationObservable -> notificationObservable) // <-- Notification has been set up, now observe value changes.
.subscribe(
bytes -> {
// Given characteristic has been changes, here is the value.
},
throwable -> {
// Handle an error here.
}
);