I'm combining two SharedFlows and then performing a long working operation.
At the start, I know the state so I emit a "starting value" for both the flows. After that user can emit to either flows.
Both flows are mostly independent but in a specific situation, the user can emit to both flows at the same time. What this does is that combine is triggered twice and the long working job is performed twice when in fact, in this case, I'm only interested receiving both values but only performing the job once.
Here is what I have:
val _numbers = MutableSharedFlow<Int>(replay = 0, extraBufferCapacity = 1, onBufferOverflow = BufferOverflow.DROP_OLDEST)
val numbers: SharedFlow<Int> = _numbers
val _strings = MutableSharedFlow<String>(replay = 0, extraBufferCapacity = 1, onBufferOverflow = BufferOverflow.DROP_OLDEST)
val strings: SharedFlow<String> = _strings
combine(numbers, strings) { (number, strings) ->
println("values $number - $strings. Starting to perform a long working job")
}
.launchIn(CoroutineScope(Dispatchers.IO))
runBlocking {
delay(500)
// This is the initial values. I always know this at start.
_numbers.emit(0)
_strings.emit("a")
// Depending of user action, number or string is emitted.
delay(100)
_numbers.emit(1)
delay(100)
_numbers.emit(2)
delay(100)
_numbers.emit(3)
delay(100)
_numbers.emit(4)
delay(100)
_strings.emit("b")
delay(100)
_strings.emit("c")
delay(100)
_strings.emit("d")
delay(100)
_strings.emit("e")
delay(100)
// In a specific situation both values need to change but I only want to trigger the long working job once
_numbers.emit(10)
_strings.emit("Z")
}
This can produce this:
values 0 - a. Starting to perform a long working job
values 1 - a. Starting to perform a long working job
values 2 - a. Starting to perform a long working job
values 3 - a. Starting to perform a long working job
values 4 - a. Starting to perform a long working job
values 4 - b. Starting to perform a long working job
values 4 - c. Starting to perform a long working job
values 4 - d. Starting to perform a long working job
values 4 - e. Starting to perform a long working job
values 10 - e. Starting to perform a long working job
values 10 - Z. Starting to perform a long working job
Or this:
values 0 - a. Starting to perform a long working job
values 1 - a. Starting to perform a long working job
values 2 - a. Starting to perform a long working job
values 3 - a. Starting to perform a long working job
values 4 - a. Starting to perform a long working job
values 4 - b. Starting to perform a long working job
values 4 - c. Starting to perform a long working job
values 4 - d. Starting to perform a long working job
values 4 - e. Starting to perform a long working job
values 10 - Z. Starting to perform a long working job
Due to the buffer overflow, sometimes I can achieve what I want (this latest one) but on others, I have the values 10 - e. Starting to perform a long working job that I'm not interested in.
Is there any way I can enforce, when emitting to the two, only start the long work once?
https://pl.kotl.in/JA1Wdhra9
If you want to keep 2 flows, the distinction between single and double events will have to be time-based. You won't be able to distinguish between a quick update of string-then-number from a "double-update".
If time-based is ok for you, using debounce before the long processing should be the way to go:
combine(numbers, strings) { (number, string) -> number to string }
.debounce(50)
.onEach { (number, string) ->
println("values $number - $string. Starting to perform a long working job")
}
.launchIn(CoroutineScope(Dispatchers.IO))
Here, combine only builds pairs from the 2 flows, but still gets all events, and then debounce ignores quick succession of events and only sends the latest of a quick series. This also introduces a slight delay, but it all depends on what you want to achieve.
If time-based distinction is not ok for you, you need a way for the producer to send double events in a way that is distinct from 2 single events. For this, you can use a single flow of events, and you can for instance define events like this:
sealed class Event {
data class SingleNumberUpdate(val value: Int): Event()
data class SingleStringUpdate(val value: String): Event()
data class DoubleUpdate(val num: Int, val str: String): Event()
}
But then you'll have to write the "combine" logic yourself (keeping the state of the latest number and string):
flow {
var num = 0
var str = "a"
emit(num to str)
events.collect { e ->
when (e) {
is Event.SingleNumberUpdate -> {
num = e.value
}
is Event.SingleStringUpdate -> {
str = e.value
}
is Event.DoubleUpdate -> {
num = e.num
str = e.str
}
}
emit(num to str)
}
}
.onEach { (number, strings) ->
println("values $number - $strings. Starting to perform a long working job")
}
.launchIn(CoroutineScope(Dispatchers.IO))
Related
I have a function with a for-loop:
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
var sum = 0
for (item in this) {
if (sumFunction(item))
sum += item
}
return sum
}
I want to know how I can write the above in functional style. I know that I have to use this.reduce(), but don't know exactly how to implement it.
return filter(sumFunction).sum()
Should be self-explanatory.
You can’t use reduce because it doesn’t let you reject the first element.
With fold it would be:
return fold(0) { a, b ->
if(sumFunction(b)) a + b else a
}
I can think if two ways to achieve that:
The first one is by using sumOf {...}:
.
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
return sumOf {
if (sumFunction(it)) it else 0
}
}
The second one is by using filter {...} then sum():
.
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
return filter(sumFunction).sum()
}
return this.reduce { sum, n -> if (sumFunction(n)) sum + n else 0}
If you really want to use reduce for some reason you can - but you need to add that 0 to the head of the list as your "start state":
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
val stuff = listOf(0) + this
return stuff.reduce { a, b -> a + if (sumFunction(b)) b else 0 }
}
You have to do that because reduce is really there to combine a bunch of items, which is why for the first iteration you get the first two items in the list. You don't get to handle them separately, which is why you need to throw that 0 in there to get past that first step, and get to a point where you can just do your checking on the second parameter and ignore the first one, treating it as an accumulator instead of another item you also need to check.
That behaviour is what fold is for - with that function you pass in an initial state (which can be a completely different type from your items, since you're not just smushing them together to create a new value like with reduce) and then on each iteration you get that state and an item.
You can handle the item as you like, and then make changes to the accumulator state depending on the result. Which is exactly the behaviour of your for loop! fold is just a functional way to write one. Tenfour04's answer is how you'd do it - it's the right tool for the job here!
How to "take(N)" iteratively - get a Sequence<Sequence>, each inner sequences having next N elements?
I am writing a high-load application in Kotlin.
I have tens of thousands of entries to insert to a database.
I want to batch them by, say, 1000.
So I created a loop:
val itemsSeq = itemsList.iterator().asSequence()
while (true) {
log.debug("Taking $BATCH_SIZE from $itemsSeq")
val batchSeq = itemsSeq.take(BATCH_SIZE)
val squareBatch = applySomething(batchSeq, something)
?: break
}
fun applySomething(batch: Sequence<Item>, something: Something) {
/* Fully consumes batch. Bulk-loads from DB by IDs, applies, bulk-saves. */
}
I thought that take() would advance the itemsSeq and the next call to take() would give me a sequence "view" of itemsSeq starting at the 10th item.
But with this code, I am getting:
DEBUG Taking 10 from kotlin.sequences.ConstrainedOnceSequence#53fe15ff
Exception in thread "main" java.lang.IllegalStateException: This sequence can be consumed only once.
at kotlin.sequences.ConstrainedOnceSequence.iterator(SequencesJVM.kt:23)
at kotlin.sequences.TakeSequence$iterator$1.<init>(Sequences.kt:411)
at kotlin.sequences.TakeSequence.iterator(Sequences.kt:409)
So it seems that the take() "opens" the itemsSeq again, while that can be consumed only once.
As a workaround, I can use chunked():
public fun <T> Sequence<T>.chunked(size: Int): Sequence<List<T>> {
But I would prefer not to create Lists, rather Sequences.
What I am looking for is something between take() and chunked().
Is there anything such in Kotlin SDK?
I can possibly create my own sequence { ... } but for readability, I would prefer something built-in.
There is a way to construct a Sequence by handing it over an Iterator, see Sequence.
Given an iterator function constructs a Sequence that returns values
through the Iterator provided by that function. The values are
evaluated lazily, and the sequence is potentially infinite.
Wrapped in an extension function it could look like this:
fun <T> Iterable<T>.toValuesThroughIteratorSequence(): Sequence<T> {
val iterator = this.iterator()
return Sequence { iterator }
}
Quick test:
data class Test(val id: Int)
val itemsList = List(45) { Test(it) }
val batchSize = 10
val repetitions = itemsList.size.div(batchSize) + 1
val itemsSeq = itemsList.toValuesThroughIteratorSequence()
(0 until repetitions).forEach { index ->
val batchSeq = itemsSeq.take(batchSize)
println("Batch no. $index: " + batchSeq.map { it.id.toString().padStart(2, ' ') }.joinToString(" "))
}
Output:
Batch no. 0: 0 1 2 3 4 5 6 7 8 9
Batch no. 1: 10 11 12 13 14 15 16 17 18 19
Batch no. 2: 20 21 22 23 24 25 26 27 28 29
Batch no. 3: 30 31 32 33 34 35 36 37 38 39
Batch no. 4: 40 41 42 43 44
Background
First of all, we need to be aware there is a big difference between an object that we can iterate over and object that represents a "live" or already running iteration process. First group means Iterable (so List, Set and all other collections), Array, Flow, etc. Second group is mostly Iterator or old Java Enumeration. The difference could be also compared to file vs file pointer when reading or database table vs database cursor.
Sequence belongs to the first group. Sequence object does not represent a live, already started iteration, but just a set of elements. These elements can be produced lazily, sequence could have unbounded size and usually internally it works by using iterators, but conceptually sequence is not an iterator itself.
If we look into the documentation about sequences it clearly compares them to Iterable, not to Iterator. All standard ways to construct sequences like: sequenceOf(), sequence {}, Iterable.asSequence() produce sequences that return the same list of items every time we iterate over them. Iterator.asSequence() also follows this pattern, but because it can't re-produce same items twice, it is intentionally protected against iterating multiple times:
public fun <T> Iterator<T>.asSequence(): Sequence<T> = Sequence { this }.constrainOnce()
Problem
Your initial attempt with using take() didn't work, because this is a misuse of sequences. We expect that subsequent take() calls on the same sequence object will produce exactly the same items (usually), not next items. Similarly as we expect multiple take() calls on a list always produce same items, each time starting from the beginning.
Being more specific, your error was caused by above constrainOnce(). When we invoke take() multiple times on a sequence, it has to restart from the beginning, but it can't do this if it was created from an iterator, so Iterator.asSequence() explicitly disallows this.
Simple solution
To fix the problem, you can just skip constrainOnce() part, as suggested by #lukas.j. This solution is nice, because stdlib already provides tools like Sequence.take(), so if used carefully, this is the easiest to implement and it just works.
However, I personally consider this a kind of workaround, because the resulting sequence doesn't behave as sequences do. It is more like an iterator on steroids than a real sequence. You need to be careful when using this sequence with existing operators or 3rd party code, because such sequence may work differently than they expect and as a result, you may get incorrect results.
Advanced solution
We can follow your initial attempt of using subsequent take() calls. In this case our object is used for live iteration, so it is no longer a proper sequence, but rather an iterator. The only thing we miss in stdlib is a way to create a sub-iterator with a single chunk. We can implement it by ourselves:
fun main() {
val list = (0 .. 25).toList()
val iter = list.iterator()
while (iter.hasNext()) {
val chunk = iter.limited(10)
println(chunk.asSequence().toList())
}
}
fun <T> Iterator<T>.limited(n: Int): Iterator<T> = object : Iterator<T> {
var left = n
val iterator = this#limited
override fun next(): T {
if (left == 0)
throw NoSuchElementException()
left--
return iterator.next()
}
override fun hasNext(): Boolean {
return left > 0 && iterator.hasNext()
}
}
I named it limited(), because take() suggests we read items from the iterator. Instead, we only create another iterator on top of the provided iterator.
Of course, sequences are easier to use than iterators and typical solution to this problem is by using chunked(). With above limited() it is pretty straightforward to implement chunkedAsSequences():
fun main() {
val list = (0 .. 25).toList()
list.asSequence()
.chunkedAsSequences(10)
.forEach { println(it.toList()) }
}
fun <T> Sequence<T>.chunkedAsSequences(size: Int): Sequence<Sequence<T>> = sequence {
val iter = iterator()
while (iter.hasNext()) {
val chunk = iter.limited(size)
yield(chunk.asSequence())
chunk.forEach {} // required if chunk was not fully consumed
}
}
Please also note there is a tricky case of chunk being not fully consumed. chunkedAsSequences() is protected against this scenario. Previous simpler solutions aren't.
How one may count how many times specific character appears in string in Kotlin?
From looking at https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-string/ there is nothing built-in and one needs to write loop every time (or may own extension function), but maybe I missed a better way to achieve this?
Easy with filter {} function
val str = "123 123 333"
val countOfSymbol = str
.filter { it == '3' } // 3 is your specific character
.length
println(countOfSymbol) // output 5
Another approach
val countOfSymbol = str.count { it == '3'} // 3 is your specific character
println(countOfSymbol) // output 5
From the point of view of saving computer resources, the count decision(second approach) is more correct.
Let's say I have the function fun someValue() : Double.
I want to run this function n times and take the maximum of all the results.
I know I can safe it all in a list and use max(), but I don't want to waste that space (because it actually returns a bigger object).
Does there exist a library-function for this?
Maybe something like repeat(n){someValue()}.max()
Here are a couple of ways that don't allocate an intermediate array or list of all the values.
var x = someValue()
repeat(n - 1) {
x = max(x, someValue())
}
val y = (1 until n).fold(someValue()) { acc, _ -> max(acc, someValue()) }
I am pretty confused with both functions fold() and reduce() in Kotlin, can anyone give me a concrete example that distinguishes both of them?
fold takes an initial value, and the first invocation of the lambda you pass to it will receive that initial value and the first element of the collection as parameters.
For example, take the following code that calculates the sum of a list of integers:
listOf(1, 2, 3).fold(0) { sum, element -> sum + element }
The first call to the lambda will be with parameters 0 and 1.
Having the ability to pass in an initial value is useful if you have to provide some sort of default value or parameter for your operation. For example, if you were looking for the maximum value inside a list, but for some reason want to return at least 10, you could do the following:
listOf(1, 6, 4).fold(10) { max, element ->
if (element > max) element else max
}
reduce doesn't take an initial value, but instead starts with the first element of the collection as the accumulator (called sum in the following example).
For example, let's do a sum of integers again:
listOf(1, 2, 3).reduce { sum, element -> sum + element }
The first call to the lambda here will be with parameters 1 and 2.
You can use reduce when your operation does not depend on any values other than those in the collection you're applying it to.
The major functional difference I would call out (which is mentioned in the comments on the other answer, but may be hard to understand) is that reduce will throw an exception if performed on an empty collection.
listOf<Int>().reduce { x, y -> x + y }
// java.lang.UnsupportedOperationException: Empty collection can't be reduced.
This is because .reduce doesn't know what value to return in the event of "no data".
Contrast this with .fold, which requires you to provide a "starting value", which will be the default value in the event of an empty collection:
val result = listOf<Int>().fold(0) { x, y -> x + y }
assertEquals(0, result)
So, even if you don't want to aggregate your collection down to a single element of a different (non-related) type (which only .fold will let you do), if your starting collection may be empty then you must either check your collection size first and then .reduce, or just use .fold
val collection: List<Int> = // collection of unknown size
val result1 = if (collection.isEmpty()) 0
else collection.reduce { x, y -> x + y }
val result2 = collection.fold(0) { x, y -> x + y }
assertEquals(result1, result2)
Another difference that none of the other answers mentioned is the following:
The result of a reduce operation will always be of the same type (or a super type) as the data that is being reduced.
We can see that from the definition of the reduce method:
public inline fun <S, T : S> Iterable<T>.reduce(operation: (acc: S, T) -> S): S {
val iterator = this.iterator()
if (!iterator.hasNext()) throw UnsupportedOperationException("Empty collection can't be reduced.")
var accumulator: S = iterator.next()
while (iterator.hasNext()) {
accumulator = operation(accumulator, iterator.next())
}
return accumulator
}
On the other hand, the result of a fold operation can be anything, because there are no restrictions when it comes to setting up the initial value.
So, for example, let us say that we have a string that contains letters and digits. We want to calculate the sum of all the digits.
We can easily do that with fold:
val string = "1a2b3"
val result: Int = string.fold(0, { currentSum: Int, char: Char ->
if (char.isDigit())
currentSum + Character.getNumericValue(char)
else currentSum
})
//result is equal to 6
reduce - The reduce() method transforms a given collection into a single result.
val numbers: List<Int> = listOf(1, 2, 3)
val sum: Int = numbers.reduce { acc, next -> acc + next }
//sum is 6 now.
fold - What would happen in the previous case of an empty list? Actually, there’s no right value to return, so reduce() throws a RuntimeException
In this case, fold is a handy tool. You can put an initial value by it -
val sum: Int = numbers.fold(0, { acc, next -> acc + next })
Here, we’ve provided initial value. In contrast, to reduce(), if the collection is empty, the initial value will be returned which will prevent you from the RuntimeException.
Simple Answer
Result of both reduce and fold is "a list of items will be transformed into a single item".
In case of fold,we provide 1 extra parameter apart from list but in case of reduce,only items in list will be considered.
Fold
listOf("AC","Fridge").fold("stabilizer") { freeGift, itemBought -> freeGift + itemBought }
//output: stabilizerACFridge
In above case,think as AC,fridge bought from store & they give stabilizer as gift(this will be the parameter passed in the fold).so,you get all 3 items together.Please note that freeGift will be available only once i.e for the first iteration.
Reduce
In case of reduce,we get items in list as parameters and can perform required transformations on it.
listOf("AC","Fridge").reduce { itemBought1, itemBought2 -> itemBought1 + itemBought2 }
//output: ACFridge
The difference between the two functions is that fold() takes an initial value and uses it as the accumulated value on the first step, whereas the first step of reduce() uses the first and the second elements as operation arguments on the first step.