I am pretty confused with both functions fold() and reduce() in Kotlin, can anyone give me a concrete example that distinguishes both of them?
fold takes an initial value, and the first invocation of the lambda you pass to it will receive that initial value and the first element of the collection as parameters.
For example, take the following code that calculates the sum of a list of integers:
listOf(1, 2, 3).fold(0) { sum, element -> sum + element }
The first call to the lambda will be with parameters 0 and 1.
Having the ability to pass in an initial value is useful if you have to provide some sort of default value or parameter for your operation. For example, if you were looking for the maximum value inside a list, but for some reason want to return at least 10, you could do the following:
listOf(1, 6, 4).fold(10) { max, element ->
if (element > max) element else max
}
reduce doesn't take an initial value, but instead starts with the first element of the collection as the accumulator (called sum in the following example).
For example, let's do a sum of integers again:
listOf(1, 2, 3).reduce { sum, element -> sum + element }
The first call to the lambda here will be with parameters 1 and 2.
You can use reduce when your operation does not depend on any values other than those in the collection you're applying it to.
The major functional difference I would call out (which is mentioned in the comments on the other answer, but may be hard to understand) is that reduce will throw an exception if performed on an empty collection.
listOf<Int>().reduce { x, y -> x + y }
// java.lang.UnsupportedOperationException: Empty collection can't be reduced.
This is because .reduce doesn't know what value to return in the event of "no data".
Contrast this with .fold, which requires you to provide a "starting value", which will be the default value in the event of an empty collection:
val result = listOf<Int>().fold(0) { x, y -> x + y }
assertEquals(0, result)
So, even if you don't want to aggregate your collection down to a single element of a different (non-related) type (which only .fold will let you do), if your starting collection may be empty then you must either check your collection size first and then .reduce, or just use .fold
val collection: List<Int> = // collection of unknown size
val result1 = if (collection.isEmpty()) 0
else collection.reduce { x, y -> x + y }
val result2 = collection.fold(0) { x, y -> x + y }
assertEquals(result1, result2)
Another difference that none of the other answers mentioned is the following:
The result of a reduce operation will always be of the same type (or a super type) as the data that is being reduced.
We can see that from the definition of the reduce method:
public inline fun <S, T : S> Iterable<T>.reduce(operation: (acc: S, T) -> S): S {
val iterator = this.iterator()
if (!iterator.hasNext()) throw UnsupportedOperationException("Empty collection can't be reduced.")
var accumulator: S = iterator.next()
while (iterator.hasNext()) {
accumulator = operation(accumulator, iterator.next())
}
return accumulator
}
On the other hand, the result of a fold operation can be anything, because there are no restrictions when it comes to setting up the initial value.
So, for example, let us say that we have a string that contains letters and digits. We want to calculate the sum of all the digits.
We can easily do that with fold:
val string = "1a2b3"
val result: Int = string.fold(0, { currentSum: Int, char: Char ->
if (char.isDigit())
currentSum + Character.getNumericValue(char)
else currentSum
})
//result is equal to 6
reduce - The reduce() method transforms a given collection into a single result.
val numbers: List<Int> = listOf(1, 2, 3)
val sum: Int = numbers.reduce { acc, next -> acc + next }
//sum is 6 now.
fold - What would happen in the previous case of an empty list? Actually, there’s no right value to return, so reduce() throws a RuntimeException
In this case, fold is a handy tool. You can put an initial value by it -
val sum: Int = numbers.fold(0, { acc, next -> acc + next })
Here, we’ve provided initial value. In contrast, to reduce(), if the collection is empty, the initial value will be returned which will prevent you from the RuntimeException.
Simple Answer
Result of both reduce and fold is "a list of items will be transformed into a single item".
In case of fold,we provide 1 extra parameter apart from list but in case of reduce,only items in list will be considered.
Fold
listOf("AC","Fridge").fold("stabilizer") { freeGift, itemBought -> freeGift + itemBought }
//output: stabilizerACFridge
In above case,think as AC,fridge bought from store & they give stabilizer as gift(this will be the parameter passed in the fold).so,you get all 3 items together.Please note that freeGift will be available only once i.e for the first iteration.
Reduce
In case of reduce,we get items in list as parameters and can perform required transformations on it.
listOf("AC","Fridge").reduce { itemBought1, itemBought2 -> itemBought1 + itemBought2 }
//output: ACFridge
The difference between the two functions is that fold() takes an initial value and uses it as the accumulated value on the first step, whereas the first step of reduce() uses the first and the second elements as operation arguments on the first step.
Related
I have a function with a for-loop:
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
var sum = 0
for (item in this) {
if (sumFunction(item))
sum += item
}
return sum
}
I want to know how I can write the above in functional style. I know that I have to use this.reduce(), but don't know exactly how to implement it.
return filter(sumFunction).sum()
Should be self-explanatory.
You can’t use reduce because it doesn’t let you reject the first element.
With fold it would be:
return fold(0) { a, b ->
if(sumFunction(b)) a + b else a
}
I can think if two ways to achieve that:
The first one is by using sumOf {...}:
.
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
return sumOf {
if (sumFunction(it)) it else 0
}
}
The second one is by using filter {...} then sum():
.
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
return filter(sumFunction).sum()
}
return this.reduce { sum, n -> if (sumFunction(n)) sum + n else 0}
If you really want to use reduce for some reason you can - but you need to add that 0 to the head of the list as your "start state":
fun List<Int>.customSum(sumFunction: (Int) -> Boolean): Int {
val stuff = listOf(0) + this
return stuff.reduce { a, b -> a + if (sumFunction(b)) b else 0 }
}
You have to do that because reduce is really there to combine a bunch of items, which is why for the first iteration you get the first two items in the list. You don't get to handle them separately, which is why you need to throw that 0 in there to get past that first step, and get to a point where you can just do your checking on the second parameter and ignore the first one, treating it as an accumulator instead of another item you also need to check.
That behaviour is what fold is for - with that function you pass in an initial state (which can be a completely different type from your items, since you're not just smushing them together to create a new value like with reduce) and then on each iteration you get that state and an item.
You can handle the item as you like, and then make changes to the accumulator state depending on the result. Which is exactly the behaviour of your for loop! fold is just a functional way to write one. Tenfour04's answer is how you'd do it - it's the right tool for the job here!
How to "take(N)" iteratively - get a Sequence<Sequence>, each inner sequences having next N elements?
I am writing a high-load application in Kotlin.
I have tens of thousands of entries to insert to a database.
I want to batch them by, say, 1000.
So I created a loop:
val itemsSeq = itemsList.iterator().asSequence()
while (true) {
log.debug("Taking $BATCH_SIZE from $itemsSeq")
val batchSeq = itemsSeq.take(BATCH_SIZE)
val squareBatch = applySomething(batchSeq, something)
?: break
}
fun applySomething(batch: Sequence<Item>, something: Something) {
/* Fully consumes batch. Bulk-loads from DB by IDs, applies, bulk-saves. */
}
I thought that take() would advance the itemsSeq and the next call to take() would give me a sequence "view" of itemsSeq starting at the 10th item.
But with this code, I am getting:
DEBUG Taking 10 from kotlin.sequences.ConstrainedOnceSequence#53fe15ff
Exception in thread "main" java.lang.IllegalStateException: This sequence can be consumed only once.
at kotlin.sequences.ConstrainedOnceSequence.iterator(SequencesJVM.kt:23)
at kotlin.sequences.TakeSequence$iterator$1.<init>(Sequences.kt:411)
at kotlin.sequences.TakeSequence.iterator(Sequences.kt:409)
So it seems that the take() "opens" the itemsSeq again, while that can be consumed only once.
As a workaround, I can use chunked():
public fun <T> Sequence<T>.chunked(size: Int): Sequence<List<T>> {
But I would prefer not to create Lists, rather Sequences.
What I am looking for is something between take() and chunked().
Is there anything such in Kotlin SDK?
I can possibly create my own sequence { ... } but for readability, I would prefer something built-in.
There is a way to construct a Sequence by handing it over an Iterator, see Sequence.
Given an iterator function constructs a Sequence that returns values
through the Iterator provided by that function. The values are
evaluated lazily, and the sequence is potentially infinite.
Wrapped in an extension function it could look like this:
fun <T> Iterable<T>.toValuesThroughIteratorSequence(): Sequence<T> {
val iterator = this.iterator()
return Sequence { iterator }
}
Quick test:
data class Test(val id: Int)
val itemsList = List(45) { Test(it) }
val batchSize = 10
val repetitions = itemsList.size.div(batchSize) + 1
val itemsSeq = itemsList.toValuesThroughIteratorSequence()
(0 until repetitions).forEach { index ->
val batchSeq = itemsSeq.take(batchSize)
println("Batch no. $index: " + batchSeq.map { it.id.toString().padStart(2, ' ') }.joinToString(" "))
}
Output:
Batch no. 0: 0 1 2 3 4 5 6 7 8 9
Batch no. 1: 10 11 12 13 14 15 16 17 18 19
Batch no. 2: 20 21 22 23 24 25 26 27 28 29
Batch no. 3: 30 31 32 33 34 35 36 37 38 39
Batch no. 4: 40 41 42 43 44
Background
First of all, we need to be aware there is a big difference between an object that we can iterate over and object that represents a "live" or already running iteration process. First group means Iterable (so List, Set and all other collections), Array, Flow, etc. Second group is mostly Iterator or old Java Enumeration. The difference could be also compared to file vs file pointer when reading or database table vs database cursor.
Sequence belongs to the first group. Sequence object does not represent a live, already started iteration, but just a set of elements. These elements can be produced lazily, sequence could have unbounded size and usually internally it works by using iterators, but conceptually sequence is not an iterator itself.
If we look into the documentation about sequences it clearly compares them to Iterable, not to Iterator. All standard ways to construct sequences like: sequenceOf(), sequence {}, Iterable.asSequence() produce sequences that return the same list of items every time we iterate over them. Iterator.asSequence() also follows this pattern, but because it can't re-produce same items twice, it is intentionally protected against iterating multiple times:
public fun <T> Iterator<T>.asSequence(): Sequence<T> = Sequence { this }.constrainOnce()
Problem
Your initial attempt with using take() didn't work, because this is a misuse of sequences. We expect that subsequent take() calls on the same sequence object will produce exactly the same items (usually), not next items. Similarly as we expect multiple take() calls on a list always produce same items, each time starting from the beginning.
Being more specific, your error was caused by above constrainOnce(). When we invoke take() multiple times on a sequence, it has to restart from the beginning, but it can't do this if it was created from an iterator, so Iterator.asSequence() explicitly disallows this.
Simple solution
To fix the problem, you can just skip constrainOnce() part, as suggested by #lukas.j. This solution is nice, because stdlib already provides tools like Sequence.take(), so if used carefully, this is the easiest to implement and it just works.
However, I personally consider this a kind of workaround, because the resulting sequence doesn't behave as sequences do. It is more like an iterator on steroids than a real sequence. You need to be careful when using this sequence with existing operators or 3rd party code, because such sequence may work differently than they expect and as a result, you may get incorrect results.
Advanced solution
We can follow your initial attempt of using subsequent take() calls. In this case our object is used for live iteration, so it is no longer a proper sequence, but rather an iterator. The only thing we miss in stdlib is a way to create a sub-iterator with a single chunk. We can implement it by ourselves:
fun main() {
val list = (0 .. 25).toList()
val iter = list.iterator()
while (iter.hasNext()) {
val chunk = iter.limited(10)
println(chunk.asSequence().toList())
}
}
fun <T> Iterator<T>.limited(n: Int): Iterator<T> = object : Iterator<T> {
var left = n
val iterator = this#limited
override fun next(): T {
if (left == 0)
throw NoSuchElementException()
left--
return iterator.next()
}
override fun hasNext(): Boolean {
return left > 0 && iterator.hasNext()
}
}
I named it limited(), because take() suggests we read items from the iterator. Instead, we only create another iterator on top of the provided iterator.
Of course, sequences are easier to use than iterators and typical solution to this problem is by using chunked(). With above limited() it is pretty straightforward to implement chunkedAsSequences():
fun main() {
val list = (0 .. 25).toList()
list.asSequence()
.chunkedAsSequences(10)
.forEach { println(it.toList()) }
}
fun <T> Sequence<T>.chunkedAsSequences(size: Int): Sequence<Sequence<T>> = sequence {
val iter = iterator()
while (iter.hasNext()) {
val chunk = iter.limited(size)
yield(chunk.asSequence())
chunk.forEach {} // required if chunk was not fully consumed
}
}
Please also note there is a tricky case of chunk being not fully consumed. chunkedAsSequences() is protected against this scenario. Previous simpler solutions aren't.
I find Lambda in Kotlin to be very confusing and on the top of it is "it".
There are two things I know about "it" and i.e.
If your Lambda has their own argument, you can replace its name with "it".
"It" is an automatically generated name for your Lambda, if it has
only one argument, and you don't specify a different argument name.
Still I don't understand what actually passes as "it".
For E.g. I wanted to apply modulo function on each element of a 3x3 matrix.
fun main(){
var result = Array(3) {
IntArray(3) { 3;2;4;6;7;9;12;11;23 }
}
result = Array(3){ IntArray(3) {it%2} }
println(result.joinToString("\n") { it.joinToString(" ") })
}
Here I assumed that "it" takes each element of the matrix which is clearly not the case as my output was:
0 1 0
0 1 0
0 1 0
So can you please explain me how "it" works, what is happening here? and what would be the correct way to implement this program?
Your line
result = Array(3){ IntArray(3) {it%2} }
isn't doing anything to the original Array that result is pointing at. You are creating a brand new group of array objects by calling the Array and IntArray constructors.
The lambda that you pass to the IntArray constructor has an input parameter that represents the array index, and the return value of your lambda is what will be put into the array at that index. So in this case it is the array index, and your lambda is returning 0 and 1 for even and odd indices respectively.
You are also instantiating your array incorrectly to begin with. Your lambda that you pass to that IntArray constructor is throwing away a bunch of pointless Int values and then returning 23 for each item. So you've created a 3x3 matrix that is completely filled with the number 23.
The correct syntax for creating an array with explicit values is to use arrayOf or intArrayOf.
val result = arrayOf(
intArrayOf(3, 2, 4),
intArrayOf(6, 7, 9),
intArrayOf(12, 11, 23)
)
To modify all the values of an array, you typically iterate the traditional way, not with a lambda:
for (innerArray in result) {
for (i in innerArray.indices)
innerArray[i] = innerArray[i] % 2
}
You were probably thinking of the map function, which lets you pass a lambda and returns a new List with the lambda function applied to every element of the input collection. Or when working with collections other than arrays, you can use forEach or onEach to iterate them without modifying them.
Let's say I have the function fun someValue() : Double.
I want to run this function n times and take the maximum of all the results.
I know I can safe it all in a list and use max(), but I don't want to waste that space (because it actually returns a bigger object).
Does there exist a library-function for this?
Maybe something like repeat(n){someValue()}.max()
Here are a couple of ways that don't allocate an intermediate array or list of all the values.
var x = someValue()
repeat(n - 1) {
x = max(x, someValue())
}
val y = (1 until n).fold(someValue()) { acc, _ -> max(acc, someValue()) }
val seq1 = sequenceOf(1, 2, 3)
val seq2 = sequenceOf(5, 6, 7)
sequenceOf(seq1, seq2).flatten().forEach { ... }
That's how I'm doing sequence concatenation but I'm worrying that it's actually copying elements, whereas all I need is an iterator that uses elements from the iterables (seq1, seq2) I gave it.
Is there such a function?
Your code doesn't copy the sequence elements, and sequenceOf(seq1, seq2).flatten() actually does what you want: it generates a sequence that takes items first from seq1 and then, when seq1 finishes, from seq2.
Also, operator + is implemented in exactly this way, so you can just use it:
(seq1 + seq2).forEach { ... }
The source of the operator is as expected:
public operator fun <T> Sequence<T>.plus(elements: Sequence<T>): Sequence<T> {
return sequenceOf(this, elements).flatten()
}
You can take a look at the implementation of .flatten() in stdlib that uses FlatteningSequence, which actually switches over the original sequences' iterators. The implementation can change over time, but Sequence is intended to be as lazy as possible, so you can expect it to behave in a similar way.
Example:
val a = generateSequence(0) { it + 1 }
val b = sequenceOf(1, 2, 3)
(a + b).take(3).forEach { println(it) }
Here, copying the first sequence can never succeed since it's infinite, and iterating over (a + b) takes items one by one from a.
Note, however, that .flatten() is implemented in a different way for Iterable, and it does copy the elements. Find more about the differences between Iterable and Sequence here.
Something else you might need to do is create a sequence of sequences:
val xs = sequence {
yield(1)
yield(2)
}
val twoXs = sequence {
yieldAll(xs)
// ... interesting things here ...
yieldAll(xs)
}
This doesn't do anything that xs + xs doesn't do, but it gives you a place to do more complex things.