Kotlin coroutines and Arrow are a nice way to avoid nesting flatmaps, introducing monadic comprehensions in Kotlin. However Kotlin's Flow type still relies on declarative flatmapping, so we get into a mixture of direct and declarative styles:
override suspend fun findAll(page: Pageable): Either<BusinessException, Flow<PageElement<ClientOut>>> = either {
val count = clientRepository.count().awaitSingle().bind()
return clientRepository.findByIdNotNull(page).asFlow()
.flatMapMerge { client ->
flow { emit(mapDetailedClientOut(client)) }
}
}
val count has been bound inside the either {...} comprehension. However, there doesn't seem to be a way to do the same with Flow, forcing us to nest a flatmapMerge().
Is there a way to do it, or is it planned to be somehow included in the near future?
Sadly there is currently no way to build comphrehensions for the KotlinX Flow datatype, since Coroutines in Kotlin only support for single-shot emission/bind.
Therefore it's only possible to build comphrensions for data types with 0..1 elements such as Either or Nullable, but not 0..N like the Flow or List data types.
Related
Is there a reason why there are multiple ways to do the same thing in Kotlin
val viaSequence = items.asSequence()
.filter { it%2 == 0 }
.map { it*2 }
.toList()
println(viaSequence)
val viaIterable = items.asIterable()
.filter { it%2 == 0 }
.map { it*2 }
.toList()
println(viaIterable)
val viaStream = items.stream()
.filter { it%2 == 0 }
.map { it*2 }
.toList()
println(viaStream)
I know that the following code creates a list on every step, which adds load to the GC, and as such should be avoided:
items.filter { it%2 == 0 }.map { it*2 }
Streams come from Java, where there are no inline functions, so Streams are the only way to use these functional chains on a collection in Java. Kotlin can do them directly on Iterables, which is better for performance in many cases because intermediate Stream objects don't need to be created.
Kotlin has Sequences as an alternative to Streams with these advantages:
They use null to represent missing items instead of Optional. Nullable values are easier to work with in Kotlin because of its null safety features. It's also better for performance to avoid wrapping all the items in the collection.
Some of the operators and aggregation functions are much more concise and avoid having to juggle generic types (compare Sequence.groupBy to Stream.collect).
There are more operators provided for Sequences, which result in performance advantages and simpler code by cutting out intermediate steps.
Many terminal operators are inline functions, so they omit the last wrapper that a Stream would need.
The sequence builder lets you create a complicated lazy sequence of items with simple sequential syntax in a coroutine. Very powerful.
They work back to Java 1.6. Streams require Java 8 or higher. This one is irrelevant for Kotlin 1.5 and higher since Kotlin now requires JDK 8 or higher.
The other answer mentions the advantages that Streams have.
Nice article comparing them here.
One of your three variants is the same as using the List itself:
items.asIterable()
.filter { it%2 == 0 }
Here, you're calling the exact same filter function as if you just called items.filter. The List itself is an Iterable so it's the Iterable filter that is called. This filter looks at all available elements and returns a complete list.
So the question is why we have both streams and sequences.
Streams are part of Java. Many of the terminal operations produce Optional. Other operations, such as toList(), will produce non-null-safe platform types such as List<Integer!>. On the other hand, sequences are native to Kotlin, and they can be used with Kotlin's own compile-time null-safety features. Also, sequences are available in non-JVM variants of Kotlin.
The Kotlin designers probably had to create a new class, because if they had just added new operations to Stream, e.g. as extension functions, they would clash with existing names (e.g. max() returns Optional in Java, and that's not ideal for Kotlin, but choosing a name other than the natural name max wouldn't be ideal either.)
So in most cases you should prefer sequences as they are more Kotlin-idiomatic. However, there are some things Java Streams can do that aren't yet available to sequences (for example, SummaryStatistics, or parallel operations). When you need an operation that is only available in Streams, but you have a Sequence, you can convert the Sequence to a Stream using asStream() (as well as vice versa).
Another advantage of Streams is that you can use primitive streams such as IntStream, to avoid unnecessary boxing/unboxing.
I am new to Arrow and try to establish my mental model of how its effects system works; in particular, how it leverages Kotlin's suspend system. My very vague understanding is as follows; if would be great if someone could confirm, clarify, or correct it:
Because Kotlin does not support higher-kinded types, implementing applicatives and monads as type classes is cumbersome. Instead, arrow derives its monad functionality (bind and return) for all of Arrow's monadic types from the continuation primitive offered by Kotlin's suspend mechanism. Ist this correct? In particular, short-circuiting behavior (e.g., for nullable or either) is somehow implemented as a delimited continuation. I did not quite get which particular feature of Kotlin's suspend machinery comes into play here.
If the above is broadly correct, I have two follow-up questions: How should I contain the scope of non-IO monadic operations? Take a simple object construction and validation example:
suspend fun mkMessage(msgType: String, appRef: String, pId: String): Message? = nullable {
val type = MessageType.mkMessageType(msgType).bind()
val ref = ApplRefe.mkAppRef((appRef)).bind()
val id = Id.mkId(pId).bind()
Message(type, ref, id)
}
In Haskell's do-notation, this would be
mkMessage :: String -> String -> String -> Maybe Message
mkMessage msgType appRef pId = do
type <- mkMessageType msgType
ref <- mkAppRef appRef
id <- mkId pId
return (Message type ref id)
In both cases, the function returns the monad type (a nullable value, resp. Maybe). However, while I can use the pure function in Haskell anywhere I see fit, the suspend function in Kotlin can only be called from within a suspend function. In this way, a simple, non-IO monad comprehension in Arrow behaves like an IO monad that must be threaded throughout my code base; I suppose this results because the suspend mechanism was designed for actual IO operations. What is the recommended way to implement non-IO monad comprehensions in Arrow without making all functions into suspend functions? Or is this actually the way to go?
Second: If in addition to non-IO monads (nullable, reader, etc.), I want to have IO - say, reading in a file and parsing it - how would i combine these two effects? Is it correct to say that there would be multiple suspend scopes corresponding to the different monads involved, and I would need to somehow nest these scopes, like I would stack monad transformers in Haskell?
The two questions above probably mean that I am still lacking a mental model that bridges between the continuation-based implementation atop the Kotlin's suspend mechanism with the generic monad-as-typeclass implementation in Haskell.
schuster,
You're correct that Arrow uses the suspension feature from Kotlin to encode something like monad comphrensions.
To answer your first question:
Kotlin has suspend in the language (and Kotlin Std), by default suspend can only be called from other suspend code. However, the compiler also has a feature called RestrictsSuspension, this disallows for mixing suspend scopes and thus disallows the ablity to combine IO and Either for example. We expose a secondary DSL, either.eager which is encoded using RestrictsSuspension and it disallows calling foreign suspend functions.
This allows you to encode mkMessage :: String -> String -> String -> Maybe Message.
fun mkMessage(msgType: String, appRef: String, pId: String): Message? = nullable.eager {
val type = MessageType.mkMessageType(msgType).bind()
val ref = ApplRefe.mkAppRef((appRef)).bind()
val id = Id.mkId(pId).bind()
Message(type, ref, id)
}
To answer your second question:
IO as a data type is not needed in Kotlin, since suspend can implement all IO operations in a referential transparent way like it works in Haskell.
The compiler also makes a lot optimisations in the runtime, just like Haskell does for IO.
So the signature suspend fun example(): Either<Error, Value> is the equivalent of EitherT IO Error Value in Haskell.
The IO operations are however not implemented in the Kotlin Std, but in a library KotlinX Coroutines, and Arrow Fx Coroutines also offers some data types and higher-level operations such as parTraverse defined on top of KotlinX Coroutines.
It's slightly different than in Haskell, since we can mix effects instead of stacking them with monad transformers. This means that we can call IO operations from within Either operations. This is due to special functionality, and optimisations the compiler can make in the suspension system. This blog explains how that optimisation works, and why it's so powerful. https://nomisrev.github.io/inline-and-suspend/
Here is also some more background on Continuations, and tagless encodings in Kotlin. https://nomisrev.github.io/continuation-monad-in-kotlin/
I hope that fully answers your question.
I don't think I can answer everything you asked, but I'll do my best for the parts that I do know how to answer.
What is the recommended way to implement non-IO monad comprehensions in Arrow without making all functions into suspend functions? Or is this actually the way to go?
you can use nullable.eager and either.eager respectively for pure code. Using nullable/either (without .eager) allows you to call suspend functions inside. Using eager means you can only call non-suspend functions. (not all effectual functions in kotlin are marked suspend)
Second: If in addition to non-IO monads (nullable, reader, etc.), I want to have IO - say, reading in a file and parsing it - how would i combine these two effects? Is it correct to say that there would be multiple suspend scopes corresponding to the different monads involved, and I would need to somehow nest these scopes, like I would stack monad transformers in Haskell?
You can use extension functions to emulate Reader. For example:
suspend fun <R> R.doSomething(i: Int): Either<Error, String> = TODO()
combines Reader + IO + Either. You can find a bigger example here from Simon, an Arrow maintainer.
I wonder if it is possible to implement something similar to the do-notation of Haskell in Kotlin on Lists or List-Like structures with monadic properties.
Take following example:
fun <A, B> cartesianProduct(xs: List<A>, ys: List<B>): List<Pair<A, B>> =
xs.flatMap { x -> ys.flatMap { y -> listOf(x to y) } }
It would be nice if I could write something like
suspend fun <A, B> cartesianProduct(xs: List<A>, ys: List<B>): List<Pair<A, B>> =
list {
val x = xs.bind()
val y = xs.bind()
yield(x to y)
}
Arrow-Kt defines similar comprehensions using coroutines for either, nullable, option and eval. I looked at the implementation and also its Effect documentation, but I have trouble to translate the concept to Lists. Is this even possible in kotlin?
It's not possible at the moment to implement monad comprehension for List, Flow, and other non-deterministic data structures that emit more than one value. The current implementation of continuations in Kotlin is single shot only. This means a continuation can resume a program with a single emitted value. Resuming the program more than once requires hijacking the continuation stack labels with reflection in order to replay their state in the second resumption. Additionally replaying a block in which a multishot data type is binding would replay all effects previous to the bind since the block has to emit again.
list {
println("printed 3 times and not cool")
val a = listOf(1, 2, 3).bind()
a
}
The arrow-continuations library already includes a MultiShot delimited scope for reset/shift but it's currently internal since is not safe until Kotlin suspension or continuations provide the ability to multishot without replaying the current block. Alternatively we would need real for comprehensions or a similar structure to enforce binds happen before other code which would also solve the block replaying issue.
The Effect interface ultimately delegates to one of these scopes for its implementation. The current versions of Reset.suspended and Reset.restricted are single shot.
New to Kotlin. I was thinking if there is a shorter way to write the following piece of code. It means to categorize enum values into a multi-value map.
fun androidPermissionsByCategory(): Map<String, List<String>> {
val result = hashMapOf<String, MutableList<String>>()
enumValues<AndroidPermission>().onEach {
result.getOrPut(it.permissionGroup, { mutableListOf() }).add(it.value())
}
return result
}
Suggestions?
That's good, solid code (especially if you're not experienced in Kotlin): good use of types, and the getOrPut() function. (The only tweak I'd suggest would be to change hashMapOf() to mutableMapOf(), since you don't care about the exact type of map. You could also replace the add() call with += operator, though that's more disputable.)
However, there's a shorter alternative in a more functional style:
fun androidPermissionsByCategory(): Map<String, List<String>>
= enumValues<AndroidPermission>()
.groupBy({ it.permissionGroup }, { it.value() })
(Disclaimer: I don't have Android libs, so I can't test this exactly.)
This uses the standard library's groupBy() function, which does exactly what you want: it compares items (using a key-selector lambda you provide), and uses it to create a multimap from them.
Here we're using the enum's permissionGroup field as the key.
We're also using the optional second parameter to transform the values in the multimap, in this case getting their value().
You'll find that Kotlin has functional alternatives to many of the common imperative patterns for constructing and transforming maps, lists, and similar structures; these are often more concise, more declarative, and easier to read. Any time you find yourself looping over a list or similar, it's worth asking whether there's a better way. (As you have here! Your intuition is clearly working well :-)
I am learning Kotlin and it is looking likely I may want to use it as my primary language within the next year. However, I keep getting conflicting research that Kotlin does or does not have immutable collections and I'm trying to figure out if I need to use Google Guava.
Can someone please give me some guidance on this? Does it by default use Immutable collections? What operators return mutable or immutable collections? If not, are there plans to implement them?
Kotlin's List from the standard library is readonly:
interface List<out E> : Collection<E> (source)
A generic ordered collection of elements. Methods in this interface
support only read-only access to the list; read/write access is
supported through the MutableList interface.
Parameters
E - the type of elements contained in the list.
As mentioned, there is also the MutableList
interface MutableList<E> : List<E>, MutableCollection<E> (source)
A generic ordered collection of elements that supports adding and
removing elements.
Parameters
E - the type of elements contained in the list.
Due to this, Kotlin enforces readonly behaviour through its interfaces, instead of throwing Exceptions on runtime like default Java implementations do.
Likewise, there is a MutableCollection, MutableIterable, MutableIterator, MutableListIterator, MutableMap, and MutableSet, see the stdlib documentation.
It is confusing but there are three, not two types of immutability:
Mutable - you are supposed to change the collection (Kotlin's MutableList)
Readonly - you are NOT supposed to change it (Kotlin's List) but something may (cast to Mutable, or change from Java)
Immutable - no one can change it (Guavas's immutable collections)
So in case (2) List is just an interface that does not have mutating methods, but you can change the instance if you cast it to MutableList.
With Guava (case (3)) you are safe from anybody to change the collection, even with a cast or from another thread.
Kotlin chose to be readonly in order to use Java collections directly, so there is no overhead or conversion in using Java collections..
As you see in other answers, Kotlin has readonly interfaces to mutable collections that let you view a collection through a readonly lens. But the collection can be bypassed via casting or manipulated from Java. But in cooperative Kotlin code that is fine, most uses do not need truly immutable collections and if your team avoids casts to the mutable form of the collection then maybe you don't need fully immutable collections.
The Kotlin collections allow both copy-on-change mutations, as well as lazy mutations. So to answer part of your questions, things like filter, map, flatmap, operators + - all create copies when used against non lazy collections. When used on a Sequence they modify the values as the collection as it is accessed and continue to be lazy (resulting in another Sequence). Although for a Sequence, calling anything such as toList, toSet, toMap will result in the final copy being made. By naming convention almost anything that starts with to is making a copy.
In other words, most operators return you the same type as you started with, and if that type is "readonly" then you will receive a copy. If that type is lazy, then you will lazily apply the change until you demand the collection in its entirety.
Some people want them for other reasons, such as parallel processing. In those cases, it might be best to look at really high performance collections designed just for those purposes. And only use them in those cases, not in all general cases.
In the JVM world it is hard to avoid interop with libraries that want standard Java collections, and converting to/from these collections adds a lot of pain and overhead for libraries that do not support the common interfaces. Kotlin gives a good mix of interop and lack of conversion, with readonly protection by contract.
So if you can't avoid wanting immutable collections, Kotlin easily works with anything from the JVM space:
Guava (https://github.com/google/guava)
Dexx a port of the Scala collections to Java (https://github.com/andrewoma/dexx) with Kotlin helpers (https://github.com/andrewoma/dexx/blob/master/kollection/README.md)
Eclipse Collections (formerly GS-Collections) a really high performance, JDK compatible, top performer in parallel processing with immutable and mutable variations (home: https://www.eclipse.org/collections/ and Github: https://github.com/eclipse/eclipse-collections)
PCollections (http://pcollections.org/)
Also, the Kotlin team is working on Immutable Collections natively for Kotlin, that effort can be seen here:
https://github.com/Kotlin/kotlinx.collections.immutable
There are many other collection frameworks out there for all different needs and constraints, Google is your friend for finding them. There is no reason the Kotlin team needs to reinvent them for its standard library. You have a lot of options, and they specialize in different things such as performance, memory use, not-boxing, immutability, etc. "Choice is Good" ... therefore some others: HPCC, HPCC-RT, FastUtil, Koloboke, Trove and more...
There are even efforts like Pure4J which since Kotlin supports Annotation processing now, maybe can have a port to Kotlin for similar ideals.
Kotlin 1.0 will not have immutable collections in the standard library. It does, however, have read-only and mutable interfaces. And nothing prevents you from using third party immutable collection libraries.
Methods in Kotlin's List interface "support only read-only access to the list" while methods in its MutableList interface support "adding and removing elements". Both of these, however, are only interfaces.
Kotlin's List interface enforces read-only access at compile-time instead of deferring such checks to run-time like java.util.Collections.unmodifiableList(java.util.List) (which "returns an unmodifiable view of the specified list... [where] attempts to modify the returned list... result in an UnsupportedOperationException." It does not enforce immutability.
Consider the following Kotlin code:
import com.google.common.collect.ImmutableList
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
fun main(args: Array<String>) {
val readOnlyList: List<Int> = arrayListOf(1, 2, 3)
val mutableList: MutableList<Int> = readOnlyList as MutableList<Int>
val immutableList: ImmutableList<Int> = ImmutableList.copyOf(readOnlyList)
assertEquals(readOnlyList, mutableList)
assertEquals(mutableList, immutableList)
// readOnlyList.add(4) // Kotlin: Unresolved reference: add
mutableList.add(4)
assertFailsWith(UnsupportedOperationException::class) { immutableList.add(4) }
assertEquals(readOnlyList, mutableList)
assertEquals(mutableList, immutableList)
}
Notice how readOnlyList is a List and methods such as add cannot be resolved (and won't compile), mutableList can naturally be mutated, and add on immutableList (from Google Guava) can also be resolved at compile-time but throws an exception at run-time.
All of the above assertions pass with exception of the last one which results in Exception in thread "main" java.lang.AssertionError: Expected <[1, 2, 3, 4]>, actual <[1, 2, 3]>. i.e. We successfully mutated a read-only List!
Note that using listOf(...) instead of arrayListOf(...) returns an effectively immutable list as you cannot cast it to any mutable list type. However, using the List interface for a variable does not prevent a MutableList from being assigned to it (MutableList<E> extends List<E>).
Finally, note that an interface in Kotlin (as well as in Java) cannot enforce immutability as it "cannot store state" (see Interfaces). As such, if you want an immutable collection you need to use something like those provided by Google Guava.
See also ImmutableCollectionsExplained · google/guava Wiki · GitHub
NOTE: This answer is here because the code is simple and open-source and you can use this idea to make your collections that you create immutable. It is not intended only as an advertisement of the library.
In Klutter library, are new Kotlin Immutable wrappers that use Kotlin delegation to wrap a existing Kotlin collection interface with a protective layer without any performance hit. There is then no way to cast the collection, its iterator, or other collections it might return into something that could be modified. They become in effect Immutable.
Klutter 1.20.0 released which adds immutable protectors for existing collections, based on a SO answer by #miensol provides a light-weight delegate around collections that prevents any avenue of modification including casting to a mutable type then modifying. And Klutter goes a step further by protecting sub collections such as iterator, listIterator, entrySet, etc. All of those doors are closed and using Kotlin delegation for most methods you take no hit in performance. Simply call myCollection.asReadonly() (protect) or myCollection.toImmutable() (copy then protect) and the result is the same interface but protected.
Here is an example from the code showing how simply the technique is, by basically delegating the interface to the actual class while overriding mutation methods and any sub-collections returned are wrapped on the fly.
/**
* Wraps a List with a lightweight delegating class that prevents casting back to mutable type
*/
open class ReadOnlyList <T>(protected val delegate: List<T>) : List<T> by delegate, ReadOnly, Serializable {
companion object {
#JvmField val serialVersionUID = 1L
}
override fun iterator(): Iterator<T> {
return delegate.iterator().asReadOnly()
}
override fun listIterator(): ListIterator<T> {
return delegate.listIterator().asReadOnly()
}
override fun listIterator(index: Int): ListIterator<T> {
return delegate.listIterator(index).asReadOnly()
}
override fun subList(fromIndex: Int, toIndex: Int): List<T> {
return delegate.subList(fromIndex, toIndex).asReadOnly()
}
override fun toString(): String {
return "ReadOnly: ${super.toString()}"
}
override fun equals(other: Any?): Boolean {
return delegate.equals(other)
}
override fun hashCode(): Int {
return delegate.hashCode()
}
}
Along with helper extension functions to make it easy to access:
/**
* Wraps the List with a lightweight delegating class that prevents casting back to mutable type,
* specializing for the case of the RandomAccess marker interface being retained if it was there originally
*/
fun <T> List<T>.asReadOnly(): List<T> {
return this.whenNotAlreadyReadOnly {
when (it) {
is RandomAccess -> ReadOnlyRandomAccessList(it)
else -> ReadOnlyList(it)
}
}
}
/**
* Copies the List and then wraps with a lightweight delegating class that prevents casting back to mutable type,
* specializing for the case of the RandomAccess marker interface being retained if it was there originally
*/
#Suppress("UNCHECKED_CAST")
fun <T> List<T>.toImmutable(): List<T> {
val copy = when (this) {
is RandomAccess -> ArrayList<T>(this)
else -> this.toList()
}
return when (copy) {
is RandomAccess -> ReadOnlyRandomAccessList(copy)
else -> ReadOnlyList(copy)
}
}
You can see the idea and extrapolate to create the missing classes from this code which repeats the patterns for other referenced types. Or view the full code here:
https://github.com/kohesive/klutter/blob/master/core-jdk6/src/main/kotlin/uy/klutter/core/common/Immutable.kt
And with tests showing some of the tricks that allowed modifications before, but now do not, along with the blocked casts and calls using these wrappers.
https://github.com/kohesive/klutter/blob/master/core-jdk6/src/test/kotlin/uy/klutter/core/collections/TestImmutable.kt
Now we have https://github.com/Kotlin/kotlinx.collections.immutable.
fun Iterable<T>.toImmutableList(): ImmutableList<T>
fun Iterable<T>.toImmutableSet(): ImmutableSet<T>
fun Iterable<T>.toPersistentList(): PersistentList<T>
fun Iterable<T>.toPersistentSet(): PersistentSet<T>