I want to create a function with two overloads:
One that takes a suspend function that takes no arguments and returns Any
One that takes a regular function that takes one argument: Any and returns Any
This is the code I tried to write:
import kotlinx.coroutines.flow.Flow
fun test(handler: suspend () -> Any) { handler.toString() }
fun test(handler: (Any) -> Flow<Any>) { handler.toString() }
And this is the error i got:
Platform declaration clash: The following declarations have the same
JVM signature (test(Lkotlin/jvm/functions/Function1;)V):
fun test(handler: (Any) -> Flow): Unit defined in se.vermiculus.vericlear.webserver.utility
fun test(handler: suspend () -> Any): Unit defined in se.vermiculus.vericlear.webserver.utility
The reason is that suspending lambda types are represented in the JVM as having an extra Continuation parameter. See also the example presented here, or just try to call your Kotlin function from a Java file in IntelliJ.
What happens under the hood when you call the suspending lambda is, all the code after the call gets wrapped up in a "continuation", and also gets passed to the lambda. That's the code that will "continue" to run after the suspension is done.
Because of the extra parameter in the suspend lambda, both of your functions have 1 parameter in the JVM representation, and hence are both represented as Function1s, hence the conflict.
In Java, suspend () -> Any would become
Function1<? super Continuation<? super Object>, ? extends Object>
and (Any) -> Flow<Any> becomes
Function1<? super Object, ? extends Flow<? extends Object>>
Though it's not like that's any useful for solving the conflict :) I recommend just sticking some #JvmNames on it.
This is because both functions are translated into the same bytecode.
Something like this:
import kotlin.jvm.functions.Function1;
import kotlin.jvm.internal.Intrinsics;
import org.jetbrains.annotations.NotNull;
public final void test(#NotNull Function1 handler) {
Intrinsics.checkNotNullParameter(handler, "handler");
handler.toString();
}
Related
With Kotlin 1.4 we now have Functional Interfaces
fun interface Todo {
fun run()
}
fun runBlock(todo: Todo){
if(condition)
todo.run()
}
fun runBlock{
println("Hello world")
}
Before i was always using (T) -> T
inline fun runBlock(block: ()-> Unit){
if(condition)
block()
}
fun runBlock{
println("Hello world")
}
So basically I can make the same task with both methods , there is any performance advantage by using Functional SAM() Interfaces over Function Type?.
It's a performance dis-advantage because the lambda is no longer inlined (unless the JIT decides to, but it won't be instant). Even if you mark runBlock as inline, the compiler will warn you that the argument won't be inlined.
There are only two reasons to use fun interfaces instead of function types:
Backwards compatibility when porting code using Java functional interfaces.
Not exposing Kotlin function types in API intended for use from Java.
To expand on point 1: before Kotlin 1.4 it was advised to keep functional interfaces as Java code, even if all your other code was Kotlin. This way you could use lambdas for parameters of those types both in Java and Kotlin code. But if you declared the interface in Kotlin, you could only use lambdas for them in Java.
https://kotlinlang.org/docs/reference/whatsnew14.html#sam-conversions-for-kotlin-interfaces
the compiler automatically converts the lambda to an instance of the class that implements the abstract member function.
So, no performance advantage, it’s the same thing as before. The compiler now does what you had to do before.
As other answers and comments have pointed out, in your case, using inlined lambda is faster, since there is no function call overhead when invoking it.
However, there is one specific use case where using SAM interface is faster, that is when you 1. do not inline the lambda and 2. the arguments/return value of the lambda is a primitive (or any other type that may cause boxing when used with generics).
For example, using SAM interface like so:
fun interface Foo() {
fun run(i: Int): Int
}
fun foo(fn: Foo) {
fn.run(42)
}
foo { it * 2 }
Will not cause any boxing when invoked, while:
fun foo(fn: (Int) -> Int) {
fn(42)
}
foo { it * 2 }
Will box the integer argument since (Int) -> Int is essentially Function1<Integer, Integer> in Java, which uses generic.
I'm looking into launch coroutine builder which takes coroutine code as block: suspend CoroutineScope.() -> Unit. We usually pass the code as lambda. However, I was wondering how to pass this function as explicit parameter to launch function.
coroutineScope {
launch(block = ::myFunction)
}
suspend fun CoroutineScope.myFunction(): Unit {
// coroutine code
}
It gives following error
Type mismatch.
Required:
suspend CoroutineScope.() → Unit
Found:
KSuspendFunction0<Unit>
What is it that i'm missing?
The syntax for extension function references is the same as for member functions:
launch(block = CoroutineScope::myFunction)
How about this way?
coroutineScope {
launch(block = myFunction())
}
fun myFunction(): suspend CoroutineScope.() -> Unit = {
for(i in 3 downTo 1) {
println("$i")
delay(1000)
}
}
According to kotlin doc, launch function with parameter is function type: CoroutineScope.() → Unit, is one function type with receiver.
Function types with receiver, such as A.(B) -> C, can be instantiated with a special form of function literals – function literals with receiver.
The same article also noted the following:
Using a callable reference to an existing declaration:
a top-level, local, member, or extension function: ::isOdd, String::toInt,
a top-level, member, or extension property: List<Int>::size,
a constructor: ::Regex
These include bound callable references that point to a member of a particular instance: foo::toString.
but not adaptive to "function literals with receiver".
so one way to make it work:
coroutineScope {
launch {
myFunction()
}
}
UPDATE2 - yep, the following extension function does what you need.
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
kotlinFunction!!.callSuspend(obj, *args)
be nice if the lib doc for callSuspend
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect.full/call-suspend.html
explicitly stated that the receiver if applicable is first in the vararg list. but i'm happy its now possible to do it in 1.3. And it's baked into the Kotlin API now so you longer have to the reflective hack to pull out the backing continuation and invoke the transformed java method through the Java reflection API.
UPDATE - i can see from another stackoverflow question that Kotlin 1.3 has KFunction.callSuspend, anyone know if that can be used in my case and invoked against a reflective method? In which case how can it be called?
val ret = method.kotlinFunction?.callSuspend(/*???*/)
how do you bind the target object? method.invoke takes the target followed by vargargs for the method parameters, but callSuspend only takes varargs.
or is callSuspend just for standalone functions?
I'm writing a fairly sophisticated remoting framework in kotlin where a class implementing an interface (with annotation similar to JAX_RS) can be efficiently remoted over several different transports including HTTP2 and VERTX, and called through a stub proxy implementing the interface so its completely transparent to the calling code. There are reasons i'm writing a custom implementation which i don't need to get into. Everything's based on suspending functions and coroutines - which are awesome.
In order to do this the kotlin interface is used to auto generate a transparent proxy stub on the client side and a dispatcher on the endpoint side. The dispatcher automatically enforces security by looking at security annotations on the interface methods. Identity data can be accessed from the implementation code through the coroutine context.
Everything's working, except the dispatcher obviously has to use reflection to invoke the suspended function on the implementing class. I cannot figure out how to propagate the coroutine context across the reflective suspended invocation. Not only that, the default ThreadPool for coroutines doesn't seem to be used either - instead it uses some fork-join pool.
Coroutines are implemented great in my opinion, but when you start doing the low level stuff you can't avoid the ugly underbelly. The other thing i noticed is a default method in a kotlin interface doesn't map to a default method in the underlying generated java interface. Which also caused my some grief, but thats a seperate issue.
Anyway - if anyone knows how to fix this final issue? Thanks.
// attach an extension function to
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutine { cont ->
println("in thread "+Thread.currentThread().name)
val ret=invoke(obj, *args, cont)
cont.resume(ret)
}
//....
withContext(kc) {
// kc NOT propagated through method invocation...
meth.invokeSuspend(rec.ob, args)!!
}
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutine { cont ->
val ret=invoke(obj, *args, cont)
cont.resume(ret)
}
There are two main mistakes here:
you expect invoke() to return the value that you must resume the continuation with
you call the user-level suspendCoroutine function instead of the low-level suspendCoroutineUninterceptedOrReturn
Behind these mistakes there may be a deeper misunderstanding of the coroutine suspension mechanism, so let me try to elaborate on that. This is a way to correct your code, taken from the implementation of KCallable.callSuspend:
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutineUninterceptedOrReturn { cont -> invoke(obj, *args, cont) }
Note the main feature of this code: it just passes the continuation to the invoked function and never tries to resume it with the result of the invocation.
Now, how does this manage to work? There are two factors:
If the called suspendable function doesn't actually suspend, it will simply return its result, and this will become the result of your invokeSuspend function.
If it does suspend, when it resumes, the suspendable function will on its own use the continuation you passed in and invoke its resume method with the result.
If it decides to suspend, the suspendable function immediately returns the special COROUTINE_SUSPENDED constant. suspendCoroutineUninterceptedOrReturn interprets this value as necessary to cooperate with the coroutine suspension mechanism. Specifically, it makes your function return the same constant to its caller (it does this whether or not your code actually returns the result of the suspendable function). This way the constant propagates all the way up the call stack until it reaches the non-suspendable function that started the coroutine. This is typically an event loop, and now it will be able to go on processing the next event.
how do you bind the target object? method.invoke takes the target followed by vargargs for the method parameters, but callSuspend only takes varargs.
The answer to this is documented under KCallable.parameters:
/**
* Parameters required to make a call to this callable.
* If this callable requires a `this` instance or an extension receiver parameter,
* they come first in the list in that order.
*/
public val parameters: List<KParameter>
So this is how simple it is to implement your invokeSuspend in terms of KCallable.callSuspend:
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
kotlinFunction!!.callSuspend(obj, *args)
Repository.kt
suspend fun getTestRepository(): X
suspend fun getTestWithParamRepository(a: String): X
Service.kt
fun getTestService(lambda: suspend () -> X) { //... }
Using
Suspend function 'getTestWithParamRepository' should be called only
from a coroutine or another suspend function
getTestService(repository::getTestRepository }
// Surround with lambda
getTestService { repository.getTestWithParamRepository("") }
GL
I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}
This code compiles with a warning (insignificant performance impact):
inline fun test(noinline f: () -> Unit) {
thread(block = f)
}
This code does not compile (illegal usage of inline-parameter):
inline fun test(crossinline f: () -> Unit) {
thread(block = f)
}
This code compiles with a warning (insignificant performance impact):
inline fun test(noinline f: () -> Unit) {
thread { f() }
}
This code compiles with no warning or error:
inline fun test(crossinline f: () -> Unit) {
thread { f() }
}
Here are my questions:
How come (2) does not compile but (4) does?
What exactly is the difference between noinline and crossinline?
If (3) does not generates a no performance improvements, why would (4) do?
From the inline functions reference:
Note that some inline functions may call the lambdas passed to them as parameters not directly from the function body, but from another execution context, such as a local object or a nested function. In such cases, non-local control flow is also not allowed in the lambdas. To indicate that, the lambda parameter needs to be marked with the crossinline modifier
Hence, example 2. doesn't compile, since crossinline enforces only local control flow, and the expression block = f violates that. Example 1 compiles, since noinline doesn't require such behavior (obviously, since it's an ordinary function parameter).
Examples 1 and 3 do not generate any performance improvements, since the only lambda parameter is marked noinline, rendering the inline modifier of the function useless and redundant - the compiler would like to inline something, but everything that could be has been marked not to be inlined.
Consider two functions, A and B
A
inline fun test(noinline f: () -> Unit) {
thread { f() }
}
B
fun test(f: () -> Unit) {
thread { f() }
}
Function A behaves like function B in the sense that the parameter f will not be inlined (the B function doesn't inline the body of test whereas in the A function, the body: thread { f() } still gets inlined).
Now, this is not true in the example 4, since the crossinline f: () -> Unit parameter can be inlined, it just cannot violate the aforementioned non-local control flow rule (like assigning new value to a global variable). And if it can be inlined, the compiler assumes performance improvements and does not warn like in the example 3.
Let me try to explain this by example: I'll go through each of your examples and describe what it orders the compiler to do. First, here's some code that uses your function:
fun main(args: Array<String>) {
test {
println("start")
println("stop")
}
}
Now let's go through your variants. I'll call the functions from your examples test1..test4 and I'll show in pseudocode what the above main function would compile into.
1. noinline, block = f
This code compiles with a warning (insignificant performance impact)
inline fun test1(noinline f: () -> Unit) {
thread(block = f)
}
fun compiledMain1() {
val myBlock = {
println("start")
println("stop")
}
thread(block = myBlock)
}
First, note there's no evidence of inline fun test1 even existing. Inline functions aren't really "called": it's as if the code of test1 was written inside main(). On the other hand, the noinline lambda parameter behaves same as without inlining: you create a lambda object and pass it to the thread function.
2. crossinline, block = f
This code does not compile (illegal usage of inline-parameter)
inline fun test2(crossinline f: () -> Unit) {
thread(block = f)
}
fun compiledMain2() {
thread(block =
println("start")
println("stop")
)
}
I hope I managed to conjure what happens here: you requested to copy-paste the code of the block into a place that expects a value. It's just syntactic garbage. Reason: with or without crossinline you request that the block be copy-pasted into the place where it's used. This modifier just restricts what you can write inside the block (no returns etc.)
3. noinline, { f() }
This code compiles with a warning (insignificant performance impact)
inline fun test3(noinline f: () -> Unit) {
thread { f() }
}
fun compiledMain3() {
val myBlock = {
println("start")
println("stop")
}
thread { myBlock() }
}
We're back to noinline here so things are straightforward again. You create a regular lambda object myBlock, then you create another regular lambda object that delegates to it: { myBlock() }, then you pass this to thread().
4. crossinline, { f() }
This code compiles with no warning or error
inline fun test4(crossinline f: () -> Unit) {
thread { f() }
}
fun compiledMain4() {
thread {
println("start")
println("stop")
}
}
Finally this example demonstrates what crossinline is for. The code of test4 is inlined into main, the code of the block is inlined into the place where it's used. But, since it's used inside the definition of a regular lambda object, it can't contain non-local control flow.
About the Performance Impact
The Kotlin team wants you to use the inlining feature sensibly. With inlining the size of the compiled code can explode dramatically and even hit the JVM limits of up to 64K bytecode instructions per method. The main use case is higher-order functions that avoid the cost of creating an actual lambda object, only to discard it right after a single function call which happens right away.
Whenever you declare an inline fun without any inline lambdas, inlining itself has lost its purpose. The compiler warns you about it.
Q1: How come (2) does not compile but (4) does?
From their doc:
Inlinable lambdas can only be called inside the inline functions or passed as inlinable arguments...
Answer:
The method thread(...) is not an inline method so you won't be able to pass f as an argument.
Q2: What exactly is the difference between noinline and crossinline?
Answer:
noinline will prevent the inlining of lambdas. This becomes useful when you have multiple lambda arguments and you want only some of the lambdas passed to an inline function to be inlined.
crossinline is used to mark lambdas that mustn't allow non-local returns, especially when such lambda is passed to another execution context. In other words, you won't be able to do a use a return in such lambdas. Using your example:
inline fun test(crossinline f: () -> Unit) {
thread { f() }
}
//another method in the class
fun foo() {
test{
//Error! return is not allowed here.
return
}
}
Q3: If (3) does not generates a no performance improvements, why would (4) do?
Answer:
That is because the only lambda you have in (3) has been marked with noinline which means you'll have the overhead cost of creating the Function object to house the body of your lamda. For (4) the lambda is still inlined (performance improvement) only that it won't allow non-local returns.
To the first and second question
How come (2) does not compile but (4) does?.. difference between noinline and crossinline
2. inline fun test(crossinline f: () -> Unit) {
thread(block = f)
}
4. inline fun test(crossinline f: () -> Unit) {
thread { f() }
}
Both cases have inline modifier instructing to inline both the function test and its argument lambda f. From kotlin reference:
The inline modifier affects both the function itself and the lambdas
passed to it: all of those will be inlined into the call site.
So the compiler is instructed to place the code (inline) instead of constructing and invoking a function object for f. crossinline modifier is only for inlined things: it just says that the passed lambda (in f parameter) should not have non-local returns (which "normal" inlined lambdas may have). crossinline can be thought of as something like this (instruction to the compiler ): “ do inline but there is a restriction that it is crossing the invoker context and so make sure the lambda does not have non-local returns.
On a side note, thread seems like a conceptually illustrative example for crossinline because obviously returning from some code (passed in f) later on a different thread cannot possibly affect the return from test, which continues to execute on the caller thread independently from what it spawned (f goes on to execute independently)..
In case #4, there is a lambda (curly braces) invoking f(). In case #2, f is passed directly as an argument to thread
So in #4, call f() can be inlined and the compiler can guarantee there is no non-local return. To elaborate, the compiler would replace f() with its definition and that code is then “wrapped” inside the enclosing lambda, in other words, { //code for f() } is sort of another (wrapper) lambda and it itself is further passed as a function object reference (to thread).
In case #2, the compiler error simply says it cannot inline f because it is being passed as a reference into an “unknown” (non-inlined) place. crossinline becomes out of place and irrelevant in this case because it could be applied only if f were inlined.
To sum up, case 2 and 4 are not the same by comparing to the example from the kotlin reference (see "Higher-Order Functions and Lambdas"): below invocations are equivalent, where curly braces (lambda expression) "replace" the wrapper function toBeSynchronized
//want to pass `sharedResource.operation()` to lock body
fun <T> lock(lock: Lock, body: () -> T): T {...}
//pass a function
fun toBeSynchronized() = sharedResource.operation()
val result = lock(lock, ::toBeSynchronized)
//or pass a lambda expression
val result = lock(lock, { sharedResource.operation() })
Case #2 and #4 in the question are not equivalent because there is no "wrapper" invoking f in #2