I am working in a Kotlin and Spring Boot project and I am trying to use Caffeine for caching. I have a service with a suspending function that makes an http call. Here is my config:
#Bean
open fun caffeineConfig(): #NonNull Caffeine<Any, Any> {
return Caffeine.newBuilder().expireAfterWrite(60, TimeUnit.SECONDS)
}
#Bean
open fun cacheManager(caffeine: Caffeine<Any, Any>): CacheManager {
val caffeineCacheManager = CaffeineCacheManager()
caffeineCacheManager.getCache("test")
caffeineCacheManager.setCaffeine(caffeine)
return caffeineCacheManager
}
And here is the function that I want to cache:
#Cacheable(value = ["test"])
open suspend fun getString(id: String): String {
return client.getString(id)
}
But it seems that the caching is not working since I can see from logs that the client gets called every time the service-function gets called. Does #Cacheable not work for suspending functions? Or am I missing something else?
The documentation of #Cacheable says:
Each time an advised method is invoked, caching behavior will be applied, checking whether the method has been already invoked for the given arguments. A sensible default simply uses the method parameters to compute the key, but a SpEL expression can be provided via the key() attribute, or a custom KeyGenerator implementation can replace the default one (see keyGenerator()).
The suspend modifier inserts an Continuation<String> parameter in the generated code which accepts input from the caller. This presumably means each invocation gets its own continuation and the cache detects this as a unique call.
However since the return value also gets changed depending on the continuation you cannot have the cache ignore the continuation parameter. A better approach is to not use suspend functions and instead returning a Deferred which consumers can share:
#Cacheable(value = ["test"])
open fun getString(id: String): Deferred<String> {
return someScope.async {
client.getString(id)
}
}
// Consumer side
getString(id).await()
This should work with the standard caching mechanism since Deferred is a normal object and no special parameters are required.
Related
I have a situation in an app, where there are a lot of network calls of the same endpoint (with different parameters) at the same time. This can cause other calls to be blocked.
The setup uses Retrofit + Kotlin Coroutines.
One solution I can think of is to run the calls with different instances of Retrofit+OkHttp using separate thread pools.
However, I'd prefer a single thread pool (and Retrofit instance) defining limitations via different kotlin coroutine dispatchers and the use of limitedParallelism().
See this code snippet:
class NetworkApi(
private val retrofitWebserviceApi: RetrofitWebserviceApi,
threadPoolExecutor: ThreadPoolExecutor,
private val dispatcher: CoroutineDispatcher = threadPoolExecutor.asCoroutineDispatcher()
.limitedParallelism(CoroutineDispatcherConfig.ioDispatcherLimit),
// A separate IO dispatcher pool so the many calls to getEntries don't block other calls
private val noParallelismDispatcher: CoroutineDispatcher = dispatcher.limitedParallelism(1),
) {
/**
* Represents an endpoint, which needs to be called with a lot of different
* parameters at the same time (about 1000 times).
* It's important these calls don't block the whole thread pool.
*/
suspend fun getEntries(description: String) = withContext(noParallelismDispatcher) {
retrofitWebserviceApi.getEntries(description)
}
/**
* This call should not be blocked by [getEntries] calls, but be executed shortly after it is called.
*/
suspend fun getCategories() = withContext(dispatcher) {
retrofitWebserviceApi.getCategories()
}
}
Full executable JVM code sample here: github sample code - question branch
So the idea here is to limit parallel requests using Kotlin Coroutine Dispatchers.
However, the project logs show that OkHttp uses its own OkHttp Dispatcher.
Is there a way to de-activate the OkHttp Dispatcher and just run a network call in the current thread (defined by a Coroutine Dispatcher here)?
Is this possible without losing the possibility to cancel requests?
Thanks for your help!
To use another dispatcher I think you need to remove suspend modifiers in RetrofitWebserviceApi for functions you want to use another dispatcher for:
internal interface RetrofitWebserviceApi {
#GET("entries")
fun getEntries(#Query("description") description: String): EntriesResponse
#GET("categories")
fun getCategories(): CategoriesResponse
}
Custom dispatcher can be set to OkhttpClient as below
private fun createDispatcher(): Dispatcher {
val dispatcher = Dispatcher(Executors.newCachedThreadPool())
dispatcher.maxRequests = 100
dispatcher.maxRequestsPerHost = 100
return dispatcher
}
private fun getOkHttpClient() = OkHttpClient.Builder()
.addInterceptor(getLoggingInterceptor())
.dispatcher(createDispatcher())
.build()
Short answer:
Yes, the OkHttp dispatcher is ignored if Retrofit calls are executed in a synchronuous way.
Long answer:
I went the same way Sergio suggested.
Besides removing the suspend keyword it's necessary to wrap the result type with Call
internal interface RetrofitWebserviceApi {
#GET("entries")
fun getEntries(#Query("description") description: String): Call<EntriesResponse>
#GET("categories")
fun getCategories(
): Call<CategoriesResponse>
}
Defining a Call<T> return type is the canonical way to define Retrofit interfaces and provides 2 options:
Synchronuous exection calling execute() on the Call object. This returns Response<T>.
Asynchronous execution calling enqueue(). This provides T in the callback.
I needed to go with option 1.
Now, the OkHttp thread pool is ignored. The caller side is now responsible to dispatch the execution of the network call to a background thread.
That was my original intention.
The functions in NetworkApi now additionally need to call execute() and body() to obtain the result:
suspend fun getEntries(description: String) =
retrofitWebserviceApi.getEntries(description)
.executeWithDispatcher(noParallelismDispatcher)
suspend fun getCategories() =
retrofitWebserviceApi.getCategories()
.executeWithDispatcher(dispatcher)
private suspend fun <T> Call<T>.executeWithDispatcher(dispatcher: CoroutineDispatcher): T =
withContext(dispatcher)
{
val response = execute()
if (response.isSuccessful) {
checkNotNull(response.body())
} else {
throw HttpException(response)
}
}
Full solution code sample
I am experimenting with coroutines and feel unsure about passing coroutineScope to plain Kotlin UseCase. Can such approach create memory leaks?
Suppose we are initialising our UseCase in VM and will try to pass viewModelScope:
class UploadUseCase(private val imagesPreparingForUploadUseCase: ImagesPreparingForUploadUseCase){
fun execute(coroutineScope: CoroutineScope, bitmap: Bitmap) {
coroutineScope.launch {
val resizedBitmap = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
}
Is it safe code? No difference if I would declare this exact code in VM instead?If no, that means I could pass coroutineScope as constructor argument....Now I initially thought that I should create my execute method in a following way:
fun CoroutineScope.execute(bitmap: Bitmap) {
launch {
val resizedBitmap = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
}
As far as I understand we use extension function in order for method to use parent coroutineScope. That means, I don't need to pass coroutineScope as argument and just change method to use extension function.
However, in my surprise VM cannot see this method available! Why this method is not available from VM to call?
This is marked as red in VM:
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
prepareDataForUploadingUseCase.execute(bitmap)
}
This is not marked red from VM:
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
prepareDataForUploadingUseCase.execute(viewModelScope, bitmap)
}
If my understanding is wrong, why would I use CoroutineScope as extension function instead of passing coroutineScope as function argument?
Passing it as a parameter vs using it as an extension function receiver is effectively the same in the end result. Extension function receivers are basically another parameter that you are passing to the function, just with rearranged syntax for convenience. So you can't use an extension function as a "cheat" to avoid passing a receiver.
But either way, I see it as kind of a clumsy design to have to provide a scope and then hiding the coroutine setup inside the function. This results in spreading coroutine scope manipulation across both sides of the function barrier. The function that calls this function has to be aware that some coroutine is going to get called on the scope it passes, but it doesn't know whether it needs to worry about how to handle cancellation and what it's allowed to do with the scope that it passed.
In my opinion, it would be cleaner to either do this:
suspend fun execute(bitmap: Bitmap) = withContext(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
so the calling function can launch the coroutine and handle the entire coroutine in one place. Or pass no coroutine scope, but have the execute function internally generate its own scope (that is dependent on lifecycleScope or viewModelScope if applicable), and handle its own cancellation behavior. Here's an example of creating a child scope of the lifecycle scope and adding it to some collection of jobs that you might want to cancel under certain circumstances.
fun execute(bitmap: Bitmap) {
lifecycleScope.launch {
bitmapScopes += coroutineScope(Dispatchers.IO) {
imagesPreparingForUploadUseCase.getResizedBitmap(bitmap, MAX_SIZE)
}
}
}
I am answering this specific question: "Why this method is not available from VM to call?"
The method is not available because it takes a receiver (CoroutineScope), but you already have an implicit receiver due to being inside a type declaration: UploadUseCase. Therefore, you cannot just call the second form of the method, because you would somehow have to specify two receivers.
Luckily, Kotlin provides an easy way to do exactly that, the with method.
private fun uploadPhoto(bitmap: Bitmap, isImageUploaded: Boolean) {
with(prepareDataForUploadingUseCase) {
viewModelScope.execute(bitmap)
}
}
However, I would say that this is quite weird, and agree with #Marko Novakovic that you should remove this responsibility from UseCase.
You can pass CoroutineScope as a function parameter, no problem with that. However I would advise you to remove that responsibility from UseCase. Launch coroutines from ViewModel, Presenter etc.
Extension functions are to be called on the instance of extension type. You don't need to call launch {} and withContext inside same function. Do either. launch(Dispatchers.IO) {}.
Extension functions are not just to access parent scope, you can use them for whatever you need them for, you choose.
UPDATE2 - yep, the following extension function does what you need.
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
kotlinFunction!!.callSuspend(obj, *args)
be nice if the lib doc for callSuspend
https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect.full/call-suspend.html
explicitly stated that the receiver if applicable is first in the vararg list. but i'm happy its now possible to do it in 1.3. And it's baked into the Kotlin API now so you longer have to the reflective hack to pull out the backing continuation and invoke the transformed java method through the Java reflection API.
UPDATE - i can see from another stackoverflow question that Kotlin 1.3 has KFunction.callSuspend, anyone know if that can be used in my case and invoked against a reflective method? In which case how can it be called?
val ret = method.kotlinFunction?.callSuspend(/*???*/)
how do you bind the target object? method.invoke takes the target followed by vargargs for the method parameters, but callSuspend only takes varargs.
or is callSuspend just for standalone functions?
I'm writing a fairly sophisticated remoting framework in kotlin where a class implementing an interface (with annotation similar to JAX_RS) can be efficiently remoted over several different transports including HTTP2 and VERTX, and called through a stub proxy implementing the interface so its completely transparent to the calling code. There are reasons i'm writing a custom implementation which i don't need to get into. Everything's based on suspending functions and coroutines - which are awesome.
In order to do this the kotlin interface is used to auto generate a transparent proxy stub on the client side and a dispatcher on the endpoint side. The dispatcher automatically enforces security by looking at security annotations on the interface methods. Identity data can be accessed from the implementation code through the coroutine context.
Everything's working, except the dispatcher obviously has to use reflection to invoke the suspended function on the implementing class. I cannot figure out how to propagate the coroutine context across the reflective suspended invocation. Not only that, the default ThreadPool for coroutines doesn't seem to be used either - instead it uses some fork-join pool.
Coroutines are implemented great in my opinion, but when you start doing the low level stuff you can't avoid the ugly underbelly. The other thing i noticed is a default method in a kotlin interface doesn't map to a default method in the underlying generated java interface. Which also caused my some grief, but thats a seperate issue.
Anyway - if anyone knows how to fix this final issue? Thanks.
// attach an extension function to
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutine { cont ->
println("in thread "+Thread.currentThread().name)
val ret=invoke(obj, *args, cont)
cont.resume(ret)
}
//....
withContext(kc) {
// kc NOT propagated through method invocation...
meth.invokeSuspend(rec.ob, args)!!
}
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutine { cont ->
val ret=invoke(obj, *args, cont)
cont.resume(ret)
}
There are two main mistakes here:
you expect invoke() to return the value that you must resume the continuation with
you call the user-level suspendCoroutine function instead of the low-level suspendCoroutineUninterceptedOrReturn
Behind these mistakes there may be a deeper misunderstanding of the coroutine suspension mechanism, so let me try to elaborate on that. This is a way to correct your code, taken from the implementation of KCallable.callSuspend:
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
suspendCoroutineUninterceptedOrReturn { cont -> invoke(obj, *args, cont) }
Note the main feature of this code: it just passes the continuation to the invoked function and never tries to resume it with the result of the invocation.
Now, how does this manage to work? There are two factors:
If the called suspendable function doesn't actually suspend, it will simply return its result, and this will become the result of your invokeSuspend function.
If it does suspend, when it resumes, the suspendable function will on its own use the continuation you passed in and invoke its resume method with the result.
If it decides to suspend, the suspendable function immediately returns the special COROUTINE_SUSPENDED constant. suspendCoroutineUninterceptedOrReturn interprets this value as necessary to cooperate with the coroutine suspension mechanism. Specifically, it makes your function return the same constant to its caller (it does this whether or not your code actually returns the result of the suspendable function). This way the constant propagates all the way up the call stack until it reaches the non-suspendable function that started the coroutine. This is typically an event loop, and now it will be able to go on processing the next event.
how do you bind the target object? method.invoke takes the target followed by vargargs for the method parameters, but callSuspend only takes varargs.
The answer to this is documented under KCallable.parameters:
/**
* Parameters required to make a call to this callable.
* If this callable requires a `this` instance or an extension receiver parameter,
* they come first in the list in that order.
*/
public val parameters: List<KParameter>
So this is how simple it is to implement your invokeSuspend in terms of KCallable.callSuspend:
suspend fun Method.invokeSuspend(obj: Any, vararg args: Any?): Any? =
kotlinFunction!!.callSuspend(obj, *args)
Repository.kt
suspend fun getTestRepository(): X
suspend fun getTestWithParamRepository(a: String): X
Service.kt
fun getTestService(lambda: suspend () -> X) { //... }
Using
Suspend function 'getTestWithParamRepository' should be called only
from a coroutine or another suspend function
getTestService(repository::getTestRepository }
// Surround with lambda
getTestService { repository.getTestWithParamRepository("") }
GL
I'm new to Kotlin/Coroutines and I've noticed two different ways to use CoroutineScope.
Option 1 is as follows, within any function:
CoroutineScope(Dispatchers.Default).launch {
expensiveOperation()
}
Option 2 is by implementing the CoroutineScope interface in your class, overriding the CoroutineContext, and then you can just launch coroutines easily with launch or async:
#Service
class ServiceImpl() : CoroutineScope {
override val coroutineContext: CoroutineContext
get() = Dispatchers.Default + Job()
fun someFunction() {
launch {
expensiveOperation()
}
}
}
I am currently developing a backend endpoint that will do the following:
take a request
save the request context to a database
launch a non blocking coroutine in the background to perform an expensive/lengthy operation on the request, and immediately return an http 200. (essentially, once we have the context saved, we can return a response and let the request process in the background)
What is the difference in the two use cases, and for this scenario, which is the preferred method for obtaining a CoroutineScope?
This endpoint may receive multiple requests per second, and the lengthy operation will take a minute or two, so there will definitely be multiple requests processing at the same time, originating from various requests.
Also, if it's option 2, do I want to pass the scope/context to the function that does the heavy processing? Or is that unnecessary? For example:
class ServiceImpl() : CoroutineScope {
override val coroutineContext: CoroutineContext
get() = Dispatchers.Default + Job()
fun someFunction() {
launch {
expensiveOperation(CoroutineScope(coroutineContext))
}
}
private fun expensiveOperation(scope: CoroutineScope)
{
// perform expensive operation
}
}
This is a Spring Boot app, and I'm using version 1.3 of Kotlin.
Please let me know if you have any thoughts/suggestions on how to best structure this service class. Thanks
I would recommend option 2. It will give you chance to clearly define parent Job for all of your coroutines. That gives a chance to shut down the whole execution correctly too.
There are several more coroutine context keys to include - CoroutineName, CoroutineExceptionHandler and so one.
Lastly, the structural concurrency may work better if you pass the CoroutineScope and the associated Job explicitly.
https://medium.com/#elizarov/structured-concurrency-722d765aa952
Also, take a look the explanation on that from Roman:
https://medium.com/#elizarov/coroutine-context-and-scope-c8b255d59055
I'm trying to poll a paginated API and provide new items to the user as they appear.
fun connect(): Sequence<T> = buildSequence {
while (true) {
// result is a List<T>
val result = dataSource.getFirstPage()
yieldAll(/* the new data in `result` */)
// Block the thread for a little bit
}
}
Here's the sample usage:
for (item in connect()) {
// do something as each item is made available
}
My first thought was to use the delay function, but I get this message:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope
This is the signature for buildSequence:
public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T>
I think this message means that I can only use the suspend functions in SequenceBuilder: yield and yieldAll and that using arbitrary suspend function calls aren't allowed.
Right now I'm using this to block the sequence building by one second after every time the API is polled:
val resumeTime = System.nanoTime() + TimeUnit.SECONDS.toNanos(1)
while (resumeTime > System.nanoTime()) {
// do nothing
}
This works, but it really doesn't seem like a good solution. Has anybody encountered this issue before?
Why does it not work? Some research
When we look at buildSequence, we can see that it takes an builderAction: suspend SequenceBuilder<T>.() -> Unit as its argument. As a client of that method, you'll be able to hand on a suspend lambda that has SequenceBuilder as its receiver (read about lambda with receiver here).
The SequenceBuilder itself is annotated with RestrictSuspension:
#RestrictsSuspension
#SinceKotlin("1.1")
public abstract class SequenceBuilder<in T> ...
The annotation is defined and commented like this:
/**
* Classes and interfaces marked with this annotation are restricted
* when used as receivers for extension `suspend` functions.
* These `suspend` extensions can only invoke other member or extension
* `suspend` functions on this particular receiver only
* and are restricted from calling arbitrary suspension functions.
*/
#SinceKotlin("1.1") #Target(AnnotationTarget.CLASS) #Retention(AnnotationRetention.BINARY)
public annotation class RestrictsSuspension
As the RestrictSuspension documentation tells, in the case of buildSequence, you can pass a lambda with SequenceBuilder as its receiver but with restricted possibilities since you'll only be able to call "other member or extension suspend functions on this particular receiver". That means, the block passed to buildSequence may call any method defined on SequenceBuilder (like yield, yieldAll). Since, on the other hand, the block is "restricted from calling arbitrary suspension functions", using delay does not work. The resulting compiler error verifies it:
Restricted suspended functions can only invoke member or extension suspending functions on their restricted coroutine scope.
Ultimately, you need to be aware that the buildSequence creates a coroutine that is an example of a synchronous coroutine. In your example, the sequence code will be executed in the same thread that consumes the sequence by calling connect().
How to delay the sequence?
As we learned, The buildSequence creates a synchronous sequence. It's fine to use regular Thread blocking here:
fun connect(): Sequence<T> = buildSequence {
while (true) {
val result = dataSource.getFirstPage()
yieldAll(result)
Thread.sleep(1000)
}
}
But, do you really want an entire thread to be blocked? Alternatively, you can implement asynchronous sequences as described here. As a result, using delay and other suspending functions will be valid.
Just for an alternate solution...
If what you're really trying to do is asynchronously produce elements, you can use Flows which are basically asynchronous sequences.
Here is a quick table:
Sync
Async
Single
Normal valuefun example(): String
suspendingsuspend fun example(): Stringorfun example(): Deferred<String>
Many
Sequencefun example(): Sequence<String>
Flowfun example(): Flow<String>
You can convert your Sequence<T> to a Flow<T> by replacing the sequence { ... } builder with the flow { ... } builder and then replace yield/yieldAll with emit/emitAll:
fun example(): Flow<String> = flow {
(1..5).forEach { getString().let { emit(it) } }
}
suspend fun getString(): String = { ... }
So, for your example:
fun connect(): Flow<T> = flow {
while (true) {
// Call suspend function to get data from dataSource
val result: List<T> = dataSource.getFirstPage()
emitAll(result)
// _Suspend_ for a little bit
delay(1000)
}
}