Correct way of locking a mutex in Kotlin - kotlin

I want to implement a simple thread-safe Buffer, using Kotlin Coroutines, because coroutines are already used within the project.
The buffer will be used both in multi-thread and single-thread contexts, so having suspend fun getMostRecentData() doesn't seem very reasonable (see code below).
This is what I have so far. The fact that I have to write all that code to lock the mutex makes me wonder if I'm doing something wrong.
Anyway here's the code:
class SafeBuffer(
private val dispatcher: CoroutineDispatcher,
private val bufferSize: Int
) {
private val buffer = LinkedList<MyDataType>()
private val mutex = Mutex()
val size: Int
get() = buffer.size
// First approach: make a suspend fun
// Not great because I will need a runBlocking{} statement somewhere, every time I want to access the buffer
suspend fun getMostRecentData() : MyDataType? {
mutex.withLock {
return if (buffer.isEmpty()) null else buffer.last
}
}
// Second approach: use a runBlocking block inside the function
// Seems like it is missing the purpose of coroutines, and I'm not
// sure it is actually thread safe if other context is used somehow?
fun getMostRecentData() : MyDataType? {
runBlocking(dispatcher) {
mutex.withLock {
return if (buffer.isEmpty()) null else buffer.last
}
}
}
/**** More code ****/
(...)
}
So what's the most idiomatic/elegant way of achieving this?

Expanding on my comment, I think it would be idiomatic to have the buffer class only expose a suspend fun, as the consumer of the class would be responsible for figuring out how they want to use it (via runBlocking or from another coroutine). If you see this use case coming up a lot, an idiomatic approach may be to have an extension function on SafeBuffer to offer this functionality.
Extension functions are used all over the place in the coroutines API. In your code example, even Mutex.withLock is defined as an extension function.
class SafeBuffer(...) {
private val buffer = LinkedList<MyDataType>()
private val mutex = Mutex()
suspend fun getMostRecentData() : MyDataType? =
mutex.withLock {
if (buffer.isEmpty()) null else buffer.last
}
}
fun SafeBuffer.getMostRecentDataBlocking(): MyDataType? =
runBlocking {
getMostRecentData()
}

Related

Shared lazy coroutine / memoized coroutine execution

Context:
import kotlinx.coroutines.CoroutineStart
import kotlinx.coroutines.async
import kotlinx.coroutines.coroutineScope
data class DatabaseData(
val a: String,
val b: String
)
interface DatabaseFetcher {
suspend fun get(): DatabaseData
}
class MyClass(
private val databaseFetcher: DatabaseFetcher
) {
suspend fun a() = coroutineScope {
val a = async { databaseFetcher.get().a }
//imagine more async{}'s here
a //imagine a gets computed in regards to the other async{}'s as well
}
suspend fun b() = databaseFetcher.get().b
}
class MyController(private val databaseFetcher: DatabaseFetcher) {
suspend fun someFun() = coroutineScope {
// the reduced example doesn't need a coroutineScope of course, imagine some async{} here
MyClass(databaseFetcher)
}
}
I am trying to call databaseFetcher.get() only once if either a() or b() are called on MyClass. Basically a lazy future that gets started when either a() or b() is called, but with coroutines.
What I have tried so far:
Can't use by lazy{} as the coroutineScope matters here and I can't use withContext(Dispatchers.IO) as I use a custom Context (multithreading, Spring request scoped data etc.) - passing my context in here seems awkward (would it be bad practice?)
I can't pass an async(start = CoroutineStart.LAZY) when constructing MyClass as it would block indefinitely if the Deferred<T> is never await()ed on, which may happen when neither a() or b() is called. It also blocks indefinitely because the corresponding coroutineScope is constructed when MyClass is constructed which would block as a() and b() are called later after MyClass has been fully constructed because (as I understand) a coroutineScope is only unblocked when all its children are done, which doesn't hold true for a lazy async thats awaited outside the curent scope
Using a wider coroutine context may leak when the lazy async is never awaited - is this true? I couldn't find much about this
This is being done in the context of GraphQL wherein either a b or both can be selected. There are boilerplatey solutions to this but as I am still learning about coroutines I wondered if there is an elegant solution to this which I don't see yet. The CoroutineStart.LAZY issue really caught me by surprise :)
I have found a solution for this:
fun <T : () -> U, U> T.memoized(): suspend () -> Deferred<U> {
val self = this
val deferred: CompletableDeferred<U> = CompletableDeferred()
val started = AtomicBoolean(false)
return suspend {
if (started.compareAndExchange(false, true)) {
deferred
} else {
coroutineScope {
async {
deferred.complete(self())
deferred.await()
}
}
}
}
}
Any () -> T function (basically any function with captured arguments) can be .memoized(). Whatever callee first calls the returned suspend fun will be used to start the Deferred<U> while allowing said callee to block whenever he sees fit:
val expensive = { someExpensiveFun(param, param2 }.memoize();
withContext(Dispatchers.IO) { // or some other context
val a = expensive()
val b = expensive()
a.await()
b.await()
}

In Kotlin, is it possible to substitute a suspend fun with a non-suspend version, without breaking the caller?

I'm learning concurrency in Kotlin, coming from C#/JavaScript background, and I can't help comparing some concepts.
In C# and JavaScript, technically we can rewrite an async function as a regular non-async version doing the same thing, using Task.ContinueWith or Promise.then etc.
The caller of the function wouldn't even notice the difference (I ranted about it in a blog post).
Is something like that possible for a suspend function in Kotlin (i.e., without changing the calling code)? I don't think it is, but I thought I'd still ask.
The closest thing I could come up with is below (Kotlin playground link), I still have to call .await():
import kotlinx.coroutines.*
suspend fun suspendableDelay(ms: Long): Long {
delay(ms);
return ms;
}
fun regularDelay(ms: Long): Deferred<Long> {
val d = CompletableDeferred<Long>()
GlobalScope.async { delay(ms); d.complete(ms) }
return d;
}
suspend fun test(ms: Long): Long {
delay(ms);
return ms;
}
fun main() {
val r1 = runBlocking { suspendableDelay(250) }
println("suspendableDelay ended: $r1");
val r2 = runBlocking { regularDelay(500).await() }
println("regularDelay ended: $r2");
}
https://pl.kotl.in/_AmzanwcB
If you're on JVM 8 or higher, you can make a function that calls the suspend function in an async job and returns a CompletableFuture, which can be used to get your result with a callback (thenApplyAsync()) or synchronously (get()).
val scope = CoroutineScope(SupervisorJob())
suspend fun foo(): Int {
delay(500)
return Random.nextInt(10)
}
fun fooAsync(): CompletableFuture<Int> = scope.async { foo() }.asCompletableFuture()
fun main() {
fooAsync()
.thenApplyAsync { println(it) }
Thread.sleep(1000)
}
The above requires the kotlinx-coroutines-jdk8 library.
I don't know of a solution that works across multiple platforms.
This can only work if you change your suspending function to a non-suspending blocking function, for example
private fun method(){
GlobalScope.launch {
val value = getInt()
}
}
// Calling coroutine can be suspended and resumed when result is ready
private suspend fun getInt(): Int{
delay(2000) // or some suspending IO call
return 5;
}
// Calling coroutine can't be suspended, it will have to wait (block)
private fun getInt(): Int{
Thread.sleep(2000) // some blocking IO
return 5;
}
Here you can simply use the non-suspending version, without any change on the caller.
But the issue here is that without suspend modifier the function becomes blocking and as such it can not cause the coroutine to suspend, basically throwing away the advantage of using coroutiens.

How can I guarantee to get latest data when I use Coroutine in Kotlin?

The Code A is from the project architecture-samples, you can see it here.
The updateTasksFromRemoteDataSource() is suspend function, so it maybe run asynchronously.
When I call the function getTasks(forceUpdate: Boolean) with the paramter True, I'm afraid that return tasksLocalDataSource.getTasks() will be fired before updateTasksFromRemoteDataSource().
I don't know if the Code B can guarantee return tasksLocalDataSource.getTasks() will be fired after updateTasksFromRemoteDataSource().
Code A
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource() {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
...
}
Code B
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
coroutineScope {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
}
return tasksLocalDataSource.getTasks()
}
}
...
}
Added Content
To Tenfour04: Thanks!
If somebody implement updateTasksFromRemoteDataSource() with lauch just like Code C, are you sure the Code C is return tasksLocalDataSource.getTasks() will be fired after updateTasksFromRemoteDataSource() when I call the function getTasks(forceUpdate: Boolean) with the paramter True?
Code C
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource()
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource() {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
launch { //I suppose that launch can be fired
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
}
New Added Content
To Joffrey: Thanks!
I think that the Code D can be compiled.
In this case, when forceUpdate is true, tasksLocalDataSource.getTasks() maybe be run before updateTasksFromRemoteDataSource() is done.
Code D
class DefaultTasksRepository(
private val tasksRemoteDataSource: TasksDataSource,
private val tasksLocalDataSource: TasksDataSource,
private val ioDispatcher: CoroutineDispatcher = Dispatchers.IO,
private val myCoroutineScope: CoroutineScope
) : TasksRepository {
override suspend fun getTasks(forceUpdate: Boolean): Result<List<Task>> {
// Set app as busy while this function executes.
wrapEspressoIdlingResource {
if (forceUpdate) {
try {
updateTasksFromRemoteDataSource(myCoroutineScope)
} catch (ex: Exception) {
return Result.Error(ex)
}
}
return tasksLocalDataSource.getTasks()
}
}
private suspend fun updateTasksFromRemoteDataSource(myCoroutineScope: CoroutineScope) {
val remoteTasks = tasksRemoteDataSource.getTasks()
if (remoteTasks is Success) {
// Real apps might want to do a proper sync, deleting, modifying or adding each task.
tasksLocalDataSource.deleteAllTasks()
myCoroutineScope.launch {
remoteTasks.data.forEach { task ->
tasksLocalDataSource.saveTask(task)
}
}
} else if (remoteTasks is Result.Error) {
throw remoteTasks.exception
}
}
...
}
suspend functions look like regular functions from the call site's point of view because they execute sequentially just like regular synchronous functions.
What I mean by this is that the instructions following a plain call to a suspend function do not execute until the called function completes its execution.
This means that code A is fine (when forceUpdate is true, tasksLocalDataSource.getTasks() will never run before updateTasksFromRemoteDataSource() is done), and the coroutineScope in code B is unnecessary.
Now regarding code C, structured concurrency is here to save you.
People simply cannot call launch without a CoroutineScope receiver.
Since TaskRepository doesn't extend CoroutineScope, the code C as-is will not compile.
There are 2 ways to make this compile though:
Using GlobalScope.launch {}: this will cause the problem you expect, indeed. The body of such a launch will be run asynchronously and independently of the caller. updateTasksFromRemoteDataSource can in this case return before the launch's body is done. The only way to control this is to use .join() on the Job returned by the call to launch (which waits until it's done). This is why it is usually not recommended to use the GlobalScope, because it can "leak" coroutines.
wrapping calls to launch in a coroutineScope {...} inside updateTasksFromRemoteDataSource. This will ensure that all coroutines launched within the coroutineScope block are actually finished before the coroutineScope call completes. Note that everything that's inside the coroutineScope block may very well run concurrently, though, depending on how launch/async are used, but this is the whole point of using launch in the first place, isn't it?
Now with Code D, my answer for code C sort of still holds. Whether you pass a scope or use the GlobalScope, you're effectively creating coroutines with a bigger lifecycle than the suspending function that starts them.
Therefore, it does create the problem you fear.
But why would you pass a CoroutineScope if you don't want implementers to launch long lived coroutines in the provided scope?
Assuming you don't do that, it's unlikely that a developer would use the GlobalScope (or any scope) to do this. It's generally bad style to create long-lived coroutines from a suspending function. If your function is suspending, callers usually expect that when it completes, it has actually done its work.

How to suspend kotlin coroutine until notified

I would like to suspend a kotlin coroutine until a method is called from outside, just like the old Java object.wait() and object.notify() methods. How do I do that?
Here: Correctly implementing wait and notify in Kotlin is an answer how to implement this with Kotlin threads (blocking). And here: Suspend coroutine until condition is true is an answer how to do this with CompleteableDeferreds but I do not want to have to create a new instance of CompleteableDeferred every time.
I am doing this currently:
var nextIndex = 0
fun handleNext(): Boolean {
if (nextIndex < apps.size) {
//Do the actual work on apps[nextIndex]
nextIndex++
}
//only execute again if nextIndex is a valid index
return nextIndex < apps.size
}
handleNext()
// The returned function will be called multiple times, which I would like to replace with something like notify()
return ::handleNext
From: https://gitlab.com/SuperFreezZ/SuperFreezZ/blob/master/src/superfreeze/tool/android/backend/Freezer.kt#L69
Channels can be used for this (though they are more general):
When capacity is 0 – it creates RendezvousChannel. This channel does not have any buffer at all. An element is transferred from sender to receiver only when send and receive invocations meet in time (rendezvous), so send suspends until another coroutine invokes receive and receive suspends until another coroutine invokes send.
So create
val channel = Channel<Unit>(0)
And use channel.receive() for object.wait(), and channel.offer(Unit) for object.notify() (or send if you want to wait until the other coroutine receives).
For notifyAll, you can use BroadcastChannel instead.
You can of course easily encapsulate it:
inline class Waiter(private val channel: Channel<Unit> = Channel<Unit>(0)) {
suspend fun doWait() { channel.receive() }
fun doNotify() { channel.offer(Unit) }
}
It is possible to use the basic suspendCoroutine{..} function for that, e.g.
class SuspendWait() {
private lateinit var myCont: Continuation<Unit>
suspend fun sleepAndWait() = suspendCoroutine<Unit>{ cont ->
myCont = cont
}
fun resume() {
val cont = myCont
myCont = null
cont.resume(Unit)
}
}
It is clear, the code have issues, e.g. myCont field is not synchonized, it is expected that sleepAndWait is called before the resume and so on, hope the idea is clear now.
There is another solution with the Mutex class from the kotlinx.coroutines library.
class SuspendWait2 {
private val mutex = Mutex(locaked = true)
suspend fun sleepAndWait() = mutex.withLock{}
fun resume() {
mutex.unlock()
}
}
I suggest using a CompletableJob for that.
My use case:
suspend fun onLoad() {
var job1: CompletableJob? = Job()
var job2: CompletableJob? = Job()
lifecycleScope.launch {
someList.collect {
doSomething(it)
job1?.complete()
}
}
lifecycleScope.launch {
otherList.collect {
doSomethingElse(it)
job2?.complete()
}
}
joinAll(job1!!, job2!!) // suspends until both jobs are done
job1 = null
job2 = null
// Do something one time
}

Notified when lateinit var has been initialised (Kotlin)

This is a straight forward question but I cannot find an answer. Is there a way to be notified when a lateinit var has been initialised in Kotlin?
I know I can check if it has been initialised with this::coolvar.isInitialized but this is not the same.
Thank you
lateinit var works well only in the simplest cases when uninitialized value doesn't make sense in the context of the app, such as dependency injection or late initialization in onCreate().
What you need is a property delegate with a particular behavior. Take a look at Delegates.observable :
var coolvar by Delegates.observable("initial value") { _, old, new ->
println("coolvar has been updated")
}
If you are using Kotlin coroutines you can handle it with delay() function.
for complex scenarios. not like a direct one if the value changes call this function.
Example:
private var lateinit extractFilesThread: Deferred<Unit>
private fun init() {
GlobalScope.async {loadData()}
GlobalScope.async {doSomeWork()}
}
private suspend fun loadData() {
extractFilesThread = GlobalScope.async {
initialFilesManager.extractDataIfNeeded()
}
}
private suspend fun doSomeWork() {
GlobalScope.async {
waitExtractFilesThread()
startThisWork()
}
}
private suspend fun waitExtractFilesThread() {
while (!::extractFilesThread.isInitialized) {
delay(HALF_SECOND_IN_MILI)
}
extractFilesThread.await()
}
so here you want to call startThisWork() after extractFilesThread Finished
and you can't use await() only if it's already initialized, so we use delay() function, It's a suspend function so it won't block other function executions.