I am trying to run the following code but it's not running because the compiler doesn't know which version of the async method to call. How do I tell it which one to call?
v
ar counter=0
val workerA=asyncIncrement(5000)
val workerB=asyncIncrement(100)
workerA.await()
workerB.await()
print("counter = $counter")
fun asyncIncrement(by:Int)=async{
for(i in 1..by){
counter++
}
}
Just copy and paste the code into a scratch file or wherever and you should see the same compiler error
From Kotlin 1.3 you need to call async on a scope. In this example I have chosen
the GlobalScope. But it doesn't matter which scope you choose, you always have to explicitly import the asyncextension function;
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.async
import kotlinx.coroutines.runBlocking
import java.util.concurrent.atomic.AtomicInteger
fun main(args: Array<String>) {
val counter=AtomicInteger(0)
fun asyncIncrement(by:Int)= GlobalScope.async{
for(i in 1..by){
counter.incrementAndGet()
}
}
val workerA=asyncIncrement(5000)
val workerB=asyncIncrement(100)
runBlocking {
workerA.await()
workerB.await()
}
print("counter = $counter")
}
BTW:
I changed the variable counter from int to an AtomicInteger because the two async blocks may run in different threads. And I introduced runBlocking, because await has to run in a suspend function.
Related
Can anyone tell me why this program doesn't finish if you run it?
import kotlinx.coroutines.*
import java.util.concurrent.Executors
fun main(args: Array<String>) {
val dispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher()
val scope = CoroutineScope(dispatcher)
scope.launch { }
scope.cancel()
}
I was writing a bit of database code that I wanted to dedicate a thread to and noticed this behavior. If I just use Dispatchers.IO instead of making my own then it finishes right away, but I wanted a dedicated single thread in this case.
Looks like #Slaw was right, it has to be manually closed. I guess its a somewhat arbitrary design choice?
import kotlinx.coroutines.*
import java.util.concurrent.Executors
fun main(args: Array<String>) {
val dispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher()
val scope = CoroutineScope(dispatcher)
scope.launch { }
scope.cancel()
dispatcher.close() // missing this
}
A bit unfortunate since the obsolete newSingleThreadContext() didn't require this.
I writing a KafkaConsumer in Kotlin using the reactive framework, the problem is the whole project structure is based on Kotlin coroutines and now the kafka consumer follows a Flux publisher pipeline,
I got it to work with runBlocking however I am aware this is not a good idea to have blocking code in our project,
I tried using #KafkaListener (fails when adding suspend modifier),
import com.github.avrokotlin.avro4k.Avro
import kotlinx.coroutines.runBlocking
import org.apache.avro.generic.GenericRecord
import org.apache.kafka.clients.consumer.ConsumerRecord
import org.springframework.boot.CommandLineRunner
import org.springframework.kafka.core.reactive.ReactiveKafkaConsumerTemplate
import org.springframework.stereotype.Component
#Component
class KafkaConsumer(
val slackNotificationService: SlackNotificationService,
val consumerTemplate: ReactiveKafkaConsumerTemplate<String, GenericRecord>
) : CommandLineRunner {
suspend fun sendNotification(record: ConsumerRecord<String, GenericRecord>) {
val tagNotification = Avro.default.fromRecord(TagNotification.serializer(), record.value())
slackNotificationService.notifyUsers(tagNotification)
}
override fun run(vararg args: String?) {
consumerTemplate
.receiveAutoAck()
.subscribe {
runBlocking {
sendNotification(it)
}
}
}
}
I can successfully receive the kafka messages and all the rest of the project is working fine, but I couldn't figure out to create this non blocking bridge here,
Does anyone know if there's a better way to handle this?
Thank you in advance :)
If you want to invoke sendNotification() asynchronously then create a CoroutineScope and launch coroutines with it:
class KafkaConsumer(
...
private val coroutineScope = CoroutineScope(Dispatchers.Default)
...
.subscribe {
coroutineScope.launch {
sendNotification(it)
}
}
If KafkaConsumer may be destroyed/shutdown then it is advised to invoke coroutineScope.cancel() when it happens.
Lets see this code:
import kotlinx.coroutines.*
fun main() = runBlocking {
val deferreds: List<Deferred<Int>> = (1..3).map {
async {
delay(50L * it)
it
}
}
val sum = awaitAll(deferreds[0], deferreds[1], deferreds[2]).sum()
println("$sum")
}
Above we call awaitAll with a varargs. This is how the source code for awaitAll looks:
public suspend fun <T> awaitAll(vararg deferreds: Deferred<T>): List<T> =
if (deferreds.isEmpty()) emptyList() else AwaitAll(deferreds).await()
But apparently we can also call awaitAll on the list of promises:
import kotlinx.coroutines.*
fun main() = runBlocking {
val deferreds: List<Deferred<Int>> = (1..3).map {
async {
delay(50L * it)
it
}
}
val sum2 = deferreds.awaitAll().sum()
println("$sum2")
}
How is this possible if awaitAll is not an extension function?
Looking at the source code for kotlinx.coroutines.awaitAll it looks like the awaitAll method is both a global function and an extension method of the Collection<Deferred<T>> template class.
Either should work just as well and do basically the same thing, it's up to you to use the one which you prefer and which fits your standard the best!
Mockk allows mocking static functions, but how does one mock a Kotlin top level function?
For example, if I have a Kotlin file called HelloWorld.kt, how do I mock the sayHello() function?
HelloWorld.kt
fun sayHello() = "Hello Kotlin!"
The following syntax has worked to me.
mockkStatic(::sayHello.javaMethod!!.declaringClass.kotlin)
I'm surprised there is nothing on the jvm-stdlib yet for this.
Edit:
This overload has now been introduced officially:
https://github.com/mockk/mockk/pull/518
mockkStatic(::sayHello)
There is way to mockk a top level function:
mockkStatic("pkg.FileKt")
every { fun() } returns 5
You just need to know which file this function goes. Check in JAR or stack trace.
To add on previous answers this is working:
mockkStatic("pkg.FileKt")
every { fun() } returns 5
Where mockStatic takes as an argument "package_name:class_file_name"
But to simplify the mockStatick call you can give your file a name for the compiler with #file:JvmName directly in the file.
HelloWorld.kt
#file:JvmName("hello")
fun sayHello() = "Hello Kotlin!"
HelloWorldTest.kt
mockkStatic("pkg.hello")
every { fun() } returns 5
More detailed explication on why this is necessary and other examples here:https://blog.kotlin-academy.com/mocking-is-not-rocket-science-mockk-advanced-features-42277e5983b5
Building on #Sergey's answer:
You could have the actual implementation of the sayHello() function in a variable that's then the default value of a function parameter to sayHello().
This example works:
package tests
import io.mockk.every
import io.mockk.mockk
import org.junit.jupiter.api.Assertions
import org.junit.jupiter.api.Test
val sayHelloKotlin = { "Hello Kotlin!" }
fun sayHello(producer: () -> String = sayHelloKotlin): String = producer()
class Tests {
interface Producer {
fun produce(): String
}
#Test
fun `Top level mocking`() {
val mock = mockk<Producer>()
every { mock.produce() } returns "Hello Mockk"
val actual = sayHello(mock::produce)
Assertions.assertEquals(actual, "Hello Mockk")
}
}
The problem with this is that you're changing production code just to cater for testing, and it feels contrived.
This code doesn't work for me with mockk version 1.10.0 but works well in 1.11.0 (of course need to change mockkStatic(::bar) )
Utils.kt
#file:JvmName("UtilsKt")
package com.example.myapplication
fun foo(): Boolean {
return bar()
}
fun bar():Boolean {
return false
}
Test
#RunWith(RobolectricTestRunner::class)
#Config(sdk = [Build.VERSION_CODES.O_MR1])
class ExampleUnitTest {
#Test
fun addition_isCorrect() {
mockkStatic("com.example.myapplication.UtilsKt")
every { bar() } returns true
assertTrue(foo())
}
}
I have a collection of objects, which I need to perform some transformation on. Currently I am using:
var myObjects: List<MyObject> = getMyObjects()
myObjects.forEach{ myObj ->
someMethod(myObj)
}
It works fine, but I was hoping to speed it up by running someMethod() in parallel, instead of waiting for each object to finish, before starting on the next one.
Is there any way to do this in Kotlin? Maybe with doAsyncTask or something?
I know when this was asked over a year ago it was not possible, but now that Kotlin has coroutines like doAsyncTask I am curious if any of the coroutines can help
Yes, this can be done using coroutines. The following function applies an operation in parallel on all elements of a collection:
fun <A>Collection<A>.forEachParallel(f: suspend (A) -> Unit): Unit = runBlocking {
map { async(CommonPool) { f(it) } }.forEach { it.await() }
}
While the definition itself is a little cryptic, you can then easily apply it as you would expect:
myObjects.forEachParallel { myObj ->
someMethod(myObj)
}
Parallel map can be implemented in a similar way, see https://stackoverflow.com/a/45794062/1104870.
Java Stream is simple to use in Kotlin:
tasks.stream().parallel().forEach { computeNotSuspend(it) }
If you are using Android however, you cannot use Java 8 if you want an app compatible with an API lower than 24.
You can also use coroutines as you suggested. But it's not really part of the language as of now (August 2017) and you need to install an external library. There is very good guide with examples.
runBlocking<Unit> {
val deferreds = tasks.map { async(CommonPool) { compute(it) } }
deferreds.forEach { it.await() }
}
Note that coroutines are implemented with non-blocking multi-threading, which mean they can be faster than traditional multi-threading. I have code below benchmarking the Stream parallel versus coroutine and in that case the coroutine approach is 7 times faster on my machine. However you have to do some work yourself to make sure your code is "suspending" (non-locking) which can be quite tricky. In my example I'm just calling delay which is a suspend function provided by the library. Non-blocking multi-threading is not always faster than traditional multi-threading. It can be faster if you have many threads doing nothing but waiting on IO, which is kind of what my benchmark is doing.
My benchmarking code:
import kotlinx.coroutines.experimental.CommonPool
import kotlinx.coroutines.experimental.async
import kotlinx.coroutines.experimental.delay
import kotlinx.coroutines.experimental.launch
import kotlinx.coroutines.experimental.runBlocking
import java.util.*
import kotlin.system.measureNanoTime
import kotlin.system.measureTimeMillis
class SomeTask() {
val durationMS = random.nextInt(1000).toLong()
companion object {
val random = Random()
}
}
suspend fun compute(task: SomeTask): Unit {
delay(task.durationMS)
//println("done ${task.durationMS}")
return
}
fun computeNotSuspend(task: SomeTask): Unit {
Thread.sleep(task.durationMS)
//println("done ${task.durationMS}")
return
}
fun main(args: Array<String>) {
val n = 100
val tasks = List(n) { SomeTask() }
val timeCoroutine = measureNanoTime {
runBlocking<Unit> {
val deferreds = tasks.map { async(CommonPool) { compute(it) } }
deferreds.forEach { it.await() }
}
}
println("Coroutine ${timeCoroutine / 1_000_000} ms")
val timePar = measureNanoTime {
tasks.stream().parallel().forEach { computeNotSuspend(it) }
}
println("Stream parallel ${timePar / 1_000_000} ms")
}
Output on my 4 cores computer:
Coroutine: 1037 ms
Stream parallel: 7150 ms
If you uncomment out the println in the two compute functions you will see that in the non-blocking coroutine code the tasks are processed in the right order, but not with Streams.
You can use RxJava to solve this.
List<MyObjects> items = getList()
Observable.from(items).flatMap(object : Func1<MyObjects, Observable<String>>() {
fun call(item: MyObjects): Observable<String> {
return someMethod(item)
}
}).subscribeOn(Schedulers.io()).observeOn(AndroidSchedulers.mainThread()).subscribe(object : Subscriber<String>() {
fun onCompleted() {
}
fun onError(e: Throwable) {
}
fun onNext(s: String) {
// do on output of each string
}
})
By subscribing on Schedulers.io(), some method is scheduled on background thread.
To process items of a collection in parallel you can use Kotlin Coroutines. For example the following extension function processes items in parallel and waits for them to be processed:
suspend fun <T, R> Iterable<T>.processInParallel(
dispatcher: CoroutineDispatcher = Dispatchers.IO,
processBlock: suspend (v: T) -> R,
): List<R> = coroutineScope { // or supervisorScope
map {
async(dispatcher) { processBlock(it) }
}.awaitAll()
}
This is suspend extension function on Iterable<T> type, which does a parallel processing of items and returns some result of processing each item. By default it uses Dispatchers.IO dispatcher to offload blocking tasks to a shared pool of threads. Must be called from a coroutine (including a coroutine with Dispatchers.Main dispatcher) or another suspend function.
Example of calling from a coroutine:
val myObjects: List<MyObject> = getMyObjects()
someCoroutineScope.launch {
val results = myObjects.processInParallel {
someMethod(it)
}
// use processing results
}
where someCoroutineScope is an instance of CoroutineScope.
Or if you want to just launch and forget you can use this function:
fun <T> CoroutineScope.processInParallelAndForget(
iterable: Iterable<T>,
dispatcher: CoroutineDispatcher = Dispatchers.IO,
processBlock: suspend (v: T) -> Unit
) = iterable.forEach {
launch(dispatcher) { processBlock(it) }
}
This is an extension function on CoroutineScope, which doesn't return any result. It also uses Dispatchers.IO dispatcher by default. Can be called using CoroutineScope or from another coroutine.
Calling example:
someoroutineScope.processInParallelAndForget(myObjects) {
someMethod(it)
}
// OR from another coroutine:
someCoroutineScope.launch {
processInParallelAndForget(myObjects) {
someMethod(it)
}
}
where someCoroutineScope is an instance of CoroutineScope.