Wait for all volley request in a for loop - kotlin

In my function, I need to return a list that is populated by a for loop with some Volley Request. So I need to wait that all of these requests to be terminated before return the list.
I think I need the async CoroutineScope to do this work but I don't know how can I wait for all of that response.
This is my code:
suspend fun getListOfAbility(pokemon: Pokemon) : MutableList<Ability> {
val listOfAbility: MutableList<Ability> = emptyList<Ability>() as MutableList<Ability>
CoroutineScope(Dispatchers.IO).launch {
/**
* get the pokemon json
*/
val pokemonJsonObjectRequest = JsonObjectRequest(
Request.Method.GET,
"$pokemonUrl${pokemon.id}",
null,
{
/**
* onResponse
*
* get the list of pokemon abilities
*/
val abilitiesJO = it.getJSONObject("abilities")
val abilityObjectType = object : TypeToken<List<PokemonGson.AbilityObjectGson>>() { }.type
val abilityListGson = Gson().fromJson<List<PokemonGson.AbilityObjectGson>>(abilitiesJO.toString(), abilityObjectType)
/**
* for each ability listed on pokemon info get the full Ability Object
*/
for((index, abilityObjectGson) in abilityListGson.withIndex()) {
val abilityJsonObjectRequest = JsonObjectRequest(
Request.Method.GET,
abilityObjectGson.ability.url,
null,
{
abilityJson ->
/**
* onResponse
*
* get the full ability info
*/
val abilityType = object : TypeToken<AbilityGson>() { }.type
val abilityGson = Gson().fromJson<AbilityGson>(abilityJson.toString(), abilityType)
/**
* fill the Ability entry of listOfAbility with the correct language
*/
val ability = Ability(abilityGson, abilityListGson[index].is_hidden)
listOfAbility.add(ability)
},
{
/**
* onError
*/
Log.d("POKEMON", "Pokemon ability error")
}
)
requestQueue.add(abilityJsonObjectRequest)
}
},
{
/**
* onError
*/
Log.d("POKEMON", "Pokemon request error")
}
)
requestQueue.add(pokemonJsonObjectRequest)
}
//wait
return listOfAbility
}

To use callback-based code in a suspend function, you need to convert it to a suspend function using suspendCoroutine or suspendCancellableCoroutine. So in this case to replace the action of creating a JSONObjectRequest and listener, queuing it to the RequestQueue, and waiting for it somehow, I would create a suspend function like this:
suspend inline fun RequestQueue.getJSONObjectOrNull(
method: Int,
url: String,
jsonRequest: JSONObject?,
crossinline onError: (VolleyError)->Unit = {}
): JSONObject? = suspendCancellableCoroutine { continuation ->
val request = JsonObjectRequest(
method,
url,
jsonRequest,
{ result: JSONObject -> continuation.resume(result) },
{ error ->
onError(error)
continuation.resume(null)
}
)
add(request)
continuation.invokeOnCancellation { request.cancel() }
}
It directly returns the JSONObject result, or null if there's a failure. You can optionally run a callback on errors in case you want to log it.
Then you can use it to write a more sequential version of your function instead of the callback-based version. You can use the pattern of coroutineScope { async { list.map { ... } } }.awaitAll() to convert each item of a list to something else using parallel coroutines.
Here is an untested version of your function. I am having it return an empty list on failure. You could alternatively return null on failure, which might be more useful so the calling function can decide to do something differently when there's a failure.
private fun VolleyError.logDebug() {
Log.d("POKEMON", "Pokemon request error: $this")
}
suspend fun getListOfAbility(pokemon: Pokemon): List<Ability> {
val pokemonJsonObject = requestQueue.getJSONObjectOrNull(Request.Method.GET, "$pokemonUrl${pokemon.id}", null, VolleyError::logDebug)
pokemonJsonObject ?: return emptyList()
val abilitiesJO = pokemonJsonObject.getJSONObject("abilities")
val abilityObjectType = object : TypeToken<List<PokemonGson.AbilityObjectGson>>() {}.type
val abilityListGson: List<Wrapper> = Gson().fromJson<List<PokemonGson.AbilityObjectGson>>(
abilitiesJO.toString(),
abilityObjectType
)
return coroutineScope {
abilityListGson.map {
async {
requestQueue.getJSONObjectOrNull(Request.Method.GET, it.ability.url, null, VolleyError::logDebug)
}
}
}
.awaitAll()
.filterNotNull()
.map { abilityJson ->
val abilityType = object : TypeToken<AbilityGson>() {}.type
val abilityGson = Gson().fromJson<AbilityGson>(abilityJson.toString(), abilityType)
Ability(abilityGson, abilityListGson[index].is_hidden)
}
}

Related

How to mock a CompletableFuture in Kotlin

We have some code to read data from DynamoDB:
suspend fun getKeys(owner: String): Set<String> {
...
val query = ...
query.subscribe { page -> foo(page) }.await()
return ...
}
The subscribe{} above is defined in AWS SDK:
default CompletableFuture<Void> subscribe(Consumer<T> consumer)
I'd like to unit test the logic in this function, for subscribe I don't care about it, hopefully just mock and do nothing.
I tried mock a callback in my unit test (irrelevant code removed):
class BucketRepoTest(
#Mock private val query: SdkPublisher<Page<DbBucket>>
) {
#Test
fun `get keys should be working`() {
val callback = mock<(Page<DbBucket>) -> Unit>()
val result = mock<CompletableFuture<Void>>()
whenever(query.subscribe(callback)).thenReturn(result)
runBlocking {
val keys = data.getKeys("Charlie")
assert(keys.isEmpty())
}
}
}
But when I run the test I got NPE:
query.subscribe { page -> foo(page) } must not be null
java.lang.NullPointerException: query.subscribe { page -> foo(page) } must not be null
at com.myApp.getKeys(myfile.kt:75)
at ...
Any idea how to fix it?
The result cannot be void.
val result = mock<CompletableFuture>()
try to create a mock data and pass as result
val result = the object or data type you want to more
Example
data Result( Charlie = "john", age = "23")
then
class BucketRepoTest(
#Mock private val query: SdkPublisher<Page<DbBucket>> ) {
#Test
fun `get keys should be working`() {
val callback = mock<(Page<DbBucket>) -> Unit>()
val result = Result()
whenever(query.subscribe(callback)).thenReturn(result)
runBlocking {
val keys = data.getKeys("Charlie")
assert(keys.isEmpty())
}
}
}

How we can mock a CoroutineDatabase in ktor?

I use the KMongo tool
How we can mock a Coroutine Database?
How can we mock our database in a koin module?
Is there a way to do this?
Thanks for guiding me
Methods I have tried and it has not worked:
The first method:
single<CoroutineDatabase> {
val client = Mockito.mock(CoroutineClient::class.java)
client.getDatabase(CoreConstants.DATABASE_NAME)
}
The second method:
single<CoroutineDatabase> {
val client = declareMock<CoroutineClient> { }
client.getDatabase(CoreConstants.DATABASE_NAME)
}
I've managed to get this working with MockK with the following approach.
TLDR
Just use a mock of MongoDatabase/MongoCollection<T> and make their coroutine extension property return a mocked CoroutineDatabase/CoroutineCollection<T>. Also need to mock the actual MongoDatabase::getCollection to return the respective MongoCollection<T>.
Suppose we have this scenario.
data class User(val id: Int, val name: String)
class Service(private val myDatabase: CoroutineDatabase) {
private val userCollection: CoroutineCollection<User> = myDatabase.getCollection("users")
suspend fun getById(id: Int): User? = userCollection.findOneById(id)
}
Since userCollection is acquired by calling the inline method CoroutineDatabase::getCollection we need to mock all the code inside that inline because inline methods cannot be mocked with MockK (at the time of writing). Looking at the method code
inline fun <reified TDocument : Any> getCollection(
collectionName: String = KMongoUtil.defaultCollectionName(TDocument::class)
): CoroutineCollection<TDocument> =
database.getCollection(collectionName, TDocument::class.java).coroutine
It just calls com.mongodb.reactivestreams.client.MongoDatabase::getCollection and then uses this extension property to map it to a CoroutineCollection. Notice it uses the field database from CoroutineDatabase which is a MongoDatabase (The CoroutineDatabase was previously obtain via a similar extension property for MongoDatabase).
val <T : Any> MongoCollection<T>.coroutine: CoroutineCollection<T> get() = CoroutineCollection(this)
val MongoDatabase.coroutine: CoroutineDatabase get() = CoroutineDatabase(this)
Having all of this we need to mock:
Both coroutine extension properties on MongoDatabase and MongoCollection<T> (see mocking extension properties with MockK)
The actual MongoDatabase::getCollection because CoroutineDatabase::getCollection is an inline function
// Arrange
val mockedMongoDd: MongoDatabase = mockk<MongoDatabase> {
mockkStatic(MongoDatabase::coroutine)
val that = this
every { coroutine } returns mockk {
every { database } returns that
}
}
val mockedMongoCol: MongoCollection<User> = mockk<MongoCollection<User>> {
mockkStatic(MongoCollection<T>::coroutine)
val that = this
every { ofType<MongoCollection<T>>().coroutine } returns mockk {
every { collection } returns that
}
}
every {
mockedMongoDb.getCollection("users", User::class.java)
} returns mockedMongoCol
val mockedCoroutineDb = mockedMongoDb.coroutine
val mockedCoroutineCol = mockedMongoCol.coroutine
val service = Service(mockedCoroutineDb)
val expectedUser = User(2, "Joe")
coEvery {
mockedCoroutineCol.findOneById(2)
} returns expectedUser
// Act
val actualUser = service.getById(2)
// Assert
assertEquals(expectedUser, actualUser)
Finally, one could make some methods like the following to hide this details from the test.
inline fun <reified T : Any> mockkCoroutineCollection(
name: String? = null,
relaxed: Boolean = false,
vararg moreInterfaces: KClass<*>,
relaxUnitFun: Boolean = false,
block: MongoCollection<T>.() -> Unit = {}
): MongoCollection<T> = mockk(name, relaxed, *moreInterfaces, relaxUnitFun = relaxUnitFun) {
mockkStatic(MongoCollection<*>::coroutine)
val that = this
every { coroutine } returns mockk(name, relaxed, *moreInterfaces, relaxUnitFun = relaxUnitFun) {
every { collection } returns that
}
block()
}
inline fun mockkCoroutineDatabase(
name: String? = null,
relaxed: Boolean = false,
vararg moreInterfaces: KClass<*>,
relaxUnitFun: Boolean = false,
block: MongoDatabase.() -> Unit = {}
): MongoDatabase = mockk(name, relaxed, *moreInterfaces, relaxUnitFun = relaxUnitFun) {
mockkStatic(MongoDatabase::coroutine)
val that = this
every { coroutine } returns mockk(name, relaxed, *moreInterfaces, relaxUnitFun = relaxUnitFun) {
every { database } returns that
}
block()
}
This would reduce the first lines to
val mockedMongoDb: MongoDatabase = mockkCoroutineDatabase()
val mockedMongoCol: MongoCollection<User> = mockkCoroutineCollection<User>()
// ...

rxjava, how to inspect the result of a Single

using kotlin, having code
fun fetchRemoteDataApi(): Single<RemoteDataResponse> = networkApi.getData()
// it is just a retrofit
#GET(".../api/getData")
fun getData() : Single<RemoteDataResponse>
fun mergeApiWithDb(): Completable = fetchRemoteDataApi()
.zipWith(localDao.getAll())
.flatMapCompletable { (remoteData, localData) ->
doMerge(remoteData, localData) //<== return a Completable
}
the code flow:
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache() //<=== would like do some inspection at this level
PublishSubject.create<Unit>().toFlowable(BackpressureStrategy.LATEST)
.compose(Transformers.flowableIO())
.switchMap {
//merge DB with api, or local default value first then listen to DB change
mergeApiDbCall.andThen(listAllTopics())
.concatMapSingle { topics -> remoteTopicUsers.map { topics to it } }
}
.flatMapCompletable { (topics, user) ->
// do something return Completable
}
.subscribe({
...
}, { throwable ->
...
})
and when making the call
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache()
the question is if would like to inspect on the Singles<RemoteDataResponse> returned from fetchRemoteDataApi() (i.e. using Log.i(...) to printout the content of RemoteDataResponse, etc.), either in got error or success case, how to do it?
/// the functions
fun listAllTopics(): Flowable<List<String>> = localRepoDao.getAllTopics()
// which a DAO:
#Query("SELECT topic FROM RemoteDataTable WHERE read = 1")
fun getAllTopics(): Flowable<List<String>>
///
private val remoteTopicUsers: Single<List<User>>
get() {
return Single.create {
networkApi.getTopicUsers(object : ICallback.IGetTopicUsersCallback {
override fun onSuccess(result: List<User>) = it.onSuccess(result)
override fun onError(errorCode: Int, errorMsg: String?) = it.onError(Exception(errorCode, errorMsg))
})
}
}
You cannot extract information about elements from the Completable. Though you can use doOnComplete() on Completable, it will not provide you any information about the element.
You can inspect elements if you call doOnSuccess() on your Single, so you need to incorporate this call earlier in your code. To inspect errors you can use doOnError() on both Completable or Single.

Destructuring instead of .bind() doesn't work in an Arrow Monad comprehension

According to Arrow's Javadoc there are three ways of binding over a monad:
/**
* All possible approaches to running [Kind] in the context of [Fx]
*
* ```
* fx {
* val one = just(1).bind() // using bind
* val (two) = just(one + 1) // using destructuring
* val three = !just(two + 1) // yelling at it
* }
* ```
*/
The first one and the last one work fine however for some reason destructuring doesn't, why?
In the next sample you can see I'm using destructuring, val (j) = helloJoey().k(),) but that value is interpreted as a Mono` instead of a String.
class HelloServiceImpl : HelloService {
private val logger = LoggerFactory.getLogger(javaClass)
override fun helloEverybody(): Mono<out String> {
return MonoK.monad().fx.monad {
val (j) = helloJoey().k()
val a = !Mono.zip(helloJohn(), helloMary()).map { "${it.t1} and ${it.t2}" }.k()
"$j and $a"
}.fix().mono
}
override fun helloJoey(): Mono<String> {
return Mono.defer {
logger.info("helloJoey()")
sleep(2000)
logger.info("helloJoey() - ready")
Mono.just("hello Joey")
}.subscribeOn(Schedulers.elastic())
}
override fun helloJohn(): Mono<String> {
return Mono.defer {
logger.info("helloJohn()")
sleep(5000)
logger.info("helloJohn() - ready")
Mono.just("hello John")
}.subscribeOn(Schedulers.elastic())
}
override fun helloMary(): Mono<String> {
return Mono.defer {
logger.info("helloMary()")
sleep(5000)
logger.info("helloMary() - ready")
Mono.just("hello Mary")
}.subscribeOn(Schedulers.elastic())
}
}
fun main() {
val countDownLatch = CountDownLatch(1)
HelloServiceImpl().helloEverybody().subscribe {
println(it)
countDownLatch.countDown()
}
countDownLatch.await()
}
This is a known problem and why we're moving away from this approach. They're marked as deprecated here.
What happens is, a MonoK is a data class for which the destructure operator is already defined as returning the wrapped Mono. When used inside an fx block this destructuring takes precedence over the one defined on BindSyntax. Check and see if hinting your expected type works, otherwise use invoke or bind instead.

produce<Type> vs Channel<Type>()

Trying to understand channels. I want to channelify the android BluetoothLeScanner. Why does this work:
fun startScan(filters: List<ScanFilter>, settings: ScanSettings = defaultSettings): ReceiveChannel<ScanResult?> {
val channel = Channel<ScanResult>()
scanCallback = object : ScanCallback() {
override fun onScanResult(callbackType: Int, result: ScanResult) {
channel.offer(result)
}
}
scanner.startScan(filters, settings, scanCallback)
return channel
}
But not this:
fun startScan(scope: CoroutineScope, filters: List<ScanFilter>, settings: ScanSettings = defaultSettings): ReceiveChannel<ScanResult?> = scope.produce {
scanCallback = object : ScanCallback() {
override fun onScanResult(callbackType: Int, result: ScanResult) {
offer(result)
}
}
scanner.startScan(filters, settings, scanCallback)
}
It tells me Channel was closed when it wants to call offer for the first time.
EDIT1: According to the docs: The channel is closed when the coroutine completes. which makes sense. I know we can use suspendCoroutine with resume for a one shot callback-replacement. This however is a listener/stream-situation. I don't want the coroutine to complete
Using produce, you introduce scope to your Channel. This means, the code that produces the items, that are streamed over the channel, can be cancelled.
This also means that the lifetime of your Channel starts at the start of the lambda of the produce and ends when this lambda ends.
In your example, the lambda of your produce call almost ends immediately, which means your Channel is closed almost immediately.
Change your code to something like this:
fun CoroutineScope.startScan(filters: List<ScanFilter>, settings: ScanSettings = defaultSettings): ReceiveChannel<ScanResult?> = produce {
scanCallback = object : ScanCallback() {
override fun onScanResult(callbackType: Int, result: ScanResult) {
offer(result)
}
}
scanner.startScan(filters, settings, scanCallback)
// now suspend this lambda forever (until its scope is canceled)
suspendCancellableCoroutine<Nothing> { cont ->
cont.invokeOnCancellation {
scanner.stopScan(...)
}
}
}
...
val channel = scope.startScan(filter)
...
...
scope.cancel() // cancels the channel and stops the scanner.
I added the line suspendCancellableCoroutine<Nothing> { ... } to make it suspend 'forever'.
Update: Using produce and handling errors in a structured way (allows for Structured Concurrency):
fun CoroutineScope.startScan(filters: List<ScanFilter>, settings: ScanSettings = defaultSettings): ReceiveChannel<ScanResult?> = produce {
// Suspend this lambda forever (until its scope is canceled)
suspendCancellableCoroutine<Nothing> { cont ->
val scanCallback = object : ScanCallback() {
override fun onScanResult(callbackType: Int, result: ScanResult) {
offer(result)
}
override fun onScanFailed(errorCode: Int) {
cont.resumeWithException(MyScanException(errorCode))
}
}
scanner.startScan(filters, settings, scanCallback)
cont.invokeOnCancellation {
scanner.stopScan(...)
}
}
}