I'm working on a Kotlin/Vertx/RxJava/Retrofit server, but some calls are blocking Vertx when calling an external API that takes too long.
The original handler makes the call:
val response = weatherService.getWeatherSummaryByCity(countryCode = queryParams[0], adminCode = queryParams[1], cityName = queryParams[2])
This in turn executes the external call:
fun getWeatherSummaryByCity(countryCode: String, adminCode: String, cityName: String): WeatherSummary? {
val citiesList = externalAPI.getLocationByCityName(countryCode = countryCode, adminCode = adminCode, cityName = cityName)
var weatherSummary : WeatherSummary? = null
citiesList
.doOnError { error -> print(error) }
.filter { cityList -> !cityList.isEmpty() }
.map { cityList -> cityList[0] }
.filter { city -> city.Key != null && !city.Key.isEmpty() }
.subscribe( { city: City -> weatherSummary = createWeatherSummary(city) } )
return weatherSummary
}
And here's the interface used by Retrofit
interface ExternalAPI {
#GET("/locations/{version}/cities/{countryCode}/{adminCode}/search.json")
fun getLocationByCityName(
#Path("version") version: String = "v1",
#Path("countryCode") countryCode: String,
#Path("adminCode") adminCode: String,
#Query("q") cityName: String,
#Query("apikey") apiKey: String = key,
#Query("details") details: String = "true",
#Query("language") language: String = "en-US"): Observable<List<City>>
}
The code as is works, but if the externalAPI takes too long it blocks Vertx. Same happens when I try this:
Json.encodePrettily(response)
and the response is too big. Any ideas to avoid the blocking?
I see two ways to handle your problem:
use an async http client to fetch the getLocationByCityName. I am not using retrofit, but looking at this: https://futurestud.io/tutorials/retrofit-synchronous-and-asynchronous-requests it has out of the box support for it.
Blocking code can always be executed on a dedicated worker thread by calling vertx.executeblocking. This can be read up here: https://vertx.io/docs/vertx-core/java/#blocking_code
I would suggest option 1 as it is more clean.
Related
In my function, I need to return a list that is populated by a for loop with some Volley Request. So I need to wait that all of these requests to be terminated before return the list.
I think I need the async CoroutineScope to do this work but I don't know how can I wait for all of that response.
This is my code:
suspend fun getListOfAbility(pokemon: Pokemon) : MutableList<Ability> {
val listOfAbility: MutableList<Ability> = emptyList<Ability>() as MutableList<Ability>
CoroutineScope(Dispatchers.IO).launch {
/**
* get the pokemon json
*/
val pokemonJsonObjectRequest = JsonObjectRequest(
Request.Method.GET,
"$pokemonUrl${pokemon.id}",
null,
{
/**
* onResponse
*
* get the list of pokemon abilities
*/
val abilitiesJO = it.getJSONObject("abilities")
val abilityObjectType = object : TypeToken<List<PokemonGson.AbilityObjectGson>>() { }.type
val abilityListGson = Gson().fromJson<List<PokemonGson.AbilityObjectGson>>(abilitiesJO.toString(), abilityObjectType)
/**
* for each ability listed on pokemon info get the full Ability Object
*/
for((index, abilityObjectGson) in abilityListGson.withIndex()) {
val abilityJsonObjectRequest = JsonObjectRequest(
Request.Method.GET,
abilityObjectGson.ability.url,
null,
{
abilityJson ->
/**
* onResponse
*
* get the full ability info
*/
val abilityType = object : TypeToken<AbilityGson>() { }.type
val abilityGson = Gson().fromJson<AbilityGson>(abilityJson.toString(), abilityType)
/**
* fill the Ability entry of listOfAbility with the correct language
*/
val ability = Ability(abilityGson, abilityListGson[index].is_hidden)
listOfAbility.add(ability)
},
{
/**
* onError
*/
Log.d("POKEMON", "Pokemon ability error")
}
)
requestQueue.add(abilityJsonObjectRequest)
}
},
{
/**
* onError
*/
Log.d("POKEMON", "Pokemon request error")
}
)
requestQueue.add(pokemonJsonObjectRequest)
}
//wait
return listOfAbility
}
To use callback-based code in a suspend function, you need to convert it to a suspend function using suspendCoroutine or suspendCancellableCoroutine. So in this case to replace the action of creating a JSONObjectRequest and listener, queuing it to the RequestQueue, and waiting for it somehow, I would create a suspend function like this:
suspend inline fun RequestQueue.getJSONObjectOrNull(
method: Int,
url: String,
jsonRequest: JSONObject?,
crossinline onError: (VolleyError)->Unit = {}
): JSONObject? = suspendCancellableCoroutine { continuation ->
val request = JsonObjectRequest(
method,
url,
jsonRequest,
{ result: JSONObject -> continuation.resume(result) },
{ error ->
onError(error)
continuation.resume(null)
}
)
add(request)
continuation.invokeOnCancellation { request.cancel() }
}
It directly returns the JSONObject result, or null if there's a failure. You can optionally run a callback on errors in case you want to log it.
Then you can use it to write a more sequential version of your function instead of the callback-based version. You can use the pattern of coroutineScope { async { list.map { ... } } }.awaitAll() to convert each item of a list to something else using parallel coroutines.
Here is an untested version of your function. I am having it return an empty list on failure. You could alternatively return null on failure, which might be more useful so the calling function can decide to do something differently when there's a failure.
private fun VolleyError.logDebug() {
Log.d("POKEMON", "Pokemon request error: $this")
}
suspend fun getListOfAbility(pokemon: Pokemon): List<Ability> {
val pokemonJsonObject = requestQueue.getJSONObjectOrNull(Request.Method.GET, "$pokemonUrl${pokemon.id}", null, VolleyError::logDebug)
pokemonJsonObject ?: return emptyList()
val abilitiesJO = pokemonJsonObject.getJSONObject("abilities")
val abilityObjectType = object : TypeToken<List<PokemonGson.AbilityObjectGson>>() {}.type
val abilityListGson: List<Wrapper> = Gson().fromJson<List<PokemonGson.AbilityObjectGson>>(
abilitiesJO.toString(),
abilityObjectType
)
return coroutineScope {
abilityListGson.map {
async {
requestQueue.getJSONObjectOrNull(Request.Method.GET, it.ability.url, null, VolleyError::logDebug)
}
}
}
.awaitAll()
.filterNotNull()
.map { abilityJson ->
val abilityType = object : TypeToken<AbilityGson>() {}.type
val abilityGson = Gson().fromJson<AbilityGson>(abilityJson.toString(), abilityType)
Ability(abilityGson, abilityListGson[index].is_hidden)
}
}
I am trying to convert my pet app to use coroutines instead of callbacks.
I am half way, but I cannot see how to get around the callback I have in this function. Is there a way to use async to get rid of callbacks or am I climbing the wrong tree?
This is what I have so far:
const val url = "https://pokeapi.co/api/v2/pokemon/"
class PokeClient {
fun getPokemonData(context: Context, successCallBack: (Pokemon) -> Unit, pokemonName: String) = runBlocking {
val queue = Volley.newRequestQueue(context)
val request = url.plus(pokemonName)
var deferredResult = async {
val stringRequest = StringRequest(Request.Method.GET, request, Response.Listener<String> { response ->
val jObj = JSONObject(response)
val imgUrl = jObj
.getJSONObject("sprites")
.getJSONObject("other")
.getJSONObject("official-artwork")
.getString("front_default")
val inputStream = URL(imgUrl).openStream()
successCallBack(Pokemon(name = jObj.getString("name"), image = BitmapFactory.decodeStream(inputStream)))
}, Response.ErrorListener {
val toast = Toast.makeText(context, "error talking to professor Oak!", Toast.LENGTH_SHORT)
toast.show()
})
queue.add(stringRequest)
}
deferredResult.await()
}
}
Any ideas?
Thank you,
Android Newbie
Essentially you need to convert network call with callback code block into a suspending function which can be called from any coroutine, this can be done using suspendCoroutine function, it basically provides you with a continuation object, which can be used to return data from inside the Response callbacks in your case
suspend fun getPokemon() = suspendCoroutine<Pokemon> { cont ->
val queue = Volley.newRequestQueue(this)
val url = "https://pokeapi.co/api/v2/pokemon/"
val stringRequest = StringRequest(Request.Method.GET, url,
Response.Listener<Pokemon> { response ->
val jObj = JSONObject(response)
val imgUrl = jObj.getJSONObject("sprites")
.getJSONObject("other")
.getJSONObject("official-artwork")
.getString("front_default")
val inputStream = URL(imgUrl).openStream()
/* call continuation.resume and pass your object */
cont.resume(Pokemon(name = jObj.getString("name"), image = BitmapFactory.decodeStream(inputStream)))
},
Response.ErrorListener {
/* if network call fails then post appropriate error */
cont.resumeWithException(YourExceptoin)
})
queue.add(stringRequest)
}
Now you can call this function from a coroutine and get a Pokemon as following
runBlocking{
try { val pokeMon = getPokemon() }
catch(e: Exception) { Log.d(TAG, "Cant get pokemon") }
}
Note : its OK to use runBlocking only for learning and exploration otherwise its not a good idea, use launch or async
Edit : As noted in comment you can also use suspendCancellableCoroutine if you need to support cancellation (which you should for structured concurrency).
using kotlin, having code
fun fetchRemoteDataApi(): Single<RemoteDataResponse> = networkApi.getData()
// it is just a retrofit
#GET(".../api/getData")
fun getData() : Single<RemoteDataResponse>
fun mergeApiWithDb(): Completable = fetchRemoteDataApi()
.zipWith(localDao.getAll())
.flatMapCompletable { (remoteData, localData) ->
doMerge(remoteData, localData) //<== return a Completable
}
the code flow:
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache() //<=== would like do some inspection at this level
PublishSubject.create<Unit>().toFlowable(BackpressureStrategy.LATEST)
.compose(Transformers.flowableIO())
.switchMap {
//merge DB with api, or local default value first then listen to DB change
mergeApiDbCall.andThen(listAllTopics())
.concatMapSingle { topics -> remoteTopicUsers.map { topics to it } }
}
.flatMapCompletable { (topics, user) ->
// do something return Completable
}
.subscribe({
...
}, { throwable ->
...
})
and when making the call
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache()
the question is if would like to inspect on the Singles<RemoteDataResponse> returned from fetchRemoteDataApi() (i.e. using Log.i(...) to printout the content of RemoteDataResponse, etc.), either in got error or success case, how to do it?
/// the functions
fun listAllTopics(): Flowable<List<String>> = localRepoDao.getAllTopics()
// which a DAO:
#Query("SELECT topic FROM RemoteDataTable WHERE read = 1")
fun getAllTopics(): Flowable<List<String>>
///
private val remoteTopicUsers: Single<List<User>>
get() {
return Single.create {
networkApi.getTopicUsers(object : ICallback.IGetTopicUsersCallback {
override fun onSuccess(result: List<User>) = it.onSuccess(result)
override fun onError(errorCode: Int, errorMsg: String?) = it.onError(Exception(errorCode, errorMsg))
})
}
}
You cannot extract information about elements from the Completable. Though you can use doOnComplete() on Completable, it will not provide you any information about the element.
You can inspect elements if you call doOnSuccess() on your Single, so you need to incorporate this call earlier in your code. To inspect errors you can use doOnError() on both Completable or Single.
I'm new to Kotlin language and I would like to know if it's a good practice have a chain of scope functions. As example, I'm writing a function that calls some API (an utilitary function), parse the string response to specific object, do a small verification and return an object.
Is a good practice have a chain of scope functions like this code above?
fun execRequest(endpoint: String, method: String = "GET", body: String? = ""): String =
defaultHttpRequestBuilder()
.uri(URI.create(endpoint))
.method(method, HttpRequest.BodyPublishers.ofString(body))
.header("Content-Type", "application/x-www-form-urlencoded")
.build()
.run { httpClient.send(this, HttpResponse.BodyHandlers.ofString()) }
.let { it.body() }
fun processLoginRequest(challenge: String) =
execRequest(buildEndpoint("login", challenge))
.let {
mapper.readValue<LoginResponse>(it)
}
.let {
val authSituation = Auth(it.skip, it.challenge)
if (it.skip) {
val acceptResponse = acceptLoginRequest(challenge, it.subject)
authSituation.redirectTo = acceptResponse.redirectTo
}
authSituation
}
This code looks awful in my opinion. Is there another way to write it in a "Kotlin way"?
I think this question is likely to be closed due to answers being a matter of opinion, but since you asked about my comment, here is how you could break up the first function.
fun execRequest(endpoint: String, method: String = "GET", body: String? = ""): String {
val request = defaultHttpRequestBuilder()
.uri(URI.create(endpoint))
.method(method, HttpRequest.BodyPublishers.ofString(body))
.header("Content-Type", "application/x-www-form-urlencoded")
.build()
val response = httpClient.send(request, HttpResponse.BodyHandlers.ofString())
return response.body()
}
I might break up the second function like this
fun processLoginRequest(challenge: String): Auth {
val httpResponse = execRequest(buildEndpoint("login", challenge))
val loginResponse: LoginResponse = mapper.readValue(httpResponse)
return Auth(loginResponse.skip, loginResponse.challenge)
.apply {
if (loginResponse.skip)
redirectTo = acceptLoginRequest(challenge, it.subject).redirectTo
}
}
all! I want to get data in DB at first and than call server if DB is empty. But I don't have any response when I use this way. I tried to call server at first and it was successful. Whats wrong??? This is my code:
private fun getDataFromRepository() {
val subscription =
carRepository.getCars()!!.
subscribeOn(Schedulers.io()).
observeOn(AndroidSchedulers.mainThread()).
subscribe(
{ cars ->
LOG.info(cars.size.toString())
carRepository.saveCarsInDB(cars)
data.postValue(cars)
},
{ e ->
loadError.postValue(e.toString())
LOG.warning(e.toString())
})
subscriptions.add(subscription)
}
Flowables:
fun getCars(): Single<List<Car>>? {
val db = getDataFromDB()
val server = getDataFromServerFlowable()
val mock = getDataFromMock()
return Flowable.concat(db, server).first(mock)
}
private fun getDataFromServerFlowable(): Flowable<List<Car>> {
return carApi.getPostsFlowable()
}
private fun getDataFromDB(): Flowable<List<Car>> {
return RealmCar().queryAllAsFlowable() //"com.github.vicpinm:krealmextensions:2.4.0"
.map { cars -> mapper.convertListRealmCarToListCar(cars) }
.filter { car -> car.isNotEmpty()}
}
private fun getDataFromMock(): List<Car> {
val cars: MutableList<Car> = mutableListOf()
val car = Car(0, 0, "Test", "Test", "Test")
cars.add(car)
return cars
}
Server call:
#GET("/photos")
fun getPostsFlowable(): Flowable<List<Car>>
Depending on your logic you should consider using merge instead of concat to interleave the elements. In your case getDataFromDB() is not emitting, so the final Flowable is waiting for it before emitting getDataFromServerFlowable(), There are plenty of good answers of merge vs concat (i.e this one)