Is there a more idiomatic way to perform a subscribe & async / await operation? - kotlin

I have a spring boot kotlin app that creates a web socket connection to another spring app, sends multiple "subscribe" messages, and then needs to wait for receipt of one response per subscription on the web socket connection. The number of subscriptions open at a given time could be up to a few thousand.
I've come up with a basic working solution using CompletableFuture and coroutines, as below. Is there a more idiomatic or concise way to do this task, or is this a fine solution? Any suggestions for improvement are appreciated.
// InputObject / ResponseObject are generic placeholders
fun getItems(inputObjects: List<InputObject>): List<ResponseObject> {
val ret: ConcurrentLinkedQueue<ResponseObject> = ConcurrentLinkedQueue()
// create a completable future for each input object
val subscriptions: MutableMap<String, CompletableFuture<ResponseObject>> = mutableMapOf()
inputObjects.forEach {
subscriptions[it.id] = CompletableFuture()
}
// create web socket client configured with a lambda handler to
// fulfill each subscription
// each responseObject.id matches one inputObject.id
val client = createWebSocketClient({
try {
val responseObject = objectMapper.readValue(it, ResponseObject::class.java)
subscriptions[responseObject.id]?.complete(responseObject)
} catch (e: Exception) {
logger.warn("Exception reading data: ${e.message}")
}
})
runBlocking {
coroutineScope {
for (item in inputObjects) {
launch {
// create and send a subscribe request
client.sendMessage(createSubscribe(item.id))
// wait for each future to complete
// uses CompletableFuture extension await() from kotlinx-coroutines-jdk8
val result = subscriptions[item.id]?.await()
if (result != null) {
ret.add(result)
}
}
}
}
}
client.close()
return ret.toList()
}
edit: I found a similar question: How to pass result as it comes using coroutines?
Which options makes the most sense?

fun getItems(inputObjects: List<InputObject>): List<ResponseObject> {
val subscriptions = ids.associateTo(mutableMapOf()) { it.id to CompletableFuture<ResponseObject>() }
val client = createWebSocketClient({
try {
val responseObject = objectMapper.readValue(it, ResponseObject::class.java)
subscriptions[responseObject.id]?.complete(responseObject)
} catch (e: Exception) {
logger.warn("Exception reading data: ${e.message}")
}
})
return runBlocking(Dispatchers.IO) {
inputObjects
.mapNotNull {
client.sendMessage(createSubscribe(item.id))
subscriptions[item.id]?.await()
}
}
}

Related

Getting data from Datastore for injection

I am trying to retrieve the base url from my proto datastore to be used to initialize my ktor client instance I know how to get the data from the datastore but I don't know how to block execution until that value is received so the client can be initialized with the base url
So my ktor client service asks for a NetworkURLS class which has a method to return the base url
Here is my property to retrieve terminalDetails from my proto datastore
val getTerminalDetails: Flow<TerminalDetails> = cxt.terminalDetails.data
.catch { e ->
if (e is IOException) {
Log.d("Error", e.message.toString())
emit(TerminalDetails.getDefaultInstance())
} else {
throw e
}
}
Normally when I want to get the values I would do something like this
private fun getTerminalDetailsFromStore() {
try {
viewModelScope.launch(Dispatchers.IO) {
localRepository.getTerminalDetails.collect {
_terminalDetails.value = it
}
}
} catch(e: Exception) {
Log.d("AdminSettingsViewModel Error", e.message.toString()) // TODO: Handle Error Properly
}
}
but in my current case what I am looking to do is return terminalDetails.backendHost from a function and that where the issue comes in I know I need to use a coroutine scope to retrieve the value so I don't need to suspend the function but how to a prevent the function returning until the coroutine scope has finished?
I have tried using async and runBlocking but async doesn't work the way I would think it would and runBlocking hangs the entire app
fun backendURL(): String = runBlocking {
var url: String = "localhost"
val job = CoroutineScope(Dispatchers.IO).async {
repo.getTerminalDetails.collect {
it.backendHost
}
}
url
}
Can anyone give me some assistance on getting this to work?
EDIT: Here is my temporary solution, I do not intend on keeping it this way, The issue with runBlocking{} turned out to be the Flow<T> does not finish so runBlocking{} continues to block the app.
fun backendURL(): String {
val details = MutableStateFlow<TerminalDetails>(TerminalDetails.getDefaultInstance())
val job = CoroutineScope(Dispatchers.IO).launch {
repo.getTerminalDetails.collect {
details.value = it
}
}
runBlocking {
delay(250L)
}
return details.value.backendHost
}
EDIT 2: I fully fixed my issue. I created a method with the same name as my val (personal decision) which utilizes runBlocking{} and Flow<T>.first() to block while the value is retrieve. The reason I did not replace my val with the function is there are places where I need the information as well where I can utilize coroutines properly where I am not initializing components on my app
val getTerminalDetails: Flow<TerminalDetails> = cxt.terminalDetails.data
.catch { e ->
if (e is IOException) {
Log.d("Error", e.message.toString())
emit(TerminalDetails.getDefaultInstance())
} else {
throw e
}
}
fun getTerminalDetails(): TerminalDetails = runBlocking {
cxt.terminalDetails.data.first()
}

How do I properly use Kotlin Flow in Ktor streaming responses?

emphasized textI am trying to use Kotlin Flow to process some data asynchronously and in parallel, and stream the responses to the client as they occur, as opposed to waiting until all the jobs are complete.
After unsuccessfully trying to just send the flow itself to the response, like this: call.respond(HttpStatusCode.OK, flow.toList())
... I tinkered for hours trying to figure it out, and came up with the following. Is this correct? It seems there should be a more idiomatic way of sending a Flow<MyData> as a response, like one can with a Flux<MyData> in Spring Boot.
Also, it seems that using the below method does not cancel the Flow when the HTTP request is cancelled, so how would one cancel it in Ktor?
data class MyData(val number: Int)
class MyService {
fun updateAllJobs(): Flow<MyData> =
flow {
buildList { repeat(10) { add(MyData(Random.nextInt())) } }
// Docs recommend using `onEach` to "delay" elements.
// However, if I delay here instead of in `map`, all elements are held
// and emitted at once at the very end of the cumulative delay.
// .onEach { delay(500) }
.map {
// I want to emit elements in a "stream" as each is computed.
delay(500)
emit(it)
}
}
}
fun Route.jobRouter() {
val service: MyService by inject() // injected with Koin
put("/jobs") {
val flow = service.updateAllJobs()
// Just using the default Jackson mapper for this example.
val mapper = jsonMapper { }
// `respondOutputStream` seems to be the only way to send a Flow as a stream.
call.respondOutputStream(ContentType.Application.Json, HttpStatusCode.OK) {
flow.collect {
println(it)
// The data does not stream without the newline and `flush()` call.
write((mapper.writeValueAsString(it) + "\n").toByteArray())
flush()
}
}
}
}
The best solution I was able to find (although I don't like it) is to use respondBytesWriter to write data to a response body channel. In the handler, a new job to collect the flow is launched to be able to cancel it if the channel is closed for writing (HTTP request is canceled):
fun Route.jobRouter(service: MyService) {
put("/jobs") {
val flow = service.updateAllJobs()
val mapper = jsonMapper {}
call.respondBytesWriter(contentType = ContentType.Application.Json) {
val job = launch {
flow.collect {
println(it)
try {
writeStringUtf8(mapper.writeValueAsString(it))
flush()
} catch (_: ChannelWriteException) {
cancel()
}
}
}
job.join()
}
}
}

How can I run Kotlin suspend fun on #ReactMethod (ReactNative)

I need to implement a custom gRPC on Kotlin native side.
#ReactMethod can't be suspend func.
How can I run it?
#ReactMethod
fun connect(ipAddress: String, port: Int) {
try {
channel = ManagedChannelBuilder.forAddress(ipAddress, port).usePlaintext().build()
var guidKey = Metadata.Key.of("GUID", Metadata.ASCII_STRING_MARSHALLER)
metadata.put(guidKey, GUID)
val stub = DBServiceGrpcKt.DBServiceCoroutineStub(channel!!)
var request = GrpcDBService.SignInRequest.newBuilder()
.setUserName("user")
.setPassword("11111")
.build()
try {
//******* this part *****
suspend fun coroutine() {
var response = stub.trySignIn(request,metadata)
}
} catch (e: Exception) {
Log.d("grpcConnect", e.localizedMessage)
}
} catch (e: Error) {
Log.d("grpcConnect ", e.localizedMessage)
}
finally {
channel?.shutdown()
}
}
You need to create a coroutine, for example, define a scope (somewhere in your class) and use it with launch:
myPluginScope.launch {
val response = stub.trySignIn(request,metadata)
// Return, the result
}
Creating the scope is easy, the tricky part is to find where to cancel it. Check the documentation on React native modules to find a good place to call cancel on your scope:
val myPluginScope = CoroutineScope(SupervisorJob() + Dispatchers.Main.immediate)
Now, is important to decide whether the connect function should behave as an asynchronous function or not. For example, if the sign in operation takes time or uses the network, connect should probably return the result through a Promise or Callback, so the Javascript side is not blocked:
#ReactMethod
fun connect(ipAddress: String, port: Int, promise: Promise) {
myPluginScope.launch {
try {
// Make the sign in happen in a separate thread:
val response = withContext(context = Dispatchers.IO) {
stub.trySignIn(request, metadata)
}
// Resolve the promise in the calling thread (The UI thread)
promise.resolve(response.hypotheticalCode)
} catch (e: Exception) {
promise.reject("Sign in error!", e)
}
}
}
See:
https://reactnative.dev/docs/native-modules-android#promises
https://kotlinlang.org/docs/async-programming.html#coroutines

Kotlin multiplatform: JobCancellationException: Parent job is Completed

I try to write a kotlin multiplatform library (android and ios) that uses ktor. Thereby I experience some issues with kotlins coroutines:
When writing tests I always get kotlinx.coroutines.JobCancellationException: Parent job is Completed; job=JobImpl{Completed}#... exception.
I use ktors mock engine for my tests:
client = HttpClient(MockEngine)
{
engine
{
addHandler
{ request ->
// Create response object
}
}
}
A sample method (commonMain module) using ktor. All methods in my library are written in a similar way. The exception occures if client.get is called.
suspend fun getData(): Either<Exception, String> = coroutineScope
{
// Exception occurs in this line:
val response: HttpResponse = client.get { url("https://www.google.com") }
return if (response.status == HttpStatusCode.OK)
{
(response.readText() as T).right()
}
else
{
Exception("Error").left()
}
}
A sample unit test (commonTest module) for the above method. The assertTrue statement is never called since the exception is thrown before.
#Test
fun getDataTest() = runTest
{
val result = getData()
assertTrue(result.isRight())
}
Actual implementation of runTest in androidTest and iosTest modules.
actual fun<T> runTest(block: suspend () -> T) { runBlocking { block() } }
I thought when I use coroutineScope, it waits until all child coroutines are done. What am I doing wrong and how can I fix this exception?
you can't cache HttpClient of CIO in client variable and reuse, It would be best if change the following code in your implementation.
val client:HttpClient get() = HttpClient(MockEngine) {
engine {
addHandler { request ->
// Create response object
}
}
}
The library must be updated, this glitch is in the fix report here: https://newreleases.io/project/github/ktorio/ktor/release/1.6.1
The problem is that you cannot use the same instance of the HttpClient. My ej:
HttpClient(CIO) {
install(JsonFeature) {
serializer = GsonSerializer()
}
}.use { client ->
return#use client.request("URL") {
method = HttpMethod.Get
}
}

Kotlin Coroutine - Ktor Server WebSocket

I made a kotlin-ktor application, what i wanted to achieve is that it is modular that anytime any pipelines inside the application maybe removed from the source code without breaking any functionalities. So i decided i want to move a websocket implementation to separate class
but i faced an issue where the coroutine inside the lambda expression terminates immediately.
link-github issue
Can someone enlighten me about the coroutine setup on this, and how I can still keep this as modular without this kind of issue
working ktor websocket
fun Application.socketModule() = runBlocking {
// other declarations
......
routing {
val sessionService = SocketSessionService()
webSocket("/home") {
val chatSession = call.sessions.get<ChatSession>()
println("request session: $chatSession")
if (chatSession == null) {
close(CloseReason(CloseReason.Codes.VIOLATED_POLICY, "empty Session"))
return#webSocket
}
send(Frame.Text("connected to server"))
sessionService.addLiveSocket(chatSession.id, this)
sessionService.checkLiveSocket()
}
thread(start = true, name = "socket-monitor") {
launch {
sessionService.checkLiveSocket()
}
}
}
}
kotlin-ktor auto-close web socket
code below closes the socket automatically
Socket Module
class WebSocketServer {
fun createWebSocket(root: String, routing: Routing) {
println("creating web socket server")
routing.installSocketRoute(root)
}
private fun Routing.installSocketRoute(root: String) {
val base = "/message/so"
val socketsWeb = SocketSessionService()
webSocket("$root$base/{type}") {
call.parameters["type"] ?: throw Exception("missing type")
val session = call.sessions.get<ChatSession>()
if (session == null) {
println( "WEB-SOCKET:: client session is null" )
close(CloseReason(CloseReason.Codes.VIOLATED_POLICY, "No Session"))
return#webSocket
}
socketsWeb.addLiveSocket(session.id, this)
thread(start= true, name = "thread-live-socket") {
launch {
socketsWeb.checkLiveSocket()
}
}
}
}
}
Application Module
fun Application.socketModule() = runBlocking {
// other delcarations
.....
install(Sessions) {
cookie<ChatSession>("SESSION")
}
intercept(ApplicationCallPipeline.Features) {
if (call.sessions.get<ChatSession>() == null) {
val sessionID = generateNonce()
println("generated Session: $sessionID")
call.sessions.set(ChatSession(sessionID))
}
}
routing {
webSocketServer.createWebSocket("/home", this)
}
}
I quite dont understand why the coroutine insdie webSocket lamda is completed.
Can someone show me other/right approach on this one.