I have a function:
fun getUpdatedStr(): DoubleArray {
var Strings : DoubleArray = doubleArrayOf()
for (i in 0..9) {
val page = Jsoup.connect("somesite.com").get()
val table = page.select("table").first().select("td").first()
Strings += table.text()
}
return Strings
}
That throws an android.os.NetworkOnMainThreadException. My problem is that if I try to put this function into a Thread then I can't return the value to use it for other functions. What's the best way to work around this?
You cannot make a network call on Android from the main thread. You must use a worker thread.
This can be done in plain old Java thread, or some "higher level" constructs such as AsyncTask, HandlerThread, RxJava, coroutines etc.
Normally you can't "return" from a thread like you are looking for, as the execution would continue on the main thread after triggering the new one.
If you use coroutines, you can do this with suspend functions.
If you don't want to learn coroutines, with RxJava you'd have to return an observable to the calling functions
Otherwise, convert your method to callback based, and invoke the callback when you're finished
Related
I have this piece of code:
// this method is used to evaluate the input string, and it returns evaluation result in string format
fun process(input: String): String {
val timeoutMillis = 5000L
val page = browser.newPage()
try {
val result = runBlocking {
withTimeout(timeoutMillis) {
val result = page.evaluate(input).toString()
return#withTimeout result
}
}
return result
} catch (playwrightException: PlaywrightException) {
return "Could not parse template! '${playwrightException.localizedMessage}'"
} catch (timeoutException: TimeoutCancellationException) {
return "Could not parse template! (timeout)"
} finally {
page.close()
}
}
It should throw exception after 5 seconds if the method is taking too long to execute (example: input potentially contains infinite loop) but it doesent (becomes deadlock I assume) coz coroutines should be cooperative. But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
So the question is: is it even possible to timeout such coroutine? if yes, then how?
Should I use java thread insted and just kill it after some time?
But the method I am calling is from another library and I have no control over its computation (for sticking yield() or smth like it).
If that is the case, I see mainly 2 situations here:
the library is aware that this is a long-running operation and supports thread interrupts to cancel it. This is the case for Thread.sleep and some I/O operations.
the library function really does block the calling thread for the whole time of the operation, and wasn't designed to handle thread interrupts
Situation 1: the library function is interruptible
If you are lucky enough to be in situation 1, then simply wrap the library's call into a runInterruptible block, and the coroutines library will translate cancellation into thread interruptions:
fun main() {
runBlocking {
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
runInterruptible {
interruptibleBlockingCall()
}
}
}
println("Done in ${elapsed}ms")
}
}
private fun interruptibleBlockingCall() {
Thread.sleep(3000)
}
Situation 2: the library function is NOT interruptible
In the more likely situation 2, you're kind of out of luck.
Should I use java thread insted and just kill it after some time?
There is no such thing as "killing a thread" in Java. See Why is Thread.stop deprecated?, or How do you kill a Thread in Java?.
In short, in that case you do not have a choice but to block some thread.
I do not know a solution to this problem that doesn't leak resources. Using an ExecutorService would not help if the task doesn't support thread interrupts - the threads will not die even with shutdownNow() (which uses interrupts).
Of course, the blocked thread doesn't have to be your thread. You can technically launch a separate coroutine on another thread (using another dispatcher if yours is single-threaded), to wrap the libary function call, and then join() the job inside a withTimeout to avoid waiting for it forever. That is however probably bad, because you're basically deferring the problem to whichever scope you use to launch the uncancellable task (this is actually why we can't use a simple withContext here).
If you use GlobalScope or another long-running scope, you effectively leak the hanging coroutine (without knowing for how long).
If you use a more local parent scope, you defer the problem to that scope. This is for instance the case if you use the scope of an enclosing runBlocking (like in your example), which makes this solution pointless:
fun main() {
val elapsed = measureTimeMillis {
doStuff()
}
println("Completely done in ${elapsed}ms")
}
private fun doStuff() {
runBlocking {
val nonCancellableJob = launch(Dispatchers.IO) {
uncancellableBlockingCall()
}
val elapsed = measureTimeMillis {
withTimeoutOrNull(100.milliseconds) {
nonCancellableJob.join()
}
}
println("Done waiting in ${elapsed}ms")
} // /!\ runBlocking will still wait here for the uncancellable child coroutine
}
// Thread.sleep is in fact interruptible but let's assume it's not for the sake of the example
private fun uncancellableBlockingCall() {
Thread.sleep(3000)
}
Outputs something like:
Done waiting in 122ms
Completely done in 3055ms
So the bottom line is either live with this long thing potentially hanging, or ask the developers of that library to handle interruption or make the task cancellable.
I am trying to write a Kotlin function that executes a HTTP request, then gives the result back to JavaScript.
Because with the IR compiler I cannot use a suspended function from JavaScript, I am trying to use a callback instead.
However, the callback is never executed when called from a coroutine.
Here's a small sample of what I am doing:
private val _httpClient = HttpClient(JsClient()) {
install(ContentNegotiation) { json() }
defaultRequest { url(settings.baseUrl) }
}
fun requestJwtVcJsonCredential(
request: JSJwtVcJsonVerifiableCredentialRequest,
callback: (JSDeferredJsonCredentialResponse?, JSJwtVcJsonVerifiableCredentialResponse?, Any?) -> Unit
) {
CoroutineScope(_httpClient.coroutineContext).launch {
// call suspend function
val response = requestCredential(convert(request))
// this never runs, even though the coroutine does run successfully
println("Coroutine received: $response")
callback(response.first, response.second, response.third)
}
}
I've noticed this question had a similar problem in Android, but the suggested fix does not apply to JavaScript... specifically, using a Channel does not help in my case because I don't have a coroutine to receive from, and trying to start a new coroutine to receive from the channel, then calling the callback from that coroutine, also doesn't work (the root problem seems to be that I cannot call a callback function from any coroutine).
What's the best way to solve this problem? Assume the function I need to call is a suspend function (the HTTP Client function) and I cannot change that, but I could change everything around it so that it works from a non-suspend function (as that's a limitation of Kotlin JS).
The root problem was that the suspend function was actually failing, but there seems to be no default exception handler so the Exception was not logged anywhere, causing the function to fail silently, making it look like the callback was being called but not executing.
However, I think it's worth it mentioning that KotlinJS supports Promise<T>, so the better way to expose a suspend function to JS is to actually write an "adapter" function that returns a Promise instead.
There is a promise extension function on CouroutineScope which can be used for this.
So, for example, if you've got a Kotlin function like this:
suspend fun makeRequest(request: Request): Response
To expose it in JavaScript you can have an adapter function like this:
#JsExport
fun makeRequestJS(request: Request): Promise<Response> {
// KTor's HttpClient itself is a CoroutineScope
return _httpClient.promise { makeRequest(request) }
}
This avoids the need to introduce a callback function.
I am using kotlin and I wanted to stream over a possibly huge resultset using flows. I found some explanations around the web:
Callbacks and Kotlin Flows
Use Flow for asynchronous data streams
I implemented it and it works fine. I also needed to batch the results before sending them to an external services, so I implemented a chunked operation on flows. Something like that:
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> {
return callbackFlow {
val listOfResult = mutableListOf<T>()
this#chunked.collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
trySendBlocking(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
trySendBlocking(listOfResult)
}
close()
}
}
To be sure that everything was working fine, I created some integration tests:
first flow + chuncked to consume all rows, passed
using the first flow (the one created from the jdbc repository) and
applying take operator just to consider few x items. It passed correctly.
using first flow + chunked operator + take operator, it hangs forever
So the last test showed that there was something wrong in the implementation.
I investigated a lot without finding nothing useful but, dumping the threads, I found a coroutine thread blocked in the trySendBlocking call on the first flow, the one created in the jdbc repository.
I am wondering in which way the chunked operator is supposed to propagate the closing to the upstream flow since it seems this part is missing.
In both cases I am propagating downstream the end of data with a close() call but I took a look the take operator and I saw it is triggering back the closing with an emitAbort(...)
Should I do something similar in the callbackFlow{...}?
After a bit of investigation, I was able to avoid the locking adding a timeout on the trySendBlocking inside the repository but I didn´t like that. At the end, I realized that I could cast the original flow (in the chunked operator) to a SendChannel and close it if the downstream flow is closed:
trySendBlocking(listOfResult.toList()).onSuccess {
LOGGER.debug("Sent")
}.onFailure {
LOGGER.warn("An error occurred sending data.", it)
}.onClosed {
LOGGER.info("Channel has been closed")
(originalFlow as SendChannel<*>).close(it)
}
Is this the correct way of closing flows backwards? Any hint to solve this issue?
Thanks!
You shouldn't use trySendBlocking instead of send. You should never use a blocking function in a coroutine without wrapping it in withContext with a Dispatcher that can handle blocking code (e.g. Dispatchers.Default). But when there's a suspend function alternative, use that instead, in this case send().
Also, callbackFlow is more convoluted than necessary for transforming a flow. You should use the standard flow builder instead (and so you'll use emit() instead of send()).
fun <T> Flow<T>.chunked(chunkSize: Int): Flow<List<T>> = flow {
val listOfResult = mutableListOf<T>()
collect {
listOfResult.add(it)
if (listOfResult.size == chunkSize) {
emit(listOfResult.toList())
listOfResult.clear()
}
}
if (listOfResult.isNotEmpty()) {
emit(listOfResult)
}
}
I have the following code (pseudocode)
fun onMapReady()
{
//do some stuff on current thread (main thread)
//get data from server
GlobalScope.launch(Dispatchers.IO){
getDataFromServer { result->
//update UI on main thread
launch(Dispatchers.Main){
updateUI(result) //BREAKPOINT HERE NEVER CALLED
}
}
}
}
As stated there as a comment, the code never enters the coroutine dispatching onto main queue. The below however works if I explicitly use GlobalScope.launch(Dispatchers.Main) instead of just launch(Dispatchers.Main)
fun onMapReady()
{
//do some stuff on current thread (main thread)
//get data from server
GlobalScope.launch(Dispatchers.IO){
getDataFromServer { result->
//update UI on main thread
GlobalScope.launch(Dispatchers.Main){
updateUI(result) //BREAKPOINT HERE IS CALLED
}
}
}
}
Why does the first approach not work?
I believe the problem here is that getDataFromServer() is asynchronous, it immediately returns and therefore you invoke launch(Dispatchers.Main) after you exited from the GlobalScope.launch(Dispatchers.IO) { ... } block. In other words: you try to start a coroutine using a coroutine scope that has finished already.
My suggestion is to not mix asynchronous, callback-based APIs with coroutines like this. Coroutines work best with suspend functions that are synchronous. Also, if you prefer to execute everything asynchronously and independently of other tasks (your onMapReady() started 3 separate asynchronous operations) then I think coroutines are not at all a good choice.
Speaking about your example: are you sure you can't execute getDataFromServer() from the main thread directly? It shouldn't block the main thread as it is asynchronous. Similarly, in some libraries callbacks are automatically executed in the main thread and in such case your example could be replaced with just:
fun onMapReady() {
getDataFromServer { result->
updateUI(result)
}
}
If the result is executed in a background thread then you can use GlobalScope.launch(Dispatchers.Main) as you did, but this is not really the usual way how we use coroutines. Or you can use utilities like e.g. runOnUiThread() on Android which probably makes more sense.
#broot already explained the gist of the problem. You're trying to launch a coroutine in the child scope of the outer GlobalScope.launch, but that scope is already done when the callback of getDataFromServer is called.
So in short, don't capture the outer scope in a callback that will be called in a place/time that you don't control.
One nicer way to deal with your problem would be to make getDataFromServer suspending instead of callback-based. If it's an API you don't control, you can create a suspending wrapper this way:
suspend fun getDataFromServerSuspend(): ResultType = suspendCoroutine { cont ->
getDataFromServer { result ->
cont.resume(result)
}
}
You can then simplify your calling code:
fun onMapReady() {
// instead of GlobalScope, please use viewModelScope or lifecycleScope,
// or something more relevant (see explanation below)
GlobalScope.launch(Dispatchers.IO) {
val result = getDataFromServer()
// you don't need a separate coroutine, just a context switch
withContext(Dispatchers.Main) {
updateUI(result)
}
}
}
As a side note, GlobalScope is probably not what you want, here. You should instead use a scope that maps to the lifecycle of your view or view model (viewModelScope or lifecycleScope) because you're not interested in the result of this coroutine if the view is destroyed (so it should just be cancelled). This will avoid coroutine leaks if for some reason something hangs or loops inside the coroutine.
I'm learning Coroutines of Kotlin.
The following content is from the artical https://developer.android.com/kotlin/coroutines.
Important: Using suspend doesn't tell Kotlin to run a function on a background thread. It's normal for suspend functions to operate on the main thread. It's also common to launch coroutines on the main thread. You should always use withContext() inside a suspend function when you need main-safety, such as when reading from or writing to disk, performing network operations, or running CPU-intensive operations.
Normally it's spend long time when I pull data from a remote server, so I need to place "the pull data function" in background thread in order not to freeze main UI.
Should I always add withContext(Dispatchers.IO) in suspend when I use suspend to pull data from remote server?
BTW,
The Code A is from the project https://github.com/googlecodelabs/kotlin-coroutines, you can see it .
But I can't find the keyword withContext() in the project, why?
Code A
fun refreshTitle() = launchDataLoad {
repository.refreshTitle()
}
private fun launchDataLoad(block: suspend () -> Unit): Unit {
viewModelScope.launch {
try {
_spinner.value = true
block()
} catch (error: TitleRefreshError) {
_snackBar.value = error.message
} finally {
_spinner.value = false
}
}
}
Should I always add withContext(Dispatchers.IO) in suspend when I use suspend to pull data from remote server?
It depends. If the you use a library like Retrofit 2.6.0 that has native support for suspend, the dispatcher is already Dispatchers.IO (or whatever the library deems more appropriate).
If the call to pull data from a remote server is blocking, you need to make sure to run it on Dispatcher.IO yourself with withContext(Dispatchers.IO) to not block the main thread.
I can't find the keyword withContext() in the project, why?
Because the project uses Retrofit, so the switch to Dispatchers.IO happens under the hood:
https://github.com/googlecodelabs/kotlin-coroutines/blob/master/coroutines-codelab/finished_code/src/main/java/com/example/android/kotlincoroutines/main/MainNetwork.kt