Use of AtomicBoolean required in RetryWhen - kotlin

In the Javadoc example of Observable.retryWhen, AtomicInteger is used for counter instead of a simpler regular Int. Is this actually necessary? Under what circumstances can errors emit on a different thread?
My reading the docs and source indicate that the takeWhile and flatMap closures are always guaranteed to run on same thread.
http://reactivex.io/RxJava/javadoc/io/reactivex/Observable.html#retryWhen-io.reactivex.functions.Function-
Observable.timer(1, TimeUnit.SECONDS)
.doOnSubscribe(s -> System.out.println("subscribing"))
.map(v -> { throw new RuntimeException(); })
.retryWhen(errors -> {
AtomicInteger counter = new AtomicInteger();
return errors
.takeWhile(e -> counter.getAndIncrement() != 3)
.flatMap(e -> {
System.out.println("delay retry by " + counter.get() + " second(s)");
return Observable.timer(counter.get(), TimeUnit.SECONDS);
});
})
.blockingSubscribe(System.out::println, System.out::println);

It is not strictly necessary but some get a heart attack when they see a one element int array used for the counter, hence the AtomicInteger.
Observable.timer(1, TimeUnit.SECONDS)
.doOnSubscribe(s -> System.out.println("subscribing"))
.map(v -> { throw new RuntimeException(); })
.retryWhen(errors -> {
int[] counter = { 0 };
return errors
.takeWhile(e -> counter[0]++ != 3)
.flatMap(e -> {
System.out.println("delay retry by " + counter[0] + " second(s)");
return Observable.timer(counter[0], TimeUnit.SECONDS);
});
})
.blockingSubscribe(System.out::println, System.out::println);
Under what circumstances can errors emit on a different thread?
The handler sequence can have its own threading so whenever you have shared external access to mutable state, you should make sure accessing that is thread safe. In the example, again, it is not necessary as the particular combination during the usage of the counter is running on a single thread and guaranteed not overlapping itself as any new error can only happen after the current sequence signaled the retry to happen.

Related

Why is the value not entering the list?

At 'urichecking2' log, I can see there is value. But in 'uriChecking' the uriList is null.
why the uriList.add not work??
private fun getPhotoList() {
val fileName = intent.getStringExtra("fileName")
Log.d("fileNameChecking", "$fileName")
val listRef = FirebaseStorage.getInstance().reference.child("image").child(fileName!!)
var tmpUrl:Uri = Uri.parse(fileName)
Log.d("firstTmpUri","$tmpUrl")
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
tmpUrl = task.result
Log.d("secondTmpUri","$tmpUrl")
Log.d("urichecking2","$task.result")
uriList.add(task.result)
} else {
}
}.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
}
Log.d("thirdTmpUri","$tmpUrl")
Log.d("urichecking", "$uriList")
}
If I do this, the log is output in the order of first, third, and second, and the desired value is in second, but when third comes out, it returns to the value of first.
The listAll method (like most cloud APIs these days, including downloadUrl which you also use) is asynchronous, since it needs to make a call to the server - which may take time. This means the code executes in a different order than you may expect, which is easiest to see if you add some logging:
Log.d("Firebase","Before starting listAll")
listRef.listAll()
.addOnSuccessListener { listResult ->
Log.d("Firebase","Got listResult")
}
Log.d("Firebase","After starting listAll")
When you run this code it outputs:
Before starting listAll
After starting listAll
Got listResult
This is probably not the order you expected, but it perfectly explains why you can't see the list result. By the time your Log.d("urichecking", "$uriList") runs, none of the uriList.add(task.result) has been called yet.
The solution for this is always the same: any code that needs the list result, has to be inside the addOnCompleteListener callback, be called from there, or be otherwise synchronized.
So in its simplest way:
listRef.listAll()
.addOnSuccessListener { listResult ->
for (item in listResult.items) {
item.downloadUrl.addOnCompleteListener { task ->
if (task.isSuccessful) {
uriList.add(task.result)
Log.d("urichecking", "$uriList")
}
}
}
}
This is an incredibly common mistake to make if you're new to programming with asynchronous APIs, so I recommend checking out
Asynchronous programming techniques in the Kotlin language guide
How to get URL from Firebase Storage getDownloadURL
Can someone help me with logic of the firebase on success listener
Why does my function that calls an API or launches a coroutine return an empty or null value?

Compare to sets of files with coroutines in Kotlin

I have written a function that scans files (pictures) from two Lists and check if a file is in both lists.
The code below is working as expected, but for large sets it takes some time. So I tried to do this in parallel with coroutines. But in sets of 100 sample files the programm was always slower than without coroutines.
The code:
private fun doJob() {
val filesToCompare = File("C:\\Users\\Tobias\\Desktop\\Test").walk().filter { it.isFile }.toList()
val allFiles = File("\\\\myserver\\Photos\\photo").walk().filter { it.isFile }.toList()
println("Files to scan: ${filesToCompare.size}")
filesToCompare.forEach { file ->
var multipleDuplicate = 0
var s = "This file is a duplicate"
s += "\n${file.absolutePath}"
allFiles.forEach { possibleDuplicate ->
if (file != possibleDuplicate) { //only needed when both lists are the same
// Files that have the same name or contains the name, so not every file gets byte comparison
if (possibleDuplicate.nameWithoutExtension.contains(file.nameWithoutExtension)) {
try {
if (Files.mismatch(file.toPath(), possibleDuplicate.toPath()) == -1L) {
s += "\n${possibleDuplicate.absolutePath}"
i++
multipleDuplicate++
println(s)
}
} catch (e: Exception) {
println(e.message)
}
}
}
}
if (multipleDuplicate > 1) {
println("This file has $multipleDuplicate duplicate(s)")
}
}
println("Files scanned: ${filesToCompare.size}")
println("Total number of duplicates found: $i")
}
How have I tried to add the coroutines?
I wrapped the code inside the first forEach in launch{...} the idea was that for each file a coroutine starts and the second loop is done concurrently. I expected the program to run faster but in fact it was about the same time or slower.
How can I achieve this code to run in parallel faster?
Running each inner loop in a coroutine seems to be a decent approach. The problem might lie in the dispatcher you were using. If you used runBlocking and launch without context argument, you were using a single thread to run all your coroutines.
Since there is mostly blocking IO here, you could instead use Dispatchers.IO to launch your coroutines, so your coroutines are dispatched on multiple threads. The parallelism should be automatically limited to 64, but if your memory can't handle that, you can also use Dispatchers.IO.limitedParallelism(n) to reduce the number of threads.

Should Flux.then and Mono.then behave differently in error case?

I encountered a case where I have a nested Flux. I don't care about the individual results of the inner flux as it returns Unit (in Kotlin / Void in Java), but I want to know if the Flux aborted due to an error or not. I thought I could use the then function, as the doc states: Error signal is replayed in the resulting Mono<V>
My problem can be reduced to the minimum (Kotlin) unit test:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Flux.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So we have 2 elements. In the inner Flux one of the elements will fail on processing and the other won't. I expect the error to propagate through the pipeline where it is catched and discarded in the onErrorContinue.
Therefore I'd expect 1 element in the resulting list, but I get the original 2. I have no clue why.
Now comes the fun part: In this particular test case, I can replace Flux.just(1) with Mono.just(1) (in my real world case this doesn't work ofc because the flux has more than 1 element) and suddenly my test passes:
#Test
fun fluxTest() {
val flux = Flux.just("willFail", "willSucceed")
.flatMap { outer ->
// In my real world example the inner flux is created via Flux.fromIterable from a property of the
// outer`-object
Mono.just(1)
.flatMap { inner ->
// this simulates a Mono.fromSupplier that can throw exceptions
if (outer == "willFail") Mono.error<Unit>(RuntimeException("bam"))
else Mono.just(Unit)
}
// We don't care about the Flux as it returns Unit/Void
// All we want to know is, whether there was an error or not
.then(Mono.just(outer))
}
.onErrorContinue { error, item -> println("$item => $error") }
.collectList()
StepVerifier.create(flux)
.expectNextMatches { it.size == 1 }
.verifyComplete()
}
So obviously there is a difference in Mono.then(Mono<T>) and in Flux.then(Mono<T>), but it shouldn't since the Javadoc is the same right?
Side note: Instead of Flux.then(Mono.just(outer)) I also tried Mono.defer but that is not changing anything.

How to modify variables outside of their scope in kotlin?

I understand that in Kotlin there is no such thing as "Non-local variables" or "Global Variables" I am looking for a way to modify variables in another "Scope" in Kotlin by using the function below:
class Listres(){
var listsize = 0
fun gatherlistresult(){
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
}
}
}
the value of listsize is always 0 (logging the result from inside of the .addOnSuccessListener scope returns 8) so clearly the listsize variable isn't being modified. I have seen many different posts about this topic on other sites , but none fit my usecase.
I simply want to modify listsize inside of the .addOnSuccessListener callback
This method will always be returned 0 as the addOnSuccessListener() listener will be invoked after the method execution completed. The addOnSuccessListener() is a callback method for asynchronous operation and you will get the value if it gives success only.
You can get the value by changing the code as below:
class Demo {
fun registerListResult() {
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
processResult(listsize)
}
listallinfo.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
fun processResult(listsize: Int) {
print(listResult+"") // you will get the 8 here as you said
}
}
What you're looking for is a way to bridge some asynchronous processing into a synchronous context. If possible it's usually better (in my opinion) to stick to one model (sync or async) throughout your code base.
That being said, sometimes these circumstances are out of our control. One approach I've used in similar situations involves introducing a BlockingQueue as a data pipe to transfer data from the async context to the sync context. In your case, that might look something like this:
class Demo {
var listSize = 0
fun registerListResult() {
val listAll = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
val dataQueue = ArrayBlockingQueue<Int>(1)
listAll.addOnSuccessListener { dataQueue.put(it.items.size) }
listSize = dataQueue.take()
}
}
The key points are:
there is a blocking variant of the Queue interface that will be used to pipe data from the async context (listener) into the sync context (calling code)
data is put() on the queue within the OnSuccessListener
the calling code invokes the queue's take() method, which will cause that thread to block until a value is available
If that doesn't work for you, hopefully it will at least inspire some new thoughts!

How Do I get total number of records in Flux of server side events

I'm using reactive programming where the client receives a flux of event streams of Server side events then those events are consumed. Functionality wise it works. I've a problem when I try to log the count of total records in the the flux stream. Below are the code snippets.
Let's create an instance connected to the server
final WebClient client = WebClient
.builder()
.baseUrl(url)
.build();
And then we start the connection, subscribing to its topic
final Flux<ServerSentEvent<SomeEvent>> eventStream = client.get()
.uri("/bus/sse?id=" + subscription)
.accept(MediaType.TEXT_EVENT_STREAM)
.exchange()
.flatMapMany(response -> response.bodyToFlux(type))
.repeat();
log(eventStream, "connectSSE");
eventStream
.doOnError(throwable -> this.onError(throwable, eventStream))
.doOnComplete(() -> this.onComplete(eventStream));
subscribe = eventStream.subscribe(someServerSideEvent -> this.onEvent(someServerSideEvent , eventStream));
The below method handles the event
private void onEvent(final ServerSentEvent<SomeEvent> content, Flux<ServerSentEvent<SomeEvent>> eventStream) {
log(eventStream, "onEvent");
//Code for handling event
}
I've a issue with the below piece of code. Actually I want to log the count of records in the stream and I was expecting it will print some numbers but it prints something like below. Need some solution without using .block(). Any help is welcome.
"Counted values in this Flux: MonoMapFuseable, caller onEvent"
private void log1(Flux<ServerSentEvent<SomeEvent>> eventStream, final String caller) {
try {
eventStream.count().map(count -> {
LOGGER.info("Counted values in this Flux: {}, caller {}", count.longValue(), caller);
return count;
});
} catch (final Exception e) {
LOGGER.info("Counted values in this Flux failed", e);
}
}