Kotlin Flow in repository pattern - kotlin

I would like to use a Flow as a return type for all functions in my repository. For ex:
suspend fun create(item:T): Flow<Result<T>>
This function should call 2 data sources: remote(to save data on the server) and local(to save returned data from the server locally). The question is how I can implement this scenario:
try to save data with RemoteDataSource
if 1. fails - try it N times with M timeout
if data has finally returned from the server - same them locally with LocalDataSource
return flow with locally saved data
RemoteDataSource and LocalDataSource both have fun create with the same signature:
suspend fun create(item:T): Flow<Result<T>>
So they both return flow of data. If you have any ideas about how to implement it, I will be grateful.
------ Update #1 ------
a part of a possible solution:
suspend fun create(item:T): Flow<T> {
// save item remotely
return remoteDataSource.create(item)
// todo: call retry if fails
// save to local a merge two flows in one
.flatMapConcat { remoteData ->
localDataSource.create(remoteData)
}
.map {
// other mapping
}
}
Is it a working idea?

I think you have the right idea but you are trying to do everything at once.
What I found works best (and easily) is to have:
an exposed flow of data coming from your local datasource (easy with Room)
one or more exposed suspend functions like create or refresh that operate on the remote data source and save to the local one (if there is no error)
For ex I have a repository that fetches vehicles in my project (the isCurrent info is only local and isLeft/isRight is because I use Either but any error handling applies):
class VehicleRepositoryImpl(
private val localDataSource: LocalVehiclesDataSource,
private val remoteDataSource: RemoteVehiclesDataSource
) : VehicleRepository {
override val vehiclesFlow = localDataSource.vehicleListFlow
override val currentVehicleFlow = localDataSource.currentVehicleFLow
override suspend fun refresh() {
remoteDataSource.getVehicles()
.fold(
ifLeft = { /* handle errors, retry, ... */ },
ifRight = { reset(it) }
)
}
private suspend fun reset(vehicles: List<VehicleEntity>) {
val current = currentVehicleFlow.first()
localDataSource.reset(vehicles)
if (current != null) localDataSource.setCurrentVehicle(current)
}
override suspend fun setCurrentVehicle(vehicle: VehicleEntity) =
localDataSource.setCurrentVehicle(vehicle)
override suspend fun clear() = localDataSource.clear()
}
Hope this helps and you can adapt it to your case :)

Related

Why can a Flow emit both Int and String value in Kotlin?

You know that Array and List only store the same data struction.
I run the Code A and get the Result A.
It seems that the Flow can emit both Int value and String value, why?
Code A
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
suspend fun performRequest(request: Int): Int {
delay(1000) // imitate long-running asynchronous work
return request
}
fun main() = runBlocking<Unit> {
(1..3).asFlow() // a flow of requests
.transform { request ->
emit("Making request $request")
if (request >1) {
emit(performRequest(request))
}
}
.collect { response -> println(response) }
}
Result A
Making request 1
Making request 2
2
Making request 3
3
This is not a question of Flow but Java/Kotling generics and type safety.
The type this flow returns is Comperable<*>
val flow: Flow<Comparable<*>> = (1..3).asFlow() // a flow of requests
.transform { request ->
emit("Making request $request")
if (request > 1) {
emit(performRequest(request))
}
If you explicitly specify which value you want to return Flow you can restrict the types.
About generics you can refer here or check any document about generics in java/kotlin, type safety you can refer this question
Also when you are in doubt what your specified type is use alt + enter with Android Studio to see avaialble options and select Specify type explicitly.
Disregarding the nature of this request, you can have the functionality you want by making your flow emit instances of some algebraic data type that is basically a "sum" (from the type-theoretic POV) of your constituent types:
sealed interface Record
data class IntData(val get: Int) : Record
data class Metadata(val get: String) : Record
// somewhere later (flow is of type Flow<Record>)
fun main() = runBlocking<Unit> {
(1..3).asFlow() // a flow of requests
.transform { request ->
emit(Metadata("Making request $request"))
if (request > 1) {
emit(IntData(performRequest(request)))
}
// probably want to handle the `else` case too
}
.collect { response -> println(response) }
}
This would be a good solution since it's extendable (i.e. you can add the other cases later on if you need to).
In your specific case though, since you just want to debug the flow, you might not want to actually emit the "metadata" and just go for the tests of your code directly.

Build Flow result based on call to a suspend function

I am learning coroutines and need some help to understand a basic use case.
Implement a non-blocking method that:
Fetches a single item from a (reactive) DB
Determines a range (i.e. the month that the item lives in) based on that item's timestamp
Fetches all items in that month
Returns the items as Flow
Approach
Because it must return a Flow I will not use suspend (like I would when returning a single item). Returning Flow and suspend (which kind of returns a Mono) are most commonly mutually exclusive, right?
So I came up with this signature:
override fun getHistory(beforeUtcMillisExclusive: Long): Flow<Item>
Trying an implementation:
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return emptyFlow()
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end)
}
This gives me on the very first line:
Suspend function 'findOneByTimestampLessThan' should be called only
from a coroutine or another suspend function
I understand the problem that we are not allowed to call a suspend function here and the proposed solution by IntelliJ "adding suspend" does not make sense, when already returning a flow.
So, from this question I got the idea of using a return flow {...}:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod == null) {
return#flow
} else {
val range = calcRange(itemInNextPeriod.timestamp)
return#flow itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start,
range.end)
}
}
The second repository call findByTimestampGreaterThanEqualAndTimestampLessThan returns Flow<Item> and I do not understand why I cannot return it.
This function must return a value of type Unit
Type mismatch.
Required:
Unit
Found:
Flow
return#flow returns from the lambda, not from enclosing function.
You need to reemit items from Flow returned by findByTimestampGreaterThanEqualAndTimestampLessThan call into Flow you're building with flow function:
return flow {
val itemInNextPeriod = itemRepository.findOneByTimestampLessThan(beforeUtcMillisExclusive)
if (itemInNextPeriod != null) {
val range = calcRange(itemInNextPeriod.timestamp)
emitAll(itemRepository.findByTimestampGreaterThanEqualAndTimestampLessThan(range.start, range.end))
}
}

Kotlin not getting called from view model

I am trying call
override suspend fun getLoginResponse(loginRequest: LoginRequest) = flow {
emit(ApiResult.Loading)
networkCall {
loginService.postLoginResponse(loginRequest)
}.let { apiResult->
apiResult.isSuccessAndNotNull().letOnTrueOnSuspend {
(apiResult.getResult() as? LoginResponse)?.let {
emit(ApiResult.Success(it))
Timber.d(it.toString())
} ?: run { emit(ApiResult.Error(TypeCastException("unknown error.")))
Timber.d(TypeCastException("unknown error."))}
}
}
}.flowOn(Dispatchers.IO)
from my viewModel like this :
private fun loginResponse(email: String, password: String, device: String){
viewModelScope.launch {
try {
var loginRequest = LoginRequest(email, password, device)
loginResponseFromServer = loginRepository.getLoginResponse(loginRequest)
.asLiveData(viewModelScope.coroutineContext+Dispatchers.Default)
Timber.d(loginResponseFromServer.toString())
}
catch (e: NetworkErrorException){
validationError.value = "Network communication error!"
}
}
}
When I debug or run the code getLoginResponse not even calling. Is there anything I am missing?
First of all, getLoginResponse doesn't need to be a suspend function since it just returns a cold Flow. If you remove the suspend modifier, you won't need a coroutine to call it or convert it to LiveData.
Second, a LiveData that is built with .asLiveData() doesn't begin to collect the Flow (remains cold) until it first becomes active. This is in the docs for the function. It becomes active when it receives its first observer, but your code has not begun to observe it, which is why the code in your flow block is never called.
You also don't need to specify a different dispatcher for your LiveData. It doesn't matter which dispatcher you're collecting in since collecting it isn't blocking code.
However, LiveData isn't something that should be collected within a ViewModel. It's for UI to interact. The LiveData should be observed from the Fragment.
You need to move your catching of the network exception into your flow builder. The exception will not be thrown at the time of creating the Flow or LiveData, but rather at the time the request is being made (in the Flow's execution).
I'm not sure exactly how to rewrite your flow builder to properly catch because it has functions I haven't seen. Just a tip, but chaining together lots of scope functions into one statement makes code hard to read and reason about.
So to do this as LiveData, you can change your code as follows:
private fun loginResponse(email: String, password: String, device: String): LiveData<LoginResponse> {
val loginRequest = LoginRequest(email, password, device)
return loginRepository.getLoginResponse(loginRequest)
.asLiveData()
}
And then observe it in your Fragment.
However
LiveData and Flow don't really fit this use case, because you want to make a single request and get a single response. Your repository should just expose a suspend function that returns the response. Then your ViewModel can have a suspend function that just passes through the response by calling the repository's suspend function.

How to inform a Flux that I have an item ready to publish?

I am trying to make a class that would take incoming user events, process them and then pass the result to whoever subscribed to it:
class EventProcessor
{
val flux: Flux<Result>
fun onUserEvent1(e : Event)
{
val result = process(e)
// Notify flux that I have a new result
}
fun onUserEvent2(e : Event)
{
val result = process(e)
// Notify flux that I have a new result
}
fun process(e : Event): Result
{
...
}
}
Then the client code can subscribe to EventProcessor::flux and get notified each time a user event has been successfully processed.
However, I do not know how to do this. I tried to construct the flux with the Flux::generate function like this:
class EventProcessor
{
private var sink: SynchronousSink<Result>? = null
val flux: Flux<Result> = Flux.generate{ sink = it }
fun onUserEvent1(e : Event)
{
val result = process(e)
sink?.next(result)
}
fun onUserEvent2(e : Event)
{
val result = process(e)
sink?.next(result)
}
....
}
But this does not work, since I am supposed to immediately call next on the SynchronousSink<Result> passed to me in Flux::generate. I cannot store the sink as in the example:
reactor.core.Exceptions$ErrorCallbackNotImplemented:
java.lang.IllegalStateException: The generator didn't call any of the
SynchronousSink method
I was also thinking about the Flux::merge and Flux::concat methods, but these are static and they create a new Flux. I just want to push things into the existing flux, such that whoever holds it, gets notified.
Based on my limited understanding of the reactive types, this is supposed to be a common use case. Yet I find it very difficult to actually implement it. This brings me to a suspicion that I am missing something crucial or that I am using the library in an odd way, in which it was not intended to be used. If this is the case, any advice is warmly welcome.

Concurrent S3 File Upload via Kotlin Coroutines

I need to upload many files to S3, it would take hours to complete that job sequentially. That's exactly what Kotlin's new coroutines excels in, so I wanted to give them a first try instead of fiddling around again with some Thread-based execution service.
Here is my (simplified) code:
fun upload(superTiles: Map<Int, Map<Int, SuperTile>>) = runBlocking {
val s3 = AmazonS3ClientBuilder.standard().withRegion("eu-west-1").build()
for ((x, ys) in superTiles) {
val jobs = mutableListOf<Deferred<Any>>()
for ((y, superTile) in ys) {
val job = async(CommonPool) {
uploadTile(s3, x, y, superTile)
}
jobs.add(job)
}
jobs.map { it.await() }
}
}
suspend fun uploadTile(s3: AmazonS3, x: Int, y: Int, superTile: SuperTile) {
val json: String = "{}"
val key = "$s3Prefix/x4/$z/$x/$y.json"
s3.putObject(PutObjectRequest("my_bucket", ByteArrayInputStream(json.toByteArray()), metadata))
}
The problem: the code is still very slow and logging reveals that requests are still executed sequentially: a job is finished before the next one is created. Only in very few cases (1 out of 10) I see jobs running concurrently.
Why does the code not run much faster / concurrently? What can I do about it?
Kotlin coroutines excel when you work with asynchronous API, while AmazonS3.putObject API that you are using is an old-school blocking, synchronous API, so you get only as many concurrent uploads as the number of threads in the CommonPool that you are using. There is no value in marking your uploadTile function with suspend modified, because it does not use any suspending functions in its body.
The first step in getting more throughput in your upload task is to start using asynchronous API for that. I'd suggest to look at Amazon S3 TransferManager for that purse. See if that gets your problem solved first.
Kotlin coroutines are designed to help you to combine your async APIs into a easy-to-use logical workflows. For example, it is straightforward to adapt asynchronous API of TransferManager for use with coroutines by writing the following extension function:
suspend fun Upload.await(): UploadResult = suspendCancellableCoroutine { cont ->
addProgressListener {
if (isDone) {
// we know it should not actually wait when done
try { cont.resume(waitForUploadResult()) }
catch (e: Throwable) { cont.resumeWithException(e) }
}
}
cont.invokeOnCompletion { abort() }
}
This extension enables you to write very fluent code that works with TransferManager and you can rewrite your uploadTile function to work with TransferManager instead of working with blocking AmazonS3 interface:
suspend fun uploadTile(tm: TransferManager, x: Int, y: Int, superTile: SuperTile) {
val json: String = "{}"
val key = "$s3Prefix/x4/$z/$x/$y.json"
tm.upload(PutObjectRequest("my_bucket", ByteArrayInputStream(json.toByteArray()), metadata))
.await()
}
Notice, how this new version of uploadTile uses a suspending function await that was defined above.