I have 2 services that I want to call back to back in a sequential fashion. Neither of them produces any data. What would be the best way to chain them in a webflux pipeline.
I want to call subscribe on the last service call to trigger the whole flow. Something like:
serviceA.methodX()
.then()
serviceB.methodY().subscribe()
This is what I figured as an alternative:
Mono.just(Boolean.TRUE)
.flatMap( success -> { serviceA.methodX(); return true; } )
.flatMap( success -> { serviceB.methodY(); return true; } )
.subscribe();
Mono<OfSomeThing> executeFirst = ... ;
Mono<OfSomeThing> onceFirstIsCompletedExcecuteSecond = ... ;
Mono<OfSomeThing> plan = executeFirst.then(onceFirstIsCompletedExcecuteSecond);
plan.block();
Related
There is query and two filters. When set search filter and query is not empty, need to one flow. When set checked filter need to other flow.
According debug, onClickSearchFilter or onClickCheckedFilter calls with query or filter changed - return new flow. But in UI no changes, collector dont work second time.
How to switch flow based on condition?
When I debugging with breakpoints in flows, app crash every time A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x14 in tid 23174 (JDWP), pid 23167 . Rebuild, clear cache, reload device - dont' help.
repeatOnStarted(viewModel.itemsFlow) {
// it doesn't work when flow is switched
pagingAdapter.submitData(it)
}
val itemsFlow = queryFlow
.debounce(1000)
.combine(filterFlow) { query, filter ->
when (filter) {
R.id.search_result_chip -> onClickSearchFilter(query)
R.id.checked_chip -> onClickCheckedFilter()
else -> throw Exception("")
}
}.flatMapMerge { // in order to switch from Flow<Flow<*>> to Flow<*>
it
}
private fun onClickSearchFilter(query: String): Flow<PagingData<ItemEntity>> {
return if (query.length < 2)
emptyFlow()
else Pager(BasePagingSource.getConfig()) {
SearchPagingSource(query, client)
}.flow.cachedIn(viewModelScope)
}
private fun onClickCheckedFilter(): Flow<PagingData<ItemEntity>> {
return Pager(
config = BasePagingSource.getConfig(),
remoteMediator = RemoteMediator(checkedIds, cacheDatabase, client)
) {
cacheDatabase.itemDao.getPagingSource(type, checkedIds)
}.flow
}
I'm using spring reactor. The code below is :
public Mono<ResponseEntity<SignUpResponse>> createSignUpForUser(SignUpRequest signUpRequest) {
return Mono.just(signUpRequest)
.map(sign -> {
Mono<UserDetailsEntity> userDetailsEntityMono = userDetailsRepository.findByPhoneNumber(sign.getMobileNumber());
userDetailsEntityMono.handle((user, sink) -> {
if (user != null) {
sink.error(new RuntimeException("Phone number already registered"));
}
});
return functionUserDetails.apply(sign);
})
.flatMap(userDetailsRepository::save)
.map(functionUser)
.flatMap(userRepository::save)
.map(usr -> ResponseEntity.ok(functionSignUpRes.apply(usr)))
.defaultIfEmpty(ResponseEntity.notFound().build())
.log();
}
Here the findByPhoneNumber(sign.getMobileNumber()) DB call is not working (the error is not throwing). The Rest of the operations are working and returning the response. Am i doing anything wrongly ? help me to fix this issue.
I think you need to rewrite your code a bit
public Mono<ResponseEntity<SignUpResponse>> createSignUpForUser(SignUpRequest signUpRequest) {
return userDetailsRepository.findByPhoneNumber(sign.getMobileNumber())
.flatMap(__ -> Mono.error(new RuntimeException("Phone number already registered")))
.switchIfEmpty(userDetailsRepository.save(functionUserDetails.apply(sign)))
.map(functionUser)
.flatMap(userRepository::save)
.map(usr -> ResponseEntity.ok(functionSignUpRes.apply(usr)))
.defaultIfEmpty(ResponseEntity.notFound().build())
.log()
;
}
Idea is that if you find that number then it should error, if result is empty then you would need to insert it so we need to use switch if empty
I have a webflux application. In one place I make 3 API calls. Every call depends on result of previous call.
Mono<List<Book>> books = bookService.find(params); //1st API call
Mono<List<User>> users = books.flatMapMany(Flux::fromIterable)
.map(User::getId)
.collectList()
.map(allUserIds -> userService.findUsers(allUserIds)) //2nd API call
.flatMap(mono -> mono);
Mono<List<User>> parents = users.flatMapMany(Flux::fromIterable)
.filter(user -> user.getParentId() != null)
.map(User::getParentId)
.collectList()
.map(parentIds -> userService.findUsers(parentIds)) //3rd API call
.flatMap(mono -> mono);
Mono<List<User>> usersWithParents = Flux.merge(users, parents)
.flatMapIterable(Function.identity())
.collectList();
This code works, but I get 4 API calls in total. I get two API calls for 2nd step.
I assume that reactor makes 1 call to calculate parents variable and 2n call for Flux.merge(users, parents).
How can I update the call to avoid extra API call?
I think you should do smth like this
public Mono<List<User>> getAllUsers(){
Mono<List<String>> booksMono = bookService.find(params)
.flatMapMany(Flux::fromIterable)
.map(User::getId)
.flatMap(book->userService.findUsers(allUserIds))
.collectList()
.flatMap(users->Mono.zip(users, getParents(users)))
.flatMap(zippedUsers-> Stream.concat(zippedUsers.getT1(), zippedUsers.getT2()));
}
private Mono<List<User>> getParents(List<User> users){
return Flux.fromIterable(users)
.filter(user -> user.getParentId() != null)
.map(User::getParentId)
.collectList()
.map(parentIds -> userService.findUsers(parentIds));
}
Found a solution. We need to combine 2nd and 3rd API calls into one chain with flatmap:
Mono<List<Book>> books = bookService.find(params); //1st API call
Mono<List<User>> users = books.flatMapMany(Flux::fromIterable)
.map(User::getId)
.collectList()
.map(allUserIds -> userService.findUsers(allUserIds)) //2nd API call
.flatMap(mono -> mono);
.flatMap(children -> {
List<Long> parentIds = children.stream()
.filter(child -> child.getParentId() != null)
.map(User::getParentId)
.collect(toList());
return userService.findUsers(parentIds);
});
I am using a RichSinkFunction to execute a SQL UPDATE query on an existing record.
This function assumes that a record already exists on the DB. However, in certain scenarios the existing record is late.
To overcome the issue of record lateness, I have added a Thread.sleep() to make the function wait and retry the DB update.
Sample code provided below for reference.
class RichSinkFact extends RichSinkFunction[FulfillmentUsagesOutput]{
private def updateFactUpcoming(
r: FulfillmentUsagesOutput,
schemaName: String
): Unit = {
var updateStmt: PreparedStatement = null
val sqlStatement =
s"""
|UPDATE $schemaName.$factUpcomingTableName
|SET unit_id = ?
|WHERE pledge_id = ?
|;
|
""".stripMargin
try {
updateStmt = connection.prepareStatement(sqlStatement)
updateStmt.setLong(1, r.unit_id)
updateStmt.setString(2, r.pledge_id)
val rows = updateStmt.executeUpdate()
if(rows == 0) {
logger.warn(s"Retrying update for ${r}")
//retry update
Thread.sleep(retrySleepTime)
val rows = updateStmt.executeUpdate()
if(rows == 0){
//raise error
logger.error(s"Unable to update row: ${r}")
}
}
} finally {
if (updateStmt != null) {
updateStmt.close()
}
}
}
}
Question : Since Flink already implements other timers and uses internal time processing functions, is this the right way of retrying a DB update?
Thanks
As you suspected, sleeping in a Flink user function can cause problems, and should be avoided. In this case there is a better solution: take a look at Sink.ProcessingTimeService. This will let you register timers that will call a callback you register when they fire.
Thanks to David for the original idea behind this approach.
Sink.ProcessingTimeService is only present from Flink 1.12 onwards. So, for anyone on a previous version of Flink looking to implement a similar solution, ProcessingTimeCallback can be used to implement timers in a Sink application.
I have included a sample approach here
https://gist.github.com/soumoks/f73694c64169c8b3494ba1842fa61f1b
I'm looking for a function in Kotlin which stops an iteration once a predicate is fulfilled:
val services = listOf(service1, service2)
...
var res: Result = null
services.stopIfPredicateFulFilled { service ->
res = service.doSomething()
res != null
}
While this example is not really nice since res is overwritten in each iteration I hope the intention is clear.
forEach doesn't do the job the way I expect it to be done. So, I was wondering if there isn't anything else.
You can use the functions find { ... } and firstOrNull { ... } (they are equivalent, just named differently). They find the first element satisfying the predicate and return that element, ignoring all the remaining elements.
services.find { service ->
res = service.doSomething()
res != null
}