I have a method which returns a Mono of Map as follows:
private Mono<Map<String,String>> allLocs(){
return all().collectList()
.flatMap(list -> {
Map<String, String> allData = new HashMap<String, String>();
if (list.size() > 0) {
list.forEach(entry -> {
if(entry.getCountryCode()!=null)
allData.put(entry.getCountryCode(), entry.getCountryName());
});
}
return Mono.just(allData);
});
}
When I'm trying to use this map in an external method, I'm getting below error in line 1 of generateComplianceResponse method
Error:
The method flatMap() in the type Mono<Map<String,String>> is not applicable for the arguments ((<no-type> entry) -> {})
Code:
private void generateComplianceResponse(Compliance compliance){
allLocs().flatMap(entry-> {
complianceResponse.setCountry(entry.get(complianceResponse.getCompliancePort().substring(0,3)));
});
}
}
Related
I am working with Kafka Parallel Consumer to consume and process messages, Now I would also like to produce new events to kafka topic. This is actually working with the ParallelStreamProcessor. But I am failing to make it work with ReactorProcessor
Here is the code that is working for me:
pConsumer = createPConsumer()
pConsumer.subscribe(UniLists.of(kafkaConsumerConfig.kafkaTopic))
pConsumer.pollAndProduceMany ({ something ->
val records = something.stream().toList()
records.map { any ->
println("Consuming ${any.partition()}:${any.offset()}")
ProducerRecord<String, JsonObject>("output", any.key(),
JsonObject(mapOf("someTest" to any.offset())))
}
}, { consumeProduceResult ->
println(
"Message ${consumeProduceResult.getOut()} saved to broker at offset " +
"${consumeProduceResult.getMeta().offset()}"
)
})
private fun createPConsumer(): ParallelStreamProcessor<String, JsonObject> {
val producer = KafkaProducerBuilder.getProducer(kafkaConsumerConfig)
val options = ParallelConsumerOptions.builder<String, JsonObject>()
.ordering(ParallelConsumerOptions.ProcessingOrder.KEY)
.maxConcurrency(parallelConsumerConfig.maxConcurrency)
.batchSize(parallelConsumerConfig.batchSize)
.consumer(buildConsumer(kafkaConsumerConfig))
.producer(producer)
.build()
return ParallelStreamProcessor.createEosStreamProcessor(options)
}
Expected this to send events, but it does not:
pConsumer.react { context ->
val events = context.stream().toList()
// do something with events
val results = events.map{any -> ProducerRecord<String, JsonObject>("output", any.key(),
JsonObject(mapOf("someTest" to any.offset())))}
Mono.just(results)
}
Will appreciate any advice
So, currently (version 0.5.2.4) it is not supported. see issue.
we did implement it in the following way if it helps anyone:
// Example usage
parallelConsumer.react(context -> {
var consumerRecord = context.getSingleRecord().getConsumerRecord();
log.info("Concurrently constructing and returning RequestInfo from record: {}", consumerRecord);
Map<String, String> params = UniMaps.of("recordKey", consumerRecord.key(), "payload", consumerRecord.value());
Mono originalResult = Mono.just(Arrays.asList(new ProducerRecord("topic", "key", "some value"));
return originalResult.map(batchProducer::produce);
});
class BatchProducer<K, V> {
Producer<K, V> producer;
public BatchProducer(Producer<K, V> producer) {
this.producer = producer;
}
public Mono<List<RecordMetadata>> produce(List<ProducerRecord<K, V>> messages) {
List<CompletableFuture<RecordMetadata>> futures = messages.stream().map(message -> {
CompletableFuture<RecordMetadata> completableFuture = new CompletableFuture<RecordMetadata>();
Callback kafkaCallback = createCallback(completableFuture);
producer.send(message, kafkaCallback);
return completableFuture;
}).toList();
CompletableFuture<List<RecordMetadata>> oneResult = sequence(futures);
return Mono.fromFuture(oneResult);
}
// From here: https://stackoverflow.com/questions/30025428/convert-from-listcompletablefuture-to-completablefuturelist
static<T> CompletableFuture<List<T>> sequence(List<CompletableFuture<T>> com) {
return CompletableFuture.allOf(com.toArray(new CompletableFuture<?>[0]))
.thenApply(v -> com.stream()
.map(CompletableFuture::join)
.collect(Collectors.toList())
);
}
private Callback createCallback(CompletableFuture<RecordMetadata> completableFuture) {
return new Callback() {
#Override
public void onCompletion(RecordMetadata metadata, Exception exception) {
if (exception != null) {
completableFuture.completeExceptionally(exception);
} else {
completableFuture.complete(metadata);
}
}
};
}
public void close() {
producer.close();
}
}
using kotlin, having code
fun fetchRemoteDataApi(): Single<RemoteDataResponse> = networkApi.getData()
// it is just a retrofit
#GET(".../api/getData")
fun getData() : Single<RemoteDataResponse>
fun mergeApiWithDb(): Completable = fetchRemoteDataApi()
.zipWith(localDao.getAll())
.flatMapCompletable { (remoteData, localData) ->
doMerge(remoteData, localData) //<== return a Completable
}
the code flow:
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache() //<=== would like do some inspection at this level
PublishSubject.create<Unit>().toFlowable(BackpressureStrategy.LATEST)
.compose(Transformers.flowableIO())
.switchMap {
//merge DB with api, or local default value first then listen to DB change
mergeApiDbCall.andThen(listAllTopics())
.concatMapSingle { topics -> remoteTopicUsers.map { topics to it } }
}
.flatMapCompletable { (topics, user) ->
// do something return Completable
}
.subscribe({
...
}, { throwable ->
...
})
and when making the call
val mergeApiDbCall = mergeApiWithDb().onErrorComplete().cache()
the question is if would like to inspect on the Singles<RemoteDataResponse> returned from fetchRemoteDataApi() (i.e. using Log.i(...) to printout the content of RemoteDataResponse, etc.), either in got error or success case, how to do it?
/// the functions
fun listAllTopics(): Flowable<List<String>> = localRepoDao.getAllTopics()
// which a DAO:
#Query("SELECT topic FROM RemoteDataTable WHERE read = 1")
fun getAllTopics(): Flowable<List<String>>
///
private val remoteTopicUsers: Single<List<User>>
get() {
return Single.create {
networkApi.getTopicUsers(object : ICallback.IGetTopicUsersCallback {
override fun onSuccess(result: List<User>) = it.onSuccess(result)
override fun onError(errorCode: Int, errorMsg: String?) = it.onError(Exception(errorCode, errorMsg))
})
}
}
You cannot extract information about elements from the Completable. Though you can use doOnComplete() on Completable, it will not provide you any information about the element.
You can inspect elements if you call doOnSuccess() on your Single, so you need to incorporate this call earlier in your code. To inspect errors you can use doOnError() on both Completable or Single.
I have an error when receiving the body parts of the multi-part request
The back is receiving a null key and all the parameters inside an array as value. I need a key value for each parameter
My code to send the multipart is the following
```
suspend inline fun<reified T> _postMultiPart(microservice: String,
function: String,
parameters: HashMap<String, Any?>,
partsData: HashMap<String, Any?>,
domain: String = MySharedPreferences().escuela
): T {
return httpClient.post(Constants.ROOT_URL+ microservice+function) {
parameters.forEach {
parameter(it.key, it.value)
}
body = MultiPartFormDataContent(
formData {
partsData.forEach {
append(it.key, it.value.toString())
}
}
)
headers {
append("referer", Constants.HEADER_REFERER+function)
append("dominio_app", domain)
}
}
}
```
I'm trying to pass a parameter through the webClient context to understand how the reactive context works.
.post()
.uri(uri)
.headers(httpHeaders -> httpHeaders.putAll(headers))
.body(BodyInserters.fromObject(payload))
.exchange()
.flatMap( s -> Mono.subscriberContext()
.map( ctx -> {
System.out.println(“SubscriberContext1 " + ctx.getOrDefault("test", "DefaultValue”));
return s;
}))
.flatMap(resp -> {
return resp
.bodyToMono(Object.class)
.defaultIfEmpty(new HashMap<>())
.map(body -> {
//I would like to access the context variable here
return doSomething(body, contextVariable);
});})
.flatMap( s -> Mono.subscriberContext()
.map( ctx -> {
System.out.println("SubscriberContext2” + ctx.getOrDefault("test", "DefaultValue”));
return s;
}))
.subscriberContext(ctx -> {
System.out.println("WRITE TEST");
ctx.put("test", "test");
return ctx;
});
This prints out:
WRITE TEST
SubscriberContext1 DefaultValue
SubscriberContext2 DefaultValue
I'd expect it to be:
WRITE TEST
SubscriberContext1 test
SubscriberContext2 test
Why is the context variable not accessible within the flatMap?
ctx.put creates a new context. Changing the code to:
.subscriberContext(ctx -> {
System.out.println("WRITE TEST");
return ctx.put("test", "test");
});
solves it.
After performing a parallelStream() on a List, I end up with a List<Map<String, Set<String>. I want to unify this into a Map<String, Set<String>> (which will only keep uniques across the List of Maps).
I am unfamiliar with the collect and reduce functions, so don't have anything to go ahead with.
Existing code:
private val TYPES = listOf("string", "integer")
private fun getLinesOfEachTypeAcrossMultipleFiles(files: List<File>): Map<String, Set<String>> {
return files
.parallelStream()
.map { file ->
TYPES.associate {
it to getRelevantTypeLinesFromFile(file)
}
}
// Converted into a Stream<String, Set<String>>
// .reduce() / collect() ?
}
private fun getRelevantTypeLinesFromFile(it: File): Set<String> {
// Sample code
return setOf()
}
If you're looking for an equivalent Java code, you can stream all the entries using flatMap and then collect them as a Map with a merge function as :
Map<String, Set<String>> some(List<Map<String, Set<String>>> listOfMap) {
return listOfMap.stream()
.flatMap(a -> a.entrySet().stream())
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue,
(s1, s2) -> {
s1.addAll(s2);
return s1;
}));
}
I figured out and implemented a Kotlin-specific solution of using the fold operator (instead of reduce or collect):
private val TYPES = listOf("string", "integer")
private fun getLinesOfEachTypeAcrossMultipleFiles(files: List<File>): Map<String, Set<String>> {
return files
.map { file ->
TYPES.associate { it to getRelevantTypeLinesFromFile(file) }
}
.fold(mutableMapOf<String, MutableSet<String>>()) { acc, map ->
acc.apply {
map.forEach { key, value ->
acc.getOrPut(key) { mutableSetOf() }.addAll(value)
}
}
}
}
private fun getRelevantTypeLinesFromFile(it: File): Set<String> {
// Sample code
return setOf()
}
A benefit of using fold is that we don't need to change the type of the data from Map to MutableMap and Set to MutableSet.