WebFlux: why do I need to use flatMap in CRUD - crud

I found examples on the Internet but this doesn't give me a full understanding. Standard CRUD when using WebFlux.
Router:
#Configuration
public class PersonRouter {
#Bean
public RouterFunction<ServerResponse> route(PersonHandler handler) {
return RouterFunctions
.route(GET("/getAllPersons").and(accept(MediaType.APPLICATION_JSON)), handler::findAll)
.andRoute(GET("/getPerson/{id}").and(accept(MediaType.APPLICATION_STREAM_JSON)), handler::findById)
.andRoute(POST("/createPerson").and(accept(MediaType.APPLICATION_JSON)), handler::save)
.andRoute(DELETE("/deletePerson/{id}").and(accept(MediaType.APPLICATION_JSON)), handler::delete);
}
}
Handler:
#Component
public class PersonHandler {
private final PersonService personService;
public PersonHandler(PersonService personService) {
this.personService = personService;
}
public Mono<ServerResponse> findById(ServerRequest request) {
String id = request.pathVariable("id");
return ok()
.contentType(MediaType.APPLICATION_JSON)
.body(personService.getById(id), Person.class);
}
public Mono<ServerResponse> findAll(ServerRequest request) {
return ok()
.contentType(MediaType.APPLICATION_JSON)
.body(personService.getAll(), Person.class);
}
public Mono<ServerResponse> save(ServerRequest request) {
final Mono<Person> person = request.bodyToMono(Person.class);
return ok()
.contentType(MediaType.APPLICATION_JSON)
.body(fromPublisher(person.flatMap(personService::save), Person.class));
}
public Mono<ServerResponse> delete(ServerRequest request) {
String id = request.pathVariable("id");
return ok()
.contentType(MediaType.APPLICATION_JSON)
.body(personService.delete(id), Void.class);
}
}
Repository:
#Repository
public interface PersonRepository extends ReactiveMongoRepository<Person, String> {
}
Service:
#Service
#Transactional
#AllArgsConstructor
public class PersonService {
private final PersonRepository personRepository;
public Flux<Person> getAll() {
return personRepository.findAll().switchIfEmpty(Flux.empty());
}
public Mono<Person> getById(final String id) {
return personRepository.findById(id);
}
public Mono update(final String id, final Person person) {
return personRepository.save(person);
}
public Mono save(final Person person) {
return personRepository.save(person);
}
public Mono delete(final String id) {
final Mono<Person> dbPerson = getById(id);
if (Objects.isNull(dbPerson)) {
return Mono.empty();
}
return getById(id).switchIfEmpty(Mono.empty()).filter(Objects::nonNull).flatMap(personToBeDeleted -> personRepository
.delete(personToBeDeleted).then(Mono.just(personToBeDeleted)));
}
}
I understand everything except the save and update methods. I don't understand why we use flatMap in this situation.
Why is this so, and how can I write the implementation of the update method in my Handler.
Updated
Let's see the method save() in Handler
public Mono<ServerResponse> save(ServerRequest request) {
final Mono<Person> person = request.bodyToMono(Person.class);
return ok()
.contentType(MediaType.APPLICATION_JSON)
.body(fromPublisher(person.flatMap(personService::save), Person.class));
}
I think the fact is that we have already received:
final Mono<Person> person = request.bodyToMono(Person.class);
and then we do:
personService::save
As a result, we get Mono< Mono< Person>>
flatMap is just like map, except that it unpacks the return value of the lambda given if the value is itself contained in a Publisher<T>. In our case, the personService.save(T) method returns a Mono<T>. If we’d used map instead of flatMap(T), we’d have a Mono< Mono< T>>, when what we really want is a Mono<T>. We can cleanly solve this problem using flatMap.
Am I right or is this statement wrong?

why you need flatMap.
These are my ideas, the answer varies depending on whether you it works on Mono or Flux.
1.
javadoc of method map and flatMap shows their usage:
map: Transform the item emitted by this {#link Mono} by applying a synchronous function to it.
flatMap: Transform the item emitted by this {#link Mono} asynchronously, returning the value emitted by another {#link Mono} (possibly changing the value type).
That says, considering flatMap and map as a pipeline with input and output, you use map when the output is the same item , otherwise, use flatMap. Check this:
public Mono<ServerResponse> influCRUD(ServerRequest req) {
return req.bodyToMono(S.class) // the pipline begins with S class.
.map(s -> {s.setF1(f1); s.setF2(f2); return s;}) // the pipeline has the same intput and output, i.e. object s, you use map.
.flatMap(s -> webClient // the pipeline has S input, and T output, you use flatMap
.post()
.uri(uri)
.body(BodyInserters.fromObject(s))
.retrive()
.bodyToMono(T.class)
).flatMap(t -> ServerResponse // now the pipeline changes again, you use flatMap.
.ok()
.contentType()
.body(BodyInserters.fromObject(t))
);
}
it deserves to mention that map can have different object as output as well.
flatMap processes every items
the above reason is usefull for Mono producer. For Flux, flatMap processes every items, while map processes all items (or one item). this is the same as they are in lambda. In case you want to process every item, you use flatMap.
flatMap takes off one layer of Mono for you.
Look at their declaration:
<R> Mono<R> map(Function<? super T, ? extends R> mapper)
and
<R> Mono<R> flatMap(Function<? super T, ? extends Mono<? extends R>> transformer)
Function does nothing but a -> b, when b is the output of another Producer/Subsciber (this is very likely when you use reactive programming) like the webClient part in the former example, it is in the form of Mono or Flux. By using flatMap, it returns Mono<R> for you, where map returns Mono<Mono<R>>, as they are stated in the function declaration.
I am a beginner in reative programming too, more than welcome to correct this.

Related

How to parse kafka message in KafkaSpout and set in tuple value

I am trying to read kafka messages from KafkaSpout and set tuple values from json that are parsed from that message. Actually, I am creating an additional Bolt that parses a tuple field called "value" with json string from KafkaSpout. Is it possible to set these values in Spout?
class ScanConfigKafkaSpout(kafkaUrl: String, kafkaGroup: String, kafkaTopic: String) : KafkaSpout<String, String>(
KafkaSpoutConfig
.builder(kafkaUrl, kafkaTopic)
.setProp(KEY_KAFKA_GROUP, "grp1")
.setProcessingGuarantee(KafkaSpoutConfig.ProcessingGuarantee.AT_MOST_ONCE)
.build()
), ComponentId {
override fun open(conf: MutableMap<String, Any>?, context: TopologyContext?, collector: SpoutOutputCollector?) {
try {
logger.debug("<${id()}> Opening ScanConfigKafkaSpout with ${conf.toString()}")
super.open(conf, context, collector)
logger.debug("<${id()}> ScanConfigKafkaSpout opened")
} catch (t: Throwable) {
logger.error("<${id()}> Error during opening CrawlScanConfigKafkaSpout", t)
}
}
override fun id(): String = SCAN_CONFIG_KAFKA_SPOUT
companion object {
private val logger = LoggerFactory.getLogger(ScanConfigKafkaSpout::class.java)
}
}
You probably need to implement the method declareOutputFields(OutputFieldsDeclarer declarer from IComponent.
It is used by Storm to serialize your attribute values and tuple configurations.
As stated here in the section Data Model , it says:
Every node in a topology must declare the output fields for the tuples it emits.
There is also a java example given for that method.
#Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("double", "triple"));
}

Gson - deserialize or default

I have a class :
data class Stam(#SerializedName("blabla") val blabla: String = "")
I want to do gson.fromJson("{\"blabla\":null}", Stam::class.java)
However, it will fail because blabla is not nullable.
I want to make it so if gson failed to deserialize some variable, it will take the default value I give it.
How to achieve that?
I don't think it is possible with GSON, this is one of the reasons why kotlinx.serialization library was created. With this library it is fairly easy:
#Serializable
data class Stam(#SerialName("blabla") val blabla: String = "") //actually, #SerialName may be omitted if it is equal to field name
Json { coerceInputValues = true }.decodeFromString<Stam>("{\"blabla\":null}")
I wouldn't say it is not possible in Gson, but Gson is definitely not the best choice:
Gson has no mention on Kotlin, its runtime and specifics, so one is better to use a more convenient and Kotlin-aware tool. Typical questions here are: how to detect a data class (if it really matters, can be easily done in Kotlin), how to detect non-null parameters and fields in runtime, etc.
Data classes in Kotlin seem to provide a default constructor resolvable by Gson therefore Gson can invoke it (despite it can instantiate classes instances without constructors using unsafe mechanics) delegating to the "full-featured" constructor with the default arguments. The trick here is removing null-valued properties from input JSON so Gson would keep "default-argumented" fields unaffected.
I do Java but I do believe the following code can be converted easily (if you believe Gson is still a right choice):
final class StripNullTypeAdapterFactory
implements TypeAdapterFactory {
// The rule to check whether this type adapter should be applied.
// Externalizing the rule makes it much more flexible.
private final Predicate<? super TypeToken<?>> isClassSupported;
private StripNullTypeAdapterFactory(final Predicate<? super TypeToken<?>> isClassSupported) {
this.isClassSupported = isClassSupported;
}
static TypeAdapterFactory create(final Predicate<? super TypeToken<?>> isClassSupported) {
return new StripNullTypeAdapterFactory(isClassSupported);
}
#Override
#Nullable
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
if ( !isClassSupported.test(typeToken) ) {
return null;
}
// If the type is supported by the rule, get the type "real" delegate
final TypeAdapter<T> delegate = gson.getDelegateAdapter(this, typeToken);
return new StripNullTypeAdapter<>(delegate);
}
private static final class StripNullTypeAdapter<T>
extends TypeAdapter<T> {
private final TypeAdapter<T> delegate;
private StripNullTypeAdapter(final TypeAdapter<T> delegate) {
this.delegate = delegate;
}
#Override
public void write(final JsonWriter out, final T value)
throws IOException {
delegate.write(out, value);
}
#Override
public T read(final JsonReader in) {
// Another disadvantage in using Gson:
// the null-stripped object must be buffered into memory regardless how big it is.
// So it may generate really big memory footprints.
final JsonObject buffer = JsonParser.parseReader(in).getAsJsonObject();
// Strip null properties from the object
for ( final Iterator<Map.Entry<String, JsonElement>> i = buffer.entrySet().iterator(); i.hasNext(); ) {
final Map.Entry<String, JsonElement> property = i.next();
if ( property.getValue().isJsonNull() ) {
i.remove();
}
}
// Now there is no null values so Gson would only use properties appearing in the buffer
return delegate.fromJsonTree(buffer);
}
}
}
Test:
public final class StripNullTypeAdapterFactoryTest {
private static final Collection<Class<?>> supportedClasses = ImmutableSet.of(Stam.class);
private static final Gson gson = new GsonBuilder()
.disableHtmlEscaping()
// I don't know how easy detecting data classes and non-null parameters is
// but since the rule is externalized, let's just lookup it
// in the "known classes" registry
.registerTypeAdapterFactory(StripNullTypeAdapterFactory.create(typeToken -> supportedClasses.contains(typeToken.getRawType())))
.create();
#Test
public void test() {
final Stam stam = gson.fromJson("{\"blabla\":null}", Stam.class);
// The test is "green" since
Assertions.assertEquals("", stam.getBlabla());
}
}
I still think Gson is not the best choice here.

JUnit 5 and Arguments.of() with functions

Writing a JUnit 5 parameterized test and need to pass functions to the test using Arguments.of(), but there are 2 compile errors that I don't know how to fix. Any help would be appreciated.
The method of(Object...) in the type Arguments is not applicable for the arguments (boolean, String::length)
The target type of this expression must be a functional interface
public static Stream<Arguments> some() {
return Stream.of(Arguments.of(true, String::length));
}
#ParameterizedTest
#MethodSource
public <T> void some(final T input, final Function<String, Integer> length) {
}
The following works as expected.
public void sample() {
some(true, String::length);
}
Wrap the arguments in a helper method
Similar to the answer "wrap it in a class", but possibly less intrusive, is to use a helper method to pass the functional interface as a java.lang.Object.
For example, the first raw method reference, Math::ciel, in this parameterized test:
#ParameterizedTest
#MethodSource("testCases")
void shouldExerciseMethod(Function<Double, Double> method, Double expected) {
assertEquals(expected, method.apply(1.5d), 1.0E-8d);
}
static Stream<Arguments> testCases() {
return Stream.of(Arguments.of(Math::ceil, 2.0d),
Arguments.of(Math::floor, 1.0d));
}
causes this compilation error:
java: method of in interface org.junit.jupiter.params.provider.Arguments cannot be applied to given types;
required: java.lang.Object[]
found: Math::ceil,double
reason: varargs mismatch; java.lang.Object is not a functional interface
which you can get around by passing the arguments through a helper method:
static <T, U> Arguments args(Function<T, U> method, U expected) {
return Arguments.of(method, expected);
}
so:
static Stream<Arguments> testCases() {
return Stream.of(args(Math::ceil, 2.0d),
args(Math::floor, 1.0d));
}
My attempts to make the idiom more general using varargs failed with variations on the same error, so I have ended up overloading it whenever I need another signature.
The function needs to be wrapped in a class.
public static class P {
private final Function<String, Integer> mFunction;
public P(final Function<String, Integer> function) {
mFunction = function;
}
public Function<String, Integer> function() {
return mFunction;
}
}
public static Stream<Arguments> some() {
return Stream.of(Arguments.of(3, "abc", new P(String::length)));
}
#ParameterizedTest
#MethodSource
public <T> void some(final int expect, final String input, final P p) {
assertEquals(expect, p.function().apply(input));
}
I liked #adrian-redgers solution, but I think overloading a method for each signature needed is a bit overkill.
You only really need to convert the functional interface to an object. So the solution I implemented was:
/**
* Helps to use {#link org.junit.jupiter.params.provider.Arguments#of(Object...)}, as functional
* interfaces cannot be converted into an object directly.
*/
public class ArgumentsWrapper {
private ArgumentsWrapper() {
throw new IllegalStateException(
ArgumentsWrapper.class + " util class cannot be instantiated");
}
public static <T, U> Function<T, U> wrap(Function<T, U> function) {
return function;
}
}
Then, it can be used as:
public static Stream<Arguments> testMapAlarmTypeConfigWithLanguage() {
return Stream.of(
// Statically imported ArgumentsWrapper#wrap
Arguments.of(null, wrap(AlarmTypeConfig::getNameInEnglish)),
Arguments.of("en-us", wrap(AlarmTypeConfig::getNameInEnglish)),
Arguments.of("es-es", wrap(AlarmTypeConfig::getNameInSpanish)));
}

Transactions with ReactiveCrudRepository with spring-data-r2dbc

I'm trying to implement transactions with spring-data-r2dbc repositories in combination with the TransactionalDatabaseClient as such:
class SongService(
private val songRepo: SongRepo,
private val databaseClient: DatabaseClient
){
private val tdbc = databaseClient as TransactionalDatabaseClient
...
...
fun save(song: Song){
return tdbc.inTransaction{
songRepo
.save(mapRow(song, albumId)) //Mapping to a row representation
.delayUntil { savedSong -> tdbc.execute.sql(...).fetch.rowsUpdated() } //saving a many to many relation
.map(::mapSong) //Mapping back to actual song and retrieve the relationship data.
}
}
}
I currently have a config class (annotated with #Configuration and #EnableR2dbcRepositories) that extends from AbstractR2dbcConfiguration. In here I override the databaseClient method to return a TransactionalDatabaseClient. This should be the same instance as in the SongService class.
When running the code in a test with just subscribing and printing, I get org.springframework.transaction.NoTransactionException: ReactiveTransactionSynchronization not active and the relationship data is not returned.
When using project Reactors stepverifier though, i get java.lang.IllegalStateException: Connection is closed. Also in this case, the relationship data is not returned.
Just for the record, I have seen https://github.com/spring-projects/spring-data-r2dbc/issues/44
Here is a working Java example:
#Autowired TransactionalDatabaseClient txClient;
#Autowired Mono<Connection> connection;
//You Can also use: #Autowired Mono<? extends Publisher> connectionPublisher;
public Flux<Void> example {
txClient.enableTransactionSynchronization(connection);
// Or, txClient.enableTransactionSynchronization(connectionPublisher);
Flux<AuditConfigByClub> audits = txClient.inTransaction(tx -> {
txClient.beginTransaction();
return tx.execute().sql("SELECT * FROM audit.items")
.as(Item.class)
.fetch()
.all();
}).doOnTerminate(() -> {
txClient.commitTransaction();
});
txClient.commitTransaction();
audits.subscribe(item -> System.out.println("anItem: " + item));
return Flux.empty()
}
I just started reactive so not too sure what I'm doing with my callbacks haha. But I decided to go with TransactionalDatabaseClient over DatabaseClient or Connection since I'll take all the utility I can get while R2dbc is in its current state.
In your code did you actually instantiate a Connection object? If so I think you would have done it in your configuration. It can be utilized throughout the app the same as DatabaseClient, but it is slightly more intricate.
If not:
#Bean
#Override // I also used abstract config
public ConnectionFactory connectionFactory() {
...
}
#Bean
TransactionalDatabaseClient txClient() {
...
}
//TransactionalDatabaseClient will take either of these as arg in
//#enableTransactionSynchronization method
#Bean
public Publisher<? extends Connection> connectionPublisher() {
return connectionFactory().create();
}
#Bean
public Mono<Connection> connection() {
return = Mono.from(connectionFactory().create());
}
If you are having problems translating to Kotlin, there is an alternative way to enable synchronization that could work:
// From what I understand, this is a useful way to move between
// transactions within a single subscription
TransactionResources resources = TransactionResources.create();
resources.registerResource(Resource.class, resource);
ConnectionFactoryUtils
.currentReactiveTransactionSynchronization()
.subscribe(currentTx -> sync.registerTransaction(Tx));
Hope this translates well for Kotlin.

Kotlin syntax confusion: passing an interface with a lambda

I am studying Kotlin programming for Android and I am trying to understand this code (that works) deeply.
It comes from the Volley library for network request:
//Network stuff
// Request a string response from the provided URL.
val jsonObjectRequest = object : JsonObjectRequest(Method.POST, http, ob,
Response.Listener<JSONObject> { response ->
// Display the first 500 characters of the response string.
Log.d("Debug","Response is: ${response.toString()} ")
},
Response.ErrorListener { error ->
Log.d("Debug","That didn't work! Code: ${error.message}")
})
{
#Throws(AuthFailureError::class)
override fun getHeaders(): Map<String, String> {
val headers = HashMap<String, String>()
headers.put("Content-Type", "application/json")
headers.put("Accept", "application/json")
return headers
}
}
My question is about the first block, right inside the constructor of the JsonObjectRequest object. I know the object construct, lambdas, classes, and interfaces but there is one little thing that I don't get here. Moreover, I have already seen this thread Pass interface as parameter in Kotlin.
My question is: what is happening in the fourth parameter used to construct the JsonObjectRequest? From what I see, there is a lambda overriding some function related to Response.Listener<JSONObject> but I don't find any reference to this syntax.
To conclude, the objectRequest has the former constructor:
public JsonObjectRequest(int method, String url, JSONObject jsonRequest,
Listener<JSONObject> listener, ErrorListener errorListener) {
super(method, url, (jsonRequest == null) ? null : jsonRequest.toString(), listener,
errorListener);
}
and the listener has the following section:
public class Response<T> {
/** Callback interface for delivering parsed responses. */
public interface Listener<T> {
/** Called when a response is received. */
void onResponse(T response);
}
/** Callback interface for delivering error responses. */
public interface ErrorListener {
/**
* Callback method that an error has been occurred with the
* provided error code and optional user-readable message.
*/
void onErrorResponse(VolleyError error);
}
Reading this I get that with this syntax we are implementing the Listener interface but I don't get why we use a lambda, since in the Listener there is no reference to it, and in particular what does this mean:
Response.Listener<JSONObject> { response ->
// Display the first 500 characters of the response string.
Log.d("Debug","Response is: ${response.toString()} ")
}
Anyone willing to explain this or pointing to some references related to this syntax?
This is a SAM-conversion. Because Response.Listener<JSONObject> is a SAM-interface (has a single method without a default implementation) and defined in Java, you can write
Response.Listener<JSONObject> /* lambda */
and the lambda is used as the implementation of the method. I.e. it's equivalent to
object : Response.Listener<JSONObject> {
override fun onResponse(response: JSONObject) {
Log.d("Debug","Response is: ${response.toString()} ")
}
}