I am trying the reactive stack with MySql at the backend.
I was expecting that query results will be a stream , t.e. a heavy query will not wait and return the result when the all the records are found, but will stream them one by one. It does not look that way. The process waits untill the query is done and then returns all the results.
This is the Spring Data repository that I created:
public interface ScopusSpringDataRepo extends ReactiveCrudRepository<Scopus, Long> {
#Query("select" + " SC1.*" + "from Scopus as SC1 join Scopus as SC2"
+ " on SC1.norma like concat(SC2.norma, '%') where ( SC2.norma is not null"
+ " and SC2.word like concat('%', :word, '%'))")
public Flux<Scopus> byNorma(String word);
}
This query is heavy on purpose.
Can someone please explain what behavior is expected from R2DBC in this case?
Thank you.
Update after the comments:
The repository method byNorma is called by a business service that is called by a REST Controller.
Service:
public class ScopusService {
protected ScopusSpringDataRepo scRepo;
public Flux<Scopus> getByNorma(String norma) {
return scRepo.byNorma(norma);
}
}
REST Controller:
public class ScopusController {
protected ScopusService scopusService;
#GetMapping(path = "/stream/scopus/{word}", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public ResponseEntity<Flux<Scopus>> getByNorma(#PathVariable String word) {
log.info("Getby norma recieved. word :{}", word);
return ResponseEntity.ok(this.scopusService.getByNorma(word));
}
}
Which is called from a Chromium browser.
Update:
I am trying to understand whether the R2DBC driver sends the result one by one or just gets all the results first and then sends them into the stream at once.
Related
I’m very new to the SpringReactor project.
Until now I've only used Mono from WebClient .bodyToMono() steps, and mostly block() those Mono's or .zip() multiple of them.
But this time I have a usecase where I need to asynchronously call methods in multiple service classes, and those multiple service classes are calling multiple backend api.
I understand Project Reactor doesn't provide asynchronous flow by default.
But we can make the publishing and/or subscribing on different thread and make code asynchronous
And that's what I am trying to do.
I tried to read the documentation here reactor reference but still not clear.
For the purpose of this question, I’m making up this imaginary scenario. that is a little closer to my use case.
Let's assume we need to get a search response from google for some texts searched under images.
Example Scenario
Let's have an endpoint in a Controller
This endpoint accepts the following object from request body
MultimediaSearchRequest{
Set<String> searchTexts; //many texts.
boolean isAddContent;
boolean isAddMetadata;
}
in the controller, I’ll break the above single request object into multiple objects of the below type.
MultimediaSingleSearchRequest{
String searchText;
boolean isAddContent;
boolean isAddMetadata;
}
This Controller talks to 3 Service classes.
Each of the service classes has a method searchSingleItem.
Each service class uses a few different backend Apis, but finally combines the results of those APIs responses into the same type of response class, let's call it MultimediaSearchResult.
class JpegSearchHandleService {
public MultimediaSearchResult searchSingleItem
(MultimediaSingleSearchRequest req){
return comboneAllImageData(
getNameApi(req),
getImageUrlApi(req),
getContentApi(req) //dont call if req.isAddContent false
)
}
}
class GifSearchHandleService {
public MultimediaSearchResult searchSingleItem
(MultimediaSingleSearchRequest req){
return comboneAllImageData(
getNameApi(req),
gitPartApi(req),
someRandomApi(req),
soemOtherRandomApi(req)
)
}
}
class VideoSearchHandleService {
public MultimediaSearchResult searchSingleItem
(MultimediaSingleSearchRequest req){
return comboneAllImageData(
getNameApi(req),
codecApi(req),
commentsApi(req),
anotherApi(req)
)
}
}
In the end, my controller returns the response as a List of MultimediaSearchResult
Class MultimediaSearchResponse{
List< MultimediaSearchResult> results;
}
If I want to use this all asynchronously using the project reactor. how to achieve it.
Like calling searchSingleItem method in each service for each searchText asynchronously.
Even within the services call each backend API asynchronously (I’m already using WebClient and converting response bodyToMono for backend API calls)
First, I will outline a solution for the upper "layer" of your scenario.
The code (a simple simulation of the scenario):
public class ChainingAsyncCallsInSpring {
public Mono<MultimediaSearchResponse> controllerEndpoint(MultimediaSearchRequest req) {
return Flux.fromIterable(req.getSearchTexts())
.map(searchText -> new MultimediaSingleSearchRequest(searchText, req.isAddContent(), req.isAddMetadata()))
.flatMap(multimediaSingleSearchRequest -> Flux.merge(
classOneSearchSingleItem(multimediaSingleSearchRequest),
classTwoSearchSingleItem(multimediaSingleSearchRequest),
classThreeSearchSingleItem(multimediaSingleSearchRequest)
))
.collectList()
.map(MultimediaSearchResponse::new);
}
private Mono<MultimediaSearchResult> classOneSearchSingleItem(MultimediaSingleSearchRequest req) {
return Mono.just(new MultimediaSearchResult("1"));
}
private Mono<MultimediaSearchResult> classTwoSearchSingleItem(MultimediaSingleSearchRequest req) {
return Mono.just(new MultimediaSearchResult("2"));
}
private Mono<MultimediaSearchResult> classThreeSearchSingleItem(MultimediaSingleSearchRequest req) {
return Mono.just(new MultimediaSearchResult("3"));
}
}
Now, some rationale.
In the controllerEndpoint() function, first we create a Flux that will emit every single searchText from the request. We map these to MultimediaSingleSearchRequest objects, so that the services can consume them with the additional metadata that was provided with the original request.
Then, Flux::flatMap the created MultimediaSingleSearchRequest objects into a merged Flux, which (as opposed to Flux::concat) ensures that all three publishers are subscribed to eagerly i.e. they don't wait for one another. It works best on this exact scenario, when several independent publishers need to be subscribed to at the same time and their order is not important.
After the flat map, at this point, we have a Flux<MultimediaSearchResult>.
We continue with Flux::collectList, thus collecting the emitted values from all publishers (we could also use Flux::reduceWith here).
As a result, we now have a Mono<List<MultimediaSearchResult>>, which can easily be mapped to a Mono<MultimediaSearchResponse>.
The results list of the MultimediaSearchResponse will have 3 items for each searchText in the original request.
Hope this was helpful!
Edit
Extending the answer with a point of view from the service classes as well. Assuming that each inner (optionally skipped) call returns a different type of result, this would be one way of going about it:
public class MultimediaSearchResult {
private Details details;
private ContentDetails content;
private MetadataDetails metadata;
}
public Mono<MultimediaSearchResult> classOneSearchSingleItem(MultimediaSingleSearchRequest req) {
return Mono.zip(getSomeDetails(req), getContentDetails(req), getMetadataDetails(req))
.map(tuple3 -> new MultimediaSearchResult(
tuple3.getT1(),
tuple3.getT2().orElse(null),
tuple3.getT3().orElse(null)
)
);
}
// Always wanted
private Mono<Details> getSomeDetails(MultimediaSingleSearchRequest req) {
return Mono.just(new Details("details")); // api call etc.
}
// Wanted if isAddContent is true
private Mono<Optional<ContentDetails>> getContentDetails(MultimediaSingleSearchRequest req) {
return req.isAddContent()
? Mono.just(Optional.of(new ContentDetails("content-details"))) // api call etc.
: Mono.just(Optional.empty());
}
// Wanted if isAddMetadata is true
private Mono<Optional<MetadataDetails>> getMetadataDetails(MultimediaSingleSearchRequest req) {
return req.isAddMetadata()
? Mono.just(Optional.of(new MetadataDetails("metadata-details"))) // api call etc.
: Mono.just(Optional.empty());
}
Optionals are used for the requests that might be skipped, since Mono::zip will fail if either of the zipped publishers emit an empty value.
If the results of each inner call extend the same base class or are the same wrapped return type, then the original answer applies as to how they can be combined (Flux::merge etc.)
I'm trying to learn Spring webflux & R2DBC. The one I try is simple use case:
have a book table
create an API (/books) that provides text stream and returning Flux<Book>
I'm hoping when I hit /books once, keep my browser open, and any new data inserted to book table, it will send the new data to browser.
Scenario 2, still from book table:
have a book table
create an API (/books/count) that returning count of data in book as Mono<Long>
I'm hoping when I hit /books/count once, keep my browser open, and any new data inserted /deleted to book table, it will send the new count to browser.
But it does not works. After I isnsert new data, no data sent to any of my endpoint.
I need to hit /books or /books/count to get the updated data.
I think to do this, I need to use Server Sent Events? But how to do this in and also querying data? Most sample I got is simple SSE that sends string every certain interval.
Any sample to do this?
Here is my BookApi.java
#RestController
#RequestMapping(value = "/books")
public class BookApi {
private final BookRepository bookRepository;
public BookApi(BookRepository bookRepository) {
this.bookRepository = bookRepository;
}
#GetMapping(produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<Book> getAllBooks() {
return bookRepository.findAll();
}
#GetMapping(value = "/count", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Mono<Long> count() {
return bookRepository.count();
}
}
BookRepository.java (R2DBC)
import org.springframework.data.r2dbc.repository.R2dbcRepository;
public interface BookRepository extends R2dbcRepository<Book, Long> {
}
Book.java
#Table("book")
#Data
#AllArgsConstructor
#NoArgsConstructor
public class Book {
#Id
private Long id;
#Column(value = "name")
private String name;
#Column(value = "author")
private String author;
}
Use a Processor or Sink to handle the Book created event.
Check my example using reactor Sinks, and read this article for the details.
Or use a tailable Mongo document.
A tailable MongoDB document can do the work automatically, check the main branch of the same repos.
My above example used the WebSocket protocol, it is easy to switch to SSE, RSocket.
Below Post would help you to achieve your first requirement
Spring WebFlux (Flux): how to publish dynamically
Let me know , if that helps you
UPD. Sorry, guys.
I have an application that acts as a SOAP server, how do I write a PHPUnit test to test it?
SOAP extension is reading data from PHP input stream. You just provide your own data there and create some integration/unit tests for your API.
Take a look at the signature of SoapServer::handle() method. It takes as an argument a string which is a request itself. This parameter is optional and if you don't pass anything in, PHP will just read the data itself. But you can simply override it.
I used streams to do it. First you wrap the SoapServer with your own class like this:
class MyServer
{
/** \SoapServer */
private $soapServer;
public function __construct(\SoapServer $soapServer)
{
$this->soapServer = $soapServer;
}
public function handle(Psr\Http\Message\StreamInterface $inputStream): void
{
$this->soapServer->handle($inputStream->getContent());
}
}
Now you are ready to mock the request.
In your test you can do:
class MyTest extends TestCase
{
public function testMyRequest(): void
{
$mySoapServer = $this->createMySoapServer();
$request = $this->createRequest();
$mySoapServer->handle($request);
}
private function createRequest(): StreamInterface
{
$requestString = '<soap:Envelope></soap:Envelope>';
$fh = fopen('php://temp', 'rw');
fwrite($fh, $requestString);
fseek($fh, SEEK_SET);
return new Psr\Http\Message\StreamInterface\Stream($fh);
}
private function createMySoapServer(): MyServer
{
return new MyServer(new \SoapServer());
}
}
One thing to keep in mind - this test will generate output. You may want to test this output or ignore it. Depends on your use case.
Another side note. What you are asking for has really nothing to do with PHPUnit. It just a matter of designing your SOAP server correctly.
If you are wondering how to set up the stream when you have a live request, this is really simple:
$server->handle(new Psr\Http\Message\StreamInterface\Stream(fopen('php://input', 'r+')));
I'm trying to make a call to a database using RequestFactory with Hibernate/JPA, and I want to retrieve a list of entities with embedded entities returned as well. I know that the .with() method works for methods like .find(), but it doesn't seem to work with custom queries.
The current way I'm doing it is as follows:
I used a named query in the entity class for the query. (Primary Entity is Name, embedded entity is a Suffix entity called nameSuffix)
#NamedQueries({ #NamedQuery(name = "Query.name", query = "select * from NameTable") })
Then in the service class, the .list() method, which is what I'd like to call with RequestFactory, is as follows.
public List<Name> list() {
return emp.get().createNamedQuery("Query.name").getResultList();
}
Finally, this is how I make the call in my client side code:
NameRequest context = requestFactory.createNameRequest();
context.list().with("nameSuffix").fire(new Receiver<List<NameProxy>>(){
public void onSuccess(List<NameProxy> response) {
String suff = response.get(0).getNameSuffix().getText();
}
});
In the above code, it says that getNameSuffix() returns null, which would imply that .with("nameSuffix") does not work with the .list() call like it does with the standard .find() method.
Is there a way to build a call that would return a list of entities and their embedded entities using .with(), or do I need to do it another way? If I need to do it another way, has anyone figured out a good way of doing it?
I think you misunderstood what the method with() is thought for, unless you had a method getNameSuffix which returns the NameSuffixentity. This is what the documentation says about it:
When querying the server, RequestFactory does not automatically populate relations in the object graph. To do this, use the with() method on a request and specify the related property name as a String
So, what you have to pass to the method is a list of the name of the child entities you want to retrieve. I hope this example could be helpful:
class A {
String getS(){return "s-a"}
B getB(){return new B();}
}
class B {
String getS(){return "s-b";}
C getC(){return new C();}
}
class C {
String getS(){return "s-c";}
}
context.getA().fire(new Receiver<A>(){
public void onSuccess(A response) {
// return 's-a'
response.getS();
// trhows a NPE
response.getB().getS();
}
});
context.getA().with("b").fire(new Receiver<A>(){
public void onSuccess(A response) {
// return 's-a'
response.getS();
// return 's-b'
response.getB().getS();
// trhows a NPE
response.getB().getC().getS();
}
});
context.getA().with("b.c").fire(new Receiver<A>(){
public void onSuccess(A response) {
// return 's-a'
response.getS();
// return 's-b'
response.getB().getS();
// return 's-c'
response.getB().getC().getS();
}
});
I'm using WCF custom Validator with HTTPS (.NET 4.5). Validate on success returns Customer object which I would like to use later. Currently I'm able to do it with Static variables which I like to avoid if possible. I tried to use HttpContext which becomes null in main thread. My understanding Validate runs under different thread. Is there any way I could share session info without involving DB or File share. See related threads here and here.
In Authentication.cs
public class CustomValidator : UserNamePasswordValidator
{
public override void Validate(string userName, string password)
{
//If User Valid then set Customer object
}
}
In Service.cs
public class Service
{
public string SaveData(string XML)
{
//Need Customer object here. Without it cannot save XML.
//HttpContext null here.
}
}
I can suggest you an alternative approach. Assuming that the WCF service is running in ASP.Net compatibility mode and you are saving the customer object to session storage. Create a class such as AppContext
The code would look something like this
public class AppContext {
public Customer CurrentCustomer {
get {
Customer cachedCustomerDetails = HttpContext.Current.Session[CUSTOMERSESSIONKEY] as Customer;
if (cachedCustomerDetails != null)
{
return cachedCustomerDetails;
}
else
{
lock (lockObject)
{
if (HttpContext.Current.Session[CUSTOMERSESSIONKEY] != null) //Thread double entry safeguard
{
return HttpContext.Current.Session[CUSTOMERSESSIONKEY] as Customer;
}
Customer CustomerDetails = ;//Load customer details based on Logged in user using HttpContext.Current.User.Identity.Name
if (CustomerDetails != null)
{
HttpContext.Current.Session[CUSTOMERSESSIONKEY] = CustomerDetails;
}
return CustomerDetails;
}
}
}
}
The basic idea here is to do lazy loading of data, when both WCF and ASP.Net pipelines have executed and HTTPContext is available.
Hope it helps.
Alright this should have been easier. Since the way UserNamePasswordValidator works, I needed to use custom Authorization to pass UserName/Password to the main thread and get customer info again from the database. This is an additional DB call but acceptable workaround for now. Please download code from Rory Primrose's genius blog entry.