How to handle null response of get command using lettuce reactive api - redis

I'm using lettuce reactive api to query redis, and need to handle the situation redis doesn't has the key. But it seems that it dosen't call the function of subscribe when the key doesn't exist
I want to query redis and then print the result,
if the result exist, it should print "result received:{result}"
and if the result doesn't exist, it should print "result received:null".
RedisURI redisURI = RedisURI.builder().withHost("10.203.0.114").withPort(6379).withPassword("123456").build();
RedisClient client = RedisClient.create(redisURI);
StatefulRedisConnection<String,String> connection = client.connect();
RedisStringReactiveCommands<String,String> reactive = connection.reactive();
reactive.get("hello").onErrorReturn("error").subscribe(res->{
System.out.println("result receive:"+res);
},Throwable::printStackTrace);
reactive.get("hell").doOnError(throwable -> {
System.out.println("do on error");
throwable.printStackTrace();
}).subscribe(r->{
System.out.println("result receive:"+r);
});
redis has the key: hello and it's value is world.
I expect the output of first get call should be "result receive:world" and actual output is the same with my expectation.
The output of second call should be "result receive:null",but nothing is printed actually
Solution:I found lettuce use project reactor api and its api behaviour is different from rxjava.It should use switchIfEmpty to handle null response
reactive.get("hell").switchIfEmpty(Mono.just("")).doOnError(throwable -> {
System.out.println("do on error");
throwable.printStackTrace();
}).subscribe(r->{
System.out.println("result receive:"+r);
});

Related

Is it possible to send an initial value?

I have a web method that returns a flux object when it will be time (it's linked to a pub/sub).
Would it at least be possible, only for the first call, to return a default?
public Flux<String> receiveStream() {
return myReactiveService.getData() //here can I return a value at start? //.map(...);
It is not that easy to do it "only for the first call". Each request is supposed to get its own sequence of Strings, unless you take specific steps to change that. And that is at two levels:
- WebFlux: each request leads to a separate invocation of the controller method, so the Flux is newly instantiated
- Reactor: most Flux are "cold", ie they don't generate data until they're subscribed to, and each subscription regenerates a separate dataset.
So even if you returned a cached Flux, it would probably still serve each request separately.
There is a way to share() a long-lived Flux so that later newcomers only see data that becomes available after they've subscribed to the shared Flux, which could help with the "only the first request" aspect of your requirement.
Assuming getData() by itself is cold (ie simply calling it doesn't trigger any meaningful processing):
AtomicReference<Flux<String>> sharedStream = new AtomicReference<>();
public Flux<String> receiveStream() {
Flux<String> result = sharedStream.get();
if (result == null) {
Flux<String> coldVersionWithInit = myReactiveService
.getData()
.startWith(FIRST_VALUE)
.map(...);
Flux<String> hotVersion = coldVersionWithInit.share();
if (sharedStream.compareAndSet(null, hotVersion))
result = hotVersion;
else
result = sharedStream.get();
}
return result;
}
i think what you are looking for is Flux#defaultIfEmpty

How to "peep" if Flux has exception?

I use Spring Webflux with Tomcat servlet container (spring-boot-starter-web + spring-boot-starter-webflux) and I would like to get the following result:
If flux of fails immediately, I would like sent to client response code 400
Otherwise, I would like to sent response code 200 and stream the flux
I tried different solutions, but no one works. v1 and v2 does not sent expected response code if failure scenario, v3 does not stream output is happy scenario.
I would like to "peep" exception on failFlux and trigger the exception before response code 200 is sent
#RequestMapping(produces = MediaType.APPLICATION_STREAM_JSON_VALUE)
public class X {
Flux<String> happyFlux = Flux.generate(s -> s.next("x"));
Flux<String> failFlux = Flux.error(new ResponseStatusException(BAD_REQUEST));
//ok: flux is streamed
#RequestMapping("/v1/happy")
Flux<String> v1Happy() {
return happyFlux;
}
//nok: http status code is 200
#RequestMapping("/v1/fail")
Flux<String> v1Fail() {
return failFlux;
}
//ok: flux is streamed
#RequestMapping("/v2/happy")
Mono<ResponseEntity<Flux<String>>> v2Happy() {
return Mono.just(ResponseEntity.ok().body(happyFlux));
}
//nok: http status code is 200
#RequestMapping("/v2/fail")
Mono<ResponseEntity<Flux<String>>> v2Fail() {
return Mono.just(ResponseEntity.ok().body(failFlux));
}
//nok: flux is not streamed but collected on server side
#RequestMapping("/v3/happy")
Mono<ResponseEntity<List<String>>> v3Happy() {
return happyFlux.collectList().map(ResponseEntity::ok);
}
//ok: http status code is 400
#RequestMapping("/v3/fail")
Mono<ResponseEntity<List<String>>> v3Fail() {
return failFlux.collectList().map(ResponseEntity::ok);
}
PS. What is interesting, v1 and v2 works with netty (only spring-boot-starter-webflux).
Update
I think "peeping" Flux is impossible. What I really is better Flux handling in Spring for servlet stack: https://jira.spring.io/browse/SPR-17440
I would advise not to use collectList() as that defeats the entire purpose of producing a Stream.
I believe you should be getting a 500 in case of exception message.
For instance, check the below code.
public Mono<ServerResponse> listPeople(ServerRequest request) {
int error = 10/0;
Flux<Person> peopleFlux = this.repository.allPeople();
peopleFlux = withDelay(peopleFlux);
return ServerResponse.ok().contentType(MediaType.TEXT_EVENT_STREAM).body(peopleFlux, Person.class);
}
The statement
int error = 10/0;
causes a 500 exception and in the client I do get 500. If I comment out the error statement then I get a 200.
So, please share you code if you are not getting 500. Please note that if the error happens after the server has started returning individual events in the stream then it will not be a 500.
You should rather use HTTP 207. https://httpstatuses.com/207

Spring WebFlux Web Client - Iterating paged REST API

I need to get the items from all pages of a pageable REST API. I also need to start processing items, as soon as they are available, not needing to wait for all the pages to be loaded. In order to do so, I'm using Spring WebFlux and its WebClient, and want to return Flux<Item>.
Also, the REST API I'm using is rate limited, and each response to it contains headers with details on the current limits:
Size of the current window
Remaining time in the current window
Request quota in window
Requests left in current window
The response to a single page request looks like:
{
"data": [],
"meta": {
"pagination": {
"total": 10,
"current": 1
}
}
}
The data array contains the actual items, while the meta object contains pagination info.
My current solution first does a "dummy" request, just to get the total number of pages, and the rate limits.
Mono<T> paginated = client.get()
.uri(uri)
.exchange()
.flatMap(response -> {
HttpHeaders headers = response.headers().asHttpHeaders();
Limits limits = new Limits();
limits.setWindowSize(headers.getFirst("X-Window-Size"));
limits.setWindowRemaining(headers.getFirst("X-Window-Remaining"));
limits.setRequestsQuota(headers.getFirst("X-Requests-Quota");
limits.setRequestsLeft(headers.getFirst("X-Requests-Remaining");
return response.bodyToMono(Paginated.class)
.map(paginated -> {
paginated.setLimits(limits);
return paginated;
});
});
Afterwards, I emit a Flux containing page numbers, and for each page, I do a REST API request, each request being delayed enough so it doesn't get past the limit, and return a Flux of extracted items:
return paginated.flatMapMany(paginated -> {
return Flux.range(1, paginated.getMeta().getPagination().getTotal())
.delayElements(Duration.ofMillis(paginated.getLimits().getWindowRemaining() / paginated.getLimits().getRequestsQuota()))
.flatMap(page -> {
return client.get()
.uri(pageUri)
.retrieve()
.bodyToMono(Item.class)
.flatMapMany(p -> Flux.fromIterable(p.getData()));
});
});
This does work, but I'm not happy with it because:
It does initial "dummy" request to get the number of pages, and then
repeats the same request to get the actual data.
It gets rate limits only with the initial request, and assumes the
limits won't change (eg, that it's the only one using the API) -
which may not be true, in which case it will get an error that it
exceeded the limit.
So my question is how to refactor it so it doesn't need the initial request (but rather get limits, page numbers and data from the first request, and continue through all pages, while updating (and respecting) the limits.
I think this code will do what you want. The idea is to make a flux that make a call to your resource server, but in the process to handle the response, to add a new event on that flux to be able to make the call to next page.
The code is composed of:
A simple wrapper to contains the next page to call and the delay to wait before executing the call
private class WaitAndNext{
private String next;
private long delay;
}
A FluxProcessor that will make HTTP call and process the response:
FluxProcessor<WaitAndNext, WaitAndNext> processor= DirectProcessor.<WaitAndNext>create();
FluxSink<WaitAndNext> sink=processor.sink();
processor
.flatMap(x-> Mono.just(x).delayElement(Duration.ofMillis(x.delay)))
.map(x-> WebClient.builder()
.baseUrl(x.next)
.defaultHeader("Accept","application/json")
.build())
.flatMap(x->x.get()
.exchange()
.flatMapMany(z->manageResponse(sink, z))
)
.subscribe(........);
I split the code with a method that only manage response: It simply unwrap your data AND add a new event to the sink (the event beeing the next page to call after the given delay)
private Flux<Data> manageResponse(FluxSink<WaitAndNext> sink, ClientResponse resp) {
if (resp.statusCode()!= HttpStatus.OK){
sink.error(new IllegalStateException("Status code invalid"));
}
WaitAndNext wn=new WaitAndNext();
HttpHeaders headers=resp.headers().asHttpHeaders();
wn.delay= Integer.parseInt(headers.getFirst("X-Window-Remaining"))/ Integer.parseInt(headers.getFirst("X-Requests-Quota"));
return resp.bodyToMono(Item.class)
.flatMapMany(p -> {
if (p.paginated.current==p.paginated.total){
sink.complete();
}else{
wn.next="https://....?page="+(p.paginated.current+1);
sink.next(wn);
}
return Flux.fromIterable(p.getData());
});
}
Now we just need to initialize the system by calling for the retrieval of the first page with no delay:
WaitAndNext wn=new WaitAndNext();
wn.next="https://....?page=1";
wn.delay=0;
sink.next(wn);

Meteor.http.get issue with Twitter API

I am using Meteor and the Twitter API for a project. I want to get information on a user from Twitter. I wrote a function that for example returns only the location of a user from Twitter. I believe this is the proper way to do a request on Meteor. Here it is :
Meteor.methods({getTwitterLocation: function (username) {
Meteor.http.get("https://api.twitter.com/1/users/show.json?screen_name="+ username +"&include_entities=true", function(error, result) {
if (result.statusCode === 200) {
var respJson = JSON.parse(result.content);
console.log(respJson.location);
console.log("location works");
return (respJson.location)
}else {
return ( "Unknown user ")
}
});
}});
Now this function will log what's in the console on my Git Bash. I get someones Location by doing a Meteor.call. But I want to post what that function returns on a page. In my case, I want to post in on a user's profile. This doesn't work. But the console.log(respJson.location) returns the location in my Git Bash but it won't display anything on the profile page. This is what I did on my profile page:
profile.js :
Template.profile.getLocation= function(){
return Meteor.call("getTwitterLocation","BillGates");
}
profile.html :
<template name="profile">
from {{getLocation}}
</template>
With that I get "Seattle, WA" and " "location works" on my Git Bash but nothing on the profile page. If anyone knows what I can do, that'd be really appreciated. Thanks.
Firstly when data is returned from the server you need to use a synchronous call, as the callback will return the data when the server already thinks the meteor method has completed. (the callback will be fired at a later time, when the data is returned from the server, by which time the meteor client would have already got a response)
var result = Meteor.http.get("https://api.twitter.com/1/users/show.json?screen_name="+ username +"&include_entities=true");
if (result.statusCode === 200) {
var respJson = JSON.parse(result.content);
console.log(respJson.location);
console.log("location works");
return (respJson.location)
}else {
return ( "Unknown user ")
}
The second is you need to use a Session hash to return the data from the template. This is because it will take time to get the response and the getLocation would expect an instant result (without a callback). At the moment client side javascript can't use synchronous api calls like on the server.
Template.profile.getLocation= function(){
return Session.get("twitterlocation");
}
Use the template created event to fire the meteor call:
Template.profile.created = function() {
Meteor.call("getTwitterLocation","BillGates", function(err,result) {
if(result && !err) {
Session.set("twitterlocation", result);
}
else
{
Session.set("twitterlocation", "Error");
}
});
});
Update:
Twitter has since updated its API to 1.1 a few modifications are required:
You now need to swap over to the 1.1 api by using 1.1 instead of 1. In addition you need to OAuth your requests. See https://dev.twitter.com/docs/auth/authorizing-request. Below contains sample data but you need to get proper keys
var authkey = "OAuth oauth_consumer_key="xvz1evFS4wEEPTGEFPHBog",
oauth_nonce="kYjzVBB8Y0ZFabxSWbWovY3uYSQ2pTgmZeNu2VS4cg",
oauth_signature="tnnArxj06cWHq44gCs1OSKk%2FjLY%3D",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp=""+(new Date().getTime()/1000).toFixed(0)+"",
oauth_token="370773112-GmHxMAgYyLbNEtIKZeRNFsMKPR9EyMZeS9weJAEb",
oauth_version="1.0"";
Be sure to remove the newlines, I've wrapped it to make it easy to read.
var result = Meteor.http.get("https://api.twitter.com/1.1/users/show.json?screen_name="+ username +"&include_entities=true",{headers:{Authorization : authkey});
If you find this a bit troublesome it might be easier to just use a package like https://github.com/Sewdn/meteor-twitter-api via meteorite to OAuth your requests for you.

Redis on Appharbor - Booksleeve GetString exception

i am trying to setup Redis on appharbor. I have followed their instructions and again i have an issue with the Booksleeve API. Here is the code i am using to make it work initially:
var connectionUri = new Uri(url);
using (var redis = new RedisConnection(connectionUri.Host, connectionUri.Port, password: connectionUri.UserInfo.Split(new[] { ':' }, 2)[1]))
{
redis.Strings.Set(1, "greeting", "welcome to remember your stuff!");
try
{
var task = redis.Strings.GetString(1, "greeting");
redis.Wait(task);
ViewBag.Message = task.Result;
}
catch (Exception)
{
// It throws an exception trying to wait for the task?
}
}
However, the issue is that it sets the string correctly, but when trying to retrieve the same string from the key value store, it throws a timeout exception waiting for the task to eexecute. However, this code works on my local redis server connection.
Am i using the API in a wrong way? or is this something related to Appharbor?
Thanks
Like a SqlConnection, you need to call Open() (otherwise your messages are queued for delivery).
Unlike SqlConnection, you should not fire up a RedisConnection each time you need it - it is intended to be used as a shared, thread-safe, multiplexer - i.e. a single connection is held somewhere and used by lots and lots of unrelated callers. Unless of course you only need to do one thing!