Not able to download a file using spring webclient but the same works with BufferedInputStream - spring-webflux

final WebClient client = WebClient.create();
Mono<byte[]> block = client.get()
.uri("urlxxx")
.header("Authorization", "xxx")
.retrieve()
.bodyToMono(byte[].class)).block()
Files.write(Paths.get("abc.tar.gz"),
Objects.requireNonNull(block.share().block()),
StandardOpenOption.CREATE);
but at this line block.share() it is giving me nullpointerexception.
I tried the same example with BufferedInputStream and can download the file. What am I missing here in the webclient code?

The better way to follow the 'non-blocking' way instead of using InputStreams. You can convert body to Flux<DataBuffer> and write data buffers to a file using DataBufferUtils
Flux<DataBuffer> dataBufferFlux = webClient.get()
.uri("http://resource.com")
.header(HttpHeaders.AUTHORIZATION, "token")
.retrieve()
.bodyToFlux(DataBuffer.class);
DataBufferUtils.write(dataBufferFlux,
Path.of("/path_to_save_file/file.txt"),
StandardOpenOption.CREATE
)
.subscribe();

Related

Spring WebClient is not emitting any data from request

This is the content type of the request
I use spring boot and this code to redirect the stream from the web server to the browser.
public Flux<String> getDataStream(String configType, String configId, String user){
Flux<String> response = null;
Optional<String> url = getLogURL(configId);
if(url.isPresent()){
logger.info("#WebClient initiation");
WebClient webClient = WebClient.create();
response = webClient.get()
.uri(url.get())
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(String.class);
}
logger.info("#WebClient response");
return response;
}
After making the request, I can see #WebClient initiation and #WebClient response in a log file.
This is the client-side JavaScript code statement, that hits the above code getDataStream()
let eventSource = new EventSource(STREAM_URL);
But it doesn't get any response from the server. I couldn't figure out what I am missing. The URL that getDataStream()->WebClient hits, confirmed to produces output.
There are several options to stream data from the server using WebFlux:
Server-sent events pushing individual events (media type: text/event-stream)
Streaming events separated by newlines (media type: application/x-ndjson)
If you need plain text content - use text/event-stream. Check WebFlux async responses from controllers for a complete example.
To see response results you need to log it as a part of the reactive flow. Also, there is no null in reactive and you need to return Flux.empty() instead.
#GetMapping(produces = TEXT_EVENT_STREAM_VALUE)
public Flux<String> getDataStream(String configType, String configId, String user){
Optional<String> url = getLogURL(configId);
if (url.isEmpty()) {
return Flux.empty();
}
log.info("#WebClient initiation");
WebClient webClient = WebClient.create();
return webClient.get()
.uri(url.get())
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(String.class)
.doOnSubscribe(s -> log.info("#WebClient request"))
.doOnNext(rec -> log.info("#WebClient response record: {}", rec));
}

How to get ClientResponse body as DataBuffer in WebClient.exchangeToMono Spring 5.3?

Before the deprecation of WebClient.exchange method I used to get ClientResponse body as Flux<DataBuffer> and manipulated it.
In Spring 5.3 the exchange() method is deprecated and I would like to change the implementation as recommended:
#deprecated since 5.3 due to the possibility to leak memory and/or
connections; please,
use {#link #exchangeToMono(Function)}, {#link #exchangeToFlux(Function)};
consider also using {#link #retrieve()} ...
Tried to do the same call in the lambda passed to exchangeToMono, but clientResponse.bodyToFlux(DataBuffer::class.java) always return an empty flux; other experiments (i.e. getting body as mono string) also could not help to get the body.
What is the standard way to get the ClientResponse body in Spring 5.3 ?
I am looking for a low-level body representation: something like "data buffer", "byte array" or "input stream"; to avoid any kind of parsing/deserialisation.
Before Spring 5.3:
webClient
.method(GET)
.uri("http://somewhere.com")
.exchange()
.flatMap { clientResponse ->
val bodyRaw: Flux<DataBuffer> = clientResponse.bodyToFlux(DataBuffer::class.java)
// ^ body as expected
// other operations
}
After Spring 5.3
webClient
.method(GET)
.uri("http://somewhere.com")
.exchangeToMono { clientResponse ->
val bodyRaw: Flux<DataBuffer> = clientResponse.bodyToFlux(DataBuffer::class.java)
// ^ always empty flux
// other operations
}
The new exchangeToMono and exchangeToFlux methods expect the body to be decoded inside the callback. Check this GitHub issue for details.
Looking at your example, probably you could use retrieve, that is safer alternative, with bodyToFlux
webClient
.method(GET)
.uri("http://somewhere.com")
.retrieve()
.bodyToFlux(DataBuffer.class)
or toEntityFlux if you need access to response details like headers and status
webClient
.method(GET)
.uri("http://somewhere.com")
.retrieve()
.toEntityFlux(DataBuffer.class)
Handling errors
Option 1. Use onErrorResume and handle WebClientResponseException
webClient
.method(GET)
.uri("http://somewhere.com")
.retrieve()
.bodyToFlux(DataBuffer.class)
.onErrorResume(WebClientResponseException.class, ex -> {
if (ex.getStatusCode().equals(HttpStatus.NOT_FOUND)) {
// ignore 404 and return empty
return Mono.empty();
}
return Mono.error(ex);
});
Option 2. Use onStatus convenience method to get access to response.
webClient
.method(GET)
.uri("http://somewhere.com")
.retrieve()
.onStatus(status -> status.equals(HttpStatus.NOT_FOUND), res -> {
// ignore 404 and return empty
return Mono.empty();
})
.bodyToFlux(DataBuffer.class)
Both methods could be used to deserialize error response Getting the response body in error case with Spring WebClient

WebClient - how to ignore a specific HTTP error

I'd like to create a Spring WebClient that ignores a specific HTTP error. From the documentation of WebClient.retrieve():
By default, 4xx and 5xx responses result in a WebClientResponseException. To customize error handling, use ResponseSpec.onStatus(Predicate, Function) handlers.
I want all calls through a WebClient instance to ignore the specific HTTP error. That is why onStatus() is of no use to me (it has to be set per response).
The best I could come up with is this:
WebClient webClient = WebClient.builder().filter((request, next) -> {
Mono<ClientResponse> response = next.exchange(request);
response = response.onErrorResume(WebClientResponseException.class, ex -> {
return ex.getRawStatusCode() == 418 ? Mono.empty() : Mono.error(ex);
});
return response;
}).build();
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
webClient.get().uri(uri).retrieve().toBodilessEntity().block();
but it does throw the exception instead of ignoring it (the lambda passed to onErrorResume() is never called).
Edited: fixed the mistake pointed out by the first answer.
After extensive debugging of spring-webflux 5.3.4 and with the help of some ideas by Martin Tarjányi, I've come to this as the only possible "solution":
WebClient webClient = WebClient.builder().filter((request, next) -> {
return next.exchange(request).flatMap(res -> {
if (res.rawStatusCode() == HttpStatus.I_AM_A_TEAPOT.value()) {
res = res.mutate().rawStatusCode(299).build();
}
return Mono.just(res);
});
}).build();
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
String body = webClient.get().uri(uri).retrieve().toEntity(String.class).block().getBody();
The background: I am migrating some code from RestTemplate to WebClient. The old code looks like this:
RestTemplate restTemplate = ...;
restTemplate.setErrorHandler(new DefaultResponseErrorHandler() {
#Override
public void handleError(ClientHttpResponse response) throws IOException {
if (response.getRawStatusCode() == HttpStatus.I_AM_A_TEAPOT.value()) {
return;
}
super.handleError(response);
}
});
URI uri = UriComponentsBuilder.fromUriString("https://httpstat.us/418").build().toUri();
String body = restTemplate.getForEntity(uri, String.class).getBody();
I believe it is a straightforward and common case.
WebClient is not yet a 100% replacement for RestTemplate.
UPDATE: Turns out this answer doesn't address the core problem of filtering out a specific status code, just addresses a general coding pattern.
The reason onErrorResume lambda is not called is that response.onErrorResume creates a brand new Mono and your code does not use the result (i.e. it's not assigned to the response variable), so in the end a Mono without the onErrorResume operator is returned.
Using Project Reactor it's usually a good practice to avoid declaring local Mono and Flux variables and use a single chain instead. This helps to avoid similar subtle bugs.
WebClient webClient = WebClient.builder()
.filter((request, next) -> next.exchange(request)
.onErrorResume(WebClientResponseException.class, ex -> ex.getRawStatusCode() == 418 ? Mono.empty() : Mono.error(ex)))
.build();

How to generate JSONP in Badgerfish format

I'm trying to create a controller for Spring MVC that will return a JSONP in Badgerfish format. My code currently creates the JSONP correctly using Jackson, but I do not know how to specify Badgerfish format. Assuming that callback is the name of the callback function and summary is my jaxb object, then my code is currently
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.writeValueAsString(new JSONPObject(callback,summary));
Is there any way to do this using Jackson or I have to use another framework? I have found an approach to generate Badgerfish using RestEasy, but only for JSON.
I actually managed to solve this with Jettison (I did not find a way to do this with Jackson). The required code is
Marshaller marshaller = null;
Writer writer = new StringWriter();
AbstractXMLStreamWriter xmlStreamWriter = new BadgerFishXMLStreamWriter(writer);
try {
marshaller = jaxbContextSummary.createMarshaller();
marshaller.marshal(myObject, xmlStreamWriter);
} catch (JAXBException e) {
logger.error("Could not construct JSONP response", e);
}

Issues performing a query to Solr via HttpClient using Solr 3.5 and HttpClient 4.2

I am trying to query my local Solr server using HttpClient and I cannot figure out why the parameters are not being added to the GET call.
My code for doing this is:
HttpRequestBase request = new HttpGet("http://localhost:8080/solr/select");
HttpParams params = new BasicHttpParams();
params.setParameter("q", query);
params.setParameter("start", String.valueOf(start));
params.setParameter("rows", String.valueOf(rows));
request.setParams(params);
HttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(request);
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
return stringToStreamConversion(is); //500 error, NullPointerException, response is empty
I have tried to return several things in hopes of seeing what I would get and trying to figure out where the problem was. I have finally realized that I was only getting back the http://localhost:8080/solr/select when I returned
return request.getURI().toURL().toString();
I cannot figure out why the parameters are not getting added. If I do
return request.getQuery();
I get nothing back...any ideas? Thanks for the help in advance!
From what I have seen you are not able to associate your paeans with the request.
So, instead of creating a new HttpParams object and associating it with request, can you try the following approach ?
httpCclient.getParams().setParameter("q", query");
....
The simpler option is to use the approach I used in HTTPPostScheduler, like this:
URL url = new URL(completeUrl);
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestMethod("POST");
conn.setRequestProperty("type", "submit");
conn.setDoOutput(true);
// Send HTTP POST
conn.connect();