Retrofit: how to parse GZIP'd response without Content-Encoding: gzip header - gzip

I'm trying to process a server response which is GZIP'd. The response comes with a header
Content-Type: application/x-gzip
but does not have header
Content-Encoding: gzip
If I add that header using a proxy, the response gets parsed just fine.
I don't have any control over the server, so I can't add the header.
Can I force Retrofit to treat it as GZIP content? Is there a better way?
The URL for the server is:
http://crowdtorch.cms.s3.amazonaws.com/4474/Updates/update-1.xml

I figured it out. The idea is to add a custom interceptor which will take the not-yet-unzipped response, and unzip it 'manually' - do the same thing that OkHttp would do automatically based on Content-Encoding header, but without requiring that header.
is like dis:
OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder()
.addInterceptor(new UnzippingInterceptor());
OkHttpClient client = clientBuilder.build();
And the Interceptor is like dis:
private class UnzippingInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
return unzip(response);
}
}
And the unzip function is like dis:
// copied from okhttp3.internal.http.HttpEngine (because is private)
private Response unzip(final Response response) throws IOException {
if (response.body() == null) {
return response;
}
GzipSource responseBody = new GzipSource(response.body().source());
Headers strippedHeaders = response.headers().newBuilder()
.removeAll("Content-Encoding")
.removeAll("Content-Length")
.build();
return response.newBuilder()
.headers(strippedHeaders)
.body(new RealResponseBody(strippedHeaders, Okio.buffer(responseBody)))
.build();
}

There is a better way than reinventing the wheel. Just add the Content-Encoding header yourself.
.addNetworkInterceptor((Interceptor.Chain chain) -> {
Request req = chain.request();
Headers.Builder headersBuilder = req.headers().newBuilder();
String credential = Credentials.basic(...);
headersBuilder.set("Authorization", credential);
Response res = chain.proceed(req.newBuilder().headers(headersBuilder.build()).build());
return res.newBuilder()
.header("Content-Encoding", "gzip")
.header("Content-Type", ""application/json")
.build();
})
In fact, your code is a classic example of the evils of using internal code (like com.sun packages from the JDK). RealResponseBody doesn't have that constructor anymore.

Related

How to send 'Origin' header in Feign Client

I am quite new in Spring Cloud Feign and trying to send HTTP header which is required by service provider. Here is the code snippet
#FeignClient(name = "authentication", url = "http://localhost:3000/api")
public interface AuthenticationService {
#PostMapping(value = "/login")
JsonNode login(#RequestHeader("Origin") String origin, #RequestBody LoginParams parameters);
}
When I try to send Origin header then server does not receive this header. But other headers like referer or x-access-token are received at server successfully.
I have also tried using RequestInterceptor and was not successful to send Origin as header.
#Component
public class HeaderInterceptor implements RequestInterceptor {
#Override
public void apply(RequestTemplate requestTemplate) {
requestTemplate.removeHeader("origin");
requestTemplate.header("origin", "http://amjad.localhost:3000/");
}
}
Any hint or help would be much appreciated.
cheers!
I had similar issue with OpenFeign. "Origin" header was blocked by defult, because it was using old Java http client.
After change to OkHttp Client, "Origin" was sent.

Unable to send modified HttpResponse back to Response.Body

I am creating a proxy using middleware in ASP.NET Core 2.1 that makes 3rd party API (OData endpoint) call to
Get data
Do some changes
Send response to Response.Body
I took a reference from here
Below is the code snippet that works fine as whatever response I am getting from API, I am sending it further
using (var responseMessage = await _httpClient.SendAsync(targetRequestMessage, HttpCompletionOption.ResponseHeadersRead, context.RequestAborted))
{
context.Response.StatusCode = (int)responseMessage.StatusCode;
CopyFromTargetResponseHeaders(context, responseMessage);
await responseMessage.Content.CopyToAsync(context.Response.Body);
}
However, If I modify the response here, for example, like this, it does not work and it shows blank page without any error.
using (var responseMessage = await _httpClient.SendAsync(targetRequestMessage, HttpCompletionOption.ResponseHeadersRead, context.RequestAborted))
{
context.Response.StatusCode = (int)responseMessage.StatusCode;
CopyFromTargetResponseHeaders(context, responseMessage);
var output = new StringContent("some sample string or may be JSON", Encoding.UTF8, "application/json");
await output.CopyToAsync(context.Response.Body);
}
It looks like we are not allowed to make any change in the response received from API call. Can anyone please tell me how can send modified content back to Response.Body?
I am able to solve the problem by updating "Content-Length" response header before rendering modified response to context.Response.Body something like this:
context.Response.Headers.Remove("Content-Length");
context.Response.Headers.Add("Content-Length", modifiedResponseStream.Length.ToString());
You might run into a System.InvalidOperationException: Response Content-Length mismatch: too few bytes written or similar exception (which you should see in the Output window). So do not use the Content-Length and maybe Content-Type headers from the response, because they probably don't match with the Content-Length and Content-Type of your modified content, e.g.:
private void CopyFromTargetResponseHeaders(HttpContext context, HttpResponseMessage responseMessage)
{
...
foreach (var header in responseMessage.Content.Headers)
{
// do not use the content headers from the response because the content will be modified
// context.Response.Headers[header.Key] = header.Value.ToArray();
}
...
}

How to remove "Server" header from the restlet/jetty response?

I use Restlet integration with Jetty in my project. I would need to remove the "Server" header from the response as it discloses server information. But since I use Restlet integration with Jetty (restlet, jetty, org.restlet.ext.jetty.jar) the HttpConfiguration object is instantiated inside Restlet and not in my code. So I am not able to set "_sendServerVersion" as false and hence not able to remove the server header from the response. How to remove the server header from the response in this case ?
The best way to create a Filter and remove the header through the Filter:
public class ServerFilter extends Filter {
public ServerFilter(Context context) {
super(context);
}
#Override
protected void afterHandle(Request request, Response response) {
response.getHeaders().set("Server", null);
super.afterHandle(request, response);
}
}
Then use it like:
ServerFilter serverFilter = new ServerFilter(getContext());
serverFilter.setNext(router);
return serverFilter;
See: https://javadocs.restlet.talend.com/2.4/jee/api/index.html for documentation

POST to REST API from my ESP8266

I've made a REST API and I'd like to do a post request to one of the endpoints from my ESP8266, but I can't manage to do so.
The code inside the loop so far:
HTTPClient http; //Declare object of class HTTPClient
http.begin("http://localhost:5000/api/users/5b1e82fb8c620238a85646fc/arduinos/5b243dc666c18a2e10eb4097/data");
http.addHeader("Content-Type", "text/plain");
http.addHeader("Authorization", "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjViMWU4MmZiOGM2MjAyMzhhODU2NDZmYyIsImlhdCI6MTUyOTEwMTc5MiwiZXhwIjoxNTI5MTE2MTkyfQ.2O6knqriuFoEW9C2JQKRlM3D0DNnzqC7e7gpidy3pWU");
http.end();
The problem is that I don't know how to set the body of the request.
It should be a json with a single key called "value". For instance:
{
"value":101
}
Anyone knows how to do it? Also it's probable that I should use the ip instead of "localhost".
Thanks in advance.
Use ArduinoJson Library here. Then you can build your HTTP body.
StaticJsonBuffer<300> JSONbuffer; //Declaring static JSON buffer
JsonObject& JSONencoder = JSONbuffer.createObject();
JSONencoder["value"] = value_var;
char JSONmessageBuffer[300];
JSONencoder.prettyPrintTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));
HTTPClient http; //Declare object of class HTTPClient
http.begin("API end point here"); //Specify request destination
http.addHeader("Content-Type", "application/json"); //Specify content-type header
int httpCode = http.POST(JSONmessageBuffer); //Send the request
String payload = http.getString(); //Get the response payload
Then use the above sample code to encapsulate JSON and send it to the API endpoint.

Retrofit 2.0 Header Interceptor vs Method Headers

There seems to be some discrepancy between using method headers and intercepting headers with OKHTTP and retrofit.
I'm looking to intercept each retrofit request going through my retrofit interface. I've written an Interceptor to go through OKHTTP like so:
OkHttpClient client = new OkHttpClient();
client.interceptors()
.add(ThisClass::onRequestIntercept);
...
private static Response onRequestIntercept(Interceptor.Chain chain) throws IOException {
Request original = chain.request();
Request request = original.newBuilder()
.header("Authorization", "auth")
.header("Accept", "json")
.method(original.method(), original.body())
.build();
return chain.proceed(request);
}
But the server throws a 401 error unless I also add the following above each and every method in the retrofit interface:
#Headers({
"Authorization: auth",
"Accept: json"
})
The logs are identical for both headers with and without the second header annotation - only that the one with the second header directly above the method goes through with 200, and if only the intercepted code is used it returns a 401 error code.
What could be the discrepancy?