ASP.NET Core Compression Middleware - Empty Reponse - asp.net-core

I am using some custom compression middleware from this repository (pasted below). Upon the first request, the content is compressed just fine. For every request after that, the response comes back as completely empty (with a Content-Length of 0).
This only started happening after migrating from ASP.NET Core RC2 to RTM.
Does anyone know why this is happening?
CompressionMiddleware:
public class CompressionMiddleware
{
private readonly RequestDelegate _next;
public CompressionMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
if (IsGZipSupported(context))
{
string acceptEncoding = context.Request.Headers["Accept-Encoding"];
var buffer = new MemoryStream();
var stream = context.Response.Body;
context.Response.Body = buffer;
await _next(context);
if (acceptEncoding.Contains("gzip"))
{
var gstream = new GZipStream(stream, CompressionLevel.Optimal);
context.Response.Headers.Add("Content-Encoding", new[] { "gzip" });
buffer.Seek(0, SeekOrigin.Begin);
await buffer.CopyToAsync(gstream);
gstream.Dispose();
}
else
{
var gstream = new DeflateStream(stream, CompressionLevel.Optimal);
context.Response.Headers.Add("Content-Encoding", new[] { "deflate" });
buffer.Seek(0, SeekOrigin.Begin);
await buffer.CopyToAsync(gstream);
gstream.Dispose();
}
}
else
{
await _next(context);
}
}
public bool IsGZipSupported(HttpContext context)
{
string acceptEncoding = context.Request.Headers["Accept-Encoding"];
return !string.IsNullOrEmpty(acceptEncoding) &&
(acceptEncoding.Contains("gzip") || acceptEncoding.Contains("deflate"));
}
}

I have found the following in "Add HTTP compression middleware" issue:
I have added gzip and it worked, but first request. I mean in the first request, the response page is null (context.Response.Body) but when you refresh the page (just once) it works correctly after that.(I don't know why but I have to solve it)
And response on question is:
You need to update
context.Response.Headers["Content-Length"] with actual compressed
buffer length.
CompressionMiddleware.cs
And above link to realisation of compression middleware contains:
if (context.Response.Headers["Content-Length"].Count > 0)
{
context.Response.Headers["Content-Length"] = compressed.Length.ToString();
}

Related

Reading RequestBody distrupts flow in ASP.NET Core 2.2 gateway

I have a middleware to track performance of my custom developed gateway in ASP.NET Core 2.2 API. I have used the this post from StackOverflow.
Basically the main part is as follows :
public class ResponseRewindMiddleware {
private readonly RequestDelegate next;
public ResponseRewindMiddleware(RequestDelegate next) {
this.next = next;
}
public async Task Invoke(HttpContext context) {
Stream originalBody = context.Response.Body;
/* MY CODE COMES HERE */
try {
using (var memStream = new MemoryStream()) {
context.Response.Body = memStream;
await next(context);
memStream.Position = 0;
string responseBody = new StreamReader(memStream).ReadToEnd();
memStream.Position = 0;
await memStream.CopyToAsync(originalBody);
}
} finally {
context.Response.Body = originalBody;
}
}
This code runs OK. But I want to log the input (a JSON body) to the gateway and I add the following lines :
using (System.IO.StreamReader rd = new System.IO.StreamReader(context.Request.Body))
{
bodyStr = rd.ReadToEnd();
}
This reads the input body from Request but the flow is broken and the rest of the process does not flow resulting in a "HTTP 500 Internal Server Error". I assume reading the Request body via a Stream breaks something.
How can I read the Request body without breaking the flow?
The idea is to call EnableBuffering to enable multiple read, and then to not dispose the request body after you have done reading it. The following works for me.
// Enable the request body to be read in the future
context.Request.EnableBuffering();
// Read the request body, but do not dispose it
var stream = new StreamReader(context.Request.Body);
string requestBody = await stream.ReadToEndAsync();
// Reset to the origin so the next read would start from the beginning
context.Request.Body.Seek(0, SeekOrigin.Begin);

Asp.net Core 2.1 HttpClientFactory: second api call is not waiting for first api call's returned result

I encountered an issue using HttpClientFactory. I need to call two web methods from one third party web api.
getOrderNumber.
getShippingLabelFile.
Call #2 depends on #1's result since it needs to pass orderNumber to it e.g.:
await _client.getAsync("http://xxx/api/getLabel?orderNumber=[returnedOrderNumber]&fileType=1")
When I set break-point and debug, it works as expected. Without debugging mode, #2 web method always failed. I have done investigation. If I pass static query parameter like:
http://xxx/api/getLabel?orderNumber=123&fileType=1
it works fine. It seems #2 evaluates the query string and execute api call before orderNumber gives to it. It is very frustrating, can you please shed on some light on this issue?
On Controller:
private readonly ISite1AuthHttpClient _site1HttpClient;
public OrderShippingOrdersController(site1AuthHttpClient)
{
_site1HttpClient=site1AuthHttpClient
}
[HttpGet("{id}")]
public async Task<IActionResult> GetShippingLabel(int id)
{
string token=await _site1HttpClient.GetToken(username.ToString(),password);
string orderNumber=await _site1HttpClient.CreateOrder(Order,token);
if (orderNumber!=null && orderNumber!="")
{
//this API call always failed during runtime. It works on debugging mode.
var streamFile=(MemoryStream)(await _site1HttpClient.getShippingLabel(orderNumber,token));
}
}
HttpClient Type Class:
public interface ISite1HttpClient
{
Task<string> CreateOrder(AueCreateOrder order,string token);
Task<Stream> GetShippingLabel(string orderNumber,string token);
}
public class Site1HttpClient:ISite1HttpClient
{
private readonly HttpClient _client;
public Site1HttpClient(HttpClient httpClient)
{
httpClient.BaseAddress = new Uri("http://abcapi.Site1.com/");
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/plain"));
_client = httpClient;
}
public async Task<string> CreateOrder(AbcCreateOrder order,string token)
{
var jsonInString=JsonConvert.SerializeObject(order);
jsonInString="[ " + jsonInString + " ]";
_client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",token);
HttpResponseMessage response = await _client.PostAsync(
"api/AgentShipmentOrder/Create", new StringContent(jsonInString, Encoding.UTF8, "application/json"));
if (response.IsSuccessStatusCode)
{
var contents = await response.Content.ReadAsStringAsync();
AbcOrderCreateResponse abcRes = JsonConvert.DeserializeObject<AbcOrderCreateResponse>(contents);
return abcRes.Message;
}
else
{
var errorResponse = await response.Content.ReadAsStringAsync();
throw new Exception(errorResponse);
}
}
public async Task<Stream> GetShippingLabel(string orderNumber,string token)
{
_client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",token);
HttpResponseMessage response = await _client.GetAsync("api/GetOrderLabel?orderId="+orderNumber+"&fileType=1");
if (response.IsSuccessStatusCode)
{
Stream streamFile= await response.Content.ReadAsStreamAsync();
return streamFile;
}
else
{
throw new Exception("failed to get label.");
}
}
}
string token = _site1HttpClient.GetToken(username.ToString(),password);
string orderNumber = await _site1HttpClient.CreateOrder(Order,token);
I guess the problem occurs because of first await keyword. When you use await for the first function call (calling an async function), you declare that your program does not need to hold on for the response. So the token variable is used in the second function when it is not set. As you can see above, you should be good to go without the first await for the token variable.

Custom batch handler in asp.net webapi

I want to write custom batch handler in my webapi.
Requirement for this : I am not able to identify weather the incoming request is part of batch or independent.
By writing custom batch handler i will be able to add value in header of each request, which i can use later to identify.
First we need to write custom batch hahttps://stackoverflow.blog/2011/07/01/its-ok-to-ask-and-answer-your-own-questions/ndler
For this we need to override HttpMessageHandler. Below is code
public class BatchHandler : HttpMessageHandler
{
HttpMessageInvoker _server;
public BatchHandler(HttpConfiguration config)
{
// BatchServer is a class which overrides
_server = new HttpMessageInvoker(new BatchServer(config));
}
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
// Return 400 for the wrong MIME type
// As batch request will always be of MIME type
if ("multipart/mixed" !=
request.Content.Headers.ContentType.MediaType)
{
return request.CreateResponse(HttpStatusCode.BadRequest);
}
// Start a multipart response
var outerContent = new MultipartContent("batch");
var outerResp = request.CreateResponse();
outerResp.Content = outerContent;
// Read the multipart request
var multipart = await request.Content.ReadAsMultipartAsync();
foreach (var httpContent in multipart.Contents)
{
HttpResponseMessage innerResp = null;
try
{
// Decode the request object
var innerReq = await
httpContent.ReadAsHttpRequestMessageAsync();
innerReq.Headers.Add("IsBatch", "true");
// Send the request through the pipeline
innerResp = await _server.SendAsync(
innerReq,
cancellationToken
);
}
catch (Exception ex)
{
// If exceptions are thrown, send back generic 400
innerResp = new HttpResponseMessage(
HttpStatusCode.BadRequest
);
}
// Wrap the response in a message content and put it
// into the multipart response
outerContent.Add(new HttpMessageContent(innerResp));
}
return outerResp;
}
}
in above code their is this line
// BatchServer is a class which overrides HttpServer
_server = new HttpMessageInvoker(new BatchServer(config));
if we don't do this we gets an error
The 'DelegatingHandler' list is invalid because the property
'InnerHandler' of 'xxhandler' is not null.\r\nParameter
name: handlers
Below is the BatchServer class which overrides HttpServer
public class BatchServer : HttpServer
{
private readonly HttpConfiguration _config;
public BatchServer(HttpConfiguration configuration)
: base(configuration)
{
_config = configuration;
}
protected override void Initialize()
{
var firstInPipeline = _config.MessageHandlers.FirstOrDefault();
if (firstInPipeline != null && firstInPipeline.InnerHandler != null)
{
InnerHandler = firstInPipeline;
}
else
{
base.Initialize();
}
}
}
Now we want to hit batch request on BatchHandler
For this we need configure route to BatchHandler
Add below code to your AppStart
var batchHandler = new BatchHandler(config);
config.Routes.MapHttpRoute("batch", "api/batch", null, null, batchHandler);

Replacing response stream in ASP.NET Core 1.0 middleware

I want to write Custom Middleware in my ASP.NET Core 1.0 project which will replace original framework's Http Response Stream to my own, so I will be able to perform read / seek / write operations on it (first 2 are not possible on the original stream) in the further code i.e. in Actions or Filters.
I've started with the following code:
public class ReplaceStreamMiddleware
{
protected RequestDelegate NextMiddleware;
public ReplaceStreamMiddleware(RequestDelegate next)
{
NextMiddleware = next;
}
public async Task Invoke(HttpContext httpContext)
{
using (var responseStream = new MemoryStream())
{
var fullResponse = httpContext.Response.Body;
httpContext.Response.Body = responseStream;
await NextMiddleware.Invoke(httpContext);
responseStream.Seek(0, SeekOrigin.Begin);
await responseStream.CopyToAsync(fullResponse);
}
}
}
The problem with the following code is that sometimes the fullResponse stream is already closed at the time of invoking await responseStream.CopyToAsync(fullResponse); so it throws an exception Cannot access a closed Stream.
This weird behaviour is easy to observe when I load the page in the browser and then refresh, before it loads completely.
I would like to know:
why this happens?
how to prevent it?
is my solution a good idea or there is another way to replace response stream?
The exception doesn't come from your CopyToAsync. It's from one of your code's callers:
You're not restoring the original response stream in HttpContext. Therefore, whoever calls your middleware will get back a closed MemoryStream.
Here's some working code:
app.Use(async (httpContext, next) =>
{
using (var memoryResponse = new MemoryStream())
{
var originalResponse = httpContext.Response.Body;
try
{
httpContext.Response.Body = memoryResponse;
await next.Invoke();
memoryResponse.Seek(0, SeekOrigin.Begin);
await memoryResponse.CopyToAsync(originalResponse);
}
finally
{
// This is what you're missing
httpContext.Response.Body = originalResponse;
}
}
});
app.Run(async (context) =>
{
context.Response.ContentType = "text/other";
await context.Response.WriteAsync("Hello World!");
});

Disable chunking in ASP.NET Core

I'm using an ASP.NET Core Azure Web App to provide a RESTful API to a client, and the client doesn't handle chunking correctly.
Is it possible to completely turn off Transfer-Encoding: chunked, either at the controller level or in file web.config?
I'm returning a JsonResult somewhat like this:
[HttpPost]
[Produces("application/json")]
public IActionResult Post([FromBody] AuthRequest RequestData)
{
AuthResult AuthResultData = new AuthResult();
return Json(AuthResultData);
}
How to get rid of chunking in .NET Core 2.2:
The trick is to read the response body into your own MemoryStream, so you can get the length. Once you do that, you can set the content-length header, and IIS won't chunk it. I assume this would work for Azure too, but I haven't tested it.
Here's the middleware:
public class DeChunkerMiddleware
{
private readonly RequestDelegate _next;
public DeChunkerMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task InvokeAsync(HttpContext context)
{
var originalBodyStream = context.Response.Body;
using (var responseBody = new MemoryStream())
{
context.Response.Body = responseBody;
long length = 0;
context.Response.OnStarting(() =>
{
context.Response.Headers.ContentLength = length;
return Task.CompletedTask;
});
await _next(context);
// If you want to read the body, uncomment these lines.
//context.Response.Body.Seek(0, SeekOrigin.Begin);
//var body = await new StreamReader(context.Response.Body).ReadToEndAsync();
length = context.Response.Body.Length;
context.Response.Body.Seek(0, SeekOrigin.Begin);
await responseBody.CopyToAsync(originalBodyStream);
}
}
}
Then add this in Startup:
app.UseMiddleware<DeChunkerMiddleware>();
It needs to be before app.UseMvC().
In ASP.NET Core, this seems to work across hosts:
response.Headers["Content-Encoding"] = "identity";
response.Headers["Transfer-Encoding"] = "identity";
Indicates the identity function (i.e., no compression, nor
modification). This token, except if explicitly specified, is always
deemed acceptable.
Content-Encoding
Transfer-Encoding
This also works when you explicitly disable response buffering:
var bufferingFeature = httpContext.Features.Get<IHttpBufferingFeature>();
bufferingFeature?.DisableResponseBuffering();
It works in .NET Core 2.0. Just set ContentLength before writing the results into the response body stream.
In the startup class:
app.Use(async (ctx, next) =>
{
var stream = new xxxResultTranslatorStream(ctx.Response.Body);
ctx.Response.Body = stream;
await Run(ctx, next);
stream.Translate(ctx);
ctx.Response.Body = stream.Stream;
});
In xxxResultTranslatorStream:
ctx.Response.Headers.ContentLength = 40;
stream.Write(writeTargetByte, 0, writeTargetByte.Length);
I found that all my chunking problems went away if I just returned a FileStream from Get() and let ASP.NET deal with the rest.
Microsoft software tends to work best if you just give up control and trust them. It tends to work worst if you actually try to control the process.