Access Raw Request Body - asp.net-core

I'm trying to access a request's raw input body/stream in ASP.net 5. In the past, I was able to reset the position of the input stream to 0 and read it into a memory stream but when I attempt to do this from the context the input stream is either null or throws an error (System.NotSupportedException => "Specified method is not supported.").
In the first example below I can access the raw request in a controller if I declare the controller method's parameter object type as dynamic. For various reasons, this is not a solution and I need to access the raw request body in an authentication filter anyways.
This Example Works, But Is Not a Reasonable Solution:
[HttpPost("requestme")]
public string GetRequestBody([FromBody] dynamic body)
{
return body.ToString();
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
var m = new MemoryStream();
Request.Body.CopyTo(m);
var contentLength = m.Length;
var b = System.Text.Encoding.UTF8.GetString(m.ToArray());
return b;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new StreamReader(Request.Body).ReadToEnd();
return input;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new MemoryStream();
Request.Body.CopyTo(input);
var inputString = System.Text.Encoding.UTF8.GetString(input.ToArray());
return inputString;
}
I need to access the raw request body of every request that comes in for an API that I am building.
Any help or direction would be greatly appreciated!
EDIT:
Here is the code that I would like to read the request body in.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNet.Mvc;
using Microsoft.AspNet.Http;
namespace API.Filters
{
public class CustomAuthorizationAttribute : Attribute, IAuthorizationFilter
{
public CustomAuthorizationAttribute()
{ }
public void OnAuthorization(AuthorizationContext context)
{
if (context == null)
throw new ArgumentNullException("OnAuthorization AuthorizationContext context can not be null.");
else
{
if (this.AuthorizeCore(context.HttpContext) == false)
{
// Do Other Stuff To Check Auth
}
else
{
context.Result = new HttpUnauthorizedResult();
}
}
}
protected virtual bool AuthorizeCore(HttpContext httpContext)
{
var result = false;
using (System.IO.MemoryStream m = new System.IO.MemoryStream())
{
try
{
if (httpContext.Request.Body.CanSeek == true)
httpContext.Request.Body.Position = 0;
httpContext.Request.Body.CopyTo(m);
var bodyString = System.Text.Encoding.UTF8.GetString(m.ToArray());
return CheckBody(bodyString); // Initial Auth Check returns true/false <-- Not Shown In Code Here on Stack Overflow
}
catch (Exception ex)
{
Logger.WriteLine(ex.Message);
}
}
return false;
}
}
}
This code would be accessed when a call is made to a controller method marked with the CustomAuthorization attribute like so.
[Filters.CustomAuthorizationAuthorization]
[HttpPost]
public ActionResult Post([FromBody]UserModel Profile)
{
// Process Profile
}

Update
The information below is pretty outdated by now. Due to performance reasons this is not possible by default, but fortunately can be changed. The latest solution should be to enable request buffering with EnableBuffering:
Request.EnableBuffering();
See also this blog post for more information: https://devblogs.microsoft.com/aspnet/re-reading-asp-net-core-request-bodies-with-enablebuffering/.
Old, outdated answer for reference
The implementation of Request.Body depends on the controller action.
If the action contains parameters it's implemented by Microsoft.AspNet.WebUtilities.FileBufferingReadStream, which supports seeking (Request.Body.CanSeek == true). This type also supports setting the Request.Body.Position.
However, if your action contains no parameters it's implemented by Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody, which does not support seeking (Request.Body.CanSeek == false). This means you can not adjust the Position property and you can just start reading the stream.
This difference probably has to do with the fact that MVC needs to extract the parameters values from the request body, therefore it needs to read the request.
In your case, your action does not have any parameters. So the Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody is used, which throws an exception if you try to set the Position property.
**Solution**: either do not set the position or check if you actually _can_ set the position first:
if (Request.Body.CanSeek)
{
// Reset the position to zero to read from the beginning.
Request.Body.Position = 0;
}
var input = new StreamReader(Request.Body).ReadToEnd();

The exceptions you see in your three last snippets are the direct consequence of trying to read the request body multiple times - once by MVC 6 and once in your custom code - when using a streamed host like IIS or WebListener. You can see this SO question for more information: Read body twice in Asp.Net 5.
That said, I'd only expect this to happen when using application/x-www-form-urlencoded, since it wouldn't be safe for MVC to start reading the request stream with lengthy requests like file uploads. If that's not the case, then it's probably a MVC bug you should report on https://github.com/aspnet/Mvc.
For workarounds, you should take a look at this SO answer, that explains how you can use context.Request.ReadFormAsync or add manual buffering: Read body twice in Asp.Net 5
app.Use(next => async context => {
// Keep the original stream in a separate
// variable to restore it later if necessary.
var stream = context.Request.Body;
// Optimization: don't buffer the request if
// there was no stream or if it is rewindable.
if (stream == Stream.Null || stream.CanSeek) {
await next(context);
return;
}
try {
using (var buffer = new MemoryStream()) {
// Copy the request stream to the memory stream.
await stream.CopyToAsync(buffer);
// Rewind the memory stream.
buffer.Position = 0L;
// Replace the request stream by the memory stream.
context.Request.Body = buffer;
// Invoke the rest of the pipeline.
await next(context);
}
}
finally {
// Restore the original stream.
context.Request.Body = stream;
}
});

I just had this same issue. Remove the parameters from the method signature, and then read the Request.Body Stream how you want to.

You need to call Request.EnableRewind() to allow the stream to be rewound so you can read it.
string bodyAsString;
Request.EnableRewind();
using (var streamReader = new StreamReader(Request.Body, Encoding.UTF8))
{
bodyAsString = streamReader.ReadToEnd();
}

I Know this my be late but in my case its Just I had a problem in routing as bellow
At startup.cs file I was beginning the routing with /api
app.MapWhen(context => context.Request.Path.StartsWithSegments(new PathString("/api")),
a =>
{
//if (environment.IsDevelopment())
//{
// a.UseDeveloperExceptionPage();
//}
a.Use(async (context, next) =>
{
// API Call
context.Request.EnableBuffering();
await next();
});
}
//and I was putting in controller
[HttpPost]
[Route("/Register", Name = "Register")]
//Just Changed the route to start with /api like my startup.cs file
[HttpPost]
[Route("/api/Register", Name = "Register")]
//and now the params are not null and I can ready the body request multiple

Related

Reading RequestBody distrupts flow in ASP.NET Core 2.2 gateway

I have a middleware to track performance of my custom developed gateway in ASP.NET Core 2.2 API. I have used the this post from StackOverflow.
Basically the main part is as follows :
public class ResponseRewindMiddleware {
private readonly RequestDelegate next;
public ResponseRewindMiddleware(RequestDelegate next) {
this.next = next;
}
public async Task Invoke(HttpContext context) {
Stream originalBody = context.Response.Body;
/* MY CODE COMES HERE */
try {
using (var memStream = new MemoryStream()) {
context.Response.Body = memStream;
await next(context);
memStream.Position = 0;
string responseBody = new StreamReader(memStream).ReadToEnd();
memStream.Position = 0;
await memStream.CopyToAsync(originalBody);
}
} finally {
context.Response.Body = originalBody;
}
}
This code runs OK. But I want to log the input (a JSON body) to the gateway and I add the following lines :
using (System.IO.StreamReader rd = new System.IO.StreamReader(context.Request.Body))
{
bodyStr = rd.ReadToEnd();
}
This reads the input body from Request but the flow is broken and the rest of the process does not flow resulting in a "HTTP 500 Internal Server Error". I assume reading the Request body via a Stream breaks something.
How can I read the Request body without breaking the flow?
The idea is to call EnableBuffering to enable multiple read, and then to not dispose the request body after you have done reading it. The following works for me.
// Enable the request body to be read in the future
context.Request.EnableBuffering();
// Read the request body, but do not dispose it
var stream = new StreamReader(context.Request.Body);
string requestBody = await stream.ReadToEndAsync();
// Reset to the origin so the next read would start from the beginning
context.Request.Body.Seek(0, SeekOrigin.Begin);

How to directly set response body to a file stream in ASP.NET Core middleware?

Sample code below to write a file stream to Response.Body in an ASP.NET Core middleware doesn't work (emits empty response):
public Task Invoke(HttpContext context)
{
context.Response.ContentType = "text/plain";
using (var fs = new FileStream("/valid-path-to-file-on-server.txt", FileMode.Open)
using (var sr = new StreamReader(fs))
{
context.Response.Body = sr.BaseStream;
}
return Task.CompletedTask;
}
Any ideas what could be wrong with this approach of directly setting the context.Response.Body?
Note: any next middleware in the pipeline is skipped for no further processing.
Update (another example): a simple MemoryStream assignment doesn't work either (empty response):
context.Response.Body = new MemoryStream(Encoding.UTF8.GetBytes(DateTime.Now.ToString()));
No. You can never do that directly.
Note that context.Response.Body is a reference to an object (HttpResponseStream) that is initialized before it becomes available in HttpContext. It is assumed that all bytes are written into this original Stream. If you change the Body to reference (point to) a new stream object by context.Response.Body = a_new_Stream, the original Stream is not changed at all.
Also, if you look into the source code of ASP.NET Core, you'll find the Team always copy the wrapper stream to the original body stream at the end rather than with a simple replacement(unless they're unit-testing with a mocked stream). For example, the SPA Prerendering middleware source code:
finally
{
context.Response.Body = originalResponseStream;
...
And the ResponseCachingMiddleware source code:
public async Task Invoke(HttpContext httpContext)
{
...
finally
{
UnshimResponseStream(context);
}
...
}
internal static void UnshimResponseStream(ResponseCachingContext context)
{
// Unshim response stream
context.HttpContext.Response.Body = context.OriginalResponseStream;
// Remove IResponseCachingFeature
RemoveResponseCachingFeature(context.HttpContext);
}
As a walkaround, you can copy the bytes to the raw stream as below:
public async Task Invoke(HttpContext context)
{
context.Response.ContentType = "text/plain";
using (var fs = new FileStream("valid-path-to-file-on-server.txt", FileMode.Open))
{
await fs.CopyToAsync(context.Response.Body);
}
}
Or if you like to hijack the raw HttpResponseStream with your own stream wrapper:
var originalBody = HttpContext.Response.Body;
var ms = new MemoryStream();
HttpContext.Response.Body = ms;
try
{
await next();
HttpContext.Response.Body = originalBody;
ms.Seek(0, SeekOrigin.Begin);
await ms.CopyToAsync(HttpContext.Response.Body);
}
finally
{
response.Body = originalBody;
}
The using statements in the question causes your stream and stream reader to be rather ephemeral, so they will both be disposed. The extra reference to the steam in "body" wont prevent the dispose.
The framework disposes of the stream after sending the response. (The medium is the message).
In net 6 I found I was getting console errors when I tried to do this e.g.:
System.InvalidOperationException: Response Content-Length mismatch: too many bytes written (25247 of 8863).
The solution was to remove the relevant header:
context.Response.Headers.Remove("Content-Length");
await context.Response.SendFileAsync(filename);

RestRequest Body not received in .net core web api

I am trying to build a service client to simplify calling my microservices in .net core.
Here is a service client sample:
public ProductServiceClient(SystemEnvironment.MachineEnvironment? environment = null)
{
this.url = ServiceEnvironment.Urls.GetUrl(ServiceEnvironment.Service.Product, environment);
}
private RestClient GetClient(string method)
{
return new RestClient(url + "/api/" + method);
}
private RestRequest GetRestRequest(Method method)
{
var restRequest = new RestRequest(method);
restRequest.RequestFormat = DataFormat.Json;
restRequest.AddHeader("Content-Type", "application/json");
return restRequest;
}
public FindProductsResponse FindProducts(FindProductsRequest request)
{
var restRequest = GetRestRequest(Method.GET);
restRequest.AddJsonBody(request);
var client = this.GetClient("Products");
var restResponse = client.Get(restRequest);
return new JsonDeserializer().Deserialize<FindProductsResponse>(restResponse);
}
public void Dispose()
{
}
And here is how I am trying to read it in my .net core api:
[HttpGet]
public ActionResult<FindProductsResponse> Get()
{
var request = "";
using (StreamReader reader = new StreamReader(Request.Body, Encoding.UTF8))
{
request = reader.ReadToEnd();
}
var buildRequest = JsonConvert.DeserializeObject<FindProductsRequest>(request);
var products = _service.FindProducts(buildRequest);
if (products != null && products.Any())
{
return new FindProductsResponse()
{
Products = products
};
}
return BadRequest("Not found");
}
However the request variable is always empty after Request.Body has been processed by the StreamReader.
If I make the same request from Postman (also using GET), I get the body just fine.
What am I doing wrong here?
EDIT: This is the unit test calling the api:
[Test]
public void Test1()
{
using (var productServiceClient = new ProductServiceClient())
{
var products = productServiceClient.FindProducts(new FindProductsRequest()
{
Id = 50
}).Products;
}
}
It can be your Request.Body has been already consumed.
Try to call Request.EnableRewind() before to open the StreamReader.
I'm not sure why you are manually doing it. It looks like you are reinventing the wheel. ASP.NET Core already does that for you.
This is what your service should look like:
[HttpGet] // oops, GET requests will not allow Bodies, this won't work
public ActionResult<FindProductsResponse> Get([FromBody]FindProductsRequest buildRequest)
{
// skip all the serialization stuff, the framework does that for you
var products = _service.FindProducts(buildRequest);
if (products != null && products.Any())
{
return new FindProductsResponse()
{
Products = products
};
}
return BadRequest("Not found");
}
And if you don't want to redo all the busy work that is retyping all the code on the client side, I suggest you read up on swagger (probably in the form of Swashbuckle). Client code can be generated. Even from within Visual Studio, if you right-click on the project and in the context menu pick "Add REST API Client...". Please don't erroneously hand-code what can be generated flawlessly by a machine instead. I don't really know what went wrong in your specific case, but searching bugs that could be avoided altogether is just busywork, that time should be spent on other parts of the program.
I just realized this is a GET request. ASP.NET will not recognize bodies for GET-Requests. You will need to make it a PUT or POST request or put your parameters in the query string.
If you happen to make that mistake as often as I did, you might want to write some unit tests that cover this. Because .NET is not helping you there. Been there, done that..

Modify middleware response

My requirement: write a middleware that filters all "bad words" out of a response that comes from another subsequent middleware (e.g. Mvc).
The problem: streaming of the response. So when we come back to our FilterBadWordsMiddleware from a subsequent middleware, which already wrote to the response, we are too late to the party... because response started already sending, which yields to the wellknown error response has already started...
So since this is a requirement in many various situations -- how to deal with it?
Replace a response stream to MemoryStream to prevent its sending. Return the original stream after the response is modified:
public async Task Invoke(HttpContext context)
{
bool modifyResponse = true;
Stream originBody = null;
if (modifyResponse)
{
//uncomment this line only if you need to read context.Request.Body stream
//context.Request.EnableRewind();
originBody = ReplaceBody(context.Response);
}
await _next(context);
if (modifyResponse)
{
//as we replaced the Response.Body with a MemoryStream instance before,
//here we can read/write Response.Body
//containing the data written by middlewares down the pipeline
//finally, write modified data to originBody and set it back as Response.Body value
ReturnBody(context.Response, originBody);
}
}
private Stream ReplaceBody(HttpResponse response)
{
var originBody = response.Body;
response.Body = new MemoryStream();
return originBody;
}
private void ReturnBody(HttpResponse response, Stream originBody)
{
response.Body.Seek(0, SeekOrigin.Begin);
response.Body.CopyTo(originBody);
response.Body = originBody;
}
It's a workaround and it can cause performance problems. I hope to see a better solution here.
A simpler version based on the code I used:
/// <summary>
/// The middleware Invoke method.
/// </summary>
/// <param name="httpContext">The current <see cref="HttpContext"/>.</param>
/// <returns>A Task to support async calls.</returns>
public async Task Invoke(HttpContext httpContext)
{
var originBody = httpContext.Response.Body;
try
{
var memStream = new MemoryStream();
httpContext.Response.Body = memStream;
await _next(httpContext).ConfigureAwait(false);
memStream.Position = 0;
var responseBody = new StreamReader(memStream).ReadToEnd();
//Custom logic to modify response
responseBody = responseBody.Replace("hello", "hi", StringComparison.InvariantCultureIgnoreCase);
var memoryStreamModified = new MemoryStream();
var sw = new StreamWriter(memoryStreamModified);
sw.Write(responseBody);
sw.Flush();
memoryStreamModified.Position = 0;
await memoryStreamModified.CopyToAsync(originBody).ConfigureAwait(false);
}
finally
{
httpContext.Response.Body = originBody;
}
}
Unfortunately I'm not allowed to comment since my score is too low.
So just wanted to post my extension of the excellent top solution, and a modification for .NET Core 3.0+
First of all
context.Request.EnableRewind();
has been changed to
context.Request.EnableBuffering();
in .NET Core 3.0+
And here's how I read/write the body content:
First a filter, so we just modify the content types we're interested in
private static readonly IEnumerable<string> validContentTypes = new HashSet<string>() { "text/html", "application/json", "application/javascript" };
It's a solution for transforming nuggeted texts like [[[Translate me]]] into its translation. This way I can just mark up everything that needs to be translated, read the po-file we've gotten from the translator, and then do the translation replacement in the output stream - regardless if the nuggeted texts is in a razor view, javascript or ... whatever.
Kind of like the TurquoiseOwl i18n package does, but in .NET Core, which that excellent package unfortunately doesn't support.
...
if (modifyResponse)
{
//as we replaced the Response.Body with a MemoryStream instance before,
//here we can read/write Response.Body
//containing the data written by middlewares down the pipeline
var contentType = context.Response.ContentType?.ToLower();
contentType = contentType?.Split(';', StringSplitOptions.RemoveEmptyEntries).FirstOrDefault(); // Filter out text/html from "text/html; charset=utf-8"
if (validContentTypes.Contains(contentType))
{
using (var streamReader = new StreamReader(context.Response.Body))
{
// Read the body
context.Response.Body.Seek(0, SeekOrigin.Begin);
var responseBody = await streamReader.ReadToEndAsync();
// Replace [[[Bananas]]] with translated texts - or Bananas if a translation is missing
responseBody = NuggetReplacer.ReplaceNuggets(poCatalog, responseBody);
// Create a new stream with the modified body, and reset the content length to match the new stream
var requestContent = new StringContent(responseBody, Encoding.UTF8, contentType);
context.Response.Body = await requestContent.ReadAsStreamAsync();//modified stream
context.Response.ContentLength = context.Response.Body.Length;
}
}
//finally, write modified data to originBody and set it back as Response.Body value
await ReturnBody(context.Response, originBody);
}
...
private Task ReturnBody(HttpResponse response, Stream originBody)
{
response.Body.Seek(0, SeekOrigin.Begin);
await response.Body.CopyToAsync(originBody);
response.Body = originBody;
}
A "real" production scenario may be found here: tethys logging middeware
If you follow the logic presented in the link, do not forget to addhttpContext.Request.EnableRewind() prior calling _next(httpContext) (extension method of Microsoft.AspNetCore.Http.Internal namespace).

Disable chunking in ASP.NET Core

I'm using an ASP.NET Core Azure Web App to provide a RESTful API to a client, and the client doesn't handle chunking correctly.
Is it possible to completely turn off Transfer-Encoding: chunked, either at the controller level or in file web.config?
I'm returning a JsonResult somewhat like this:
[HttpPost]
[Produces("application/json")]
public IActionResult Post([FromBody] AuthRequest RequestData)
{
AuthResult AuthResultData = new AuthResult();
return Json(AuthResultData);
}
How to get rid of chunking in .NET Core 2.2:
The trick is to read the response body into your own MemoryStream, so you can get the length. Once you do that, you can set the content-length header, and IIS won't chunk it. I assume this would work for Azure too, but I haven't tested it.
Here's the middleware:
public class DeChunkerMiddleware
{
private readonly RequestDelegate _next;
public DeChunkerMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task InvokeAsync(HttpContext context)
{
var originalBodyStream = context.Response.Body;
using (var responseBody = new MemoryStream())
{
context.Response.Body = responseBody;
long length = 0;
context.Response.OnStarting(() =>
{
context.Response.Headers.ContentLength = length;
return Task.CompletedTask;
});
await _next(context);
// If you want to read the body, uncomment these lines.
//context.Response.Body.Seek(0, SeekOrigin.Begin);
//var body = await new StreamReader(context.Response.Body).ReadToEndAsync();
length = context.Response.Body.Length;
context.Response.Body.Seek(0, SeekOrigin.Begin);
await responseBody.CopyToAsync(originalBodyStream);
}
}
}
Then add this in Startup:
app.UseMiddleware<DeChunkerMiddleware>();
It needs to be before app.UseMvC().
In ASP.NET Core, this seems to work across hosts:
response.Headers["Content-Encoding"] = "identity";
response.Headers["Transfer-Encoding"] = "identity";
Indicates the identity function (i.e., no compression, nor
modification). This token, except if explicitly specified, is always
deemed acceptable.
Content-Encoding
Transfer-Encoding
This also works when you explicitly disable response buffering:
var bufferingFeature = httpContext.Features.Get<IHttpBufferingFeature>();
bufferingFeature?.DisableResponseBuffering();
It works in .NET Core 2.0. Just set ContentLength before writing the results into the response body stream.
In the startup class:
app.Use(async (ctx, next) =>
{
var stream = new xxxResultTranslatorStream(ctx.Response.Body);
ctx.Response.Body = stream;
await Run(ctx, next);
stream.Translate(ctx);
ctx.Response.Body = stream.Stream;
});
In xxxResultTranslatorStream:
ctx.Response.Headers.ContentLength = 40;
stream.Write(writeTargetByte, 0, writeTargetByte.Length);
I found that all my chunking problems went away if I just returned a FileStream from Get() and let ASP.NET deal with the rest.
Microsoft software tends to work best if you just give up control and trust them. It tends to work worst if you actually try to control the process.