.NET Core - Increase Http web api response limit - asp.net-core

I have an API that returns a large number of data in a list.
when I am calling the API from postman I can't see the response.
i seen some solutions on google but in that they are setting
maxContentLength in web.config file.
but i haven't web.config file i have project in .net core (5.0)
There is anything that I need to do to increase API response?
here in this screenshot while debugging I can see the response.
here is the code :
[HttpPost]
[Route("ReconcileUpload")]
[Consumes("multipart/form-data")]
[RequestFormLimits(MultipartBodyLengthLimit = 268435456)]
[RequestSizeLimit(52428800)]
[hasAccess("OPRT_TXN")]
public ReconcileResponse ReconcileUpload([FromForm] IFormFile file, [FromQuery] BankChallanType FileType)
{
ReconcileResponse response = new ReconcileResponse();
string storePath = "";
using (var session = db.Database.BeginTransaction())
{
try
{
//response filling
}
catch (Exception e)
{
session.Rollback();
Logger.Error($"Error in {MethodBase.GetCurrentMethod().Name} {e.Message} at {e.StackTrace}");
throw;
}
finally
{
if (!string.IsNullOrEmpty(storePath))
System.IO.File.Delete(storePath);
}
var x= response;
return x;
}
}

Related

Net Core 3.1 OData returns error 500 when using both $filter and $select and output is xml

I'm currently developing an OData API in .Net Core 3.1 which fetches data from SQL server. Using postman, I'm sending GET requests to the API with Accept headers text/xml and application/json.
With this url: <http://localhost:8008/odata/Contact?$filter=No_ eq 'T20-1234567'&$select=No_> and an application/json Accept-Header (or No Accept-Header) the response is
json response
But with Accept-Header application/xml or text/xml:
An unhandled exception was thrown by the application.
System.ArgumentException: Object of type 'System.Linq.EnumerableQuery1[Microsoft.AspNet.OData.Query.Expressions.SelectExpandBinder+SelectSome1[Models.Contact]]' cannot be converted to type 'System.Collections.Generic.IEnumerable`1[Models.Contact]'.
The strange thing is that when the $select part is removed from the url, the request is correctly handled by the application (200).
My controller action:
[HttpGet]
[ODataRoute(nameof(Contact))]
public IQueryable<Contact> GetContact()
{
return _context.Contact;
}
Has anyone seen this type of behaviour?
To answer my own question, here's a possible workaround:
services
.AddMvc(config =>
{
...
config.OutputFormatters.Add(new CustomXmlOutputFormatter());
config.RespectBrowserAcceptHeader = true;
});
public class CustomXmlOutputFormatter : TextOutputFormatter
{
public CustomXmlOutputFormatter()
{
SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse("application/xml"));
SupportedMediaTypes.Add(MediaTypeHeaderValue.Parse("text/xml"));
SupportedEncodings.Add(Encoding.UTF8);
SupportedEncodings.Add(Encoding.Unicode);
}
protected override bool CanWriteType(Type type)
{
return true;
}
public override async Task WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
{
if (context == null) throw new ArgumentNullException(nameof(context));
if (selectedEncoding == null) throw new ArgumentNullException(nameof(selectedEncoding));
var httpContext = context.HttpContext;
var json = JsonConvert.SerializeObject(new { item = context.Object });
var xml = JsonConvert.DeserializeXNode(json, "root");
var buffer = new StringBuilder(xml.ToString());
await httpContext.Response.WriteAsync(buffer.ToString());
}
}

RestRequest Body not received in .net core web api

I am trying to build a service client to simplify calling my microservices in .net core.
Here is a service client sample:
public ProductServiceClient(SystemEnvironment.MachineEnvironment? environment = null)
{
this.url = ServiceEnvironment.Urls.GetUrl(ServiceEnvironment.Service.Product, environment);
}
private RestClient GetClient(string method)
{
return new RestClient(url + "/api/" + method);
}
private RestRequest GetRestRequest(Method method)
{
var restRequest = new RestRequest(method);
restRequest.RequestFormat = DataFormat.Json;
restRequest.AddHeader("Content-Type", "application/json");
return restRequest;
}
public FindProductsResponse FindProducts(FindProductsRequest request)
{
var restRequest = GetRestRequest(Method.GET);
restRequest.AddJsonBody(request);
var client = this.GetClient("Products");
var restResponse = client.Get(restRequest);
return new JsonDeserializer().Deserialize<FindProductsResponse>(restResponse);
}
public void Dispose()
{
}
And here is how I am trying to read it in my .net core api:
[HttpGet]
public ActionResult<FindProductsResponse> Get()
{
var request = "";
using (StreamReader reader = new StreamReader(Request.Body, Encoding.UTF8))
{
request = reader.ReadToEnd();
}
var buildRequest = JsonConvert.DeserializeObject<FindProductsRequest>(request);
var products = _service.FindProducts(buildRequest);
if (products != null && products.Any())
{
return new FindProductsResponse()
{
Products = products
};
}
return BadRequest("Not found");
}
However the request variable is always empty after Request.Body has been processed by the StreamReader.
If I make the same request from Postman (also using GET), I get the body just fine.
What am I doing wrong here?
EDIT: This is the unit test calling the api:
[Test]
public void Test1()
{
using (var productServiceClient = new ProductServiceClient())
{
var products = productServiceClient.FindProducts(new FindProductsRequest()
{
Id = 50
}).Products;
}
}
It can be your Request.Body has been already consumed.
Try to call Request.EnableRewind() before to open the StreamReader.
I'm not sure why you are manually doing it. It looks like you are reinventing the wheel. ASP.NET Core already does that for you.
This is what your service should look like:
[HttpGet] // oops, GET requests will not allow Bodies, this won't work
public ActionResult<FindProductsResponse> Get([FromBody]FindProductsRequest buildRequest)
{
// skip all the serialization stuff, the framework does that for you
var products = _service.FindProducts(buildRequest);
if (products != null && products.Any())
{
return new FindProductsResponse()
{
Products = products
};
}
return BadRequest("Not found");
}
And if you don't want to redo all the busy work that is retyping all the code on the client side, I suggest you read up on swagger (probably in the form of Swashbuckle). Client code can be generated. Even from within Visual Studio, if you right-click on the project and in the context menu pick "Add REST API Client...". Please don't erroneously hand-code what can be generated flawlessly by a machine instead. I don't really know what went wrong in your specific case, but searching bugs that could be avoided altogether is just busywork, that time should be spent on other parts of the program.
I just realized this is a GET request. ASP.NET will not recognize bodies for GET-Requests. You will need to make it a PUT or POST request or put your parameters in the query string.
If you happen to make that mistake as often as I did, you might want to write some unit tests that cover this. Because .NET is not helping you there. Been there, done that..

Access Raw Request Body

I'm trying to access a request's raw input body/stream in ASP.net 5. In the past, I was able to reset the position of the input stream to 0 and read it into a memory stream but when I attempt to do this from the context the input stream is either null or throws an error (System.NotSupportedException => "Specified method is not supported.").
In the first example below I can access the raw request in a controller if I declare the controller method's parameter object type as dynamic. For various reasons, this is not a solution and I need to access the raw request body in an authentication filter anyways.
This Example Works, But Is Not a Reasonable Solution:
[HttpPost("requestme")]
public string GetRequestBody([FromBody] dynamic body)
{
return body.ToString();
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
var m = new MemoryStream();
Request.Body.CopyTo(m);
var contentLength = m.Length;
var b = System.Text.Encoding.UTF8.GetString(m.ToArray());
return b;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new StreamReader(Request.Body).ReadToEnd();
return input;
}
Throws Error:
[HttpPost("requestme")]
public string GetRequestBody()
{
Request.Body.Position = 0;
var input = new MemoryStream();
Request.Body.CopyTo(input);
var inputString = System.Text.Encoding.UTF8.GetString(input.ToArray());
return inputString;
}
I need to access the raw request body of every request that comes in for an API that I am building.
Any help or direction would be greatly appreciated!
EDIT:
Here is the code that I would like to read the request body in.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNet.Mvc;
using Microsoft.AspNet.Http;
namespace API.Filters
{
public class CustomAuthorizationAttribute : Attribute, IAuthorizationFilter
{
public CustomAuthorizationAttribute()
{ }
public void OnAuthorization(AuthorizationContext context)
{
if (context == null)
throw new ArgumentNullException("OnAuthorization AuthorizationContext context can not be null.");
else
{
if (this.AuthorizeCore(context.HttpContext) == false)
{
// Do Other Stuff To Check Auth
}
else
{
context.Result = new HttpUnauthorizedResult();
}
}
}
protected virtual bool AuthorizeCore(HttpContext httpContext)
{
var result = false;
using (System.IO.MemoryStream m = new System.IO.MemoryStream())
{
try
{
if (httpContext.Request.Body.CanSeek == true)
httpContext.Request.Body.Position = 0;
httpContext.Request.Body.CopyTo(m);
var bodyString = System.Text.Encoding.UTF8.GetString(m.ToArray());
return CheckBody(bodyString); // Initial Auth Check returns true/false <-- Not Shown In Code Here on Stack Overflow
}
catch (Exception ex)
{
Logger.WriteLine(ex.Message);
}
}
return false;
}
}
}
This code would be accessed when a call is made to a controller method marked with the CustomAuthorization attribute like so.
[Filters.CustomAuthorizationAuthorization]
[HttpPost]
public ActionResult Post([FromBody]UserModel Profile)
{
// Process Profile
}
Update
The information below is pretty outdated by now. Due to performance reasons this is not possible by default, but fortunately can be changed. The latest solution should be to enable request buffering with EnableBuffering:
Request.EnableBuffering();
See also this blog post for more information: https://devblogs.microsoft.com/aspnet/re-reading-asp-net-core-request-bodies-with-enablebuffering/.
Old, outdated answer for reference
The implementation of Request.Body depends on the controller action.
If the action contains parameters it's implemented by Microsoft.AspNet.WebUtilities.FileBufferingReadStream, which supports seeking (Request.Body.CanSeek == true). This type also supports setting the Request.Body.Position.
However, if your action contains no parameters it's implemented by Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody, which does not support seeking (Request.Body.CanSeek == false). This means you can not adjust the Position property and you can just start reading the stream.
This difference probably has to do with the fact that MVC needs to extract the parameters values from the request body, therefore it needs to read the request.
In your case, your action does not have any parameters. So the Microsoft.AspNet.Loader.IIS.FeatureModel.RequestBody is used, which throws an exception if you try to set the Position property.
**Solution**: either do not set the position or check if you actually _can_ set the position first:
if (Request.Body.CanSeek)
{
// Reset the position to zero to read from the beginning.
Request.Body.Position = 0;
}
var input = new StreamReader(Request.Body).ReadToEnd();
The exceptions you see in your three last snippets are the direct consequence of trying to read the request body multiple times - once by MVC 6 and once in your custom code - when using a streamed host like IIS or WebListener. You can see this SO question for more information: Read body twice in Asp.Net 5.
That said, I'd only expect this to happen when using application/x-www-form-urlencoded, since it wouldn't be safe for MVC to start reading the request stream with lengthy requests like file uploads. If that's not the case, then it's probably a MVC bug you should report on https://github.com/aspnet/Mvc.
For workarounds, you should take a look at this SO answer, that explains how you can use context.Request.ReadFormAsync or add manual buffering: Read body twice in Asp.Net 5
app.Use(next => async context => {
// Keep the original stream in a separate
// variable to restore it later if necessary.
var stream = context.Request.Body;
// Optimization: don't buffer the request if
// there was no stream or if it is rewindable.
if (stream == Stream.Null || stream.CanSeek) {
await next(context);
return;
}
try {
using (var buffer = new MemoryStream()) {
// Copy the request stream to the memory stream.
await stream.CopyToAsync(buffer);
// Rewind the memory stream.
buffer.Position = 0L;
// Replace the request stream by the memory stream.
context.Request.Body = buffer;
// Invoke the rest of the pipeline.
await next(context);
}
}
finally {
// Restore the original stream.
context.Request.Body = stream;
}
});
I just had this same issue. Remove the parameters from the method signature, and then read the Request.Body Stream how you want to.
You need to call Request.EnableRewind() to allow the stream to be rewound so you can read it.
string bodyAsString;
Request.EnableRewind();
using (var streamReader = new StreamReader(Request.Body, Encoding.UTF8))
{
bodyAsString = streamReader.ReadToEnd();
}
I Know this my be late but in my case its Just I had a problem in routing as bellow
At startup.cs file I was beginning the routing with /api
app.MapWhen(context => context.Request.Path.StartsWithSegments(new PathString("/api")),
a =>
{
//if (environment.IsDevelopment())
//{
// a.UseDeveloperExceptionPage();
//}
a.Use(async (context, next) =>
{
// API Call
context.Request.EnableBuffering();
await next();
});
}
//and I was putting in controller
[HttpPost]
[Route("/Register", Name = "Register")]
//Just Changed the route to start with /api like my startup.cs file
[HttpPost]
[Route("/api/Register", Name = "Register")]
//and now the params are not null and I can ready the body request multiple

Dropwizard / JerseyClient ignored JsonProperty when sending http request

I have two REST services implemented with Dropwizard-0.8.
Both share an API dependency with following POJO:
public class Report{
private String text;
#JsonProperty("t")
public String getText()
{
return text;
}
public void setText(String tx)
{
text = tx;
}
}
My Server has a rest recourse:
#POST
#Consumes(MediaType.APPLICATION_JSON + ";charset=UTF-8")
#Produces(MediaType.TEXT_PLAIN + ";charset=UTF-8")
#Timed
public Response receive(Report dto) {
//do some stuff with dto
}
My Client has a method :
sendReport(report);
with:
private void sendReport(Report report) {
final String uri = "http://localhost:8080/.....";
Response response = null;
try {
response = client.target(uri).request().post(Entity.entity(report, MediaType.APPLICATION_JSON), Response.class);
final int status = response.getStatus();
if (status != Status.ACCEPTED.getStatusCode()) {
final StatusType statusInfo = response.getStatusInfo();
throw new SomeException();
}
}
catch (Exception e) {
LOGGER.error(e.getMessage());
}
finally {
if (response != null) {
response.close();
}
}
}
The Client is made in the Dropwizard application class with:
service.client = new JerseyClientBuilder(env).using(conf.getJerseyClient()).withProvider(JacksonJaxbJsonProvider.class).build(getName());
env.jersey().register(service);
Where 'service' is my rest class calling the 'sendReport' method.
Problem
When I call the rest service of my server from a browser or with curl etc it works perfectly as expected with following messagebody:
{"t":"some text for the server"}
But when I run my application to call the rest service I get a 400 "unable to process JSON".
Debugging and the log messages showed me that the application sends the following JSON to my server:
{"text":"some text for the server"}
Which leads to the error that Jackson cant find a property "text".
Why is the JerseyClient ignoring the JsonProperty annotation?
From what I understand you using Entity.entity from jersey which has no idea about the #JsonProperty annotation(which is from jackson library) . What you need to do is do serialisation using a jackson library and give it to post call .

JavaFX: File upload to REST service / servlet fails because of missing boundary

I'm trying to upload a file using JavaFX using the HttpRequest. For this purpose I have written the following function.
function uploadFile(inputFile : File) : Void {
// check file
if (inputFile == null or not(inputFile.exists()) or inputFile.isDirectory()) {
return;
}
def httpRequest : HttpRequest = HttpRequest {
location: urlConverter.encodeURL("{serverUrl}");
source: new FileInputStream(inputFile)
method: HttpRequest.POST
headers: [
HttpHeader {
name: HttpHeader.CONTENT_TYPE
value: "multipart/form-data"
}
]
}
httpRequest.start();
}
On the server side, I am trying to handle the incoming data using the Apache Commons FileUpload API using a Jersey REST service. The code used to do this is a simple copy of the FileUpload tutorial on the Apache homepage.
#Path("Upload")
public class UploadService {
public static final String RC_OK = "OK";
public static final String RC_ERROR = "ERROR";
#POST
#Produces("text/plain")
public String handleFileUpload(#Context HttpServletRequest request) {
if (!ServletFileUpload.isMultipartContent(request)) {
return RC_ERROR;
}
FileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
List<FileItem> items = null;
try {
items = upload.parseRequest(request);
}
catch (FileUploadException e) {
e.printStackTrace();
return RC_ERROR;
}
...
}
}
However, I get a exception at items = upload.parseRequest(request);:
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
I guess I have to add a manual boundary info to the InputStream. Is there any easy solution to do this? Or are there even other solutions?
Have you tried just using the InputStream from HttpServletRequest like so
InputStream is = httpRequest.getInputStream();
BufferedInputStream in = new BufferedInputStream(is);
//Write out bytes
out.close();
is.close();