I am working on an ASP.Net core application where one of the actions is responsible to upload large files. The Limits.MaxRequestBodySize property is set to 100MB in the Startup.cs for Kestrel. The action that uploads the file is already decorated with [DisableRequestSizeLimit] and [RequestFormLimits(MultipartBodyLengthLimit = int.MaxValue)].
Despite ignoring the limit at action level and setting the maximum body size to 100MB at global level, the request fails with 500: BadHttpRequestException "Request body too large" when the file I am trying to upload is only 34MB. The exception occurs in one of the middleware at "await context.Request.Body.CopyToAsync(stream)". And the exception stack trace also mentions the Content-length is 129MB. The exception does not occur if I set the Limits.MaxRequestBodySize to 200MB or to null.
Questions:
Why is the request size 129MB when I am uploading only 34MB file? what makes the remaining ~100MB?
When the request is already in context.Request.Body, why is it throwing error while copying it ("await context.Request.Body.CopyToAsync(stream)")to a new stream?
I really appreciate any help with these. Please let me know if anything is unclear, I can provide more details.
Regards,
Siva
The issue could be that the default request limit in you application or the webserver is to low. Looks like the default maxAllowedContentLength is approx. 30MB.
Perhaps these link can help you out:
https://stackoverflow.com/a/59840618/432074
https://github.com/dotnet/aspnetcore/issues/20369#issuecomment-607057822
here is the solution:
There was nothing wrong with the MaxRequestBodySize or maxAllowedContentLength. It was the size of the request that was causing the issue. Eventhough I was uploading file size of ~34MB, the file was converted to byte array and then to base64. This resulted in increased request size. I used IFormFile interface to send the file instead of byte array/base64, it is working fine now.
Related
I have an API endpoint for uploading large files, streaming then directly to DB. I use ASP.NET Core's IFormFeature to do this, calling IFormFile.OpenReadStream() to get a Stream that I pass to SqlClient for streaming.
I want to enforce a a maximum file size to avoid abuse. I know IFormFile has a Length property, but I assume that is based on Content-Length or similar and can not be trusted (please correct me if I'm wrong, but AFAIK the only way to be 100% sure about the file size is to actually read the data; the client could send an incorrect Content-Length.)
I must therefore ensure that when the stream is read, it does not read more than what is specified in IFormFile.Length (ideally it should throw if it encounters additional bytes). I have not found a way to do this. Is this possible, or is there perhaps a better way to ensure the server doesn't read enormous amounts of data from clients sending incorrect Content-Length headers?
(It should go without saying that this must not entail reading the entire file into memory.)
We have an API endpoint that allows users to upload images; one of its parameters is an IFormFileCollection.
We'd like to validate the file size to make sure that the endpoint isn't being abused so I'm checking the Length property of each IFormFile, but I don't know whether I can trust this property or not, i.e. does this come from the request? Is it considered 'input', much like Content-Length is?
If you have an IFormFileCollection parameter, and you send data using a "form-data" content-type in the request, that parameter will be bound by a whole lot of plumbing that's hard to dig through online, but if you just debug the action method that accepts the IFormFileCollection (or any collection of IFormFile, really)and inspect the collection, you'll see that the uploaded files will already have been saved on your server's disk.
That's because the entire multi-part form request's body has to be read to determine how many files there are, if any, and form parameters, and validate the request body's format while it's reading it.
So yes, by the time your code ends up there, you can trust IFormFile.Length, because it's pointing to a local file that exists and contains that many bytes.
You're too late there to reject the request though, as it's been already entirely read. You better fix rate and size limits lower in the stack, like on the web server or firewall.
Content-Length is compressed number of bytes of data in the body , it is not reliable since it may include extra data ,for example , you are sending multipart request . Just use the IFormFile.length for features like calculation or validation .
I have code that does async file uploads which works fine on my dev vm but after I deployed it to the client system, I keep getting this error:
"error writing mime multipart body part to output stream"
I know this is the line that is throwing the error but I can't seem to figure out why:
//Read the form data and return an async task.
await Request.Content.ReadAsMultipartAsync(provider);
The file size was only 1MB and I even tried different file types with much smaller sizes. Why would this occur, I need ideas
Since the error message is mentioning about an error while writing to output stream, can you check if the folder to where the response is being written out has necessary permissions for your application to write.
You can also get this error if a file with the same name already exists in the destination folder.
I had this issue but I had already set permissions on the destination folder.
I fixed the problem by setting permissions on the App_Data folder (I think this is where the file gets temporarily stored after being uploaded).
I set up an app proxy for my app deployed on a normal shared hosted server (not Heroku or anything like that). It works like a charm (as do my other apps) until I set the content type to application/liquid.
As soon as I do that I get a 411 Length Required error by nginx which is generated by my server (my guess). I tried to resolve it by setting content length to 0. It worked for a while but then it stopped. I tried other values and it works depending on its mood. Funnily, sometimes the output is truncated at content length, and at times I get the whole output (a simple page refresh can give different outputs). Also, sometimes it doesn't work AT ALL and shopify throws a "we're having tech. difficulties" error.
To summarize, content length is not reliable at all.
Now I am not sure exactly what causes a 411 error and what can I do about it. And why is it thrown only when content type is liquid. Moreover, content-length doesn't result in a consistent output (no output/predictable output/truncated output/shopify error).
Anyone knows what's up?
Perhaps your responses are using chunked transfer encoding. I don't think nginx supports this by default, so would return a 411 error in this case because chunked encoding doesn't use a Content-Length header.
If you do want to use chunked responses, there is the http://wiki.nginx.org/HttpChunkinModule module that should add support for this. Otherwise, disable the chunked encoding in your app, and make sure the Content-Length header is consistent with the length of the body of the response.
I have a project set up using ASP.NET WebAPI on top of Azure, and am having a problem whenever I try to make an HTTP Post where the content-length is too long in the header.
Normally I would've just ignored this problem, because you should be correctly setting the content-length on POST, but it turns out that when this happens, it causes the session to hang indefinitely, and then the Azure emulator crashes.
I have a custom JSON Formatter which extends MediaTypeFormatter, and I set a breakpoint on the first line of my implementation of OnReadFromStreamAsync(). However, the breakpoint is never hit because the hangup happens before ever hitting the JSON Deserializer.
I really have no idea where this hanging is occurring from because I receive no exception, just an indefinite hang and occasional Azure emulator crash.
Thank you in advance for any help or insight you might provide!
This sounds like a bug. The good thing is that you can get updated developer bits form codeplex.
There is a chance what your experiencing is related to one of these:
WebAPI: Stream uploading under webhost is not working
DevDiv 388456 -- WebHost should not use Transfer-Encoding chunked when
content length is known.
Zero ContentLength without content type header in body is throwing
If the updated bits don't fix your problem I suggest you try the standard media formatters to rule in/out your formatter. Failing that, then submit an issue.