Is there a way to ensure the HTTP request body can be loaded into memory? There are multiple middle-wares which will use same HTTP Request body to log at multiple levels.
I remember doing following in .Net Framework 4.6
await request.Content.LoadIntoBufferAsync();
/// *** Method snapshot from HttpContent - System.Net.Http library ***
/// <summary>Serialize the HTTP content to a memory buffer as an asynchronous operation.</summary>
/// <returns>The task object representing the asynchronous operation.</returns>
public Task LoadIntoBufferAsync()
{
return this.LoadIntoBufferAsync((long) int.MaxValue);
}
Can anyone help me find similar behavior in .Net Core?
EDIT
-- I think right answer here is to use EnableBuffering, but I am not to able to figure out which overloaded method should I use for EnableBuffering?
public static void EnableBuffering(this HttpRequest request)
{
BufferingHelper.EnableRewind(request);
}
public static void EnableBuffering(this HttpRequest request, int bufferThreshold)
{
BufferingHelper.EnableRewind(request, bufferThreshold);
}
public static void EnableBuffering(this HttpRequest request, long bufferLimit)
{
BufferingHelper.EnableRewind(request, bufferLimit: bufferLimit);
}
Size of HTTP Request in our application is varying from 50kb to 300mb.
For asp.net core 2.x you can use :
HttpContext.Request.EnableRewind();
For asp.net core 3.x you can use :
HttpContext.Request.EnableBuffering();
That methods ensure the request Body can be read multiple times. Normally buffers request bodies in memory :
https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.http.httprequestrewindextensions.enablebuffering?view=aspnetcore-3.1
Related
I need to process the raw request body in and MVC core controller that has route parameters
[HttpPut]
[Route("api/foo/{fooId}")]
public async Task Put(string fooId)
{
reader.Read(Request.Body).ToList();
await _store.Add("tm", "test", data);
}
but It seems like the model binder has already consumed the request stream by the time it gets to the controller.
If I remove the route parameters, I can then access the request stream (since the framework will no longer process the stream to look for parameters).
How can I specify both route parameters and be able to access Request Body without having to manually parse request URI etc.?
I have tried decorating my parameters with [FromRoute] but it had no effect.
Please note I cannot bind the request body to an object and have framework handle the binding, as I am expecting an extremely large payload that needs to be processed in chunks in a custom manner.
There are no other controller, no custom middle-ware, filters, serialzier, etc.
I do not need to process the body several times, only once
storing the stream in a temp memory or file stream is not an options, I simply want to process the request body directly.
How can I get the framework to bind paramters from Uri, QueryString, etc. but leave the request body to me?
Define this attribute in your code:
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
{
public void OnResourceExecuting(ResourceExecutingContext context)
{
var factories = context.ValueProviderFactories;
factories.RemoveType<FormValueProviderFactory>();
factories.RemoveType<JQueryFormValueProviderFactory>();
}
public void OnResourceExecuted(ResourceExecutedContext context)
{
}
}
If you're targeting .Net Core 3 you also need to add
factories.RemoveType<FormFileValueProviderFactory>();
Now decorate your action method with it:
[HttpPut]
[Route("api/foo/{fooId}")]
[DisableFormValueModelBinding]
public async Task Put(string fooId)
{
reader.Read(Request.Body).ToList();
await _store.Add("tm", "test", data);
}
The attribute works by removing Value Providers which will attempt to read the request body, leaving just those which supply values from the route or the query string.
HT #Tseng for the link Uploading large files with streaming which defines this attribute
As I suspected the root cause was MVC inspecting the request body in order to try to bind route parameters. This is how model binding works by default for any routes that are not parameter-less, as per documentation.
The framework however does this only when the request content type is not specified, or when it is form data (multipart or url-encoded I assume).
Changing my request content-type to any thing other than form data (e.g. application/json) I can get the framework to ignore the body unless specifically required (e.g. with a [FromBody] route parameter). This is an acceptable solution for my case since I am only interested accepting JSON payloads with content-type application/json.
Implementation of DisableFormValueModelBindingAttribute in Uploading large files with streaming pointed out by #Tseng seems to be a better approach however, so I will look into using that instead, for complete
I'm writing a .Net Web Api (2) that have this one POST method. This method is currently deserializing it's only parameter by using the standard JSON formatter. We are also writing the Client that will consume this Api a C# Client using System.Net.Http.HttpClient to communicate.
There is the potential to be moving a large volume of data. This made us look into reducing the footprint of the request.
After searching this site, I came across some alternatives using gzip compression. I already have a working proof of concept:
Client side something down the lines of this
Server side something down the lines of this
So, my question...
Do I really need to write all this custom code for this? Is there a built in way to accomplish lowering the footprint of the request?
Some articles that came across mention about enabling gzip (or deflate) in IIS (see Enable IIS7 gzip). This was not working for me (I enabled it, I'm still doing the compression on the Client side, removed the DelegatingHandler from the Server...but nothing, I end up with a null parameter in the controller method)
I ended up implementing a DelegatingHandler to look for a header with ContentEncoding "gzip" and decompress accordingly.
using System;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
namespace WebApi.MessageHandlers
{
/// <summary>
/// GZip message handler.
/// </summary>
public class GZipMessageHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (IsRequestCompressed(request))
{
request.Content = Descompress(request.Content);
}
return base.SendAsync(request, cancellationToken);
}
private bool IsRequestCompressed(HttpRequestMessage request)
{
return request.Content.Headers.ContentEncoding.Contains("gzip", StringComparer.OrdinalIgnoreCase);
}
private HttpContent Descompress(HttpContent content)
{
// Handle compression...
throw new NotImplementedException();
}
}
}
Is there a way to configure a rest web service to allow only one connection at a time?
I am using Wildfly 9.0.1-Final with the resteasy 3.0.11.Final implementation.
You can use synchronized block on static field:
private static final Object LOCK = new Object();
#GET
#Path("find")
#Produces(MediaType.APPLICATION_JSON)
public Response find(){
synchronized(LOCK){
//your code
}
}
I am using DevForce 2010 and Silverlight 4.
When saving entities that contain large amount of binary data, I get this error:
Unhandled Error in Silverlight Application The remote server returned an error: NotFound.
When debuging the application I see these errors:
Unhandled Error in Silverlight Application Insufficient memory to continue the execution of the program.
Bad CRC32 in GZIP stream.
I found this thread on Ideablades forum that discusses the problem: http://www.ideablade.com/forum/forum_posts.asp?TID=3361&PN=1&title=bad-crc32-in-gzip-stream
Is this a problem on the server or client?
Is this a problem that has been resolved in any new version of DevForce 2010?
My server has 4 GB memory. Would increasing the memory resolve the problem?
Or what would be the right solution?
Yes, the OnEndpointCreated overrides on both client and server are where you should add the customization. You can add the following to remove GZIP from the binding:
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint)
{
if (endpoint.Binding is CustomBinding)
{
var binding = endpoint.Binding as CustomBinding;
var elements = binding.CreateBindingElements();
// Swap out existing (GZIP) message encoding for binary
var encoding = elements.Find<MessageEncodingBindingElement>();
if (encoding != null)
{
elements.Remove(encoding);
encoding = new BinaryMessageEncodingBindingElement();
elements.Insert(0, encoding);
endpoint.Binding = new CustomBinding(elements);
}
}
}
DevForce will find your classes if they're in an assembly probed on the client/server.
This will turn off compression for everything from your DevForce client to the EntityServer, so may be a bit heavy-handed. You can turn on IIS compression to compress data sent to the client to help.
There haven't been any changes to GZIP processing since the 6.1.7 release of DevForce 2010. That thread still contains the best information of how to work around the problem: 1) modify the save logic or your entity definition to reduce the amount of data saved; 2) turn off use of GZIP; or 3) write a custom message encoder with another compression library.
Thank you Kim Johnson,
I have looked at the samples and I feel uncomfortable adding those config files and maybe breaking something that works fine today.
If I go the code-way, will I be ably to switch off GZIP and still retain the rest of the default settings for DevForce?
I guess the code below is what I should go for?
If I save these classes on the client and server, will DevForce automatically find these classes?
//Client
using System.ServiceModel.Channels;
using IdeaBlade.Core.Wcf.Extensions;
public class ProxyEvents : IdeaBlade.EntityModel.ServiceProxyEvents {
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint) {
base.OnEndpointCreated(endpoint);
// My client code turning GZIP off comes here?
}
public override void OnFactoryCreated(System.ServiceModel.ChannelFactory factory) {
base.OnFactoryCreated(factory);
}
}
//Server
using System.ServiceModel.Channels;
using IdeaBlade.Core.Wcf.Extensions;
public class ServiceEvents : IdeaBlade.EntityModel.Server.ServiceHostEvents {
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint) {
base.OnEndpointCreated(endpoint);
// My server code turning GZIP off comes here?
}
public override void OnServiceHostCreated(System.ServiceModel.ServiceHost host) {
base.OnServiceHostCreated(host);
}
}
I started developing an application in Silverlight that was dealing with downloading the HTML of a website and then parsing it. With Silverlight 4 this can be achieved easily by simply requesting elevated permissions. With Silverlight 3, however, the only way to get the HTML of a website is via a WebService call. My initial idea was to do the following:
public class Service1
{
[OperationContract]
public void GetHtml()
{
Uri targetUri = new Uri("http://www.google.com", UriKind.RelativeOrAbsolute);
WebClient webClient = new WebClient();
webClient.DownloadStringCompleted += this.WebClient_DownloadStringCompleted;
webClient.DownloadStringAsync(targetUri);
}
private void WebClient_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
}
}
However, I realized that as soon as I make the call, which is async as well, from my Silverlight application, there is no way for me to retrieve the HTML of the website. That is why I changed to the following:
public class Service1
{
[OperationContract]
public string GetHtml()
{
Uri targetUri = new Uri("http://www.google.com", UriKind.RelativeOrAbsolute);
WebClient webClient = new WebClient();
return webClient.DownloadString(targetUri);
}
}
I believe the last approach is not that fine since it will freeze the thread. So, my question, is there a way to achieve the first approach a.k.a. make async call from an async call :). Any help would be greatly appreciated.
Best Regards,
Kiril
You can achieve your goal by implementig a Duplex Service. There is some useful information about it on the msdn site and a wonderful podcast entry by Mike Taulty. In general, you would have to modify your operation contract by splitting it into two parts. First part would initiate your WebClient download on the server. Then, on the server, after the html has been downloaded, the server would call back a contract that is implemented on the client side with the payload consisting of the required html content.