.Net Web API 2 POST consuming gzip compressed content - asp.net-web-api2

I'm writing a .Net Web Api (2) that have this one POST method. This method is currently deserializing it's only parameter by using the standard JSON formatter. We are also writing the Client that will consume this Api a C# Client using System.Net.Http.HttpClient to communicate.
There is the potential to be moving a large volume of data. This made us look into reducing the footprint of the request.
After searching this site, I came across some alternatives using gzip compression. I already have a working proof of concept:
Client side something down the lines of this
Server side something down the lines of this
So, my question...
Do I really need to write all this custom code for this? Is there a built in way to accomplish lowering the footprint of the request?
Some articles that came across mention about enabling gzip (or deflate) in IIS (see Enable IIS7 gzip). This was not working for me (I enabled it, I'm still doing the compression on the Client side, removed the DelegatingHandler from the Server...but nothing, I end up with a null parameter in the controller method)

I ended up implementing a DelegatingHandler to look for a header with ContentEncoding "gzip" and decompress accordingly.
using System;
using System.Linq;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
namespace WebApi.MessageHandlers
{
/// <summary>
/// GZip message handler.
/// </summary>
public class GZipMessageHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (IsRequestCompressed(request))
{
request.Content = Descompress(request.Content);
}
return base.SendAsync(request, cancellationToken);
}
private bool IsRequestCompressed(HttpRequestMessage request)
{
return request.Content.Headers.ContentEncoding.Contains("gzip", StringComparer.OrdinalIgnoreCase);
}
private HttpContent Descompress(HttpContent content)
{
// Handle compression...
throw new NotImplementedException();
}
}
}

Related

Third party IIS Module not printing to log files

I have an asp.net core application hosted and webdav enabled in IIS for storing files as well. I want to know when a user saves a file in the webdav server (ie, if opened and saved in Microsoft word). I have put together this IIS module to try and intercept all http requests so I can find out what a webdav HTTP requests look like and then hopefully create a if statement using this logic once I can identify the save request. So far here is the IIS module I have created:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Web;
namespace WebDAVTargetDetectorIIS
{
public class LoggingModule : IHttpModule
{
#region IHttpModule Members
public void Dispose()
{
}
public void Init(HttpApplication context)
{
context.PreRequestHandlerExecute += new EventHandler(OnPreRequestHandlerExecute);
context.PostLogRequest += new EventHandler(PostLogEvent);
}
#endregion
public void OnPreRequestHandlerExecute(Object source, EventArgs e)
{
HttpApplication app = (HttpApplication)source;
HttpRequest request = app.Context.Request;
if (!String.IsNullOrEmpty(request["HTTP_CF_CONNECTING_IP"]))
{
request.ServerVariables.Set("REMOTE_ADDR", request["HTTP_CF_CONNECTING_IP"]);
}
}
public void PostLogEvent(Object source, EventArgs e)
{
HttpApplication app = (HttpApplication)source;
HttpRequest request = app.Context.Request;
app.Response.AppendToLog("This is a test");
}
}
}
I have created the BIN folder in the root folder of my web application and used the IIS Manager to add the new module. It is added to the end of the ordered list. The log files in IIS seem to save to "C:\inetpub\logs\LogFiles\W3SVC1" however my module does not seem to be printing my test line in there. Is there a particular reason the code above would not implement on any even a login request? Or should I be attaching this to some other event?

Is there a way to ensure the HTTP request body can be loaded into memory for possible multiple read requests?

Is there a way to ensure the HTTP request body can be loaded into memory? There are multiple middle-wares which will use same HTTP Request body to log at multiple levels.
I remember doing following in .Net Framework 4.6
await request.Content.LoadIntoBufferAsync();
/// *** Method snapshot from HttpContent - System.Net.Http library ***
/// <summary>Serialize the HTTP content to a memory buffer as an asynchronous operation.</summary>
/// <returns>The task object representing the asynchronous operation.</returns>
public Task LoadIntoBufferAsync()
{
return this.LoadIntoBufferAsync((long) int.MaxValue);
}
Can anyone help me find similar behavior in .Net Core?
EDIT
-- I think right answer here is to use EnableBuffering, but I am not to able to figure out which overloaded method should I use for EnableBuffering?
public static void EnableBuffering(this HttpRequest request)
{
BufferingHelper.EnableRewind(request);
}
public static void EnableBuffering(this HttpRequest request, int bufferThreshold)
{
BufferingHelper.EnableRewind(request, bufferThreshold);
}
public static void EnableBuffering(this HttpRequest request, long bufferLimit)
{
BufferingHelper.EnableRewind(request, bufferLimit: bufferLimit);
}
Size of HTTP Request in our application is varying from 50kb to 300mb.
For asp.net core 2.x you can use :
HttpContext.Request.EnableRewind();
For asp.net core 3.x you can use :
HttpContext.Request.EnableBuffering();
That methods ensure the request Body can be read multiple times. Normally buffers request bodies in memory :
https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.http.httprequestrewindextensions.enablebuffering?view=aspnetcore-3.1

Trying to implement a Breeze controller in asp.net mvc

I am trying to implement a simple Breeze controller in Asp.Net MVC4, but can't seem to access it. Is it possibly conflicting with .Net's standard Web.Api ?
If my url is http://localhost:49479/api/values then I get a good return value from Web Api.
However if my url is http://localhost:49479/breeze/Breeze I get "Http 404" error "Resource not found".
If my url is http://localhost:49479/breeze/Breeze/5 I get error No HTTP resource was found that matches the request URI 'http://localhost:49479/breeze/Breeze/5'.
Your advice is greatly appreciate.
Here's what I have in ..Controllers/BreezeController.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Web.Http;
using Breeze.ContextProvider;
using Breeze.WebApi2;
using Newtonsoft.Json;
namespace RageSys.Controllers
{
[BreezeController]
public class BreeezeController : ApiController
{
// GET api/values
public string Get(int id)
{
return "value";
}
public IEnumerable<string> GetMtm(int id)
{
return new string[] { "value1", "value2" };
}
}
}
and in BreezeWebApiConfig.cs :
using System.Web.Http;
[assembly: WebActivator.PreApplicationStartMethod(
typeof(RageSys.App_Start.BreezeWebApiConfig), "RegisterBreezePreStart")]
namespace RageSys.App_Start {
///<summary>
/// Inserts the Breeze Web API controller route at the front of all Web API routes
///</summary>
public static class BreezeWebApiConfig {
public static void RegisterBreezePreStart() {
GlobalConfiguration.Configuration.Routes.MapHttpRoute(
name: "BreezeApi",
routeTemplate: "breeze/{controller}/{action}"
);
}
}
}
The result you are getting from your api/values request is not coming from the listed controller. You must have the default ValuesController and WebApiConfig (which defines a route that takes a parameter) still in your project.
You do not have a route for http://localhost:49479/breeze/Breeze/5. The third segment (currently 5) needs to be the name of an Action method. For you, that means GetMtm. You do not have a route that takes any parameters, so you'll get nothing from: http://localhost:49479/breeze/Breeze/GetMtm/5 unless you define such a route. You probably don't want to do this though, because Breeze coupled with Entity Framework will make life very easy. You should implement the simplest possible Breeze / Entity Framework application and see how it works from there.
If you are using parameters and using Breeze, then ensure you use the .withParameters({ ParameterName: "Fred"}) or .withParameters({ id: id-value}), for example, in your Breeze query and ensure the parameter name in your function to be called (GetMtm in your case) at the server matches the parameter name you are using at the client.

Bad CRC32 in GZIP stream

I am using DevForce 2010 and Silverlight 4.
When saving entities that contain large amount of binary data, I get this error:
Unhandled Error in Silverlight Application The remote server returned an error: NotFound.
When debuging the application I see these errors:
Unhandled Error in Silverlight Application Insufficient memory to continue the execution of the program.
Bad CRC32 in GZIP stream.
I found this thread on Ideablades forum that discusses the problem: http://www.ideablade.com/forum/forum_posts.asp?TID=3361&PN=1&title=bad-crc32-in-gzip-stream
Is this a problem on the server or client?
Is this a problem that has been resolved in any new version of DevForce 2010?
My server has 4 GB memory. Would increasing the memory resolve the problem?
Or what would be the right solution?
Yes, the OnEndpointCreated overrides on both client and server are where you should add the customization. You can add the following to remove GZIP from the binding:
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint)
{
if (endpoint.Binding is CustomBinding)
{
var binding = endpoint.Binding as CustomBinding;
var elements = binding.CreateBindingElements();
// Swap out existing (GZIP) message encoding for binary
var encoding = elements.Find<MessageEncodingBindingElement>();
if (encoding != null)
{
elements.Remove(encoding);
encoding = new BinaryMessageEncodingBindingElement();
elements.Insert(0, encoding);
endpoint.Binding = new CustomBinding(elements);
}
}
}
DevForce will find your classes if they're in an assembly probed on the client/server.
This will turn off compression for everything from your DevForce client to the EntityServer, so may be a bit heavy-handed. You can turn on IIS compression to compress data sent to the client to help.
There haven't been any changes to GZIP processing since the 6.1.7 release of DevForce 2010. That thread still contains the best information of how to work around the problem: 1) modify the save logic or your entity definition to reduce the amount of data saved; 2) turn off use of GZIP; or 3) write a custom message encoder with another compression library.
Thank you Kim Johnson,
I have looked at the samples and I feel uncomfortable adding those config files and maybe breaking something that works fine today.
If I go the code-way, will I be ably to switch off GZIP and still retain the rest of the default settings for DevForce?
I guess the code below is what I should go for?
If I save these classes on the client and server, will DevForce automatically find these classes?
//Client
using System.ServiceModel.Channels;
using IdeaBlade.Core.Wcf.Extensions;
public class ProxyEvents : IdeaBlade.EntityModel.ServiceProxyEvents {
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint) {
base.OnEndpointCreated(endpoint);
// My client code turning GZIP off comes here?
}
public override void OnFactoryCreated(System.ServiceModel.ChannelFactory factory) {
base.OnFactoryCreated(factory);
}
}
//Server
using System.ServiceModel.Channels;
using IdeaBlade.Core.Wcf.Extensions;
public class ServiceEvents : IdeaBlade.EntityModel.Server.ServiceHostEvents {
public override void OnEndpointCreated(System.ServiceModel.Description.ServiceEndpoint endpoint) {
base.OnEndpointCreated(endpoint);
// My server code turning GZIP off comes here?
}
public override void OnServiceHostCreated(System.ServiceModel.ServiceHost host) {
base.OnServiceHostCreated(host);
}
}

Calling WebClient within WebService

I started developing an application in Silverlight that was dealing with downloading the HTML of a website and then parsing it. With Silverlight 4 this can be achieved easily by simply requesting elevated permissions. With Silverlight 3, however, the only way to get the HTML of a website is via a WebService call. My initial idea was to do the following:
public class Service1
{
[OperationContract]
public void GetHtml()
{
Uri targetUri = new Uri("http://www.google.com", UriKind.RelativeOrAbsolute);
WebClient webClient = new WebClient();
webClient.DownloadStringCompleted += this.WebClient_DownloadStringCompleted;
webClient.DownloadStringAsync(targetUri);
}
private void WebClient_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
}
}
However, I realized that as soon as I make the call, which is async as well, from my Silverlight application, there is no way for me to retrieve the HTML of the website. That is why I changed to the following:
public class Service1
{
[OperationContract]
public string GetHtml()
{
Uri targetUri = new Uri("http://www.google.com", UriKind.RelativeOrAbsolute);
WebClient webClient = new WebClient();
return webClient.DownloadString(targetUri);
}
}
I believe the last approach is not that fine since it will freeze the thread. So, my question, is there a way to achieve the first approach a.k.a. make async call from an async call :). Any help would be greatly appreciated.
Best Regards,
Kiril
You can achieve your goal by implementig a Duplex Service. There is some useful information about it on the msdn site and a wonderful podcast entry by Mike Taulty. In general, you would have to modify your operation contract by splitting it into two parts. First part would initiate your WebClient download on the server. Then, on the server, after the html has been downloaded, the server would call back a contract that is implemented on the client side with the payload consisting of the required html content.