Generic passthrough/forwarding of form data in ApsNet.Core - asp.net-core

I'm attempting to create a webhook to receive messages from a Twilio phone number. But instead of just needing a webhook that will modify the data and immediately return a result to Twilio, I need this webhook to pass Twilio's message into an internal API, wait for the response, and then return the result to Twilio.
Here's some generic code I came up with that I hoped would work.
public async Task<HttpResponseMessage> ReceiveAndForwardSms(HttpContent smsContent)
{
var client = new HttpClient();
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", smsContent);
return response;
}
The problem with this code is that Twilio immediately returns a 415 error code (Unsupported Media Type) before entering the function.
When I try to accept the "correct type" (Twilio.AspNet.Common.SmsRequest), I am unable to stuff the SmsRequest back into a form-encoded object and send it via client.PostAsync()...
Ex.:
public async Task<HttpResponseMessage> ReceiveAndForwardSms([FromForm]SmsRequest smsRequest)
{
var client = new HttpClient();
var stringContent = new StringContent(smsRequest.ToString());
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", stringContent);
return response;
}
Is there anything I can do to "mask" the function's accepted type or keep this first function generic?
How do I go about shoving this SmsRequest back into a "form-encoded" object so I can accept it the same way in my consuming service?

TLDR
Your options are:
Use an existing reverse proxy like NGINX, HAProxy, F5
Use YARP to add reverse proxy functionality to an ASP.NET Core project
Accept the webhook request in a controller, map the headers and data to a new HttpRequestMessage and send it to your private service, then map the response of your private service, to the response back to Twilio.
It sounds like what you're trying to build is a reverse proxy. It is very common to put a reverse proxy in front of your web application for SSL termination, caching, routing based on hostname or URL, etc.
The reverse proxy will receive the Twilio HTTP request and then forwards it to the correct private service. The private service responds which the reverse proxy forwards back to Twilio.
I would recommend using an existing reverse proxy instead of building this functionality yourself. If you really want to build it yourself, here's a sample I was able to get working:
In your reverse proxy project, add a controller as such:
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
namespace ReverseProxy.Controllers;
public class SmsController : Controller
{
private static readonly HttpClient HttpClient;
private readonly ILogger<SmsController> logger;
private readonly string twilioWebhookServiceUrl;
static SmsController()
{
// don't do this in production!
var insecureHttpClientHandler = new HttpClientHandler();
insecureHttpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
HttpClient = new HttpClient(insecureHttpClientHandler);
}
public SmsController(ILogger<SmsController> logger, IConfiguration configuration)
{
this.logger = logger;
twilioWebhookServiceUrl = configuration["TwilioWebhookServiceUrl"];
}
public async Task Index()
{
using var serviceRequest = new HttpRequestMessage(HttpMethod.Post, twilioWebhookServiceUrl);
foreach (var header in Request.Headers)
{
serviceRequest.Headers.TryAddWithoutValidation(header.Key, header.Value.ToArray());
}
serviceRequest.Content = new FormUrlEncodedContent(
Request.Form.ToDictionary(
kv => kv.Key,
kv => kv.Value.ToString()
)
);
var serviceResponse = await HttpClient.SendAsync(serviceRequest);
Response.ContentType = "application/xml";
var headersDenyList = new HashSet<string>()
{
"Content-Length",
"Date",
"Transfer-Encoding"
};
foreach (var header in serviceResponse.Headers)
{
if(headersDenyList.Contains(header.Key)) continue;
logger.LogInformation("Header: {Header}, Value: {Value}", header.Key, string.Join(',', header.Value));
Response.Headers.Add(header.Key, new StringValues(header.Value.ToArray()));
}
await serviceResponse.Content.CopyToAsync(Response.Body);
}
}
This will accept the Twilio webhook request, and forward all headers and content to the private web service. Be warned, even though I was able to hack this together until it works, it is probably not secure and not performant. You'll probably have to do a lot more to get this to become production level code. Use at your own risk.
In the ASP.NET Core project for your private service, use a TwilioController to accept the request:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Common;
using Twilio.AspNet.Core;
using Twilio.TwiML;
namespace Service.Controllers;
public class SmsController : TwilioController
{
private readonly ILogger<SmsController> logger;
public SmsController(ILogger<SmsController> logger)
{
this.logger = logger;
}
public IActionResult Index(SmsRequest smsRequest)
{
logger.LogInformation("SMS Received: {SmsId}", smsRequest.SmsSid);
var response = new MessagingResponse();
response.Message($"You sent: {smsRequest.Body}");
return TwiML(response);
}
}
Instead of proxying the request using the brittle code in the reverse proxy controller, I'd recommend installing YARP in your reverse proxy project, which is an ASP.NET Core based reverse proxy library.
dotnet add package Yarp.ReverseProxy
Then add the following configuration to appsettings.json:
{
...
"ReverseProxy": {
"Routes": {
"SmsRoute" : {
"ClusterId": "SmsCluster",
"Match": {
"Path": "/sms"
}
}
},
"Clusters": {
"SmsCluster": {
"Destinations": {
"SmsService1": {
"Address": "https://localhost:7196"
}
}
}
}
}
}
This configuration will forward any request to the path /Sms, to your private ASP.NET Core service, which on my local machine is running at https://localhost:7196.
You also need to update your Program.cs file to start using YARP:
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddReverseProxy()
.LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));
var app = builder.Build();
app.MapReverseProxy();
app.Run();
When you run both projects now, the Twilio webhook request to /sms is forwarded to your private service, your private service will respond, and your reverse proxy service will forward the response back to Twilio.
Using YARP you can do a lot more through configuration or even programmatically, so if you're interested I'd check out the YARP docs.
If you already have a reverse proxy like NGINX, HAProxy, F5, etc. it may be easier to configure that to forward your request instead of using YARP.
PS: Here's the source code for the hacky and YARP solution

Related

Setup JWT from headers to be used by default in HttpClient

I use .NET 6 and add HttpClient in Program.cs like this:
builder.Services.AddHttpClient<IUserClient, UserClient>(client =>
{
client.BaseAddress = new Uri(builder.Configuration["Clients:UserBaseUrl"]);
});
In my onion architecture when I want to create an order a request is coming to OrderService.API and to check userId is correct in IUserService from OrderService.BL I call user microservice API with the help of registered HttpClient. So the problem is that for now, I need to transfer JWT to the business logic layer via method parameters.
var createdOrder = await _orderService.Add(model.MapToDto(), HttpContext.Request.Headers["Authorization"]);
I don't like it because, for every method using the HttpClient, it's necessary to provide an extra parameter. I think maybe there is a way to set up HttpClient default authentication during the current request.
I tried to setup default request headers during HttpClient registration:
builder.Services.AddHttpClient<IUserClient, UserClient>(client =>
{
client.BaseAddress = new Uri(builder.Configuration["Clients:UserBaseUrl"]);
client.DefaultRequestHeaders.Add("Authorization", token); // setup token
});
but I don't know how to get JWT from HttpRequest headers there.
Additionally, I thought maybe I can set up the header for HttpClient in some additional BaseController which would be nested by any other my controller but it doesn't seem to be a great solution.
Maybe there is a way for middleware use but as I understand we handle an incoming request to OrderService and can't handle outcoming requests from HttpClient.
So would be grateful for any of your ideas!
Thanks to #Rena who provided a link to an existing similar problem: https://stackoverflow.com/a/62324677/11398810
So I created a message helper like this and it seems work for me:
public sealed class HttpClientsAuthHelper : DelegatingHandler
{
private readonly IHttpContextAccessor _accessor;
public HttpClientsAuthHelper(IHttpContextAccessor accessor)
{
_accessor = accessor;
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var token = _accessor.HttpContext.Request.Headers["Authorization"].First();
request.Headers.Add("Authorization", token);
return await base.SendAsync(request, cancellationToken);
}
}
And added these lines to Program.cs:
builder.Services.TryAddSingleton<IHttpContextAccessor, HttpContextAccessor>();
builder.Services.AddTransient<HttpClientsAuthHelper>();
builder.Services.AddHttpClient<IUserClient, UserClient>(client =>
{
client.BaseAddress = new Uri(builder.Configuration["Clients:UserBaseUrl"]);
}).AddHttpMessageHandler<HttpClientsAuthHelper>();
builder.Services.AddHttpClient<IProductClient, ProductClient>(client =>
{
client.BaseAddress = new Uri(builder.Configuration["Clients:ProductBaseUrl"]);
}).AddHttpMessageHandler<HttpClientsAuthHelper>();
I'm not sure how correct such approach so I'll dive a bit into this logic later =)

Why can't my xamarin app recieve data from my API call?

Im new to connecting an API to my xamarin app.
When I try to call the API visual studio & the app do not give a response.
Visual studio keeps running but nothing happens.
I've changed the firewall settings, and set my IP adres in all the desired places. Still not luck.
If I go to my API using swager or postman and I just the same Uri as I want to pass trough with my app I get the correct response.
What could be the reason for this?
my code:
Material service:
private readonly string _baseUri;
public APIMaterialService()
{
_baseUri = "https://192.168.1.9:5001/api";
}
public async Task<Material> GetById(Guid id)
{
return await WebApiClient
.GetApiResult<Material>($"{_baseUri}/Materials/{id}");
}
WebApiClient:
public class WebApiClient
{
private static HttpClientHandler ClientHandler()
{
var httpClientHandler = new HttpClientHandler();
#if DEBUG
//allow connecting to untrusted certificates when running a DEBUG assembly
httpClientHandler.ServerCertificateCustomValidationCallback =
(message, cert, chain, errors) => { return true; };
#endif
return httpClientHandler;
}
private static JsonMediaTypeFormatter GetJsonFormatter()
{
var formatter = new JsonMediaTypeFormatter();
formatter.SerializerSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
return formatter;
}
public async static Task<T> GetApiResult<T>(string uri)
{
using (HttpClient httpClient = new HttpClient(ClientHandler()))
{
//Gets stuck finding the response
string response = await httpClient.GetStringAsync(uri);
return JsonConvert.DeserializeObject<T>(response, GetJsonFormatter().SerializerSettings);
}
}
I'll also add some images of the postman and swager response:
This is the code fo my controller.
return OK (material) shows me the data retrieved from the API
public async Task<IActionResult> GetMaterialByPartOfMaterialNumberOP(string partOfMaterialNumber)
{
var material = await _materialService.GetMaterialListbyPartOfMaterialNumber(partOfMaterialNumber);
return Ok(material);
}
The symptom you have (stuck on result from calling a method of HttpClient class) suggests a deadlock.
I believe the deadlock happens if you create multiple instances of HttpClient.
Doc HttpClient Class says:
// HttpClient is intended to be instantiated once per application, rather than per-use. See Remarks.
And shows this code:
static readonly HttpClient client = new HttpClient();
HOWEVER a deadlock would only happen the SECOND time your code does new HttpClient. And using ... new HttpClient should protect you, at least in simple situations.
Here are ways there might be TWO HttpClients active:
Is it possible that GetApiResult gets called A SECOND TIME, before the first one finishes?
Does your app do new HttpClient ELSEWHERE?
Here is what the technique might look like in your app:
public class WebApiClient
{
static HttpClient _client = new HttpClient(ClientHandler());
public async static Task<T> GetApiResult<T>(string uri)
{
string response = await _client.GetStringAsync(uri);
return JsonConvert.DeserializeObject<T>(response, GetJsonFormatter().SerializerSettings);
}
}

How to pass authorization token in every request

After the user logs in I verify their info and generate a JWT token.
Authentication process happens with Authentication (it's not my custom handler).
Where and how do I save this token so it will be sent along the http calls? I don't want to save it in the client side because of XSS attacks. The following doesn't seem to work either as I wont be in every request
HttpContext.Request.Headers.Append("Authorization", MyGeneratedJWTTokenAsString);
I have found answers that use HttpClient.Request but is there any other secure way of doing this?
When using HttpClient in a backend service, it is always good to use the IHttpClientFactory to generate clients.
So, what we are going to do is use this factory (in conjunction with IHttpContextAccessor) to produce HttpClient objects that have the current user's authorization scheme and token. So, add this to your ConfigureServices method in Startup.cs
public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddHttpClient("UserAuthorizedHttpClient", (sp, httpClient) =>
{
var accessor = sp.GetRequiredService<IHttpContextAccessor>();
if (accessor.HttpContext.Request.Headers.TryGetValue(
"Authorization", out var authHeaderValue) &&
AuthenticationHeaderValue.TryParse(
authHeaderValue, out var auth))
{
httpClient.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue(auth.Scheme, auth.Parameter);
}
else
{
// incase there is a value from a previous generation
if(httpClient.DefaultRequestHeaders.Contains("Authorization"))
{
httpClient.DefaultRequestHeaders.Remove("Authorization");
}
}
});
services.AddHttpContextAccessor();
// ...
}
In order to use these special clients, you simply inject IHttpClientFactory in to the service that needs to make the HTTP requests:
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace YouApplicationNamespace.Services
{
public interface IMyHttpRequesterService
{
Task DoSomethingCoolAsync();
}
public sealed class MyHttpRequesterService : IMyHttpRequesterService
{
private readonly IHttpClientFactory _httpClientFactory;
public MyHttpRequesterService(IHttpClientFactory httpClientFactory) =>
_httpClientFactory = httpClientFactory;
public async Task DoSomethingCoolAsync()
{
var authroizedHttpClient =
_httpClientFactory.CreateClient("UserAuthorizedHttpClient");
var resp = await authroizedHttpClient.GetAsync(new Uri("https://www.example.com/"));
// ...
}
}
}
As long as you use the same name, you will get a client that uses the AddHttpClient routine in your configuration.
(Please note: this code is not tested. It is more of a guideline)

In ASP.NET Core, can the application generate a new request to itself without doing IO?

I'm implementing a custom ASP.NET Core middleware to handle the ETag/If-Match pattern to prevent lost updates. My HTTP GET operations will return an ETag value in the header and every PUT operation will be required to include that ETag in the If-Match header. I'm hashing the body of the GET responses to generate the ETag value. I've currently implemented the middleware using the HttpClient to perform the GET operation when I need to check the If-Match header. This works but requires a network/out-of-process call. Shouldn't there be a better performing way to call the ASP.NET Core HTTP pipeline without leaving the process? Can the application generate a new request to itself without doing IO? Here's the code currently for my middleware:
public class ETagIfMatchMiddleware : IMiddleware
{
//client for my asp.net core application
public static HttpClient client = new HttpClient { BaseAddress = new Uri("https://localhost:5001") };
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var request = context.Request;
if (request.Method == HttpMethods.Put)
{
var ifMatch = request.Headers[HeaderNames.IfMatch];
//requires network out of process call
var response = await client.GetAsync(request.GetEncodedUrl());
string eTag = response.Headers.ETag.Tag;
if (eTag != ifMatch)
{
context.Response.StatusCode = StatusCodes.Status412PreconditionFailed;
return;
}
}
await next(context);
}
}

How to mock WepAPi response (JSON) in client MVC client project

I want to test my service proxy class that is calling a MVC WebApi and gets back a JSON response.
public class CarServiceProxy
{
public CarsCollection GetCars()
{
CarsCollection cars = new CarsCollection();
string api = "api/Car/GetCars";
var response = httpClient.GetAsync(api).Result;
if (response.IsSuccessStatusCode)
{
cars = response.Content.ReadAsAsync<CarsCollection >().Result;
}
return cars;
}
}
then my MVC WebSite controller class calls the above proxy class as:
public ActionResult Index()
{
CarsCollection cars = this.carsServiceProxy.GetCars();
return View(cars);
}
Now to test both in isolation, I can test my controller using MOQ and mocking carServiceProxy and faking the CarsCollection with some fake data in my tests. I am ok so far.
But, how do I test the proxy class using MOQ to mock the API response (JSON)? I think I do need to test because the conversion form JSON to C# happens in that class, and if some one changes my model CarsCollection, that might break. So I do need to test the proxy class.
You will want to create a "Fake" HTTP handler for your client. This way you can control exactly what HTTP response is returned. You can inject an HTTP handler when you construct an HTTP client.
For testing I typically use an internal constructor for testing, which accepts an HTTP handler. This way I can easily test my class that consumes an HTTP Client with a fake handler. If you take this approach and your unit tests are in a separate assembly, you'll need to add the following to the AssemblyInfo.cs for your target project:
[assembly: InternalsVisibleTo("NameSpace.MyUnitTestProject")]
Fake Handler:
public class FakeHttpMessageHandler : HttpMessageHandler
{
private HttpResponseMessage _response;
public FakeHttpMessageHandler(HttpResponseMessage response)
{
_response = response;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var responseTask = new TaskCompletionSource<HttpResponseMessage>();
responseTask.SetResult(_response);
return responseTask.Task;
}
}
And then to consume (may want to use JSON serialisation helpers here):
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StringContent(#"{'Cars':[{'Name':'BMW'}]");
var handler = new FakeHttpMessageHandler(response);
var client = new HttpClient(handler);
// client.GetAsync().result will return the response
You can change the httpClient.GetAsync(api).Result part to be moackable and testable to test the proxy class. In other words, you can make your code independent of HTTP request response and just work on JSON data for testing.