Generic passthrough/forwarding of form data in ApsNet.Core - asp.net-core

Question

I'm attempting to create a webhook to receive messages from a Twilio phone number. But instead of just needing a webhook that will modify the data and immediately return a result to Twilio, I need this webhook to pass Twilio's message into an internal API, wait for the response, and then return the result to Twilio.
Here's some generic code I came up with that I hoped would work.
public async Task<HttpResponseMessage> ReceiveAndForwardSms(HttpContent smsContent)
{
var client = new HttpClient();
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", smsContent);
return response;
}
The problem with this code is that Twilio immediately returns a 415 error code (Unsupported Media Type) before entering the function.
When I try to accept the "correct type" (Twilio.AspNet.Common.SmsRequest), I am unable to stuff the SmsRequest back into a form-encoded object and send it via client.PostAsync()...
Ex.:
public async Task<HttpResponseMessage> ReceiveAndForwardSms([FromForm]SmsRequest smsRequest)
{
var client = new HttpClient();
var stringContent = new StringContent(smsRequest.ToString());
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", stringContent);
return response;
}
Is there anything I can do to "mask" the function's accepted type or keep this first function generic?
How do I go about shoving this SmsRequest back into a "form-encoded" object so I can accept it the same way in my consuming service?

Answer 1

TLDR
Your options are:
Use an existing reverse proxy like NGINX, HAProxy, F5
Use YARP to add reverse proxy functionality to an ASP.NET Core project
Accept the webhook request in a controller, map the headers and data to a new HttpRequestMessage and send it to your private service, then map the response of your private service, to the response back to Twilio.
It sounds like what you're trying to build is a reverse proxy. It is very common to put a reverse proxy in front of your web application for SSL termination, caching, routing based on hostname or URL, etc.
The reverse proxy will receive the Twilio HTTP request and then forwards it to the correct private service. The private service responds which the reverse proxy forwards back to Twilio.
I would recommend using an existing reverse proxy instead of building this functionality yourself. If you really want to build it yourself, here's a sample I was able to get working:
In your reverse proxy project, add a controller as such:
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
namespace ReverseProxy.Controllers;
public class SmsController : Controller
{
private static readonly HttpClient HttpClient;
private readonly ILogger<SmsController> logger;
private readonly string twilioWebhookServiceUrl;
static SmsController()
{
// don't do this in production!
var insecureHttpClientHandler = new HttpClientHandler();
insecureHttpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
HttpClient = new HttpClient(insecureHttpClientHandler);
}
public SmsController(ILogger<SmsController> logger, IConfiguration configuration)
{
this.logger = logger;
twilioWebhookServiceUrl = configuration["TwilioWebhookServiceUrl"];
}
public async Task Index()
{
using var serviceRequest = new HttpRequestMessage(HttpMethod.Post, twilioWebhookServiceUrl);
foreach (var header in Request.Headers)
{
serviceRequest.Headers.TryAddWithoutValidation(header.Key, header.Value.ToArray());
}
serviceRequest.Content = new FormUrlEncodedContent(
Request.Form.ToDictionary(
kv => kv.Key,
kv => kv.Value.ToString()
)
);
var serviceResponse = await HttpClient.SendAsync(serviceRequest);
Response.ContentType = "application/xml";
var headersDenyList = new HashSet<string>()
{
"Content-Length",
"Date",
"Transfer-Encoding"
};
foreach (var header in serviceResponse.Headers)
{
if(headersDenyList.Contains(header.Key)) continue;
logger.LogInformation("Header: {Header}, Value: {Value}", header.Key, string.Join(',', header.Value));
Response.Headers.Add(header.Key, new StringValues(header.Value.ToArray()));
}
await serviceResponse.Content.CopyToAsync(Response.Body);
}
}
This will accept the Twilio webhook request, and forward all headers and content to the private web service. Be warned, even though I was able to hack this together until it works, it is probably not secure and not performant. You'll probably have to do a lot more to get this to become production level code. Use at your own risk.
In the ASP.NET Core project for your private service, use a TwilioController to accept the request:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Common;
using Twilio.AspNet.Core;
using Twilio.TwiML;
namespace Service.Controllers;
public class SmsController : TwilioController
{
private readonly ILogger<SmsController> logger;
public SmsController(ILogger<SmsController> logger)
{
this.logger = logger;
}
public IActionResult Index(SmsRequest smsRequest)
{
logger.LogInformation("SMS Received: {SmsId}", smsRequest.SmsSid);
var response = new MessagingResponse();
response.Message($"You sent: {smsRequest.Body}");
return TwiML(response);
}
}
Instead of proxying the request using the brittle code in the reverse proxy controller, I'd recommend installing YARP in your reverse proxy project, which is an ASP.NET Core based reverse proxy library.
dotnet add package Yarp.ReverseProxy
Then add the following configuration to appsettings.json:
{
...
"ReverseProxy": {
"Routes": {
"SmsRoute" : {
"ClusterId": "SmsCluster",
"Match": {
"Path": "/sms"
}
}
},
"Clusters": {
"SmsCluster": {
"Destinations": {
"SmsService1": {
"Address": "https://localhost:7196"
}
}
}
}
}
}
This configuration will forward any request to the path /Sms, to your private ASP.NET Core service, which on my local machine is running at https://localhost:7196.
You also need to update your Program.cs file to start using YARP:
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddReverseProxy()
.LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));
var app = builder.Build();
app.MapReverseProxy();
app.Run();
When you run both projects now, the Twilio webhook request to /sms is forwarded to your private service, your private service will respond, and your reverse proxy service will forward the response back to Twilio.
Using YARP you can do a lot more through configuration or even programmatically, so if you're interested I'd check out the YARP docs.
If you already have a reverse proxy like NGINX, HAProxy, F5, etc. it may be easier to configure that to forward your request instead of using YARP.
PS: Here's the source code for the hacky and YARP solution