Recently I have transferred my simple application which consists of angular 7 client and asp .net core web api (.net core v2.2) to production environment and I encountered a strange problem. When I tried to make an action which should do a post request with authentication token in header i got a 401 - Unauthorized server error.
My production environment is Ubuntu 18.04 with Apache2 server with proxy set to redirect from port 80 to port 5000 (this is the place where API is).
On development environment everything worked perfectly without any error so I guess that there is something with transferring request header from apache to kestrel.
I tried to google this problem and on Microsoft's website I found that I need to use forwarded headers method in my startup.cs file like this:
app.UseForwardedHeaders(new ForwardedHeadersOptions
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});
app.UseAuthentication();
I configured my startup class like that but without any success.
Any help or direction where to try to find the solution would be appreciated.
Thank you very much in advance.
try to add a custome header throgh middleware.
namespace Middleware
{
public class CustomHeader
{
private readonly RequestDelegate _next;
public CustomHeader(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context,
{
var dictionary = new Dictionary<string, string[]>()
{
{
"Access-Control-Allow-Headers",new string[]{ "authorization" }
}
};
//To add Headers AFTER everything you need to do this
context.Response.OnStarting(state => {
var httpContext = (HttpContext)state;
foreach (var item in dictionary)
{
httpContext.Response.Headers.Add(item.Key, item.Value);
}
return Task.FromResult(0);
}, context);
await _next(context);
}
}
}
Related
I'm attempting to create a webhook to receive messages from a Twilio phone number. But instead of just needing a webhook that will modify the data and immediately return a result to Twilio, I need this webhook to pass Twilio's message into an internal API, wait for the response, and then return the result to Twilio.
Here's some generic code I came up with that I hoped would work.
public async Task<HttpResponseMessage> ReceiveAndForwardSms(HttpContent smsContent)
{
var client = new HttpClient();
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", smsContent);
return response;
}
The problem with this code is that Twilio immediately returns a 415 error code (Unsupported Media Type) before entering the function.
When I try to accept the "correct type" (Twilio.AspNet.Common.SmsRequest), I am unable to stuff the SmsRequest back into a form-encoded object and send it via client.PostAsync()...
Ex.:
public async Task<HttpResponseMessage> ReceiveAndForwardSms([FromForm]SmsRequest smsRequest)
{
var client = new HttpClient();
var stringContent = new StringContent(smsRequest.ToString());
HttpResponseMessage response = await client.PostAsync(Environment.GetEnvironmentVariable("requestUriBase") + "/api/SmsHandler/PostSms", stringContent);
return response;
}
Is there anything I can do to "mask" the function's accepted type or keep this first function generic?
How do I go about shoving this SmsRequest back into a "form-encoded" object so I can accept it the same way in my consuming service?
TLDR
Your options are:
Use an existing reverse proxy like NGINX, HAProxy, F5
Use YARP to add reverse proxy functionality to an ASP.NET Core project
Accept the webhook request in a controller, map the headers and data to a new HttpRequestMessage and send it to your private service, then map the response of your private service, to the response back to Twilio.
It sounds like what you're trying to build is a reverse proxy. It is very common to put a reverse proxy in front of your web application for SSL termination, caching, routing based on hostname or URL, etc.
The reverse proxy will receive the Twilio HTTP request and then forwards it to the correct private service. The private service responds which the reverse proxy forwards back to Twilio.
I would recommend using an existing reverse proxy instead of building this functionality yourself. If you really want to build it yourself, here's a sample I was able to get working:
In your reverse proxy project, add a controller as such:
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Primitives;
namespace ReverseProxy.Controllers;
public class SmsController : Controller
{
private static readonly HttpClient HttpClient;
private readonly ILogger<SmsController> logger;
private readonly string twilioWebhookServiceUrl;
static SmsController()
{
// don't do this in production!
var insecureHttpClientHandler = new HttpClientHandler();
insecureHttpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
HttpClient = new HttpClient(insecureHttpClientHandler);
}
public SmsController(ILogger<SmsController> logger, IConfiguration configuration)
{
this.logger = logger;
twilioWebhookServiceUrl = configuration["TwilioWebhookServiceUrl"];
}
public async Task Index()
{
using var serviceRequest = new HttpRequestMessage(HttpMethod.Post, twilioWebhookServiceUrl);
foreach (var header in Request.Headers)
{
serviceRequest.Headers.TryAddWithoutValidation(header.Key, header.Value.ToArray());
}
serviceRequest.Content = new FormUrlEncodedContent(
Request.Form.ToDictionary(
kv => kv.Key,
kv => kv.Value.ToString()
)
);
var serviceResponse = await HttpClient.SendAsync(serviceRequest);
Response.ContentType = "application/xml";
var headersDenyList = new HashSet<string>()
{
"Content-Length",
"Date",
"Transfer-Encoding"
};
foreach (var header in serviceResponse.Headers)
{
if(headersDenyList.Contains(header.Key)) continue;
logger.LogInformation("Header: {Header}, Value: {Value}", header.Key, string.Join(',', header.Value));
Response.Headers.Add(header.Key, new StringValues(header.Value.ToArray()));
}
await serviceResponse.Content.CopyToAsync(Response.Body);
}
}
This will accept the Twilio webhook request, and forward all headers and content to the private web service. Be warned, even though I was able to hack this together until it works, it is probably not secure and not performant. You'll probably have to do a lot more to get this to become production level code. Use at your own risk.
In the ASP.NET Core project for your private service, use a TwilioController to accept the request:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Common;
using Twilio.AspNet.Core;
using Twilio.TwiML;
namespace Service.Controllers;
public class SmsController : TwilioController
{
private readonly ILogger<SmsController> logger;
public SmsController(ILogger<SmsController> logger)
{
this.logger = logger;
}
public IActionResult Index(SmsRequest smsRequest)
{
logger.LogInformation("SMS Received: {SmsId}", smsRequest.SmsSid);
var response = new MessagingResponse();
response.Message($"You sent: {smsRequest.Body}");
return TwiML(response);
}
}
Instead of proxying the request using the brittle code in the reverse proxy controller, I'd recommend installing YARP in your reverse proxy project, which is an ASP.NET Core based reverse proxy library.
dotnet add package Yarp.ReverseProxy
Then add the following configuration to appsettings.json:
{
...
"ReverseProxy": {
"Routes": {
"SmsRoute" : {
"ClusterId": "SmsCluster",
"Match": {
"Path": "/sms"
}
}
},
"Clusters": {
"SmsCluster": {
"Destinations": {
"SmsService1": {
"Address": "https://localhost:7196"
}
}
}
}
}
}
This configuration will forward any request to the path /Sms, to your private ASP.NET Core service, which on my local machine is running at https://localhost:7196.
You also need to update your Program.cs file to start using YARP:
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddReverseProxy()
.LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));
var app = builder.Build();
app.MapReverseProxy();
app.Run();
When you run both projects now, the Twilio webhook request to /sms is forwarded to your private service, your private service will respond, and your reverse proxy service will forward the response back to Twilio.
Using YARP you can do a lot more through configuration or even programmatically, so if you're interested I'd check out the YARP docs.
If you already have a reverse proxy like NGINX, HAProxy, F5, etc. it may be easier to configure that to forward your request instead of using YARP.
PS: Here's the source code for the hacky and YARP solution
I have a REST API and an IdentityServer set up. I would like to be able to display items in my client from the API without having to sign in. However, I would also like to protect my API from external clients that don't belong to me. Is it possible to AllowAnonymous but only from my client?
[HttpGet]
[AllowAnonymous]
public List<Item> GetItems()
{
return new List<Item> { "item1", "item2" };
}
Edit w/ Solution
As mentioned by Tore Nestenius, I changed the grant types from Code to CodeAndClientCredentials and added the Authorize attribute to my controller so that only my client can access it.
Controller:
[HttpGet]
[Authorize]
public List<Item> GetItems()
{
return new List<Item> { "item1", "item2" };
Identity Server 4 Config File:
public static IEnumerable<Client> Clients =>
new Client[]
{
new Client
{
ClientId = "postman-api",
AllowedGrantTypes = GrantTypes.CodeAndClientCredentials,
ClientSecrets =
{
new Secret("secret".Sha256())
},
}
};
}
CORS only works for requests from browsers, if a non browser application makes a request, then CORS will not be involved.
if you use [AllowAnonymous], then any client can access that API endpoint. Either you create separate client for the general things, perhaps using the Client Credentials flow, so that the client can authenticate, get its own token without any user involved.
Turns out this is handled by CORS.
public void ConfigureServices(IServiceCollection services)
{
services.AddCors(options =>
{
options.AddDefaultPolicy(
builder => builder
.WithOrigins("yourURL")
.AllowAnyMethod());
})
}
I am newbie in ASP.NET Core, and I have a controller I need to authorise it only on my machine, for the test purposes, however, deny on other...
I have the following config:
public void ConfigureServices(IServiceCollection services)
{
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
services.AddMvc().AddJsonOptions(options =>
{
options.SerializerSettings.DateFormatString= "yyyy-MM-ddTHH:mm:ssZ";
});
services.AddAuthentication("Cookie")
.AddScheme<CookieAuthenticationOptions, CookieAuthenticationHandler>("Cookie", null);
services.AddLogging(builder => { builder.AddSerilog(dispose: true); });
And on the test controlled I enabled the [Authorise] attrubute
[Authorize]
public class OrderController : Controller
Is there a way to allow my local machine to be autorised to acces the controller's actions? Something like [Authorize(Allow=localhost)]
You can create an action filter like so:
public class LocalhostAttribute : ActionFilterAttribute
{
public override void OnActionExecuting(ActionExecutingContext context)
{
var ip = context.HttpContext.Connection.RemoteIpAddress;
if (!IPAddress.IsLoopback(ip)) {
context.Result = new UnauthorizedResult();
return;
}
base.OnActionExecuting(context);
}
}
And then use the tag Localhost:
//[Authorize]
[Localhost]
public class OrderController : Controller
I believe this will work, restricting the access to the machine where it's executed.
This is more whitelisting than authorization. Authorization means checking whether a user has permission to do something. To do that, the user must be identified first, ie authenticated.
The article Client IP Safelist in the docs shows how you can implement IP safelists through middleware, an action filter or a Razor Pages filter.
App-wide Middleware
The middleware option applies to the entire application. The sample code retrieves the request's endpoint IP, checks it against a list of safe IDs and allows the call to proceed only if it comes from a "safe" list. Otherwise it returns a predetermined error code, in this case 401:
public async Task Invoke(HttpContext context)
{
if (context.Request.Method != "GET")
{
var remoteIp = context.Connection.RemoteIpAddress;
_logger.LogDebug("Request from Remote IP address: {RemoteIp}", remoteIp);
string[] ip = _adminSafeList.Split(';');
var bytes = remoteIp.GetAddressBytes();
var badIp = true;
foreach (var address in ip)
{
var testIp = IPAddress.Parse(address);
if(testIp.GetAddressBytes().SequenceEqual(bytes))
{
badIp = false;
break;
}
}
if(badIp)
{
_logger.LogInformation(
"Forbidden Request from Remote IP address: {RemoteIp}", remoteIp);
context.Response.StatusCode = 401;
return;
}
}
await _next.Invoke(context);
}
The article shows registering it before UseMvc() which means the request will be rejected before reaching the MVC middleware :
app.UseMiddleware<AdminSafeListMiddleware>(Configuration["AdminSafeList"]);
app.UseMvc();
This way we don't waste CPU time routing and processing a request that's going to be rejected anyway. The middleware option is a good choice for implementing a blacklist too.
Action Filter
The filtering code is essentially the same, this time defined in a class derived from ActionFilterAttribute. The filter is defined as a scoped service :
services.AddScoped<ClientIpCheckFilter>();
services.AddMvc(options =>
{
options.Filters.Add
(new ClientIpCheckPageFilter
(_loggerFactory, Configuration));
}).SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
In this case the request will reach the MVC infrastructure before it's accepted or rejected.
Razor Pages Filter
The code is once more the same, this time deriving from IPageFilter
I used ABP CLI generated a MVC template, with which I would like to try a Blazor Server project. I do add a MyProjBlazorModule which was as same as every common Module, just like the ConsoleTestApp project did:
namespace MyProj.Blazor
{
[DependsOn(
typeof(MyProjHttpApiClientModule),
typeof(AbpHttpClientIdentityModelModule)
)]
public class MyProjBlazorModule : AbpModule
{
}
}
Then I added the module as service to ConfigureServices method:
public void ConfigureServices(IServiceCollection services)
{
services.AddSyncfusionBlazor();
services.AddRazorPages();
services.AddServerSideBlazor();
services.AddSingleton<WeatherForecastService>();
services.AddApplication<TaurusBlazorModule>();
}
for a rapid test, I also copied ClientDemoService class from template project MyProj.HttpApi.Client.ConsoleTestApp , and I consume it in my index.razor like this:
#inject ClientDemoService _clientService
...
protected override async Task OnInitializedAsync()
{
await base.OnInitializedAsync();
profile = await _clientService.RunAsync();
}
But it couldn't work, with a error message in browser:
InvalidOperationException: No authenticationScheme was specified, and there was no DefaultAuthenticateScheme found. The default schemes can
be set using either AddAuthentication(string defaultScheme) or
AddAuthentication(Action configureOptions).
while If I copy code identical to the console test project like this:
using (var application = AbpApplicationFactory.Create<MyProjConsoleApiClientModule>())
{
application.Initialize();
var demo = application.ServiceProvider.GetRequiredService<ClientDemoService>();
profile = AsyncHelper.RunSync(() => demo.RunAsync());
}
and it worked. I would like to know the difference between using ABP module and explicitly calling an ugly ServiceProvider method here, and how can I fix this issue in some correct and beautiful way?
Thanks for everyone's help!
Finally, I have got what's wrong with that. In the template source code from abp CLI, the MyProjHttpApiHostModule's ConfigureAuthentication method register authenticate service like this:
private void ConfigureAuthentication(ServiceConfigurationContext context, IConfiguration configuration)
{
context.Services.AddAuthentication()
.AddIdentityServerAuthentication(options =>
{
options.Authority = configuration["AuthServer:Authority"];
options.RequireHttpsMetadata = false;
options.ApiName = "MyProj";
options.JwtBackChannelHandler = new HttpClientHandler()
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator
};
});
}
where AddAuthentication() method used empty parameter overload, that caused the No authenticationScheme was specified error. I referenced IdentityServer4 official document and found the right way to do:
context.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
.AddIdentityServerAuthentication(options =>
{
...
});
That's easy, I should set the default scheme JwtBearerDefaults.AuthenticationScheme
using a different overload of AddAuthentication method just as the error had reported.
I hope this post could help someone facing the same or similar issue.
My startup's middleware configuration looks something like this:
public void Configure(IApplicationBuilder app)
{
app.UseCompression();
app.UseIISPlatformHandler();
app.UseApplicationInsightsRequestTelemetry();
app.UseCors("CorsPolicy");
app.UseStaticFiles();
app.UseMvc();
app.UseApplicationInsightsExceptionTelemetry();
}
Since adding the app.UseCompression() middleware, static html files in wwwroot aren't loading correctly anymore. They don't resolve and tend to load indefinitely.
The compression middleware looks as follows and was sourced from here:
public class CompressionMiddleware
{
private readonly RequestDelegate nextDelegate;
public CompressionMiddleware(RequestDelegate next)
{
nextDelegate = next;
}
public async Task Invoke(HttpContext httpContext)
{
var acceptEncoding = httpContext.Request.Headers["Accept-Encoding"];
//Checking that we have been given the correct header before moving forward
if (!(String.IsNullOrEmpty(acceptEncoding)))
{
//Checking that it is gzip
if (acceptEncoding.ToString().IndexOf("gzip", StringComparison.CurrentCultureIgnoreCase) >= 0)
{
using (var memoryStream = new MemoryStream())
{
var stream = httpContext.Response.Body;
httpContext.Response.Body = memoryStream;
await nextDelegate(httpContext);
using (var compressedStream = new GZipStream(stream, CompressionLevel.Optimal))
{
httpContext.Response.Headers.Add("Content-Encoding", new string[] { "gzip" });
memoryStream.Seek(0, SeekOrigin.Begin);
await memoryStream.CopyToAsync(compressedStream);
}
}
}
else
{
await nextDelegate(httpContext);
}
}
//If we have are given to Accept Encoding header or it is blank
else
{
await nextDelegate(httpContext);
}
}
}
Does anyone know why this could be happening?
Note: I am using DNX 1.0.0-rc1-update1 and the 1.0.0-rc1-final libraries.
I just enabled compression in web.config in /wwwroot folder (like we used to do in IIs applicationHost.config), and it works. There's no need to add a compression middleware.
http://www.brandonmartinez.com/2015/08/20/enable-gzip-compression-for-azure-web-apps/
Compression defined in web.config only works with IIS NOT the self hosted web server.
I tested the CompressionMiddleware example and the problem You are seeing is caused by the Content-Length header.
If the Content-Length is already set, the browser gets confused as the actual size of the response doesn't match the value defined in the header as the content is zipped.
Removing the content-length header solved the problem:
httpContext.Response.Headers.Remove("Content-Length");
httpContext.Response.Headers.Add("Content-Encoding", new string[] { "gzip" });
...even better you could try to specify the actual content-length when the content is zipped.