Configure ForwardHeadersMiddleware from appsettings? - asp.net-core

I'm trying to configure ASP.NET Core 5's ForwardedHeadersMiddleware from appsettings.config. I'm having trouble to set KnownProxies (IList<IPAddress> KnownProxies { get; }) and it keeps reverting back to the default value. I assume it has to do with the options machinery not knowing how to convert the string to an IPAddress, or KnownProxies only having a getter.
{
"ForwardedHeaders": {
"ForwardedHeaders": "All"
"KnownProxies": ["10.0.0.1"]
}
}
services.Configure<ForwardedHeadersOptions>(Configuration.GetSection("ForwardedHeaders"));
How can I achieve what I want, without doing the parsing manually?
Can I specify the mapping somewhere generic?
Why doesn't this throw an exception that some of my configuration could not be parsed / is invalid?

I may propose my recipe for this:
Define your own options class like the following:
public class ForwardedForKnownNetworks
{
public class Network
{
public string Prefix { get; set; }
public int PrefixLength { get; set; }
}
public List<Network> Networks { get; set; } = new List<Network>();
public List<string> Proxies { get; set; } = new List<string>();
}
To make your code shorter and more agile you may need kind of helper method to resolve hostnames and/or convert parse IP addressed strings:
public static class Extensions
{
public static IPAddress[] ResolveIP(this string? host)
{
return (!string.IsNullOrWhiteSpace(host))
? Dns.GetHostAddresses(host)
: new IPAddress[0];
}
}
When configuring your services, you can add something similar to the following:
var forwardedForKnownNetworks = builder.Configuration.GetSection(nameof(ForwardedForKnownNetworks)).Get<ForwardedForKnownNetworks>();
_ = builder.Services.Configure<ForwardedHeadersOptions>(options => {
options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
forwardedForKnownNetworks?.Networks?.ForEach((network) => options.KnownNetworks.Add(new IPNetwork(IPAddress.Parse(network.Prefix), network.PrefixLength)));
forwardedForKnownNetworks?.Proxies?.ForEach((proxy) => proxy.ResolveIP().ToList().ForEach((ip) => options.KnownProxies.Add(ip)));
});
And finally your appsettings.json may appear like the following:
{
"ForwardedForKnownNetworks": {
"Networks": [
{
"Prefix": "172.16.0.0",
"PrefixLength": 12
},
{
"Prefix": "192.168.0.0",
"PrefixLength": 16
},
{
"Prefix": "10.0.0.0",
"PrefixLength": 8
}
],
"Proxies": ["123.234.32.21", "my.proxy.local"] // Here you can mention IPs and/or hostnames as the ResolveIP() helper will take care of that and resolve any hostname to its IP(s)
}
}

Related

OData empty object with PUT/PATCH

When I am trying to send PUT or PATCH request with JSON in body I am getting object with default values.
Everything is fine with get requests. And PUT request is working if specify data as parameters in URL.
I am using .NET Core and Microsoft.AspNetCore.OData 7.5.0 NuGet package
The example:
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
}
public class OdataModelConfigurations : IModelConfiguration
{
public void Apply(ODataModelBuilder builder, ApiVersion apiVersion)
{
var product = builder.EntitySet<Product>("Products").EntityType;
product.HasKey(p => p.Id);
product.Property(p => p.Name);
}
}
[ODataRoutePrefix("Products")]
public class ProductController : ODataController
{
[ODataRoute]
[HttpPut]
[EnableQuery(AllowedQueryOptions = AllowedQueryOptions.All)]
public async Task<IActionResult> Put([FromBody] Product update)
{
// some code omitted
}
}
I'v tried to use different body content and to add different headers (Specify OData-Version for example).
Here is one of body examples that I'v tried to use:
{
"#odata.context": "https://localhost:5001/odata/$metadata#Product",
"Name": "put tested",
"Id":"1"
}
Or another one:
{
"#odata.type": "#ODataAPI.Models.Product",
"Name#odata.type": "String",
"Name": "patch tested"
}
Everything works like a charm without Versioning and Swagger. Even if I am just sending simple body:
{
"Name": "put tested",
"Id":1,
"CategoryId":1
}
Was able to find next one project that shows how it is possible to combine OData and Swagger:
https://github.com/microsoft/aspnet-api-versioning/tree/master/samples/aspnetcore/SwaggerODataSample

How to make an IOptions section optional in .NET Core?

Consider an example service that optionally supports LDAP authentication, otherwise, it does something like local Identity authentication. When LDAP is completely configured, appsettings.json might look like this...
{
"LdapOptions": {
"Host": "ldap.example.com",
"Port": 389
}
}
With an options class.
public class LdapOptions
{
public string Host { get; set; }
public int Port { get; set; } = 389;
}
And Startup has the expected Configure call.
service.Configure<LdapOptions>(nameof(LdapOptions));
This work great when I have a complete valid "LdapOptions" section. But, it's not so great if I intentionally leave the section out of my appsettings.
An IOptions<TOptions> instance resolves even if I leave the section out of my appsettings entirely; it even resolves if I remove the Startup configure call entirely! I get an object that appears, based on property values, to be default(TOptions).
public AuthenticationService(IOptions<LdapOptions> ldapOptions)
{
this.ldapOptions = ldapOptions.Value; // never null, sometimes default(LdapOptions)!
}
I don't want to depend on checking properties if a section is intentionally left out. I can imagine scenarios where all of the properties in an object have explicit defaults and this wouldn't work. I'd like something like a Maybe<TOptions> with a HasValue property, but I'll take a null.
Is there any way to make an options section optional?
Update: Be aware that I also intend to validate data annotations...
services.AddOptions<LdapOptions>()
.Configure(conf.GetSection(nameof(LdapOptions)))
.ValidateDataAnnotations();
So, what I really want is for optional options to be valid when the section is missing (conf.Exists() == false) and then normal validations to kick in when the section is partially or completely filled out.
I can't imagine any solution working with data annotation validations that depends on the behavior of creating a default instance (for example, there is no correct default for Host, so a default instance will always be invalid).
The whole idea of IOptions<T> is to have non-null default values, so that your settings file doesn't contain hundreds/thousands sections to configure the entire ASP pipeline
So, its not possible to make it optional in the sense that you will get null, but you can always defined some "magic" property to indicate whether this was configured or not:
public class LdapOptions
{
public bool IsEnabled { get; set; } = false;
public string Host { get; set; }
public int Port { get; set; } = 389;
}
and your app settings file:
{
"LdapOptions": {
"IsEnabled: true,
"Host": "ldap.example.com",
"Port": 389
}
}
Now, if you keep 'IsEnabled' consistently 'true' in your settings, if IsEnabled is false, that means the section is missing.
An alternative solution is to use a different design approach, e.g. put the auth type in the settings file:
public class LdapOptions
{
public string AuthType { get; set; } = "Local";
public string Host { get; set; }
public int Port { get; set; } = 389;
}
And your app settings:
{
"LdapOptions": {
"AuthType : "LDAP",
"Host": "ldap.example.com",
"Port": 389
}
}
This is IMO a cleaner & more consistent approach
If you must have a logic that is based on available/missing section, you can also configure it directly:
var section = conf.GetSection(nameof(LdapOptions));
var optionsBuilder = services.AddOptions<LdapOptions>();
if section.Value != null {
optionsBuilder.Configure(section).ValidateDataAnnotations();
}
else {
optionsBuilder.Configure(options => {
// Set defaults here
options.Host = "Deafult Host";
}
}
I wanted to avoid lambdas in Startup that would need to be copy/pasted correctly for every "optional" section and I wanted to be very explicit about optionality (at the expense of some awkward naming).
Startup.cs
public void ConfigureServices(IServiceCollection services)
{
services.AddOption<Optional<LdapOptions>>()
.ConfigureOptional(conf.GetSection(nameof(LdapOptions)))
.ValidateOptionalDataAnnotations();
}
The Optional type is pretty straightforward, but may need a better name (to avoid interfering with other implementations of the generic Option/Some/Maybe pattern). I thought about just using null, but that seemed contrary to Options insistence on returning something no matter what.
Optional.cs
public class Optional<TOptions> where TOptions : class
{
public TOptions Value { get; set; }
public bool HasValue { get => !(Value is null); }
}
The configure extension method takes into account section existence.
OptionalExtensions.cs
public static class OptionalExtensions
{
public static OptionsBuilder<Optional<TOptions>> ConfigureOptional<TOptions>(this OptionsBuilder<Optional<TOptions>> optionsBuilder, IConfigurationSection config) where TOptions : class
{
return optionsBuilder.Configure(options =>
{
if (config.Exists())
{
options.Value = config.Get<TOptions>();
}
});
}
public static OptionsBuilder<Optional<TOptions>> ValidateOptionalDataAnnotations<TOptions>(this OptionsBuilder<Optional<TOptions>> optionsBuilder) where TOptions : class
{
optionsBuilder.Services.AddSingleton<IValidateOptions<Optional<TOptions>>>(new DataAnnotationValidateOptional<TOptions>(optionsBuilder.Name));
return optionsBuilder;
}
}
The validate extension method works with a custom options validator that also takes into account how missing sections work (like the comment says, "missing optional options are always valid").
DataAnnotationValidateOptional.cs
public class DataAnnotationValidateOptional<TOptions> : IValidateOptions<Optional<TOptions>> where TOptions : class
{
private readonly DataAnnotationValidateOptions<TOptions> innerValidator;
public DataAnnotationValidateOptional(string name)
{
this.innerValidator = new DataAnnotationValidateOptions<TOptions>(name);
}
public ValidateOptionsResult Validate(string name, Optional<TOptions> options)
{
if (options.Value is null)
{
// Missing optional options are always valid.
return ValidateOptionsResult.Success;
}
return this.innerValidator.Validate(name, options.Value);
}
}
Now, anywhere you need to use an optional option, like, say, a login controller, you can take the following actions...
LdapLoginController.cs
[ApiController]
[Route("/api/login/ldap")]
public class LdapLoginController : ControllerBase
{
private readonly Optional<LdapOptions> ldapOptions;
public LdapLoginController(IOptionsSnapshot<Optional<LdapOptions>> ldapOptions)
{
// data annotations should trigger here and possibly throw an OptionsValidationException
this.ldapOptions = ldapOptions.Value;
}
[HttpPost]
public void Post(...)
{
if (!ldapOptions.Value.HasValue)
{
// a missing section is valid, but indicates that this option was not configured; I figure that relates to a 501 Not Implemented
return StatusCode((int)HttpStatusCode.NotImplemented);
}
// else we can proceed with valid options
}
}

I'm trying to write a POST request to my API so that it can act on two separate objects(an insert and an update)

I'm trying to write this controller that accepts a POST request.
I need this controller to add a new book, and also add that new books bookId to another object called a StoreList.
So I am trying to pass in the new bookList, and the storeList that needs the bookId added to it.
// POST: api/BookList
[HttpPost]
public async Task<ActionResult<BookList>> PostBookList(BookList bookList, StoreList storeList)
{
_context.BookList.Add(bookList);
await _context.SaveChangesAsync();
_context.Entry(storeList).State = EntityState.Modified;
try
{
await _context.SaveChangesAsync();
}
catch (DbUpdateConcurrencyException)
{
if (!StoreListExists(storeId))
{
return NotFound();
}
else
{
throw;
}
}
return CreatedAtAction("GetBookList", new { id = bookList.BookId }, bookList);
}
Here is my API endpoint:
https://localhost:44362/api/BookList/
And these are the two objects I'm passing in the BODY of the request (the new bookList and the existing storeList):
{
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7",
"bookTitle": "Is this a test 2?"
},
{
"storeId": "0001f801-6909-4b6e-8652-e1b49745280f",
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7"
}
But whenever I try to 'hit' that endpoint, I get this error:
System.InvalidOperationException HResult=0x80131509 Message=Action
'DocumentStorageAPI.Controllers.Book.BookListController.PostBookList
(DocumentStorageAPI)' has more than one parameter that was
specified or inferred as bound from request body. Only one
parameter per action may be bound from body. Inspect the following
parameters, and use 'FromQueryAttribute' to specify bound from query,
'FromRouteAttribute' to specify bound from route, and
'FromBodyAttribute' for parameters to be bound from body: BookList
bookList StoreList storeList
How can I get my controller to allow me to add a new bookList and update the needed storeList?
Thanks!
The body of the request should be just one object and the PostBookList method must have only one parameter (with the [FromBody] attribute). If you need both classes to use within the method create a new class like this:
public class PostBookListRequest
{
public BookList BookList { get; set; }
public StoreList StoreList { get; set; }
}
change the PostBookList method to
public async Task<ActionResult<BookList>> PostBookList([FromBody]PostBookListRequest request)
{
// Your logic here
}
And in the BODY of the request do
{
"bookList": {
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7",
"bookTitle": "Is this a test 2?"
},
"storeList": {
"storeId": "0001f801-6909-4b6e-8652-e1b49745280f",
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7"
}
}
You can use body like this:
{
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7",
"title": "Is this a test 2?",
"storeId" "0001f801-6909-4b6e-8652-e1b49745280f"
}
In Controller you need additional class BookListBinding with this 3 fields, which you will use for create you 2 objects, for example.
[HttpPost]
public async Task<ActionResult> PostBookList(BookListBinding binding)
{
var bookList = new BookList
{
Id = binding.BookId,
Title = binding.Title
});
var storeList = new StoreList
{
Id = binding.StoreId,
BookId = binding.BookId
}
// you work with _context
return CreatedAtAction("GetBookList", new { id = binding.BookId }, bookList);
}
Why you need change StoreList? What is it?
You have to create a new model like so-
public class AddBookToStoreModel
{
public Book BookToAdd { get; set; }
public StoreList BookStoreList { get; set; }
}
You have to add the same to the controller definition, like this-
[HttpPost]
public async Task<ActionResult<BookStore>> PostBookList(AddBookToStoreModel model)
Also you need to create the individual Models for Book and StoreList like below-
public class Book
{
public Guid BookId { get; set; }
public string BookTitle { get; set; }
}
public class StoreList
{
public Guid StoreId { get; set; }
public Guid BookId { get; set; }
}
The Json that you post to the controller will look like below-
{
"BookToAdd": {
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7",
"bookTitle": "Is this a test 2?"
},
"BookStoreList": {
"storeId": "0001f801-6909-4b6e-8652-e1b49745280f",
"bookId": "bc381612-c63b-4438-b35b-161a3a568fc7"
}
}
I hope this works for you.

Returning sub-properties from document in RavenDb via Index using Lucene

I'm using RavenDB 2.5 and what I want to do is query a Group (see below) providing a valid Lucene search term and get back a collection of Member instances (or even just Ids) that match. So, class definition:
public class Group {
public string Id { get; set; }
public IList<Member> Members { get; set; }
}
public class Member {
public string Name { get; set; }
public string Bio { get; set; }
}
And they are stored in the database as session.Store(groupInstance); as you'd expect. What I'd like to do is query and return the Member instances which match a given search term.
So, something like:
public class GroupMembers_BySearchTerm : AbstractIndexCreationTask {
public override IndexDefinition CreateIndexDefinition(){
return new IndexDefinition {
Map = "from g in docs.Groups select new { Content = new [] { g.Members.Select(m => m.Name), g.Members.Select(m => m.Bio) }, Id = g.Id }",
Indexes = { { "Id", FieldIndexing.Default }, { "Content", FieldIndexing.Analyzed } }
}
}
}
If I call this using something like:
session.Advanced.LuceneQuery<Group, GroupMembers_BySearchTerm>().Where("Id: myId AND Content: search terms").ToList();
I obviously get back a Group instance, but how can I get back the Members instead?
What about an index like this:
public class Members_BySearchTermAndGroup : AbstractIndexCreationTask {
public override IndexDefinition CreateIndexDefinition(){
return new IndexDefinition {
Map = "from g in docs.Groups
from member in g.Members
select new {
GroupdId = g.Id,
Name = member.Name,
Bio = member.Bio,
Content = new [] {member.Name, member.Bio },
}",
Indexes = {
{ "GroupId", FieldIndexing.Default },
{ "Content", FieldIndexing.Analyzed }
},
Stores = {
{ "Name", FieldStorage.Yes },
{ "Bio", FieldStorage.Yes }
}
}
}
}
If you take a closer look you'll see that we are creating a new lucene entry for each member inside of a group. Consequently, you'll be able to query on those elements and retrieve them.
Finally you can query your store like this (more info about searching):
session.Query<Member, Members_BySearchTermAndGroup>()
.Search(x => x.Content, "search terms")
.ProjectFromIndexFieldsInto<Member>()
.ToList();
I cannot check this right now but I guess that you need to project your results using the ProjectFromIndexFieldsInto(). Some more information about projections in this link.
or, following your example:
session.Advanced
.LuceneQuery<Member, Members_BySearchTermAndGroup>()
.Where("GroupId: myId AND Content: search terms")
.SelectFields<Member>()
.ToList();

ADO.NET Entity Framework Hierarchical Data

Consider following database model:
And following query code:
using (var context = new DatabaseEntities())
{
return context.Feed.ToHierarchy(f => f.Id_Parent, null);
}
Where ToHierarchy is an extension to ObjectSet<TEntity> as:
public static List<TEntity> ToHierarchy<TEntity, TProperty>(this ObjectSet<TEntity> set, Func<TEntity, TProperty> parentIdProperty, TProperty idRoot) where TEntity : class
{
return set.ToList().Where(f => parentIdProperty(f).Equals(idRoot)).ToList();
}
This would result in example JSON formatted response:
[
{
"Description":"...",
"Details":[ ],
"Id":1,
"Id_Parent":null,
"Title":"...",
"URL":"..."
},
{
"Description":"...",
"Details":[
{
"Description":"...",
"Details":[ ],
"Id":4,
"Id_Parent":3,
"Title":"...",
"URL":"..."
},
{
"Description":"...",
"Details":[
{
"Description":"...",
"Details":[
{
"Description":"...",
"Details":[ ],
"Id":7,
"Id_Parent":6,
"Title":"...",
"URL":"..."
}
],
"Id":6,
"Id_Parent":5,
"Title":"...",
"URL":"..."
}
],
"Id":5,
"Id_Parent":3,
"Title":"...",
"URL":null
}
],
"Id":3,
"Id_Parent":null,
"Title":"...",
"URL":null
}
]
As you may have noticed ToHierarchy method is supposed to (and apparently do indeed) retrieve all rows from a given set (flat) and return hierarchical representation of these as per "parent property".
When I was in the middle of my implementation I quick tried my code and surprisingly it worked! Now, I imagine how weird does this sound to many of you, but I really don't understand why or how that piece of code works, even though I kinda wrote it down on my own...
Could you explain how does it work?
P.S.: if you look closer, ToHierarchy is not near the same as .Include("Details").
It works because set.ToList will load all records from the database table to your application and the rest is done be EF and its change tracking mechanism which should ensure correct referencing between related entities.
Btw. you are filtering records in the memory of your application, not in the database. For example if your table contains 10000 records and your filter should return only 10, you will still load all 10000 from the database.
You will find that implementing this with EF is quite hard because EF has no support for hierarchical data. You will always end with bad solution. The only good solution is using stored procedure and some support for hierarchical queries in the database - for example common table expressions (CTE) in SQL server.
I just made this very simple example and it works as I described in comment:
public class SelfReferencing
{
public int Id { get; set; }
public string Name { get; set; }
public SelfReferencing Parent { get; set; }
public ICollection<SelfReferencing> Children { get; set; }
}
public class Context : DbContext
{
public DbSet<SelfReferencing> SelfReferencings { get; set; }
}
public class Program
{
static void Main(string[] args)
{
using (var context = new Context())
{
context.Database.Delete();
context.Database.CreateIfNotExists();
context.SelfReferencings.Add(
new SelfReferencing()
{
Name = "A",
Children = new List<SelfReferencing>()
{
new SelfReferencing()
{
Name = "AA",
Children = new List<SelfReferencing>()
{
new SelfReferencing()
{
Name = "AAA"
}
}
},
new SelfReferencing()
{
Name = "AB",
Children = new List<SelfReferencing>()
{
new SelfReferencing()
{
Name = "ABA"
}
}
}
}
});
context.SaveChanges();
}
using (var context = new Context())
{
context.Configuration.LazyLoadingEnabled = false;
context.Configuration.ProxyCreationEnabled = false;
var data = context.SelfReferencings.ToList();
}
}
}
It uses code first approach but internally it is same as when using EDMX. When I get data I have 5 entities in list and all have correctly configured Parent and Children navigation properties.