I haven't messed with OData in a while, but I remember it being really useful. So I've gone for a .NetCore 3.1 EFCore + OData architecture for my API. With a view to make it fully generic etc. etc.
Doing a little test, I can get the expected results from my browser: e.g.
https://localhost:44310/things?someidfield=44
Cool I get back the JSON I was expecting! But it's sooo slow, why? Looking at the SQL (Profiler) I can see it has no WHERE clause, it's getting everything from the database and filtering it in memory, on half a million records?
What am I missing here? I've tried a number of ways to write the GET method. Starting with not passing any queryOptions (which worked! but same result underneath) and then the below where I explicitly apply the options to by EFCore entity.
[HttpGet]
[EnableQuery]
public async Task<IEnumerable<thing>> GetThingsAsync(ODataQueryOptions<thing> queryOptions)
{
return await queryOptions.ApplyTo(DB.thing).Cast<thing>().ToListAsync();
}
The result set is being loaded in memory because you're calling ToListAsync() and returning an IEnumerable.
If your GetThingsAsync method returns an IQueryable<T> (instead of IEnumerable<T>), then the query will be applied to the database and only the filtered data will be fetched.
If DB.thing is an EFCore DbSet (which implements IQueryable<T>), then you can simplify your method as
[HttpGet]
[EnableQuery]
public Task GetThingsAsync()
{
return DB.thing;
}
Furthermore, like some in the comment already mentioned, the correct syntax for filtering in your case would be ?$filter=someidfield eq 44
Related
As is the recommended pattern in EF Core these days when you have a many to many join we do something like this:
Fluent API, many-to-many in Entity Framework Core
Having done that I'm faced with the issue as to how I go about exposing that in the OData model.
Since technically the Entity type definition has no key property (as it uses a composite key the OData framework doesn't like me adding the set to the model.
Whats the recommended approach to this problem?
It seems that EF and OData have become somewhat synced up, what would be even better would be if they could literally share model building code.
To that end I found that in the OData model build after calling AddSet I was able to define the key in the same way as I did in EF ...
Builder.EntityType<UserRole>().HasKey(ac => new { ac.UserId, ac.RoleId });
This is somewhat clean, I have not yet tried posting or directly requesting such a type yet, but expanding from either side of the relationship chain seems to work fine.
EDIT: Moredetails on the definition of this in controller ...
The Url needs to contain both key parts ...
HTTP GET ~/UserRole(123, 456)
the same with PUTs and DELETEs but POST's don't contain the key they are part of the object in the body.
The method signature must contain both key parts, here's some examples ...
public IActionResult Get([FromODataUri]int keyUserId, [FromODataUri]Guid keyRoleId)
{
...
}
public IActionResult Put([FromODataUri]int keyUserId, [FromODataUri]Guid keyRoleId)
{
...
}
public IActionResult Post(UserRole entity)
{
...
}
I have a data source provider :
public class DSProvider
{
public IQueryable<Product> Products
{
get
{
return _repo.Products.AsQueryable();
}
}
}
The repository in the above example currently gets ALL the records (of Products) from DB and then applies the filters, this just does not sound right if you had 50000 requests/sec from a website.How can you limit the repository to just return required info from DB without converting the service to a tightly coupled request option i.e. opposite of what you try to achieve by using oData?
So to summarize I would like to know if its possible to query the DB on the oData options supplied by the user so that my request does not always have to get all products and then apply filters of oData.
I found out after doing a small POC that Entity framework takes care of building dynamic query based on the request.
I doubt this is just specific to NHibernate. But I have code as follows....
public class ClientController : ApiController
{
// GET /api/<controller>
public IQueryable<Api.Client> Get()
{
return Repositories.Clients.Query().Select(c => Mapper.Map<Client, Api.Client>(c));
}
I basically want to Query the database using the Odata criteria.... get the relevant 'Client' objects, and the convert them to the DTO 'Api.Client'.
But... the code as is, doesn't work. Because NHibernate doesn't know what to do the with the Mapper.... It really wants the query to come before the .Select. But I'm not sure I can get the Odata Query first?
It will work if I do
return Repositories.Clients.Query().Select(c => Mapper.Map<Client, Api.Client>(c)).ToList().AsQueryable();
But that's a bit sucky as you have to get ALL the clients from the database to do the OData query on.
Is there anyway to get the "Select" to happen after the OData query? Or another way to approach this?
I did not test it yet but the Open Source project NHibernate.OData could be useful for you.
The problem is that you are trying to execute C# code (Mapper.Map) inside the NH call (that is translated to SQL)
You'd have to map Api.Client manually, or create a Mapper implementation that returns an Expression<Func<Client, Api.Client>> and pass it directly as a parameter to Select().
Even with that, I'm not sure if NHibernate will translate it. But you can try.
AutoMapper supports this scenario with the Queryable Extensions
public IQueryable<Api.Client> Get() {
return Repositories.Clients.Query().Select(c => Mapper.Map<Client, Api.Client>(c));
}
becomes
public IQueryable<Api.Client> Get() {
return Repositories.Clients.Query().ProjectTo<Api.Client>(mapper.ConfigurationProvider);
}
All,
I have a requirement to hide my EF implementation behind a Repository. My simple question: Is there a way to execute a 'find' across both a DbSet AND the DbSet.Local without having to deal with them both.
For example - I have standard repository implementation with Add/Update/Remove/FindById. I break the generic pattern by adding a FindByName method (for demo purposes only :). This gives me the following code:
Client App:
ProductCategoryRepository categoryRepository = new ProductCategoryRepository();
categoryRepository.Add(new ProductCategory { Name = "N" });
var category1 = categoryRepository.FindByName("N");
Implementation
public ProductCategory FindByName(string s)
{
// Assume name is unique for demo
return _legoContext.Categories.Where(c => c.Name == s).SingleOrDefault();
}
In this example, category1 is null.
However, if I implement the FindByName method as:
public ProductCategory FindByName(string s)
{
var t = _legoContext.Categories.Local.Where(c => c.Name == s).SingleOrDefault();
if (t == null)
{
t = _legoContext.Categories.Where(c => c.Name == s).SingleOrDefault();
}
return t;
}
In this case, I get what I expect when querying against both a new entry and one that is only in the database. But this presents a few issues that I am confused over:
1) I would assume (as a user of the repository) that cat2 below is not found. But it is found, and the great part is that cat2.Name is "Goober".
ProductCategoryRepository categoryRepository = new ProductCategoryRepository();
var cat = categoryRepository.FindByName("Technic");
cat.Name = "Goober";
var cat2 = categoryRepository.FindByName("Technic");
2) I would like to return a generic IQueryable from my repository.
It just seems like a lot of work to wrap the calls to the DbSet in a repository. Typically, this means that I've screwed something up. I'd appreciate any insight.
With older versions of EF you had very complicated situations that could arise quite fast due to the required references. In this version I would recomend not exposing IQueryable but ICollections or ILists. This will contain EF in your repository and create a good seperation.
Edit: furthermore, by sending back ICollection IEnumerable or IList you are restraining and controlling the queries being sent to the database. This will also allow you to fine tune and maintain the system with greater ease. By exposing IQueriable, you are exposing yourself to side affects which occur when people add more to the query, .Take() or .Where ... .SelectMany, EF will see these additions and will generate sql to reflect these uncontrolled queries. Not confining the queries can result in queries getting executed from the UI and is more complicated tests and maintenance issues in the long run.
since the point of the repository pattern is to be able to swap them out at will. the details of DbSets should be completly hidden.
I think that you're on a good path. The only thing I probaly ask my self is :
Is the context long lived? if not then do not worry about querying Local. An object that has been Inserted / Deleted should only be accessible once it has been comitted.
if this is a long lived context and you need access to deleted and inserted objects then querying the Local is a good idea, but as you've pointed out, you may run into difficulties at some point.
I am using NHibernate and Rhinomocks and having trouble testing what I want. I would like to test the following repository method without hitting the database (where _session is injected into the repository as ISession):
public class Repository : IRepository
{
(... code snipped for brevity ...)
public T FindBy<T>(Expression<Func<T, bool>> where)
{
return _session.Linq<T>().Where(where).FirstOrDefault();
}
}
My initial approach is to mock ISession, and return an IQueryable stub (hand coded) when Linq is called. I have a IList of Customer objects I would like to query in memeory to test my Linq query code without hitting the db. And I'm not sure what this would look like. Do I write my own implementation of IQueryable? If so, has someone done this for this approach? Or do I need to look at other avenues?
Thanks!
How I've done this test is to not pass the expression to the repository, instead expose IQueryable giving the repository an interface such as:
public interface IRepository<T>
{
IQueryable<T> All();
// whatever else you want
}
Easily implemented like so:
public IQueryable<T> All()
{
return session.Linq<T>();
}
This means that instead of calling your method on the repository like:
var result = repository.FindBy(x => x.Id == 1);
You can do:
var result = repository.All().Where(x => x.Id == 1);
Or the LINQ syntax:
var result = from instance in repository.All()
where instance.Id == 1
select instance;
This then means you can get the same test by mocking the repository out directly which should be easier. You just get the mock to return a list you have created and called AsQueryable() on.
As you have pointed out, the point of this is to let you test the logic of your queries without involving the database which would slow them down dramatically.
From my point of view is this would be considered Integration Testing. NHibernate has it's own tests that it passes and it seems to me like you're trying duplicate some of those tests in your own test suite. I'd either add the NHibernate code and tests to your project and add this there along with their tests, thats if they don't have one very similiar, and use their methods of testing or move this to an Integration testing scenario and hit the database.
If it's just the fact you don't want to have to setup a database to test against you're in luck since you're using NHibernate. With some googling you can find quite a few examples of how to use SQLite to "kinda" do integration testing with the database but keep it in memory.