How to add post processing to .Net Core OData? - asp.net-core

I have an OData Controller which looks pretty standard.
[HttpGet]
[ODataRoute("GridData")]
[EnableQuery]
public async Task<IQueryable<GridData>> GetGridData(ODataQueryOptions<GridData> odataOptions)
{
var query = odataOptions.ApplyTo(_service.GetGridDataQueryable()) as IQueryable<GridData>
return query;
}
Projection looks like this :
.Select(async x =>
{
//Pretty resource heavy
x.Ownership = await _ownershipService.ComputeAsync(_currentUser));
return x;
})
.Select(t => t.Result)
.ToList();
Now the problem is that I need to actually return a GridDataDTO object from this call. There is some processing that cannot be done at the database level. The processing is pretty heavy so I would not like to add it inside the GetGridDataQueryable().Also the processing is async, and need a materialized result set to be able to apply it.
I also need to return the IQueryable in the controller to be able to benefit from $count, $select, etc .
This hooks up to a pretty complex grid with a lot of options for filtering/sorting so I would not like to remove the OData functionality.
Is there a simple way to add postprocessing here ? After the result is materialized, project it to my GridDataDTO ?
There is no need for insert/update/delete support, as this will be only used for read operations.

There is no requirement for your controller method to only pass through a query from the database, in fact your method does not need to return an IQueryable<T> result at all!
You can still benefit from OData $select, $expand and $filter operators on result sets that are not IQueryable<T>, but you lose most of the performance benefits of doing so and you have to prepare you data so that the operators can be processed, and you will have to explicitly decorate your endpoint with the [EnableQuery] attribute.
In the following example you current query is materialized into memory, after applying the query options, then we can iterate over the set and manipulate it as we need to.
In the end the same recordset, with the modified records is returned, cast as queryable to match the method signature, however the method would still function the same if the result was IEnumerable<T>
There is a strong argument that says you should return IEnumerable<T> because it conveys the correct information that the recordset has been materialized and is not deferred.
[HttpGet]
[ODataRoute("GridData")]
public async Task<IQueryable<GridDataDTO>> GetGridData(ODataQueryOptions<GridData> odataOptions)
{
// NOTE: GridDataDTO : GridData
// apply $filter, $top and $skip to the DB query
IQueryable<GridData> query = odataOptions.ApplyTo(_service.GetGridDataQueryable());
// materialize
var list = query.ToList();
// project into DTO
List<GridDataDTO> output = list.Select(async x =>
{
var o = new GridDataDTO(x);
o.Ownership = await _ownershipService.ComputeAsync(_currentUser));
}).ToList();
// return, as Queryable
return output.AsQueryable();
}
UPDATE:
When the manipulations involve projection into a new type, then to properly support OData query options the type defined in your ODataQueryOptions<> needs to be assignable from the output element type. You can do this through inheritance or with implicit cast definitions.
If an explicit cast is required (or no cast is available at all) then you will have to manually validate the ApplyTo logic, the ODataQueryOptions must be a valid type reference to match the output.

Related

Efficient way to bring parameters into controller action URL's

In ASP.Net Core you have multiple ways to generate an URL for controller action, the newest being tag helpers.
Using tag-helpers for GET-requests asp-route is used to specify route parameters. It is from what I understand not supported to use complex objects in route request. And sometimes a page could have many different links pointing to itself, possible with minor addition to the URL for each link.
To me it seems wrong that any modification to controller action signature requires changing all tag-helpers using that action. I.e. if one adds string query to controller, one must add query to model and add asp-route-query="#Model.Query" 20 different places spread across cshtml-files. Using this approach is setting the code up for future bugs.
Is there a more elegant way of handling this? For example some way of having a Request object? (I.e. request object from controller can be put into Model and fed back into action URL.)
In my other answer I found a way to provide request object through Model.
From the SO article #tseng provided I found a smaller solution. This one does not use a request object in Model, but retains all route parameters unless explicitly overridden. It won't allow you to specify route through an request object, which is most often not what you want anyway. But it solved problem in OP.
<a asp-controller="Test" asp-action="HelloWorld" asp-all-route-data="#Context.GetQueryParameters()" asp-route-somestring="optional override">Link</a>
This requires an extension method to convert query parameters into a dictionary.
public static Dictionary GetQueryParameters(this HttpContext context)
{
return context.Request.Query.ToDictionary(d => d.Key, d => d.Value.ToString());
}
There's a rationale here that I don't think you're getting. GET requests are intentionally simplistic. They are supposed to describe a specific resource. They do no have bodies, because you're not supposed to be passing complex data objects in the first place. That's not how the HTTP protocol is designed.
Additionally, query string params should generally be optional. If some bit of data is required in order to identify the resource, it should be part of the main URI (i.e. the path). As such, neglecting to add something like a query param, should simply result in the full data set being returned instead of some subset defined by the query. Or in the case of something like a search page, it generally will result in a form being presented to the user to collect the query. In other words, you action should account for that param being missing and handle that situation accordingly.
Long and short, no, there is no way "elegant" way to handle this, I suppose, but the reason for that is that there doesn't need to be. If you're designing your routes and actions correctly, it's generally not an issue.
To solve this I'd like to have a request object used as route parameters for anchor TagHelper. This means that all route links are defined in only one location, not throughout solution. Changes made to request object model automatically propagates to URL for <a asp-action>-tags.
The benefit of this is reducing number of places in the code we need to change when changing method signature for a controller action. We localize change to model and action only.
I thought writing a tag-helper for a custom asp-object-route could help. I looked into chaining Taghelpers so mine could run before AnchorTagHelper, but that does not work. Creating instance and nesting them requires me to hardcode all properties of ASP.Net Cores AnchorTagHelper, which may require maintenance in the future. Also considered using a custom method with UrlHelper to build URL, but then TagHelper would not work.
The solution I landed on is to use asp-all-route-data as suggested by #kirk-larkin along with an extension method for serializing to Dictionary. Any asp-all-route-* will override values in asp-all-route-data.
<a asp-controller="Test" asp-action="HelloWorld" asp-all-route-data="#Model.RouteParameters.ToDictionary()" asp-route-somestring="optional override">Link</a>
ASP.Net Core can deserialize complex objects (including lists and child objects).
public IActionResult HelloWorld(HelloWorldRequest request) { }
In the request object (when used) would typically have only a few simple properties. But I thought it would be nice if it supported child objects as well. Serializing object into a Dictionary is usually done using reflection, which can be slow. I figured Newtonsoft.Json would be more optimized than writing simple reflection code myself, and found this implementation ready to go:
public static class ExtensionMethods
{
public static IDictionary ToDictionary(this object metaToken)
{
// From https://geeklearning.io/serialize-an-object-to-an-url-encoded-string-in-csharp/
if (metaToken == null)
{
return null;
}
JToken token = metaToken as JToken;
if (token == null)
{
return ToDictionary(JObject.FromObject(metaToken));
}
if (token.HasValues)
{
var contentData = new Dictionary();
foreach (var child in token.Children().ToList())
{
var childContent = child.ToDictionary();
if (childContent != null)
{
contentData = contentData.Concat(childContent)
.ToDictionary(k => k.Key, v => v.Value);
}
}
return contentData;
}
var jValue = token as JValue;
if (jValue?.Value == null)
{
return null;
}
var value = jValue?.Type == JTokenType.Date ?
jValue?.ToString("o", CultureInfo.InvariantCulture) :
jValue?.ToString(CultureInfo.InvariantCulture);
return new Dictionary { { token.Path, value } };
}
}

Unwrapping breeze Entity properties

I'm very new to breeze/knockout, but I'm 99% of the way to doing what I need to do. I'm using the Hot Towel template, and I'm successfully retrieving a list of items via breeze. The entity (ITBAL) is a database first Entity Framework entity. When I look at the JSON coming back in Fiddler, I see the correct data. The problem is that all of the properties of data.results are dependentobservables, and not the raw values themselves.
We have a custom grid control that is trying to display the data.results array. Because it is not expecting observables, it is simply displaying "function dependentobservable" instead of the value.
I tried to unwrap the object, but keep getting the circular reference error. I don't know why that is, since ITBAL isn't associated with anything.
The data as Fiddler reports it:
[{"$id":"1","$type":"WebUIHtml5HotTowel.Models.ITBAL, WebUIHtml5HotTowel","IBITNO":"A100 ","IBWHID":"1 ","IBITCL":"50","IBITSC":"3 ","IBSUSP":" ","IBVNNO":"100 ","IBPRLC":" ","IBSCLC":" ","IBCCCD":"P","IBPICD":" ","IBSAFL":"Y","IBSTCS":399.99000,"IBUSCS":0.00000,"IBAVCS":414.95214,"IBLCST":7.00000,"IBLCCC":20.0,"IBLCDT":110923.0,"IBLSCC":20.0,"IBLSDT":130111.0,"IBLXCC":19.0,"IBLXDT":990102.0,"IBMXO1":2100.000,"IBMXO2":0.000,"IBMXO3":0.000,"IBMNO1":5.000,"IBMNO2":0.000,"IBMNO3":0.000,"IBFOQ1":0.000,"IBFOQ2":0.000,"IBFOQ3":0.000,"IBOHQ1":327.000,"IBOHQ2":0.000,"IBOHQ3":0.000,"IBAQT1":1576.000,"IBAQT2":0.000,"IBAQT3":0.000,"IBBOQ1":50.000,"IBBOQ2":0.000,"IBBOQ3":0.000,"IBPOQ1":448.000,"IBPOQ2":0.000,"IBPOQ3":0.000,"IBIQT1":1446.000,"IBIQT2":0.000,"IBIQT3":0.000,"IBRMD1":10.000,"IBRMD2":0.000,"IBRMD3":0.000,"IBRYD1":10.000,"IBRYD2":0.000,"IBRYD3":0.000,"IBISM1":0.000,"IBISM2":0.000,"IBISM3":0.000,"IBISY1":0.000,"IBISY2":0.000,"IBISY3":0.000,"IBAMD1":0.000,"IBAMD2":0.000,"IBAMD3":0.000,"IBAYD1":0.000,"IBAYD2":0.000,"IBAYD3":0.000,"IBMMD1":0.000,"IBMMD2":0.000,"IBMMD3":0.000,"IBMYD1":0.000,"IBMYD2":0.000,"IBMYD3":0.000,"IBSMD1":1.0,"IBSMD2":0.0,"IBSMD3":0.0,"IBSYD1":1.0,"IBSYD2":0.0,"IBSYD3":0.0,"IBBLME":335.000,"IBBLYO":2680.000,"IBBLLY":1441.000,"IBNMTY":8.0,"IBNMLY":11.0,"IBQSMD":21.000,"IBQSYD":21.000,"IBQSLY":20.000,"IBISMD":16318.19,"IBISYD":16318.19,"IBISLY":45714.87,"IBCSMD":373.46,"IBCSYD":373.46,"IBCSLY":67.00,"IBDQMD":0.000,"IBDQYD":0.000,"IBDQLY":0.000,"IBDSMD":0.00,"IBDSYD":0.00,"IBDSLY":0.00,"IBDCMD":0.00,"IBDCYD":0.00,"IBDCLY":0.00,"IBNOMD":18.0,"IBNOYD":18.0,"IBNOLY":18.0,"IBPKMD":15.0,"IBPKYD":15.0,"IBPKLY":14.0,"IBINUS":" ","IBIAID":0.0,"IBSAID":0.0,"IBCQT1":1527.000,"IBCQT2":0.000,"IBCQT3":0.000,"IBFCST":"Y","IBDRSH":" ","IBWMIU":"JP","IBFL15":" ","IBUS20":" ","IBLPR1":0.00000,"IBLPR2":0.00000,"IBLPR3":0.00000,"IBLPR4":0.00000,"IBLPR5":0.00000,"IBLPCD":" ","IBABCC":"B","IBPRCL":0.0,"IBQBCL":" ","IBACDC":"Y","IBTDCD":" ","IBDOUM":" ","IBTP01":0.0,"IBTP02":0.0,"IBTP03":0.0,"IBTP04":0.0,"IBLMCC":20.0,"IBLMDT":130513.0,"IBTMPH":"Y","IBCOMC":" ","IBCOMF":0.00000,"IBITCT":" ","IBEOQT":0.000,"IBITCM":0.0,"IBBRVW":" ","IBPTID":" ","IBQTLT":0.0000,"IBCTY1":"AUS","IBCTY2":"AUS","IBTXCD":"1","IBREVS":"Y","IBITXC":" ","IBMNOQ":0.000,"IBSTUS":0.000,"IBUS30":" ","IBPSLN":" ","IBPLIN":"N","IBUPDP":"Y","IBDFII":"2011-08-11T00:00:00.000","IBLHRK":"A","IBPLNC":" "}]
My Controller:
[BreezeController]
public class ItemInquiryController : ApiController
{
readonly EFContextProvider<AplusEntities> _contextProvider = new EFContextProvider<AplusEntities>();
[System.Web.Http.HttpGet]
public string Metadata()
{
return _contextProvider.Metadata();
}
[HttpGet]
public IQueryable<ITBAL> ItemBalances(string itemNumber, string warehouse)
{
return _contextProvider.Context.ITBALs.Where(i => i.IBITNO == itemNumber && i.IBWHID == warehouse)
.OrderBy(i => i.IBWHID)
.ThenBy(i => i.IBITNO);
}
}
The relevant portion from the viewmodel:
var manager = new breeze.EntityManager("api/ItemInquiry");
var store = manager.metadataStore;
var itbalInitializer = function (itbal) {
itbal.CompositeKey = ko.computed(function () {
return itbal.IBITNO() + itbal.IBWHID();
});
};
store.registerEntityTypeCtor("ITBAL", null, itbalInitializer);
var index = "0" + (args.pageNum * args.pageSize);
var query = new breeze.EntityQuery("ItemBalances")
.withParameters({ itemNumber: "A100", warehouse: "1" })
.take(args.pageSize);
if (index > 0) {
query = query.skip(index);
}
manager.executeQuery(query).then(function (data) {
vm.itbals.removeAll();
var itbals = data.results;//[0].Data;
itbals.forEach(function (itbal) {
vm.itbals.push(itbal);
});
vm.totalRecords(1);
itemBalancesGrid.mergeData(vm.itbals(), args.pageNum, parseInt(vm.totalRecords()));
}).fail(function (e) {
logger.log(e, null, loggerSource, true, 'error');
});
I figure I must be missing something fairly simple, but it is escaping me.
UPDATE: I removed the BreezeController attribute from the ApiController, and it works correctly.
Jon, removing the [Breeze] attribute effectively disables breeze for your application so I don't think that is the long term answer to your problem.
If you don't actually want entities for this scenario - you just want data - than a Breeze projection that mentions just the data to display in the grid would seem to be the best choice. Projections return raw data that are not wrapped in KO observables and are not held in the Breeze EntityManager cache.
If you want the data as cached entities and you also want to display them in a grid that doesn't like KO observable properties ... read on.
You can unwrap a KO'd object with ko.toJS. However, the grid is likely to complain about circular references (or throw an "out of memory" exception as some grids do) ... even if the entity has no circular navigation paths. The difficulty stems from the fact that every Breeze entity has an inherent circularity by way of its entityAspect property:
something.entityAspect.entity //'entity' points back to 'something'
Because you are using Knockout for your model library and because you say ITBAL has no navigation properties ("is not related to anything"), I think the following will work for you:
manager.executeQuery(query).then(success) ...
function success(data) {
var unwrapped = ko.toJS(data.results).map(
function(entity) {
delete entity.entityAspect;
return entity;
});
vm.itbals(unwrapped);
vm.totalRecords(1); // huh? What is that "parseInt ..." stuff?
itemBalancesGrid.mergeData(vm.itbals(), args.pageNum, parseInt(vm.totalRecords()));
})
ko.toJS is a Knockout function that recursively unwraps an object or collection of objects, returning copies of values. Then we iterate over the copied object graphs, deleting their entityAspect properties. The array of results is stuffed into the vm.itbals observable and handed along.
You can imagine how to generalize this to remove anything that is giving you trouble.
Extra
What the heck is vm.totalRecords? I sense that this is supposed to be the total number of matching records before paging. You can get that from Breeze by adding .inlineCount() to the breeze query definition. You get the value after the query returns from the data.inlineCount property.
Do you really need vm.itbals()? If all you do here is pass values to the grid, why not do that and cut out the middle man?
The following success callback combines these thoughts
function success(data) {
var unwrapped = ko.toJS(data.results).map(
function(entity) {
delete entity.entityAspect;
return entity;
});
itemBalancesGrid.mergeData(unwrapped, args.pageNum, data.inlineCount);
})

WebAPI and OData - returning Queryable with preconditions

I have a simple GET method, which returns IQueryable, and has some preconditions on query:
[Queryable(HandleNullPropagation = HandleNullPropagationOption.False)]
public IQueryable<Message> Get()
{
using (var session = RavenStore.GetSession())
{
var messages = session.Query<Message>().Where(x => x.TargetUserId == this.User.Identity.Name || x.SourceUserId == this.User.Identity.Name);
return messages;
}
}
This is RavenDB, btw. The issue I'm having is that upon execution the user id is replaced with "[EMPTY_STRING]", so the actual query its running is this:
'TargetUserId:[[EMPTY_STRING]] OR SourceUserId:[[EMPTY_STRING]]' on
index .....
which is obviously wrong.
If I'm returning List instead of IQueriable - it works fine, so something later in the pipeline changes the query. Does anyone have any insight on how to make this work ?
It should work when the values are copied to a local variable first:
var userName = this.User.Identity.Name;
return session.Query<Message>()
.Where(x => x.TargetUserId == userName ||
x.SourceUserId == userName);
This is because by the time the query is executed, the Raven Client query translator can't resolve the objects expressed in the predicate. By copying them into a local variable, you are passing a constant value into the expression.
I believe this is related to closures. Perhaps someone with more direct knowledge of expression trees can explain better in comments.

Using the return type IQueryable<TABLE_1>

I am new to silverlight, many posts indicate using observablecollection is the best.
Domainservice1 returns IQUERYABLE type.
How to work with this return type in
silverlight side?
How to convert/cast the data returned
to observable collection?
The DomainServices1.cs
public IQueryable<TABLE_1> GetTABLE_1()
{
return this.ObjectContext.TABLE_1;
}
*The HOME.XAML.CS***
public Home()
{
InitializeComponent();
this.Title = ApplicationStrings.HomePageTitle;
Web.DomainService1 dservice = new Web.DomainService1();
EntityQuery<Web.TABLE_1> query=new EntityQuery<Web.TABLE_1>();
query = dservice.GetTABLE_1Query();
//Convert result to ObservableCollection
//bind the grid ITEM SOURCE
}
The IQueryable does not return results until you enumerate the collection. so for instance if you wanted to limit the results of that dservice.getTable_1Query with a .where() you could...
to get the object into an observable collection you .tolist the query like this
observablecollection<Table1> t=new observablecollection<Table1>(query.ToList());
I actually think there is a little more that you have to do(a loadoperation is how I do mine)
I am in the learning stages of the linq dynamic, but from other applications that i have had to convert returned results to an observable collection; that is how i did it. I actually wrote an exension so that I could .ToObservableCollection

How to intercept and modify SQL query in Linq to SQL

I was wondering if there is any way to intercept and modify the sql generated from linq to Sql before the query is sent off?
Basically, we have a record security layer, that given a query like 'select * from records' it will modify the query to be something like 'select * from records WHERE [somesecurityfilter]'
I am trying to find the best way to intercept and modify the sql before its executed by the linq to sql provider.
Ok, first to directly answer your question (but read on for words of caution ;)), there is a way, albeit a finicky one, to do what you want.
// IQueryable<Customer> L2S query definition, db is DataContext (AdventureWorks)
var cs = from c in db.Customers
select c;
// extract command and append your stuff
DbCommand dbc = db.GetCommand(cs);
dbc.CommandText += " WHERE MiddleName = 'M.'";
// modify command and execute letting data context map it to IEnumerable<T>
var result = db.ExecuteQuery<Customer>(dbc.CommandText, new object[] { });
Now, the caveats.
You have to know which query is generated so you would know how to modify it, this prolongs development.
It falls out of L2S framework and thus creates a possible gaping hole for sustainable development, if anyone modifies a Linq it will hurt.
If your Linq causes parameters (has a where or other extension causing a WHERE section to appear with constants) it complicates things, you'll have to extract and pass those parameters to ExecuteQuery
All in all, possible but very troublesome. That being said you should consider using .Where() extension as Yaakov suggested. If you want to centrally controll security on object level using this approach you can create an extension to handle it for you
static class MySecurityExtensions
{
public static IQueryable<Customer> ApplySecurity(this IQueryable<Customer> source)
{
return source.Where(x => x.MiddleName == "M.");
}
}
//...
// now apply it to any Customer query
var cs = (from c in db.Customers select c).ApplySecurity();
so if you modify ApplySecurity it will automatically be applied to all linq queries on Customer object.
If you want to intercept the SQL generated by L2S and fiddle with that, your best option is to create a wrapper classes for SqlConnection, SqlCommand, DbProviderFactory etc. Give a wrapped instance of SqlConnection to the L2S datacontext constructor overload that takes a db connection. In the wrapped connection you can replace the DbProviderFactory with your own custom DbProviderFactory-derived class that returns wrapped versions of SqlCommand etc.
E.g.:
//sample wrapped SqlConnection:
public class MySqlConnectionWrapper : SqlConnection
{
private SqlConnecction _sqlConn = null;
public MySqlConnectionWrapper(string connectString)
{
_sqlConn = new SqlConnection(connectString);
}
public override void Open()
{
_sqlConn.Open();
}
//TODO: override everything else and pass on to _sqlConn...
protected override DbProviderFactory DbProviderFactory
{
//todo: return wrapped provider factory...
}
}
When using:
using (SomeDataContext dc = new SomeDataContext(new MySqlConnectionWrapper("connect strng"))
{
var q = from x in dc.SomeTable select x;
//...etc...
}
That said, do you really want to go down that road? You'll need to be able to parse the SQL statements and queries generated by L2S in order to modify them properly. If you can instead modify the linq queries to append whatever you want to add to them, that is probably a better alternative.
Remember that Linq queries are composable, so you can add 'extras' in a separate method if you have something that you want to add to many queries.
first thing come to my mind is to modify the query and return the result in Non-LINQ format
//Get linq-query as datatable-schema
public DataTable ToDataTable(System.Data.Linq.DataContext ctx, object query)
{
if (query == null)
{
throw new ArgumentNullException("query");
}
IDbCommand cmd = ctx.GetCommand((IQueryable)query);
System.Data.SqlClient.SqlDataAdapter adapter = new System.Data.SqlClient.SqlDataAdapter();
adapter.SelectCommand = (System.Data.SqlClient.SqlCommand)cmd;
DataTable dt = new DataTable("sd");
try
{
cmd.Connection.Open();
adapter.FillSchema(dt, SchemaType.Source);
adapter.Fill(dt);
}
finally
{
cmd.Connection.Close();
}
return dt;
}
try to add your condition to the selectCommand and see if it helps.
Try setting up a view in the DB that applies the security filter to the records as needed, and then when retrieving records through L2S. This will ensure that the records that you need will not be returned.
Alternatively, add a .Where() to the query before it is submitted that will apply the security filter. This will allow you to apply the filter programmatically (in case it needs to change based on the scenario).