How to use a dynamically-determined Type as parameter to a Lambda<Func<>>? - dynamic

I am dynamically creating a Lambda expression (based on user input but at the moment using dummy values for a proof-of-concept) for a type which I will only know at runtime. I therefore need to pass the T portion of the Func<T,TResult> as a dynamic type, since I won't know the type until runtime (TResult will always be a bool).
It seems that I cannot pass in a Type variable or use typeof with generics. Basically I'm trying to do something like this:
// (f => f.Baz == 1)
Type theType = Type.GetType("Foo");
ParameterExpression pe = Expression.Parameter(theType, "f");
Expression left = Expression.Property(pe, theType.GetProperty("Baz"));
Expression right = Expression.Constant(1);
Expression expr = Expression.Equal(left, right);
// This works fine but uses a hard-coded type, which I won't know until runtime:
// var lambda = Expression.Lambda<Func<Foo,bool>>(expr, new ParameterExpression[] { pe }).Compile();
var lambda = Expression.Lambda<Func<theType, bool>>(expr, new ParameterExpression[] { pe }).Compile();
However, I cannot use the variable theType as the T portion of the Func. How can I fix this?

No you can't.
For example, in C#, you can't:
Type t = typeof(int);
List<t> list = new List<t>();
or
object list = new List<t>();
Unless you use reflection, but then you have to put the list in an object, and you can use it only through reflection.
So if you want you can save your Func<> in an object (or a dynamic) but nothing more.
What you COULD do is always return Func<object, bool> and cast the object to the desidered type IN the lambda function (so use a Expression.Convert(pe, theType));
Or you could use the dynamic:
// lambda == Func<Foo, bool>
dynamic lamdba = Expression.Lambda(expr, new ParameterExpression[] { pe }).Compile();
bool res = lambda(myvalue);
or
// lambda == Func<Foo, bool>
Delegate lamdba = Expression.Lambda(expr, new ParameterExpression[] { pe }).Compile();
bool res = (bool)lambda2.DynamicInvoke(t);
To be taken "not as real" some benchmarks (in StopWatch.Ticks, look at them only for proportions) (Release Mode + Start Without Debugging + some useless cycles so that they are "hot"):
236384685 dynamic
56773593 Func<object, bool> + cast
10556024247 DynamicInvoke
as a note, Func<Foo, bool> has the same speed, so there isn't any speed lost in the extra cast.
You can see the code here http://ideone.com/qhnVP3

Related

jooq query using bind variables

I using bind variable to do a batch update using below code
`var balanceUpdate = dslContext.batch(
dslContext.update(BALANCE)
.set(BALANCE.BALANCE, (BigDecimal) null)
.where(BALANCE.ID.eq((String) null)));
balances.forEach(balance -> {
balanceUpdate.bind(
balance.getAmount()
balance.Id);
});
int[] execute = balanceUpdate.execute();
`
Above code work well, but now i want to use bind with array of arguments like
var balanceUpdate = dslContext.batch(
dslContext.update(BALANCE)
.set(BALANCE.BALANCE, (BigDecimal) null)
.where(BALANCE.ID.eq((String) null)));
var arguments = balances.stream()
.map(balance ->
new Object[] {
balance.getAmount(),
balance.Id
}).collect(Collectors.toList());
int[] execute = balanceUpdate.bind(arguments).execute();
I get exception
java.lang.ArrayStoreException: arraycopy: element type mismatch: can not cast one of the elements of java.lang.Object[] to the type of the destination array, java.math.BigDecimal
at org.jooq_3.14.4.ORACLE12C.debug(Unknown Source)
at java.base/java.util.Arrays.copyOf(Arrays.java:3722)
at org.jooq.tools.Convert.convertArray(Convert.java:357)
at org.jooq.tools.Convert.convertArray(Convert.java:345)
at org.jooq.tools.Convert$ConvertAll.from(Convert.java:603)
at org.jooq.tools.Convert.convert0(Convert.java:392)
at org.jooq.tools.Convert.convert(Convert.java:384)
at org.jooq.tools.Convert.convert(Convert.java:458)
at org.jooq.tools.Convert.convertArray(Convert.java:363)
at org.jooq.tools.Convert.convertArray(Convert.java:345)
at org.jooq.tools.Convert$ConvertAll.from(Convert.java:614)
at org.jooq.tools.Convert.convert0(Convert.java:392)
at org.jooq.tools.Convert.convert(Convert.java:384)
at org.jooq.tools.Convert.convert(Convert.java:458)
at org.jooq.impl.AbstractDataType.convert(AbstractDataType.java:534)
at org.jooq.impl.DefaultDataType.convert(DefaultDataType.java:86)
at org.jooq.impl.DSL.val(DSL.java:24409)
at org.jooq.impl.DSL.val(DSL.java:24377)
at org.jooq.impl.Tools.field(Tools.java:1794)
at org.jooq.impl.Tools.fields(Tools.java:1865)
at org.jooq.impl.BatchSingle.executePrepared(BatchSingle.java:226)
at org.jooq.impl.BatchSingle.execute(BatchSingle.java:170)
According docs it should work ? Atleast it works without explicit casting when using jdbc. Is there anyway to get it work to sent bind variables only once instead of many times like in first example?
I think you're calling the wrong BatchBindStep.bind(Object...) method, or at least not in the way you're expecting. There's currently no overload accepting a collection of type List<Object[]>. So, what you should do instead is create an Object[][] type for your bind variable sets:
Object[][] arguments = balances
.stream()
.map(balance -> new Object[] {
balance.getAmount(),
balance.Id
}).toArray();

Another failure at deserializing data with discriminated unions, in F#

Following a question where the answer provided a working solution to serialize / deserialize discriminated unions (IgnoreMissingMember setting doesn't seem to work with FSharpLu.Json deserializer)
I have now a practical case where this fails (although it works in simpler cases).
here is the test code:
open System.Collections.Generic
open Microsoft.FSharpLu.Json
open Newtonsoft.Json
open Newtonsoft.Json.Serialization
// set up the serialization / deserialization based on answer from:
// https://stackoverflow.com/questions/62364229/ignoremissingmember-setting-doesnt-seem-to-work-with-fsharplu-json-deserializer/62364913#62364913
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true) |]
)
let serialize object =
JsonConvert.SerializeObject(object, settings)
let deserialize<'a> object =
JsonConvert.DeserializeObject<'a>(object, settings)
// define the type used
type BookSide =
| Bid
| Ask
type BookEntry =
{
S : float
P : float
}
type BookSideData =
Dictionary<int, BookEntry>
type BookData =
{
Data: Dictionary<BookSide, BookSideData>
}
static member empty =
{
Data = Dictionary<BookSide, BookSideData>(dict [ (BookSide.Bid, BookSideData()); (BookSide.Ask, BookSideData()) ])
}
// make some sample data
let bookEntry = { S=3.; P=5. }
let bookData = BookData.empty
bookData.Data.[BookSide.Bid].Add(1, bookEntry)
// serialize. This part works
let s = serialize bookData
// deserialize. This part fails
deserialize<BookData> s
the serialized test data will look like this:
{"Data":{"Bid":{"1":{"S":3.0,"P":5.0}},"Ask":{}}}
but deserializing will crash like this:
Could not convert string 'Bid' to dictionary key type 'FSI_0023+BookSide'. Create a TypeConverter to convert from the string to the key type object.
although the serialization / deserialization of the DU through FSharpLu which has a DU converter.
The reason I am trying to find some automated solution, vs writing a custom TypeConverter (besides the fact I've never done it) is that I have a lot of types I do not control to go through.
Here is a fiddle:
https://dotnetfiddle.net/Sx0k4x
Your basic problem is that you are using BookSide as a dictionary key -- but this is an f# union which makes it a complex key -- one not immediately convertible to and from a string. Unfortunately Json.NET does not support complex dictionary keys out of the box as is stated in its Serialization Guide:
When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString() for the key type or by implementing a TypeConverter. A TypeConverter will also support converting a custom string back again when deserializing a dictionary.
There are two basic approaches to handling this issue:
Implement a TypeConverter as is shown in, e.g., Not ableTo Serialize Dictionary with Complex key using Json.net.
Serialize the dictionary as an array of key/value pair objects e.g. as is shown in Serialize dictionary as array (of key value pairs).
Since your data model includes dictionaries with a variety of keys (DU, strings and ints) the second solution would appear to be the only possibility. The following DictionaryConverter should have the necessary logic:
let inline isNull (x:^T when ^T : not struct) = obj.ReferenceEquals (x, null)
type Type with
member t.BaseTypesAndSelf() =
t |> Seq.unfold (fun state -> if isNull state then None else Some(state, state.BaseType))
member t.DictionaryKeyValueTypes() =
t.BaseTypesAndSelf()
|> Seq.filter (fun i -> i.IsGenericType && i.GetGenericTypeDefinition() = typedefof<Dictionary<_,_>>)
|> Seq.map (fun i -> i.GetGenericArguments())
type JsonReader with
member r.ReadAndAssert() =
if not (r.Read()) then raise (JsonReaderException("Unexpected end of JSON stream."))
r
member r.MoveToContentAndAssert() =
if r.TokenType = JsonToken.None then r.ReadAndAssert() |> ignore
while r.TokenType = JsonToken.Comment do r.ReadAndAssert() |> ignore
r
type internal DictionaryReadOnlySurrogate<'TKey, 'TValue>(i : IDictionary<'TKey, 'TValue>) =
interface IReadOnlyDictionary<'TKey, 'TValue> with
member this.ContainsKey(key) = i.ContainsKey(key)
member this.TryGetValue(key, value) = i.TryGetValue(key, &value)
member this.Item with get(index) = i.[index]
member this.Keys = i.Keys :> IEnumerable<'TKey>
member this.Values = i.Values :> IEnumerable<'TValue>
member this.Count = i.Count
member this.GetEnumerator() = i.GetEnumerator()
member this.GetEnumerator() = i.GetEnumerator() :> IEnumerator
type DictionaryConverter () =
// ReadJson adapted from this answer https://stackoverflow.com/a/28633769/3744182
// To https://stackoverflow.com/questions/28451990/newtonsoft-json-deserialize-dictionary-as-key-value-list-from-datacontractjsonse
// By https://stackoverflow.com/users/3744182/dbc
inherit JsonConverter()
override this.CanConvert(t) = (t.DictionaryKeyValueTypes().Count() = 1) // If ever implemented for IReadOnlyDictionary<'TKey, 'TValue> then reject DictionaryReadOnlySurrogate<'TKey, 'TValue>
member private this.ReadJsonGeneric<'TKey, 'TValue> (reader : JsonReader, t : Type, existingValue : obj, serializer : JsonSerializer) : obj =
let contract = serializer.ContractResolver.ResolveContract(t)
let dict = if (existingValue :? IDictionary<'TKey, 'TValue>) then existingValue :?> IDictionary<'TKey, 'TValue> else contract.DefaultCreator.Invoke() :?> IDictionary<'TKey, 'TValue>
match reader.MoveToContentAndAssert().TokenType with
| JsonToken.StartArray ->
let l = serializer.Deserialize<List<KeyValuePair<'TKey, 'TValue>>>(reader)
for p in l do dict.Add(p)
dict :> obj
| JsonToken.StartObject ->
serializer.Populate(reader, dict)
dict :> obj
| JsonToken.Null -> null // Or throw an exception if you prefer
| _ -> raise (JsonSerializationException(String.Format("Unexpected token {0}", reader.TokenType)))
override this.ReadJson(reader, t, existingValue, serializer) =
let keyValueTypes = t.DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("ReadJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| reader; t; existingValue; serializer |])
member private this.WriteJsonGeneric<'TKey, 'TValue> (writer : JsonWriter, value : obj, serializer : JsonSerializer) =
let dict = value :?> IDictionary<'TKey, 'TValue>
let keyContract = serializer.ContractResolver.ResolveContract(typeof<'Key>)
// Wrap the value in an enumerator or read-only surrogate to prevent infinite recursion.
match keyContract with
| :? JsonPrimitiveContract -> serializer.Serialize(writer, new DictionaryReadOnlySurrogate<'TKey, 'TValue>(dict))
| _ -> serializer.Serialize(writer, seq { yield! dict })
()
override this.WriteJson(writer, value, serializer) =
let keyValueTypes = value.GetType().DictionaryKeyValueTypes().Single(); // Throws an exception if not exactly one.
let m = typeof<DictionaryConverter>.GetMethod("WriteJsonGeneric", BindingFlags.NonPublic ||| BindingFlags.Instance ||| BindingFlags.Public);
m.MakeGenericMethod(keyValueTypes).Invoke(this, [| writer; value; serializer |])
()
Which you would add to settings as follows:
let settings =
JsonSerializerSettings(
NullValueHandling = NullValueHandling.Ignore,
Converters = [| CompactUnionJsonConverter(true, true); DictionaryConverter() |]
)
And generates the following JSON for your bookData:
{
"Data": [
{
"Key": "Bid",
"Value": [
{
"Key": 1,
"Value": {
"S": 3.0,
"P": 5.0
}
}
]
},
{
"Key": "Ask",
"Value": []
}
]
}
Notes:
The converter works for all Dictionary<TKey, TValue> types (and subtypes).
The converter detects whether the dictionary keys will be serialized using a primitive contract, and if so, serializes the dictionary compactly as a JSON object. If not the dictionary is serialized as an array. You can observe this in the JSON shown above: the Dictionary<BookSide, BookSideData> dictionary is serialized as a JSON array, and the Dictionary<int, BookEntry> dictionary is serialized as a JSON object.
During deserialization the converter detects whether the incoming JSON value is an array or object, and adapts as required.
The converter is only implemented for the mutable .Net Dictionary<TKey, TValue> type. The logic would require some slight modification to deserialize the immutable Map<'Key,'Value> type.
Demo fiddle here.

Dynamic parameter in parsed expression

A quick sample to understand my situation:
static Interpreter ParseInterpreter = new Interpreter();
...
var func = ParseInterpreter.Parse("ctx.SomeProp", new Parameter("ctx", typeof(???1)).Compile<Func<???2, object>>;
...
var token = JToken.Parse(s);
dynamic dToken = token;
var obj = func(dToken);
In other words, is there a way to pass some dynamic parameter to Parse method and then get a functor which accepts such parameters?
You can directly use the Lambda class returned by the Parse method, and not call the Compile function:
var interpreter = new Interpreter()
string expression = "ctx.SomeProp";
Lambda parsedExpression = interpreter.Parse(expression, new Parameter("ctx", typeof(object)));
var token = JToken.Parse(s);
var result = parsedExpression.Invoke(token);
I have not tested exactly your code but for example I have a test like this that works correctly:
dynamic dyn = new ExpandoObject();
dyn.Foo = "bar";
var interpreter = new Interpreter()
.SetVariable("dyn", (object)dyn);
Assert.AreEqual(dyn.Foo, interpreter.Eval("dyn.Foo"));
Consider that this only works on .NET 4.x, on .NET Standard/Core dynamics are not supported.

How to re-filter a sequence asynchrounously in Rx

I have an observable sequence that should be filtered using the Where operator based on some criteria, and should populate a list with those filtered elements.
I want to change the filtering criteria on the fly, so when this happens, the list should be cleared, the previous elements of the observable sequence should be evaluated for the new criteria, as well as new elements produced by the sequence.
The list should be repopulated with past elements which now passes the new criteria, continuing with new elements (filtered as well). I don't care that new elements might be delayed, while previous elements are re-evaluated, but i do care about order.
Is this something that can be done with Reactive Extensions?
Here's a nice extension method that does what you need:
public static IObservable<T> Refilterable<T>(
this IObservable<T> source, IObservable<Func<T, bool>> filters)
{
return
Observable
.Create<T>(o =>
{
var replay = new ReplaySubject<T>();
var replaySubscription = source.Subscribe(replay);
var query = filters.Select(f => replay.Where(f)).Switch();
var querySubscription = query.Subscribe(o);
return new CompositeDisposable(replaySubscription, querySubscription);
});
}
I tested this with this code:
var source = new Subject<int>();
var filters = new Subject<Func<int, bool>>();
var subscription = source.Refilterable(filters).Subscribe(x => Console.WriteLine(x));
source.OnNext(1);
source.OnNext(2);
source.OnNext(3);
filters.OnNext(x => x % 2 == 0);
source.OnNext(4);
source.OnNext(5);
filters.OnNext(x => x % 2 == 1);
source.OnNext(6);
filters.OnNext(x => x % 3 == 0);
source.OnNext(7);
filters.OnNext(x => x % 2 == 1);
subscription.Dispose();
filters.OnNext(x => x % 2 == 0);
I got this output:
2
4
1
3
5
3
6
1
3
5
7
Which seems to be what you're after.
I just noticed the requirement to produce lists. Here's an update:
public static IObservable<IList<T>> Refilterable<T>(this IObservable<T> source, IObservable<Func<T, bool>> filters)
{
return
Observable
.Create<IList<T>>(o =>
{
var replay = new ReplaySubject<T>();
var replaySubscription = source.Subscribe(replay);
var query =
filters
.Select(f =>
replay
.Synchronize()
.Where(f)
.Scan(new List<T>(), (a, x) =>
{
a.Add(x);
return a;
}))
.Switch();
var querySubscription = query.Subscribe(o);
return new CompositeDisposable(replaySubscription, querySubscription);
});
}
The only other thing I noticed was the VB.NET tag. I'll see if I can convert later if need be.
This should be the right VB:
<System.Runtime.CompilerServices.Extension> _
Public Shared Function Refilterable(Of T)(source As IObservable(Of T), filters As IObservable(Of Func(Of T, Boolean))) As IObservable(Of IList(Of T))
Return Observable.Create(Of IList(Of T))( _
Function(o)
Dim replay = New ReplaySubject(Of T)()
Dim replaySubscription = source.Subscribe(replay)
Dim query = filters.[Select](Function(f) replay.Synchronize().Where(f).Scan(New List(Of T)(), _
Function(a, x)
a.Add(x)
Return a
End Function)).Switch()
Dim querySubscription = query.Subscribe(o)
Return New CompositeDisposable(replaySubscription, querySubscription)
End Function)
End Function
Unless I missed something crucial, the specified requirements eliminate the need for immutable collections and simplify the implementation because you need to buffer all emitted values.
private List<T> values = new List<T>();
private IObservable<T> _valueSource;
public List<T> ValidValues => values.Where(MatchesCriteria).ToList();
private void StartSubscriptions()
{
var addNewValuesSub = _valueSource.Subscribe(values.Add); //todo disposing
}
If you believe that IEnumerable.Where is too slow and we know all possible criterias beforehand. We could GroupBy separate values to their respective Observables / data structures. It would look something like this.
_valueSource.GroupBy(CriteriaSelector)
.Subscribe(i => UpdateDataStructure(i.Key(), i.Latest()) );
IObservable is not good for buffering lots of values which should later be accessed by some criteria. That's a job for IEnumerable.
Finally, if you think memory usage will be an issue consider refactoring values as a memory object caching system.
I have two initial reactions
That is just a .Replay() that is resubscribed to when the filter condition changes
That also sounds like an unbounded buffer :-/
To address my first thought, you could just have this code
public class Foo
{
private readonly IDisposable _replayConnection;
private readonly IConnectableObservable<int> _replaySource;
private readonly SerialDisposable _subscription = new SerialDisposable();
private readonly List<int> _values = new List<int>();
//the Ctor or some initialise method...
public Foo(IObservable<int> source)
{
_replaySource = source.Replay();
_replayConnection = _replaySource.Connect()
}
public void SetFilter(Func<int, bool> predicate)
{
//Not thread safe. If required, then a scheduler can solve that.
_values.Clear();
_subscription.Disposable = _replaySource.Where(predicate)
.Subscribe(value => _values.Add(value),
ex => {/*Do something here to handle errors*/},
() => {/*Do something here if you want to cater for completion of the sequence*/},
}
}
However, this just makes me more concerned about point 2. If you are expecting millions of values, then if they are just ints, then it is about 3MB/1mil items of memory you will be using. If they are ints, then copy by value semantics, you will get copies in the Replay buffer and in your final list (IIRC). If this kind of memory pressure is ok, then I think the Replay code above will be fine. Also note that this Replay usage will throw if you try to buffer more than int.MaxValue values.

dynamic linq contains

i search for a dynamic linq-to-sql Contains (StartsWith / EndsWith) method.
I’ve tried the following code, but it didn't work.
Any ideas?
public static IQueryable<T> WhereContains<T, V>(this IQueryable<T> queryable, string propertyName, V propertyValue)
{
ParameterExpression pe = Expression.Parameter(typeof(T), "p");
Expression left = Expression.Property(pe, propertyName);
Expression right = Expression.Constant(propertyValue, typeof(V));
IQueryable<T> x = queryable.Where<T>(
Expression.Lambda<Func<T, bool>>(
Expression.Call(
typeof(T).GetMethod("Contains"),
left,
right),
new ParameterExpression[] { pe }));
return x;
}
LINQ-to-SQL knows how to translate StartsWith, EndsWith and Contains for strings.
For example:
View.Customers = from c in db.Customers
where c.ContactName.EndsWith("c")
orderby c.CompanyName
select c;
Ref: http://davidhayden.com/blog/dave/archive/2007/11/23/LINQToSQLLIKEOperatorGeneratingLIKESQLServer.aspx
If you are trying to achieve a sort of Reflection style anything that has a Contains Method that takes two parameters, you're going to have a harder time...