LinQ to SQL and CLR User Defined Types - sql

I have created a User Defined Type in .Net 3.5 as per my blog entry at :
http://jwsadlerdesign.blogspot.com/2009/04/this-is-how-you-register.html
This works fine when using SQL with technologies like nHibernate.
However, when I try to map my LinQ to SQL class to use this UDT (with attribute defintions not XML), and I setup the property as the enumeration. I cannot get LinQ to map to this type. I have tried Image, Binary, varchar and integer all of which seem to issue Invalid Cast errors.
In particular I get the error 'Unable to cast object of type 'ISTD.InstallManager.Common.Classes.SQLUDTTargetType' to type 'System.Byte[]' any ideas or help would be much appreciated.
James.

UPDATE: I ran into this myself recently and found that the previous solution wasn't quite complete. Despite what all of the documentation says, it is possible to do this, but somewhat painful.
The first step, for your own convenience, is to implement some conversion operators:
public class MyUDT : INullable, IBinarySerialize
{
// Class implementation would go here
// ...
public static explicit operator MyUDT(byte[] data)
{
using (MemoryStream stream = new MemoryStream(data))
{
using (BinaryReader reader = new BinaryReader(stream))
{
MyUDT result = new MyUDT();
result.Read(reader);
return result;
}
}
}
public static explicit operator byte[](MyUDT x)
{
using (MemoryStream ms = new MemoryStream())
{
using (BinaryWriter writer = new BinaryWriter(ms))
{
x.Write(writer);
}
return ms.ToArray();
}
}
}
Linq to SQL will still flat-out refuse to give you the UDT field, no matter how you declare the property. So you have to give it a binary field instead. You don't need a stored procedure or any custom SQL for this, just add a computed column to your table:
ALTER TABLE MyTable
ADD UDTField_Data AS CAST(UDTField AS varbinary(len))
Where len is whatever your UDT defines in the MaxByteSize attribute.
Now you can finally get access to the column data. You might be tempted to use your UDT as the return type of the new property, thinking that Linq to SQL will find your conversion operator and automatically convert from the byte array; don't bother. Linq to SQL will decide that it's actually a serialized .NET object and spit out a message to the effect of "input stream is not a valid binary format." Instead, you need another layer of indirection:
private MyUDT udtField;
[Column(Name = "UDTField_Data", DbType = "varbinary(len)")]
private byte[] UdtFieldData
{
get { return (byte[])udtField; }
set { udtField = (MyUDT)value; }
}
public MyUDT UdtProperty
{
get { return udtField; }
set { udtField = value; }
}
A few notes to make it clear what's going on here:
The actual field data (udtField) is declared as the UDT itself, not a byte array. The reason for this is that we only want the conversion to happen when loading from or saving to the database. If you had to convert the byte array to the UDT every time you accessed it, it would not only hurt performance, but it would cause inconsistencies if the UDT declares any mutable fields.
The raw byte[] property (UdtFieldData) is declared private, so consumers only see the UDT itself. Linq to SQL will still read it as long as it has the [Column] attribute.
The UdtFieldData property does not declare a storage property. This is critical; if you try to use the UDT field as the storage property, you'll just get the same type conversion error.
Finally, the UdtProperty property is how consumers actually get to access the data. To them it looks like any other property.
It's unfortunate that you have to jump through so many hoops to get this to work, but it does work. You'll probably have difficulties doing this kind of massaging through the Linq surface designer, which is just one of several reasons why I don't use it; better to write the classes yourself and use SqlMetal to help you along if necessary.

Related

SQL-Query returns 'sql-select' string instead of record-set

I'm querying a SQL table via an API and I expect the API to return a <IQueryable> collection of 'Outfit' objects however, the response object is actually just a string containing the sql-select statement.
Questions:
Why is the response object returning the SQL-select statement as a string?
Should the IQueryable<> contain a collection of 'Outfit' objecy-types? And should the object-type 'Outfit' be a C# class OR can I use a generic type like 'object' to hold each 'outfit' in the collection?
//Endpoint setup within the APIcontroller.cs file
[Route("api/getSummerOutfits")]
[HttpGet]
[Authorize]
public string getSummerOutfits()
{
IQueryable<Outfit> outfits = _dbContext.Outfits.Where(outfit => outfit.Type == 'Summer');
return outfits;
}
//Setup for the service within the api.service.ts file
getSummerOutfits(): Observable<Object>
{
return this.httpClient.get('/api/getSummerOutfits').pipe();
}
//A snippet of the response-string when the API is called within Postman
"SELECT \r\n ....... WHERE Outfit.Type = 'Summer'"
I have tried setting the IQueryable<> to contain objects of-type 'outfit' however the response continues to be a string containing the sql-select statement.
The query is declared but never executed.
IQueryable<T> (Remarks section)
Enumeration causes the expression tree associated with an IQueryable object to be executed.
Queries that do not return enumerable results are executed when the Execute method is called.
You have to materialize the query with .ToList() or .AsEnumerable().
public List<Outfit> getSummerOutfits()
{
List<Outfit> outfits = _dbContext.Outfits
.Where(outfit => outfit.Type == 'Summer')
.ToList();
return outfits;
}
While I suggest removing.pipe() as you didn't perform any operation in the response. And return the value of Observable<any[]> or Observable<Outfit[]> if you have write Outfit class/interface.
getSummerOutfits(): Observable<any[]>
{
return this.httpClient.get<any[]>('/api/getSummerOutfits');
}
I'm surprised that even worked. Essentially it passed back an IQueryable<Outfit>.ToString() result.
To return a collection of Outfits Yong Shun's answer covers that using a ToList() and having the return type being an IEnumerable<Outfit>/ ICollection<Outfit>.
As a general rule though I don't recommend passing entities back to views or API, especially for asynchronous AJAX calls as this will typically send far more information than the consumer needs, and potentially opens you up to serialization "traps" with lazy loading.
Instead, define a view model or a DTO, which is a serializable POCO C# object containing just the fields your consumer needs. Then your method becomes:
public IEnumerable<OutfitDto> getSummerOutfits()
{
var outfits = _dbContext.Outfits
.Where(outfit => outfit.Type == 'Summer')
.Select(outfit => new OutfitDto
{
// copy values across here.
}).ToList();
return outfits;
}
This avoids the possibility that a serializer attempts to lazy load navigation properties in Outfit. It reduces the size of the payload by just including the fields that you need, and removes any need to eager-load entire related entities if there are details you want from those. (just reference the related entity properties, EF will build the suitable query) It also avoids confusion especially if you go to pass an outfit DTO back to the server rather than attempting to send what you think is an Entity which is actually nothing more than a serialized JSON object or set of parameters type-cast into an Entity class.

Object fingerprinting: serialization + untouchable legacy code + Getter-only auto-properties = cornered?

I have found myself cornered, so here we go.
Context
I need to produce a fingerprint hash code for object diffing. Comparing the hashes of two sets of objects will need to tell me if there are identical objects with the same hash.
The fingerprint hash must be platform-independent. So I went for MD5 hashing.
I am working with a large Object model code base that is out of my control. All types that I will be passed for this fingerprinting can not be modified by me. I cannot add attribute or constructors or modify anything. That does not exclude that the types will change in the future. So any approach must be programmatic -- I cannot just create a Surrogate class to avoid the problem; at least, not manually.
However, performance is not a concern, so reflection has complete green-light.
In addition, I will need to be able to control the exclusion of properties from the hashing. If I exclude a certain property, two object that have all the properties identical to each other except that one will still need to get the same hash.
Issue: serializing to Byte[] with hands tied on the legacy code
MD5 hashing requires the object to be Serialised in Byte[].
The serialisation requires the class to be marked as [Serializable]. Which I cannot add to the legacy code, and naturally it can not be added at runtime either.
So I went for protobuf-net.
Protobuf rightly fails when encountering types that implement an interface with Getter-only auto-properties:
public interface ISomeInterface
{
double Vpy { get; }
double Vy { get; }
double Vpz { get; }
...
}
Being this Interface implemented by many types, using Surrogates seems also a no-go (impractical, non maintainable).
I would just need to serialize, not to deserialize, so I don't see why the limitation of protobuf-net in this case. I understand protobuf-net would not be able to round-trip if needed, but I don't need to round-trip!
Question
Am I really cornered?
Is there any alternative?
My code
As I said, this works perfectly but only if the objects do not have any property (or nested property) that is a type with a Getter-only auto property.
public static byte[] ToByteArray(this object obj, List<PropertyInfo> exclusionsProps = null)
{
if (exclusionsProps == null)
exclusionsProps = new List<PropertyInfo>();
// Protobuf-net implementation
ProtoBuf.Meta.RuntimeTypeModel model = ProtoBuf.Meta.TypeModel.Create();
AddPropsToModel(model, obj.GetType(), exclusionsProps);
byte[] bytes;
using (var memoryStream = new MemoryStream())
{
model.Serialize(memoryStream, obj);
bytes = memoryStream.GetBuffer();
}
return bytes;
}
public static void AddPropsToModel(ProtoBuf.Meta.RuntimeTypeModel model, Type objType, List<PropertyInfo> exclusionsProps = null)
{
List<PropertyInfo> props = new List<PropertyInfo>();
if (exclusionsProps != null)
props.RemoveAll(pr => exclusionsProps.Exists(t => t.DeclaringType == pr.DeclaringType && t.Name == pr.Name));
props
.Where(prop => prop.PropertyType.IsClass || prop.PropertyType.IsInterface).ToList()
.ForEach(prop =>
{
AddPropsToModel(model, prop.PropertyType, exclusionsProps); //recursive call
}
);
var propsNames = props.Select(p => p.Name).OrderBy(name => name).ToList();
model.Add(objType, true).Add(propsNames.ToArray());
}
Which I will then use as such:
foreach (var obj in objs)
{
byte[] objByte = obj.ToByteArray(exclusionTypes);
using (MD5 md5Hash = MD5.Create())
{
string hash = GetMd5Hash(md5Hash, objByte);
Console.WriteLine(obj.GetType().Name + ": " + hash);
}
}
The simple solution here is to completely sidestep the root cause of your issue.
When you can't modify the existing classes, but you need some modifications for them, the easiest way to do that is to create a new and improved subclass, where the modifications you require are available.
Considering that the legacy codebase apparently will change outside of your control, the only way to deal with these changes is to generate these types at runtime. Luckily C# allows you to emit intermediate language which can solve exactly this problem.
You'd start with the DefineType method available from the ModuleBuilder class. Specifically you want to use the overload taking a String, TypeAttributes and a Type (representing the class you extend)
You pointed out that
If two objects have the same hash, you consider them exact copies of each other
Please realise that a hash has a finite entropy, while the source objects have infinite entropy. Hash collisions are bound to happen. Let's have a look at some examples:
public class Point
{
public int X;
public int Y;
}
public class Coordinate
{
public int X;
public int Y;
}
Let's say we calculate the hash as X ^ Y. Instances of both classes could have the same hash, even though they represent different classes. Even when taking just one of these classes, if we take one instance with X = 1, Y = 2 and the other X = 2, Y = 1, they have the same hash. Sure you could optimize the hash algorithm to mitigate the risk on collissions, but you cannot ensure that such collisions can be avoided at all time.
Instead, I would implement a DeepEquals method. This takes more effort (if writing it yourself). But when implemented correctly, it can ensure two objects to be copies.

How to deserialize polymorphic collections in JsonFX?

My JsonFX serialization code works, but the object that I'm serializing contains a list of polymorphic entities, and they're all deserialized as their base type and not their actual type.
Here's my serialization code:
public static string Serialize(System.Object obj)
{
StringBuilder builder = new StringBuilder();
using (TextWriter textWriter = new StringWriter(builder))
{
JsonWriter writer = new JsonWriter(textWriter);
writer.Write(obj);
return builder.ToString();
}
}
public static T Deserialize<T>(string json)
{
using (TextReader textReader = new StringReader(json))
{
var jsonReader = new JsonReader(textReader);
return jsonReader.Deserialize<T>();
}
}
As you can see it's pretty straightforward. I'm also not decorating my classes with any attributes or anything special to make them serializable. Besides the polymorphic problem, it all just seems to be working properly.
So how can I get my polymorphic types to deserialize properly?.
Thanks.
You need to turn on type hinting. Here's example code (this is relevant to JsonFx v1.4, may or may not work with your version):
StringBuilder result = new StringBuilder(string.Empty);
JsonWriterSettings settings = JsonDataWriter.CreateSettings(true);
settings.TypeHintName = "__type";
JsonWriter writer = new JsonWriter(result, settings);
writer.Write(obj);
return result.ToString();
This will add extra data to your JSON string which looks something like:
"__type": "MyNamespace.MyClass, MyAssembly",
Which means it finds out what derived type it is based on the class name. If you change your class name or namespace name, it won't work anymore. Worse, you can't deserialize from your old JSON text data anymore, unless you mass replace all occurrences of the old class name and replace it with the new.
So you have to be careful with it.
EDIT: Forgot to mention that you have to edit the source code of JsonFx for this to work.
In JsonReader.cs, find the ReadArray method:
Change:
object value = this.Read(arrayItemType, isArrayTypeAHint);
to:
object value = this.Read(null, false);
This will ensure that JsonFx will always attempt to figure out the type of each element in an array/list. If you want this to work for just single variables, well you'd need to do the changes on the appropriate code (haven't tried that).

How do I convert an object from Reflection to a generic collection?

I'm trying to write a Compare method to compare properties in some POCOs using Reflection to ensure that they've been persisted to the database correctly. For example, let's say I have this POCO:
public class NoahsArk
{
public string Owner { get; set; }
public ICollection<Animal> Animals { get; set; }
}
What I want to do is this:
[Test]
public class Saves_Correctly_To_Database()
{
var noahsArk = new NoahsArk { // some setup code here };
db.Save(noahsArk);
var dbNoahsArk = db.Get<NoahsArk>(noahsArk.Id);
Assert.That(Compare(noahsArk, dbNoahsArk), Is.True);
}
The ORM I'm using is NHibernate. My Compare method looks like this so far:
public static bool EqualsProperties<T>(this T x, T y)
{
var xType = x.GetType();
foreach (var property in xType.GetProperties())
{
if (property.GetValue(x, null).Implements(typeof(ICollection<>)))
{
var xValue = property.GetValue(x, null) as ICollection<T>;
var yValue = property.GetValue(y, null) as ICollection<T>;
}
Object.Implements() is an extension method I wrote to determine if a type implements an interface. As you can see, the method is incomplete. The problem I'm running into is that when I use property.GetValue(x, null), it returns an object, and I don't know how to cast it into its specific generic ICollection type. I need to be able to do this so I can use LINQ to do a x.Contains(y) to compare the two collections for equality. Any idea on how to do this?
P.S. I tried using Compare .NET Objects, but it's giving me a null reference exception somewhere deep within NHibernate. It doesn't properly handle how NHibernate proxies the ICollection for lazy loading. To make matters worse, NHibernate modifies the POCO to support lazy-loading, but this is all done at runtime. In the source code, it looks like you're just working with a regular ICollection, but NHibernate changes this to NHibernate.Collections.Generic.PersistentSet at runtime, and this is what's causing the comparer to fail.
Your question is a bit confusing because you don't need the type parameter T in the declaration of your EqualsProperties method. You just need
public static bool EqualsProperties(this object x, object y)
You then go on to use the same parameter T to cast properties of x and y to ICollection<T>; however, the objects in these collections obviously may have a different type than x and y.
Now to answer your question: you don't need to cast to the correct generic type to use the LINQ Contains method. You can do something like this:
xValue = property.GetValue(x, null);
yValue = property.GetValue(y, null);
if (typeof(IEnumerable).IsInstanceOf(x))
{
IEnumerable<object> xEnumerable = (x as IEnumerable).Cast<object>();
IEnumerable<object> yEnumerable = (y as IEnumerable).Cast<object>();
// use any LINQ method you like now
}
You should also make sure you use the LINQ overloads that take an equality comparer, as your domain objects obviously do not override the Equals method themselves. Otherwise you wouldn't be writing this unit testing code to compare them.
Sharp architecture framework use attribute to decor properties which should be taken into the equals method. See the source code of DomainSignatureAttribute class and EntityWithTypedId<>.Equals method.

Encapsulation within class definitions

For example, do you use accessors and mutators within your method definitions or just access the data directly? Sometimes, all the time or when in Rome?
Always try to use accessors, even inside the class. The only time you would want to access state directly and not through the public interface is if for some reason you needed to bypass the validation or other logic contained in the accessor method.
Now if you find yourself in the situation where you do need to bypass that logic you ought to step back and ask yourself whether or not this need betrays a flaw in your design.
Edit: Read Automatic vs Explicit Properties by Eric Lippert in which he delves into this very issue and explains things very clearly. It is about C# specifically but the OOP theory is universal and solid.
Here is an excerpt:
If the reason that motivated the
change from automatically implemented
property to explicitly implemented
property was to change the semantics
of the property then you should
evaluate whether the desired semantics
when accessing the property from
within the class are identical to or
different from the desired semantics
when accessing the property from
outside the class.
If the result of that investigation is
“from within the class, the desired
semantics of accessing this property
are different from the desired
semantics of accessing the property
from the outside”, then your edit has
introduced a bug. You should fix the
bug. If they are the same, then your
edit has not introduced a bug; keep
the implementation the same.
In general, I prefer accessors/mutators. That way, I can change the internal implementation of a class, while the class functions in the same way to an external user (or preexisting code that I dont want to break).
The accessors are designed so that you can add property specific logic. Such as
int Degrees
{
set
{
_degrees = value % 360;
}
}
So, you would always want to access that field through the getter and setter, and that way you can always be certain that the value will never get greater than 360.
If, as Andrew mentioned, you need to skip the validation, then it's quite possible that there is a flaw in the design of the function, or in the design of the validation.
Accessors and Mutators are designed to ensure consistency of your data, so even within your class you should always strive to make sure that there's no possible way of injecting unvalidated data into those fields.
EDIT
See this question as well:
OO Design: Do you use public properties or private fields internally?
I don't tend to share with the outside world the 'innards' of my classes and so my internal needs for data (the private method stuff) tends to not do the same sort of stuff that my public interface does, typically.
It is pretty uncommon that I'll write an accessor/mutator that a private method will call, but I suspect I'm in the minority here. Maybe I should do more of this, but I don't tend to.
Anyway, that's my [patina covered] two cents.
I will often start with private auto properties, then refactor if necessary. I'll refactor to a property with a backing field, then replace the backing field with the "real" store, for instance Session or ViewState for an ASP.NET application.
From:
private int[] Property { get; set; }
to
private int[] _property;
private int[] Property
{
get { return _property; }
set { _property = value; }
}
to
private int[] _property;
private int[] Property
{
get
{
if (_property == null)
{
_property = new int[8];
}
return _property;
}
set { _property = value; }
}
to
private int[] Property
{
get
{
if (ViewState["PropertyKey"] == null)
{
ViewState["PropertyKey"] = new int[8];
}
return (int[]) ViewState["PropertyKey"];
}
set { ViewState["PropertyKey"] = value; }
}
Of course, I use ReSharper, so this takes less time to do than to post about.