How to serialize large nested arrays with protobuf? - serialization

I am using protobuf-net for binary serialization. I am gettig OutOfMemory while serializing Class A. The same object is well serialized with BinaryFormatter.
Below is class example:
[ProtoContract]
class A:
[ProtoMember(1, DataFormat = DataFormat.Group)]
B[] Array1 {get; set;}
....
class B:
[ProtoMember(1)]
string Field1 {get; set;}
[ProtoMember(2)]
string Field1 {get; set;}
[ProtoMember(3, DataFormat = DataFormat.Group)]
C[] Array2 {get; set;} // 20000 elements
....
class C:
[ProtoMember(1)]
string Field1 {get; set;}
[ProtoMember(2)]
string Field1 {get; set;}

Wow. Simply wow. Thanks for asking this question. There was a glitch where-by it was incorrectly not applying group-encoding in a few scenarios, yours included. For protobuf-net, this isn't a biggie since group and string are treated interchangeably, but this is still a bit of an embarrassing glitch, not least because "group" is (as you have used correctly) key in making things forwards-only, for serializing large graphs.
I have fixed this locally and in the source - however, I want to do a bit more stability testing before doing a formal release. If you are happy to build from source, it should work fine now - or I can email you a dll if you want.

Related

Should entities in a AggregateRoot also become an AggregateRoot

I'm trying to setup my domain models around the DDD principles. Right now I have the following class:
class Customer : AggregateRoot
{
public string CustomerReference {get;set;}
public string CustomerName {get;set;}
public string List<Adult> Adults {get;set;}
public string List<Child> Children {get;set;}
}
Creating a Customer is the essence of the application, so I made that class an AggregateRoot. Because a lot of
times the end user will want to find a customer by its CustomerReference key.
But then again, sometimes the end user wants to search by the name of a Child, to find out the Customer info. Or even sometimes
by the name of an Adult.
I'm not sure if that means that I should make the Child and Adult classes also an AggregateRoot? Or should I always
start searching from the Customer AggregateRoot if I want to search by a Child or Adult name?
No - aggregate root members are only accessible via the aggregate root.
however, If you are wanting to manipulate Adult/Child entities on their own, it is likely you don't need the full Adult and Child entities as part of the customer root. If this is the case replace those collections of entities with collections of Id's and rely on the fully entities to be provided to any functions that require their attributes.
class Customer : AggregateRoot
{
public string CustomerReference {get; private set;}
public string CustomerName {get; private set;}
public string IEnrumerable<AdultId> Adults {get; private set;}
public string IEnrumerable<ChildId> Children {get; private set;}
public void RegisterAnAdult(Adult adult) {...}
public void RegisterAChild(Child child) {...}
}
I emphasised If because this looks a little bit off unless your system is huge and Adult and Child can belong to multiple customers. (How do you handle when an child grows up to transition to an adult?)
As #mgonzalezbaile said, don't model your domain based on queries - searching is a whole different thing - model it on business behavior. (For more info start with [http://www.zankavtaskin.com/2016/06/applied-domain-driven-design-ddd-part-7.html])
Finally, in your example the properties are publicly set-able, if this is on purpose then it might be worth stepping back and re-reading the literature on DDD a few more times, public settable properties potentially allow your entity to move to an invalid state, something that DDD tries to avoid.

Why is there extra schema information on DataMember properties of base class when using WCF?

I have some DataContracts defined, one is inherited from the other. These are not the exact classes but there are something like this:
[DataContract]
public class BaseModel
{
[DataMember]
public String Id {get;set;}
}
[DataContract]
public class MyModel : BaseModel
{
[DataMember]
public String Name {get;set;}
}
I am using WCF with the basicHTTPbinding to move data from server to client. When I use fiddler to look at the data being sent it is doing something a bit strange.
In the raw data being returned, when I examine the Name property I can see 'Name' followed by the data. For the Id property I see 'Id http://schema.datacontract.org/2004/07/MyService' followed by the data. I only every see the schema.datacontract.org part for the fields from the base class. Why do only the inherited fields get that schema part? Both classes are in the same namespace. I don't want it on ANY properties as its adding significant size to my response. Is there anyway I can turn it off?
I changed each [DataContract] to include Namespace with the same value, e.g.
[DataContract(Namespace = "http://schemas.datacontract.org/2004/07/MyCode.Service")]
and this has fixed my issue. Not sure why it needed to be specified since they were in the same namespace already but it does work.

Fluent NHibernate automapping: One-to-many entities, many-to-many backend?

My goal is to use NHibernate schema generation along with Fluent NHibernate's automapper to generate my database. I'm having trouble with what I'll call "unidirectional many-to-many relationships."
Many of my entities have localized resources. A single class might look like this:
public class Something {
public virtual int Id {get; private set;}
public virtual Resource Title {get;set;}
public virtual Resource Description {get;set;}
public virtual IList<Resource> Bullets {get;set;}
}
The Resource class doesn't have any references back; these are entirely unidirectional.
public class Resource {
public virtual int Id {get; private set;}
public virtual IList<LocalizedResource> LocalizedResources {get;set;}
// etc.
}
public class LocalizedResource { //
public virtual int Id {get; private set; }
public virtual string CultureCode {get;set;}
public virtual string Value {get;set;}
public virtual Resource Resource {get;set;}
}
Without the IList<Resource>, everything is generated as I'd want -- Resource ID's are in the Title and Description fields. When I add in the IList though, NHibernate adds the field something_id to the Resource table. I understand why it does this, but in this situation it's not a sustainable approach.
What I want is to create a junction table for the bullets. Something like:
CREATE TABLE SomethingBullet (
Id int NOT NULL PRIMARY KEY IDENTITY(1,1),
Something_Id int NOT NULL,
Resource_Id int NOT NULL
)
This way when I add the other twenty-odd entities into the database I won't end up with a ridiculously wide and sparse Resource table.
How do I instruct the Automapper to treat all IList<Resource> properties this way?
Every many-to-many is in fact composed with one-to-many's in object model. If your relationship doesn't need to be bidirectional, just don't map the second side. The mapping on your mapped side is not affected at all:
HasManyToMany(x => x.Bullets).AsSet();
In this case, NHibernate already knows that it needs to generate the intermediate table.
See also this article for many-to-many tips.
:)
The only way I found to make this work with automapping is by constructing your own custom automapping step and replacing the "native" HasManyToManyStep. It's either that or an override, I'm afraid.
I lifted mine off of Samer Abu Rabie, posted here.
The good news is that Samer's code, so far, seems to work flawlessly with my conventions and whatnots, so, once it was in place, it was completely transparent to everything else in my code.
The bad news is that it costs you the ability to have unidirectional one-to-many relationships, as Samer's code assumes that all x-to-many unidirectional relationships are many-to-many. Depending on your model, this may or may not be a good thing.
Presumably, you could code up a different implementation of ShouldMap that would distinguish between what you want to be many-to-many and what you want to be one-to-many, and everything would then work again. Do note that that would require having two custom steps to replace the native HasManyToManyStep, although, again, Samer's code is a good starting point.
Let us know how it goes. :)
Cheers,
J.

NHibernate foreign key set

I have a:
class Garage {
int garageID;
ISet<Car> cars
}
should the cars class have garageID as one of its properties, as in:
class Car {
int carID;
int garageID;
String name;
}
That is how it appears in the database, but wondering if the classes on the many side are supposed to have the foreign key as a property or if the ORM just adds that in when performing the SQL (assuming you can specify it in the mappings file).
In short: No. (and yes the ORM will take care of FK based on your mapping files)
The "Car" table will have the GarageId in it but you should not add it to the Car class.
You can have a bi-directional relationship (although opinions vary on whether these are a good idea).
A bi-directional relationship would make the Car class look like this:
public class Car {
public virtual int Id {get; set;}
public virtual Garage Garage {get; set;} //You can traverse back up to Garage
public virtual string Name {get; set;}
}
If you would like me to post the Fluent/XML maps let me know.

Mapping list of ints in NHibernate

In NHibernate manual I have found mapping like this:
<bag name="Sizes" table="SIZES" order-by="SIZE ASC">
<key column="OWNER"/>
<element column="SIZE" type="Int32"/>
</bag>
I can't help wondering why would anyone want to do something like this? Is there something better about mapping plain integers than creating an entity corresponding to a given integer (Size entity in this scenario) and creating true one-to-many relationship?
The requirements of your business domain that your application operates in will dictate whether you should have a list of ints (a collection of value types), or a list of entities. If there's a good use case for a IList<int> then by all means go for it, and let NHibernate map this association accordingly. Otherwise just remove it.
However, removing it because it seems unfamiliar to you, isn't a valid reason.
I use this a lot in my domain models. Right now I have a lot of 'Twitter Applications' that index tweets based on search 'Keywords', so I have mapped it like this:
public class TwitterApplication
{
public virtual int Id { get; set; }
public virtual string ApplicationName { get; set; }
// other properties (snip)
public virtual ISet<string> Keywords { get; set; }
}
I use this mapping because I know that:
The number of Keywords will be small
(about 4 - 6)
I'm not interested in storing
Keyword DateAdded etc.
I'm not going to be doing Paging or
Querying on the Keywords, just
retrieve them all at the same time,
or not at all
On this basis, I decided mapping it as a collection of strings was appropriate.
The question is not if you want to map it like this, the question is, if you need a list of ints in your model.
When you have a list of ints in your model, then you want to map it like this. You don't want to write complex code in your model, just because of the mapping.
So, do you think it is useful to have a list of ints in a class? Or a list of Guids, enums, doubles?
class Graph
{
IList<double> Values { get; private set; }
}
Doesn't it make sense to you?