design pattern advice: graph -> computation - oop

I have a domain model, persisted in a database, which represents a graph. A graph consists of nodes (e.g. NodeTypeA, NodeTypeB) which are connected via branches. The two generic elements (nodes and branches will have properties). A graph will be sent to a computation engine. To perform computations the engine has to be initialised like so (simplified pseudo code):
Engine Engine = new Engine() ;
Object ID1 = Engine.AddNodeTypeA(TypeA.Property1, TypeA.Property2, …, TypeA.Propertyn);
Object ID2 = Engine.AddNodeTypeB(TypeB.Property1, TypeB.Property2, …, TypeB.Propertyn);
Engine.AddBranch(ID1,ID2);
Finally the computation is performed like this:
Engine.DoSomeComputation();
I am just wondering, if there are any relevant design patterns out there, which help to achieve the above using good design principles. I hope this makes sense. Any feedback would be very much appreciated.

Engine looks like it's breaking the Open/Closed principal and probably the Single Responsibility principal too. If Engine's purpose is to do some computation then it probably shouldn't know how to create ID1 or ID2 from different node types which would be an SRP violation. The names AddNodeTypeA and AddNodeTypeB lead me to think that there could be a C, D, etc. This means that any new NodeType would require the Engine class to change which is where OCP appears to be broken.
The Adapter pattern would resolve both of those concerns
class Engine
{
public void AddBranch(Branch branch1, Branch branch2)
{
//do something
}
public void DoSomeComputation();
{
//do something
}
}
interface BranchAdapter
{
Branch GetBranch();
}
class NodeTypeABranchAdapter : BranchAdapter
{
private readonly TypeA _typeA;
public NodeTypeABranchCreator(TypeA typeA)
{
_typeA = typeA;
}
public Branch GetBranch()
{
//do processing to take _typeA and transform it into Branch
}
}
class NodeTypeBBranchAdapter : BranchAdapter
{
private readonly TypeB _typeB;
public NodeTypeABranchCreator(TypeB typeB)
{
_typeB = typeB;
}
public Branch GetBranch()
{
//do processing to take _typeB and transform it into Branch
}
}
The calling code could then work like so
var engine = new Engine();
Branch id1 = new NodeTypeABranchAdapter(TypeA).GetBranch();
Branch id2 = new NodeTypeBBranchAdapter(TypeB).GetBranch();
engine.AddBranch(id1, id2);
engine.DoSomeComputation();

Related

How to implement a Saga on Topos?

Topos it .NET Event Processing library, similar to Rebus. Unlike Rebus, it is not so much for messages, as for event processing.
Rebus supports Sagas out of the "box", including in terms of persistence, correlation and concurrency. How to implement a Saga on Topos?
If Topos supports Sagas, is there an example of a Saga implementation somewhere?
Topos does not have any kind of built-in sagas, unfortunately.
In Fleet Manager (the Rebus management app that comes with Rebus Pro, and the reason I made Topos) I made a saga-like event processor that uses MongoDB or LiteDB for persistence.
This implementation is completely proprietary though, as it's part of a commercial software product, and it's not quite generic enough to be suited for reuse. I can tell you a little bit about it here anyway, hopefully to give you some inspiration on how you could go about building something like it yourself. 🙂
The event processor is hosted in a Topos consumer, which dispatches all received events to a bunch of "projections", thus implementing the classic event sourced "left-fold" (current_state + event => new_state).
Fleet Manager has projections in two flavors: process managers (i.e. projections that cause other events to be emitted by issuing commands) and views. The two types combined would be what you call a "saga" 🙂
One possible view could be implemented like this (with lots of stuff removed for brevity):
public class QueueInstanceView : ViewInstance<InstancePerQueue>, IExpire, IHaveAccountId, IHaveQueueName, ICanBeHidden
{
public string AccountId { get; set; }
public string QueueName { get; set; }
public DateTime LastActivity { get; set; }
public bool Hidden { get; set; }
protected override void DispatchEvent(AuditEvent auditEvent)
{
if (auditEvent.Body is EntityHidden entityHidden)
{
QueueName ??= entityHidden.Id;
Hidden = !entityHidden.Reverse;
}
else
{
QueueName ??= auditEvent.GetQueueName();
}
LastActivity = auditEvent.GetTime();
}
}
Note how the view class inherits from the generic ViewInstance<> class, closing it with the InstancePerQueue type. The base class keeps track of the ID of the view instance and some other stuff used to implement idempotency, and then InstancePerQueue defines how events are mapped to view instances.
It looks like this:
public class InstancePerQueue : ViewLocator
{
public override string[] GetViewIds(AuditEvent auditEvent)
{
if (auditEvent.Body is EntityHidden entityHidden)
{
if (entityHidden.HasType(EntityTypeNames.Queue))
{
var accountId = auditEvent.GetAccountId();
return new[] { $"{accountId}/{entityHidden.Id}" };
}
return Array.Empty<string>();
}
var queueName = auditEvent.GetQueueNameOrNull();
if (queueName == null) return Array.Empty<string>();
var accountId = auditEvent.GetAccountId();
return new[] { $"{accountId}/{queueName}" };
}
}
thus correlating events with IDs on the form "/" (where "account" in Fleet Manager terminology is basically just an environment, i.e. the queue names get to IDENTIFY the queues within an account).
Of course lots of logic is then implemented in the projection implementations, but while it's lengthy, it's also fairly straightforward.
I hope that this could give you some inspiration on how you might want to approach building "sagas" for Topos. 🙂
Btw. I cannot take credit for this particular design. I was originally exposed to a design very similar to this back in 2013-2014 by Emil Krog Ingerslev, who came up with it for an event-sourced application we were building at d60.
I later imitated all of the moving parts to implement persistent projections for Cirqus, which we used for a couple of event-sourced apps.
And finally I made my current implementation for Fleet Manager, back in 2016 when I needed something similar, only without the aggregate root stuff present in Cirqus, and working on Kafka instead of normal databases.

Returning the class which is a foreign key in the database

I want to ask this question and I tried to search for a while without concrete answers.
I have made a database and used LINQ2SQL to auto-generate the classes needed.
I have set the serialization mode to unidirectional to make sure the classes are being serialized and making the datamembers.
Now, what I want to know is, how I can send the references to the other classes (which has been made through LINQ2SQL).
F.x. I have a Class called Scheduler which is referencing Reservation, and Seat, because Reservation and Seat have foreign keys.
You can see the dbml here:
http://imgur.com/rR6OxDi
The dbml file. This is the model of our database
Also you can see that when I run the WCF test client it does not return the objects of Seats and Reservation.
http://imgur.com/brxNBz7
Hopefully you can all help.
UPDATE
Here is the snippet of the code provided by LINQ2SQL.
This is the fields for the scheduler
[global::System.Data.Linq.Mapping.TableAttribute(Name="dbo.Scheduler")]
[global::System.Runtime.Serialization.DataContractAttribute()]
public partial class Scheduler : INotifyPropertyChanging, INotifyPropertyChanged
{
private static PropertyChangingEventArgs emptyChangingEventArgs = new PropertyChangingEventArgs(String.Empty);
private int _SchID;
private System.Nullable<System.DateTime> _Date;
private System.Nullable<System.TimeSpan> _Starttime;
private System.Nullable<int> _MovieID;
private System.Nullable<int> _HallID;
private EntitySet<Seat> _Seats;
private EntitySet<Reservation> _Reservations;
private EntityRef<Hall> _Hall;
private EntityRef<Movie> _Movie;
private bool serializing;
And here is the snippet part of the code where it references to Reservation and Seat:
[global::System.Data.Linq.Mapping.AssociationAttribute(Name="Scheduler_Seat", Storage="_Seats", ThisKey="SchID", OtherKey="SchedulerID")]
[global::System.Runtime.Serialization.DataMemberAttribute(Order=6, EmitDefaultValue=false)]
public EntitySet<Seat> Seats
{
get
{
if ((this.serializing
&& (this._Seats.HasLoadedOrAssignedValues == false)))
{
return null;
}
return this._Seats;
}
set
{
this._Seats.Assign(value);
}
}
[global::System.Data.Linq.Mapping.AssociationAttribute(Name="Scheduler_Reservation", Storage="_Reservations", ThisKey="SchID", OtherKey="SchedulerID")]
[global::System.Runtime.Serialization.DataMemberAttribute(Order=7, EmitDefaultValue=false)]
public EntitySet<Reservation> Reservations
{
get
{
if ((this.serializing
&& (this._Reservations.HasLoadedOrAssignedValues == false)))
{
return null;
}
return this._Reservations;
}
set
{
this._Reservations.Assign(value);
}
}
Update 2
Here is the Reservation class which LINQ2SQL made:
Here is the fields:
[global::System.Data.Linq.Mapping.TableAttribute(Name="dbo.Reservation")]
[global::System.Runtime.Serialization.DataContractAttribute()]
public partial class Reservation : INotifyPropertyChanging, INotifyPropertyChanged
{
private static PropertyChangingEventArgs emptyChangingEventArgs = new PropertyChangingEventArgs(String.Empty);
private int _ResID;
private System.Nullable<int> _CustomerID;
private System.Nullable<int> _SchedulerID;
private string _Row;
private string _Seat;
private EntityRef<Customer> _Customer;
private EntityRef<Scheduler> _Scheduler;
And here is the Scheduler reference part of the class
[global::System.Data.Linq.Mapping.AssociationAttribute(Name="Scheduler_Reservation", Storage="_Scheduler", ThisKey="SchedulerID", OtherKey="SchID", IsForeignKey=true, DeleteRule="SET DEFAULT")]
public Scheduler Scheduler
{
get
{
return this._Scheduler.Entity;
}
set
{
Scheduler previousValue = this._Scheduler.Entity;
if (((previousValue != value)
|| (this._Scheduler.HasLoadedOrAssignedValue == false)))
{
this.SendPropertyChanging();
if ((previousValue != null))
{
this._Scheduler.Entity = null;
previousValue.Reservations.Remove(this);
}
this._Scheduler.Entity = value;
if ((value != null))
{
value.Reservations.Add(this);
this._SchedulerID = value.SchID;
}
else
{
this._SchedulerID = default(Nullable<int>);
}
this.SendPropertyChanged("Scheduler");
}
}
}
All of these things should lead to where I could get the object like this:
Scheduler[] schedulers = client.GetAllSchedulers();
Reservation reservation = schedulers[0].Reservations.First();
But get this error due to WCF not sending the object, (which you could see in picture one).
Which is this error:
Description: An unhandled exception occurred during the execution of
the current web request. Please review the stack trace for more
information about the error and where it originated in the code.
Exception Details: System.InvalidOperationException: Sequence contains
no elements
UPDATE 3:
Ok so it appears that it works somehow.
I just had to make a join between the Scheduler and Reservation.
Also whenever I debug the code I can see the variables are there. (Due to my reputation I can not post links).
But some of you might recognize the following whenever you try to view a result in debug mode:
"expanding the results view will enumerate the ienumerable c#"
Whenever I do this, it works, but not if I run it in release mode.
Looks like only object types (Reservation,Seat) have null values.
I'm guessing either you are missing DataContract/DataMember attributes in your complex types or you might need to include KnownTypeAttribute
It'd be easier to tell if you could provide some code.
EDIT
What your are talking about later is deferred loading. See this blog for more information on deferred vs immediate loading.
When you expand the IEnumerable in debug mode, that makes the request to retrieve/load the objects.
What your probably want is to load your Reservation,Seat objects along with your Scheduler object. Something like the following:
YourDatabaseContext database = new YourDatabaseContext ())
{
DataLoadOptions options = new DataLoadOptions();
options.LoadWith<Scheduler>(sch=> sch.Reservation);
options.LoadWith<Scheduler>(sch=> sch.Seat);
database.LoadOptions = options;
}
See DataLoadOptions for more details.
If you want to understand deferred execution. See this article for more details.
Quote from the article:
By default LINQ uses deferred query execution. This means when you write a LINQ query it doesn't execute. LINQ queries execute when you 'touch' the query results. This means you can change the underlying collection and run the same query subsequent times in the same scope. Touching the data means accessing the results, for instance in a for loop or by using an aggregate operator like Average or AsParallel on the results.

Use MEF to compose parts but postpone the creation of the parts

As explained in these questions I'm trying to build an application that consists of a host and multiple task processing clients. With some help I have figured out how to discover and serialize part definitions so that I could store those definitions without having to have the actual runtime type loaded.
The next step I want to achieve (or next two steps really) is that I want to split the composition of parts from the actual creation and connection of the objects (represented by those parts). So if I have a set of parts then I would like to be able to do the following thing (in pseudo-code):
public sealed class Host
{
public CreationScript Compose()
{
CreationScript result;
var container = new DelayLoadCompositionContainer(
s => result = s);
container.Compose();
return script;
}
public static void Main()
{
var script = Compose();
// Send the script to the client application
SendToClient(script);
}
}
// Lives inside other application
public sealed class Client
{
public void Load(CreationScript script)
{
var container = new ScriptLoader(script);
container.Load();
}
public static void Main(string scriptText)
{
var script = new CreationScript(scriptText);
Load(script);
}
}
So that way I can compose the parts in the host application, but actually load the code and execute it in the client application. The goal is to put all the smarts of deciding what to load in one location (the host) while the actual work can be done anywhere (by the clients).
Essentially what I'm looking for is some way of getting the ComposablePart graph that MEF implicitly creates.
Now my question is if there are any bits in MEF that would allow me to implement this kind of behaviour? I suspect that the provider model may help me with this but that is a rather large and complex part of MEF so any guidelines would be helpful.
From lots of investigation it seems that is not possible to separate the composition process from the instantiation process in MEF so I have had to create my own approach for this problem. The solution assumes that the scanning of plugins results in having the type, import and export data stored somehow.
In order to compose parts you need to keep track of each part instance and how it is connected to other part instances. The simplest way to do this is to make use of a graph data structure that keeps track of which import is connected to which export.
public sealed class CompositionCollection
{
private readonly Dictionary<PartId, PartDefinition> m_Parts;
private readonly Graph<PartId, PartEdge> m_PartConnections;
public PartId Add(PartDefinition definition)
{
var id = new PartId();
m_Parts.Add(id, definition);
m_PartConnections.AddVertex(id);
return id;
}
public void Connect(
PartId importingPart,
MyImportDefinition import,
PartId exportingPart,
MyExportDefinition export)
{
// Assume that edges point from the export to the import
m_PartConnections.AddEdge(
new PartEdge(
exportingPart,
export,
importingPart,
import));
}
}
Note that before connecting two parts it is necessary to check if the import can be connected to the export. In other cases MEF does that but in this case we'll need to do that ourselves. An example of how to approach that is:
public bool Accepts(
MyImportDefinition importDefinition,
MyExportDefinition exportDefinition)
{
if (!string.Equals(
importDefinition.ContractName,
exportDefinition.ContractName,
StringComparison.OrdinalIgnoreCase))
{
return false;
}
// Determine what the actual type is we're importing. MEF provides us with
// that information through the RequiredTypeIdentity property. We'll
// get the type identity first (e.g. System.String)
var importRequiredType = importDefinition.RequiredTypeIdentity;
// Once we have the type identity we need to get the type information
// (still in serialized format of course)
var importRequiredTypeDef =
m_Repository.TypeByIdentity(importRequiredType);
// Now find the type we're exporting
var exportType = ExportedType(exportDefinition);
if (AvailableTypeMatchesRequiredType(importRequiredType, exportType))
{
return true;
}
// The import and export can't directly be mapped so maybe the import is a
// special case. Try those
Func<TypeIdentity, TypeDefinition> toDefinition =
t => m_Repository.TypeByIdentity(t);
if (ImportIsCollection(importRequiredTypeDef, toDefinition)
&& ExportMatchesCollectionImport(
importRequiredType,
exportType,
toDefinition))
{
return true;
}
if (ImportIsLazy(importRequiredTypeDef, toDefinition)
&& ExportMatchesLazyImport(importRequiredType, exportType))
{
return true;
}
if (ImportIsFunc(importRequiredTypeDef, toDefinition)
&& ExportMatchesFuncImport(
importRequiredType,
exportType,
exportDefinition))
{
return true;
}
if (ImportIsAction(importRequiredTypeDef, toDefinition)
&& ExportMatchesActionImport(importRequiredType, exportDefinition))
{
return true;
}
return false;
}
Note that the special cases (like IEnumerable<T>, Lazy<T> etc.) require determining if the importing type is based on a generic type which can be a bit tricky.
Once all the composition information is stored it is possible to do the instantiation of the parts at any point in time because all the required information is available. Instantiation requires a generous helping of reflection combined with the use of the trusty Activator class and will be left as an exercise to the reader.

LINQ to SQL Datacontext mapping to WCF DataContract

I'm writing a simple movie database which has a three tier stack of service layer, data access layer and SQL DB.
I'm using LINQ to SQL to access the DB and return a Film from the Film table in the DB. This will then be returned as a Film object from the DataContracts in the service layer.
I thought this would work ok, but its resulted in some awkward code which doesn't look right. Can someone sanity check this please?
Is it best practice to map every LINQ result to it's DataContract?
public static class DBConnection
{
private static RMDB_LINQDataContext _db;
static DBConnection()
{
_db = new RMDB_LINQDataContext();
}
public static RMDB.DTO.Film GetFilm(string name)
{
var LINQ_film = from film in _db.GetTable<Film>()
where film.name == name
select film;
if (LINQ_film.ToList().Count != 1)
{
// TODO - faultException
}
else
{
foreach (Film f in LINQ_film.ToList())
{
// Yuck
return new RMDB.DTO.Film(f.name,
f.releaseDate.GetValueOrDefault(), "foo", f.rating.GetValueOrDefault());
}
}
return null;
}
this is a bit neater and more efficient as your formulation performs the query twice
public static List<Film> GetFilm(string name)
{
var LINQ_film = from film in _db.GetTable<Film>()
where film.name == name
select new Film(film.name,
film.releaseDate.GetValueOrDefault(),
"foo",
film.rating.GetValueOrDefault());
var list = LINQ_film.ToList();
if (list.Count != 1)
{
// TODO - faultException
}
return list;
}
Copying one instance of a type to another is standard practice when mapping from a domain object to a DTO. It may seem like a lot more work but it is a purely memory based mapping and provides a layer of isolation between your business layer and your clients. Without it clients will be impacted by refactoring of the business layer

Does NHibernate really deliver transparent persistency

Starting to use Nhibernate for persistency being seduced by the promise that it respects your domain model, I tried to implement a relation manager for my domain objects. Basically, to DRY my code with respect to managing bidirectional one to many and many to many relations, I decided to have those relations managed by a separate class. When a one to many or many to one property is set an entry for the two objects is made in an dictionary, the key is either a one side with a collection value to hold the many sides, or a many side with a value of the one side.
A one to many relation for a specific combination of types looks as follows:
public class OneToManyRelation<TOnePart, TManyPart> : IRelation<IRelationPart, IRelationPart>
where TOnePart : class, IRelationPart
where TManyPart : class, IRelationPart
{
private readonly IDictionary<TOnePart, Iesi.Collections.Generic.ISet<TManyPart>> _oneToMany;
private readonly IDictionary<TManyPart, TOnePart> _manyToOne;
public OneToManyRelation()
{
_manyToOne = new ConcurrentDictionary<TManyPart, TOnePart>();
_oneToMany = new ConcurrentDictionary<TOnePart, Iesi.Collections.Generic.ISet<TManyPart>>();
}
public void Set(TOnePart onePart, TManyPart manyPart)
{
if (onePart == null || manyPart == null) return;
if (!_manyToOne.ContainsKey(manyPart)) _manyToOne.Add(manyPart, onePart);
else _manyToOne[manyPart] = onePart;
}
public void Add(TOnePart onePart, TManyPart manyPart)
{
if (onePart == null || manyPart == null) return;
if (!_manyToOne.ContainsKey(manyPart)) _manyToOne.Add(manyPart, onePart);
else _manyToOne[manyPart] = onePart;
if (!_oneToMany.ContainsKey(onePart)) _oneToMany.Add(onePart, new HashedSet<TManyPart>());
_oneToMany[onePart].Add(manyPart);
}
public Iesi.Collections.Generic.ISet<TManyPart> GetManyPart(TOnePart onePart)
{
if (!_oneToMany.ContainsKey(onePart)) _oneToMany[onePart] = new HashedSet<TManyPart>();
return _oneToMany[onePart];
}
public TOnePart GetOnePart(TManyPart manyPart)
{
if(!_manyToOne.ContainsKey(manyPart)) _manyToOne[manyPart] = default(TOnePart);
return _manyToOne[manyPart];
}
public void Remove(TOnePart onePart, TManyPart manyPart)
{
_manyToOne.Remove(manyPart);
_oneToMany[onePart].Remove(manyPart);
}
public void Set(TOnePart onePart, Iesi.Collections.Generic.ISet<TManyPart> manyPart)
{
if (onePart == null) return;
if (!_oneToMany.ContainsKey(onePart)) _oneToMany.Add(onePart, manyPart);
else _oneToMany[onePart] = manyPart;
}
public void Clear(TOnePart onePart)
{
var list = new HashedSet<TManyPart>(_oneToMany[onePart]);
foreach (var manyPart in list)
{
_manyToOne.Remove(manyPart);
}
_oneToMany.Remove(onePart);
}
public void Clear(TManyPart manyPart)
{
if (!_manyToOne.ContainsKey(manyPart)) return;
if (_manyToOne[manyPart] == null) return;
_oneToMany[_manyToOne[manyPart]].Remove(manyPart);
_manyToOne.Remove(manyPart);
}
}
On the many side a code snippet looks like:
public virtual SubstanceGroup SubstanceGroup
{
get { return RelationProvider.SubstanceGroupSubstance.GetOnePart(this); }
protected set { RelationProvider.SubstanceGroupSubstance.Set(value, this); }
}
On the one side, so, in this case the SubstanceGroup, the snippet looks like:
public virtual ISet<Substance> Substances
{
get { return RelationProvider.SubstanceGroupSubstance.GetManyPart(this); }
protected set { RelationProvider.SubstanceGroupSubstance.Set(this, value); }
}
Just using my domain objects, this works excellent. In the domain object I just have to reference an abstract factory that retrieves the appropriate relation and I can set the relation from one side, wich automatically becomes thus bidirectional.
However, when NH kicks in the problem is that I get duplicate keys in my dictionaries. Somehow NH sets a relation property with a null value(!) with a new copy(?) of a domain object. So when the domain object gets saved, I have two entries of that domain object in, for example the many side of the relation, i.e. _manyToOne dictionary.
This problem makes me lose my hair, I do not get it what is happening??
To answer your first, very general question: "Does NHibernate really deliver transparent persistency", I just can say: nothing is perfect. NH tries its best to be as transparent as possible, by also trying to keep its complexity as low as possible.
There are some assumptions, particularly regarding collections: Collections and their implementations are not considered to be part of your domain model. NH provides its own collection implementations. You are not only expected to use the interfaces like ISet and IList. You should also take the instances given by NH when the object is read from the database and never replace it with your own. (I don't know what your relation class is actually used for, so I don't know if this is the problem here.)
Domain objects are unique within the same instance of the session. If you get new instances of domain objects each time, you probably implemented the "session-per-call" anti-pattern, which creates a new session for each database interaction.
I don't have a clue what you actually are doing. How is this OneToManyRelation actually used for? What are you doing when NH doesn't behave as expected? This is a very specific problem to your specific implementation.
Besides the comments on 'convoluted code' and 'what the heck are you doing'. The problem was that I was replacing the persistence collections of NH like in the below code snippet:
public void Add(TOnePart onePart, TManyPart manyPart)
{
if (onePart == null || manyPart == null) return;
if (!_manyToOne.ContainsKey(manyPart)) _manyToOne.Add(manyPart, onePart);
else _manyToOne[manyPart] = onePart;
if (!_oneToMany.ContainsKey(onePart)) _oneToMany.Add(onePart, new HashedSet<TManyPart>());
_oneToMany[onePart].Add(manyPart);
}
I create a new Hashed set for the many part. And that was the problem. If just has set the many part with the collection coming in (in case of the persistence collection implementation of NH) than it would have worked.
As a NH newbie, this replacing of collections with a special implementation from NH has been an important source of errors. Just as a warning to other NH newbies.