I'm using protocol buffer as a wire data-format in a client-server architecture. Domain objects (java beans) will go through following life-cycle.
Used in client side business logic
Converted to protobuf format
Transmitted to the server
Converted back to domain object
Used in server side business logic
"Protocol Buffers and O-O Design" section in ProtoBuf documentation recommends wrapping generated class inside proper domain model.
I'd like to find-out the best appoach.
For e.g. I have a simple proto definition.
package customer;
option java_package = "com.example";
option java_outer_classname = "CustomerProtos";
message Customer {
required string name = 1;
optional string address = 2;
}
This is how domain model is defined. As you can see, the data is completely stored in proto builder object.
package com.example;
public class CustomerModel
{
private CustomerProtos.Customer.Builder builder = CustomerProtos.Customer.newBuilder();
public String getName()
{
return builder.getName();
}
public void setName(String name)
{
builder.setName(name);
}
public String getAddress()
{
return builder.getAddress();
}
public void setAddress(String address)
{
builder.setAddress(address);
}
public byte[] serialize()
{
return builder.build().toByteArray();
}
}
Is this a good practice? because these objects are used in all phases of life-cycle, but we only requires protocolbuf format at client-server transmission phase.
Is there any performance issue when accessing proto builder class getter/setter methods specially when proto definition is complex and nested?
I have no experience with protocol buffers, but I would not recommend implementing your domain objects tailored to a specific serialization/transfer framework. You might regret that in the future.
The domain objects and logic of a software application should be as independent as possible from specific implementation issues (in your case serialization/transfer), because you want your domain to be easy to understand and be reusable/maintainable in the future.
If you want to define your domain objects independent of serialization/transfer, you have two options:
Before serialization/transfer, you copy the information to protocol
buffers specific objects and send them to your server. There you
would have to copy the information back to your domain objects.
Use a non-protocol serialization library like Kryo or
ProtoStuff to directly transfer your domain objects to the
server.
The disadvantages of option 1 are that your domain is defined two times (which is undesirable with respect to modifications) and the copying of information (which produces error-prone and non maintainable code).
The disadvantages of option 2 are that you lose schema evolution (although ProtoStuff apparently supports it) and the complete (potentially large) object graph is serialized and transferred. Although you could prune the object graph (manually or with JGT) before serialization/transfer.
We've made a protobuf-converter to solve the problem of transformation of your Domain Model Objects into Google Protobuf Messages and vice versa.
How to use it:
Domain model classes that have to be transformed into protobuf messages must satisfy conditions:
Class has to be marked by #ProtoClass annotaion that contains
reference on related protobuf message class.
Class fields has to be marked by #ProtoField annotaion. These fields must have getters and setters.
E.g.:
#ProtoClass(ProtobufUser.class)
public class User {
#ProtoField
private String name;
#ProtoField
private String password;
// getters and setters for 'name' and 'password' fields
...
}
Code for conversion User instance into related protobuf message:
User userDomain = new User();
...
ProtobufUser userProto = Converter.create().toProtobuf(ProtobufUser.class, userDomain);
Code for backward conversion:
User userDomain = Converter.create().toDomain(User.class, userProto);
Conversion of lists of objects is similar to single object conversion.
Related
What is the differences between these two Properties?
I can use HttpContext.Items instead of HttpContext.Features to share data between middlewares. The only difference I see is that I tell Items for a key and it gives me an object and I have to cast it. This casting can be done in Features automatically.
Is there something else behind them?
The biggest difference is that the HttpContext.Items is designed to store Key-Value-Pair, while the HttpContext.Features is designed to store Type-Instance-Pair.
To be more clear, HttpContext.Items is designed to share items within the scope of current request, while the HttpContext.Features, which is an instance of IFeatureCollection, is by no means to be used like that .
The IFeatureCollection interface represents a collection of HTTP features, such as:
IAuthenticationFeature which stores original PathBase and original Path.
ISessionFeature which stores current Session.
IHttpConnectionFeature which stores the underlying connection.
and so on.
To help store and retrieve a Type-Instance-Pair, the interface has three important methods:
public interface IFeatureCollection : IEnumerable<KeyValuePair<Type, object>>{
// ...
object this[Type key] { get; set; }
TFeature Get<TFeature>();
void Set<TFeature>(TFeature instance);
}
and the implementation (FeatureCollection) will simply cast the value into required type:
public class FeatureCollection : IFeatureCollection
{
// ... get the required type of feature
public TFeature Get<TFeature>()
{
return (TFeature)this[typeof(TFeature)]; // note: cast here!
}
public void Set<TFeature>(TFeature instance)
{
this[typeof(TFeature)] = instance; // note!
}
}
This is by design. Because there's no need to store two IHttpConnectionFeature instances or two ISession instances.
While you can store some Type-Value pairs with FeatureCollection, you'd better not . As you see, the Set<TFeature>(TFeature instance) will simply replace the old one if the some type already exists in the collection; it also means there will be a bug if you have two of the same type.
HttpContext.Items is designed to share short-lived per-request data, as you mentioned.
HttpContext.Features is designed to share various HTTP features that allow middleware to create or modify the application's hosting pipeline. It's already filled with several features from .NET, such as IHttpSendFileFeature.
You should use HttpContext.Items to store data, and HttpContext.Features to add any new HTTP features that another middleware class might need.
I am working on a distributed algorithm and decided to use a Akka to scale it across machines. The machines need to exchange messages very frequently and these messages reference some immutable objects that exist on every machine. Hence, it seems sensible to "compress" the messages in the sense that the shared, replicated objects should not be serialized in the messages. Not only would this save on network bandwidth but it also would avoid creating duplicate objects in the receiver side whenever a message is deserialized.
Now, my question is how to do this properly. So far, I could think of two options:
Handle this on the "business layer", i.e., converting my original message objects to some reference objects that replace references to the shared, replicated objects by some symbolic references. Then, I would send those reference objects rather than the original messages. Think of it as replacing some actual web resource with a URL. Doing this seems rather straight-forward in terms of coding but it also drags serialization concerns into the actual business logic.
Write custom serializers that are aware of the shared, replicated objects. In my case, it would be okay that this solution would introduce the replicated, shared objects as global state to the actor systems via the serializers. However, the Akka documentation does not describe how to programmatically add custom serializers, which would be necessary to weave in the shared objects with the serializer. Also, I could imagine that there are a couple of reasons, why such a solution would be discouraged. So, I am asking here.
Thanks a lot!
It's possible to write your own, custom serializers and let them do all sorts of weird things, then you can bind them at the config level as usual:
class MyOwnSerializer extends Serializer {
// If you need logging here, introduce a constructor that takes an ExtendedActorSystem.
// class MyOwnSerializer(actorSystem: ExtendedActorSystem) extends Serializer
// Get a logger using:
// private val logger = Logging(actorSystem, this)
// This is whether "fromBinary" requires a "clazz" or not
def includeManifest: Boolean = true
// Pick a unique identifier for your Serializer,
// you've got a couple of billions to choose from,
// 0 - 40 is reserved by Akka itself
def identifier = 1234567
// "toBinary" serializes the given object to an Array of Bytes
def toBinary(obj: AnyRef): Array[Byte] = {
// Put the code that serializes the object here
//#...
Array[Byte]()
//#...
}
// "fromBinary" deserializes the given array,
// using the type hint (if any, see "includeManifest" above)
def fromBinary(
bytes: Array[Byte],
clazz: Option[Class[_]]): AnyRef = {
// Put your code that deserializes here
//#...
null
//#...
}
}
But this raises an important question: if your messages all references data that is shared on the machines already, why would you want to put in the message the pointer to the object (very bad! messages should be immutable, and a pointer isn't!), rather than some sort of immutable, string objectId (kinda your option 1) ? This is a much better option when it comes to preserving the immutability of the messages, and there is little change in your business logic (just put a wrapper over the shared state storage)
for more info, see the documentation
I finally went with the solution proposed by Diego and want to share some more details on my reasoning and solution.
First of all, I am also in favor of option 1 (handling the "compaction" of messages in the business layer) for those reasons:
Serializers are global to the actor system. Making them stateful is actually a most severe violation of Akka's very philosophy as it goes against the encapsulation of behavior and state in actors.
Serializers have to be created upfront, anyway (even when adding them "programatically").
Design-wise, one can argue that "message compaction is not a responsibility of the serializer, either. In a strict sense, serialization is merely the transformation of runtime-specific data into a compact, exchangable representation. Changing what to serialize, is not a task of a serializer, though.
Having settled upon this, I still strived for a clear separation of "message compaction" and the actual business logic in the actors. I came up with a neat way to do this in Scala, which I want to share here. The basic idea is to make the message itself look like a normal case class but still allow these messages to "compactify" themselves. Here is an abstract example:
class Sender extends ActorRef {
def context: SharedContext = ... // This is the shared data present on every node.
// ...
def someBusinessLogic(receiver: ActorRef) {
val someData = computeData
receiver ! MyMessage(someData)
}
}
class Receiver extends ActorRef {
implicit def context: SharedContext = ... // This is the shared data present on every node.
def receiver = {
case MyMessage(someData) =>
// ...
}
}
object Receiver {
object MyMessage {
def apply(someData: SomeData) = MyCompactMessage(someData: SomeData)
def unapply(myCompactMessage: MyCompactMessage)(implicit context: SharedContext)
: Option[SomeData] =
Some(myCompactMessage.someData(context))
}
}
As you can see, the sender and receiver code feels just like using a case class and in fact, MyMessage could be a case class.
However, by implementing apply and unapply manually, one can insert its own "compactification" logic and also implicitly inject the shared data necessary to do the "uncompactification", without touching the sender and receiver. For defining MyCompactMessage, I found Protocol Buffers to be especially suited, as it is already a dependency of Akka and efficient in terms of space and computation, but any other solution would do.
I'm new to serialization concept, please help in understanding concept.
What exactly serialization means? I have read the definition, but could not understand in details.
How basic types (int, string) are serialized?
If we don't use serialization in our code how data will be transmitted?
Is there any implicit serialization process involved while accessing database from front end Java/C# code? example insert/delete from database.
Serialization just takes an object and translates it into something simpler. Imagine that you had an object in C# like so:
class Employee
{
public int age;
public string fullname;
}
public static void Main()
{
var john = new Employee();
john.age = 21;
john.fullname = "John Smith";
var matt = new Employee();
matt.age = 44;
matt.fullname = "Matt Rogers";
...
This is C# friendly. But if you wanted to save that information in a text file in CSV format, you would end up with something like this:
age,fullname
21,John Smith
44,Matt Rogers
When you write a CSV, you are basically serializing information into a different format - in this case a CSV file. You can serialize your object to XML, JSON, database table(s), memory or something else. Here's an example from Udemy regarding serialization.
If you don't serialize, confusion will be transmitted. Perhaps your object's ToString() will be implictly called before transmission and whatever result gets transmitted. Therefore it is vital to convert your data to something that is receiver friendly.
There's always some serialization happening. When you execute a query that populates a DataTable, for example, serialization occurred.
Concept :
Serialization is the process of converting an object into series of bytes.
Usually the objects we use in application will be complex and all of them can be easily represented in the form of series of bytes which can be stored in the file/database or transfered over network.
You can make a class Serializable just by making it implement Serializable interface.
For a class to be serialized successfully, two conditions must be met:
The class must implement the java.io.Serializable interface.
All of the fields in the class must be serializable. If a field is not serializable, it must be marked transient.
When the program is done serializing, and if it is stored in a file with extension .ser then it can be used for deserializing.
Serialization gives an serialVersionUID to the serialized object which has to match for deserialization
In my Google Web Toolkit project, I got the following error:
com.google.gwt.user.client.rpc.SerializationException: Type ‘your.class.Type’ was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.
What are the possible causes of this error?
GWT keeps track of a set of types which can be serialized and sent to the client. your.class.Type apparently was not on this list. Lists like this are stored in .gwt.rpc files. These lists are generated, so editing these lists is probably useless. How these lists are generated is a bit unclear, but you can try the following things:
Make sure your.class.Type implements java.io.Serializable
Make sure your.class.Type has a public no-args constructor
Make sure the members of your.class.Type do the same
Check if your program does not contain collections of a non-serializable type, e.g. ArrayList<Object>. If such a collection contains your.class.Type and is serialized, this error will occur.
Make your.class.Type implement IsSerializable. This marker interface was specifically meant for classes that should be sent to the client. This didn't work for me, but my class also implemented Serializable, so maybe both interfaces don't work well together.
Another option is to create a dummy class with your.class.Type as a member, and add a method to your RPC interface that gets and returns the dummy. This forces the GWT compiler to add the dummy class and its members to the serialization whitelist.
I'll also add that if you want to use a nested class, use a static member class.
I.e.,
public class Pojo {
public static class Insider {
}
}
Nonstatic member classes get the SerializationException in GWT 2.4
I had the same issue in a RemoteService like this
public List<X> getX(...);
where X is an interface. The only implementation did conform to the rules, i.e. implements Serializable or IsSerializable, has a default constructor, and all its (non-transient and non-final) fields follow those rules as well.
But I kept getting that SerializationException until I changed the result type from List to X[], so
public X[] getX(...);
worked. Interestingly, the only argument being a List, Y being an interface, was no problem at all...
I have run into this problem, and if you per chance are using JPA or Hibernate, this can be a result of trying to return the query object and not creating a new object and copying your relavant fields into that new object. Check the following out, which I saw in a google group.
#SuppressWarnings("unchecked")
public static List<Article> getForUser(User user)
{
List<Article> articles = null;
PersistenceManager pm = PMF.get().getPersistenceManager();
try
{
Query query = pm.newQuery(Article.class);
query.setFilter("email == emailParam");
query.setOrdering("timeStamp desc");
query.declareParameters("String emailParam");
List<Article> results = (List<Article>) query.execute(user.getEmail
());
articles = new ArrayList<Article>();
for (Article a : results)
{
a.getEmail();
articles.add(a);
}
}
finally
{
pm.close();
}
return articles;
}
this helped me out a lot, hopefully it points others in the right direction.
Looks like this question is very similar to what IsSerializable or not in GWT?, see more links to related documentation there.
When your class has JDO annotations, then this fixed it for me (in addition to the points in bspoel's answer) : https://stackoverflow.com/a/4826778/1099376
Is it a violation of the Persistance igorance to inject a repository interface into a Entity object Like this. By not using a interface I clearly see a problem but when using a interface is there really a problem? Is the code below a good or bad pattern and why?
public class Contact
{
private readonly IAddressRepository _addressRepository;
public Contact(IAddressRepository addressRepository)
{
_addressRepository = addressRepository;
}
private IEnumerable<Address> _addressBook;
public IEnumerable<Address> AddressBook
{
get
{
if(_addressBook == null)
{
_addressBook = _addressRepository.GetAddresses(this.Id);
}
return _addressBook;
}
}
}
It's not exactly a good idea, but it may be ok for some limited scenarios. I'm a little confused by your model, as I have a hard time believing that Address is your aggregate root, and therefore it wouldn't be ordinary to have a full-blown address repository. Based on your example, you probably are actually using a table data gateway or dao rather than a respository.
I prefer to use a data mapper to solve this problem (an ORM or similar solution). Basically, I would take advantage of my ORM to treat address-book as a lazy loaded property of the aggregate root, "Contact". This has the advantage that your changes can be saved as long as the entity is bound to a session.
If I weren't using an ORM, I'd still prefer that the concrete Contact repository implementation set the property of the AddressBook backing store (list, or whatever). I might have the repository set that enumeration to a proxy object that does know about the other data store, and loads it on demand.
You can inject the load function from outside. The new Lazy<T> type in .NET 4.0 comes in handy for that:
public Contact(Lazy<IEnumerable<Address>> addressBook)
{
_addressBook = addressBook;
}
private Lazy<IEnumerable<Address>> _addressBook;
public IEnumerable<Address> AddressBook
{
get { return this._addressBook.Value; }
}
Also note that IEnumerable<T>s might be intrinsically lazy anyhow when you get them from a query provider. But for any other type you can use the Lazy<T>.
Normally when you follow DDD you always operate with the whole aggregate. The repository always returns you a fully loaded aggregate root.
It doesn't make much sense (in DDD at least) to write code as in your example. A Contact aggregate will always contain all the addresses (if it needs them for its behavior, which I doubt to be honest).
So typically ContactRepository supposes to construct you the whole Contact aggregate where Address is an entity or, most likely, a value object inside this aggregate.
Because Address is an entity/value object that belongs to (and therefore managed by) Contact aggregate it will not have its own repository as you are not suppose to manage entities that belong to an aggregate outside this aggregate.
Resume: always load the whole Contact and call its behavior method to do something with its state.
Since its been 2 years since I asked the question and the question somewhat misunderstood I will try to answer it myself.
Rephrased question:
"Should Business entity classes be fully persistance ignorant?"
I think entity classes should be fully persistance ignorant, because you will instanciate them many places in your code base so it will quickly become messy to always have to inject the Repository class into the entity constructor, neither does it look very clean. This becomes even more evident if you are in need of injecting several repositories. Therefore I always use a separate handler/service class to do the persistance jobs for the entities. These classes are instanciated far less frequently and you usually have more control over where and when this happens. Entity classes are kept as lightweight as possible.
I now always have 1 Repository pr aggregate root and if I have need for some extra business logic when entities are fetched from repositories I usually create 1 ServiceClass for the aggregate root.
By taking a tweaked example of the code in the question as it was a bad example I would do it like this now:
Instead of:
public class Contact
{
private readonly IContactRepository _contactRepository;
public Contact(IContactRepository contactRepository)
{
_contactRepository = contactRepository;
}
public void Save()
{
_contactRepository.Save(this);
}
}
I do it like this:
public class Contact
{
}
public class ContactService
{
private readonly IContactRepository _contactRepository;
public ContactService(IContactRepository contactRepository)
{
_contactRepository = contactRepository;
}
public void Save(Contact contact)
{
_contactRepository.Save(contact);
}
}