How can I serialize/deserialize java.util.stream.Stream using Jackson? - jackson

Assuming I have the following object
public class DataObjectA {
private Stream<DataObjectB> dataObjectBStream;
}
How can I serialize them using Jackson?

As others have pointed out, you can only iterate once over a stream. If that works for you, you can use this to serialize:
new ObjectMapper().writerFor(Iterator.class).writeValueAsString(dataObjectBStream.iterator())
If you're using a Jackson version prior to 2.5, use writerWithType() instead of writerFor().

See https://github.com/FasterXML/jackson-modules-java8/issues/3 for the open issue to add java.util.Stream support to Jackson. There's a preliminary version of the code included. (edit: this is now merged and supported in 2.9.0).
Streaming support feels like it would work naturally/safely if the stream is the top level object you were (de)serializing, eg returning a java.util.stream.Stream<T> from a JAX-RS resource, or reading a Stream from a JAX-RS client.
A Stream as a member variable of a (de)serialized object, as you have in your example, is trickier, because it's mutable and single use:
private Stream<DataObjectB> dataObjectBStream;
Assuming it was supported, all of the caveats around storing references to streams would apply. You wouldn't be able to serialize the object more than once, and once you deserialized the wrapping object presumably it's stream member would retain a live connection back through the JAX-RS client and HTTP connection, which could create surprises.

You don’t.
A Stream is a single-use chain of operations and never meant to be persistent. Even storing it into an instance field like in your question is an indicator for a misunderstanding of it’s purpose. Once a terminal operation has been applied on the stream, it is useless and streams can’t be cloned. This, there is no point in remembering the unusable stream in a field then.
Since the only operations offered by Stream are chaining more operations to the pipeline and finally evaluating it, there is no way of querying its state such that it would allow to create an equivalent stream regarding its behavior. Therefore, no persistence framework can store it. The only thing a framework could do, is traversing the resulting elements of the stream operation and store them but that means effectively storing a kind of collection of objects rather than the Stream. Besides that, the single-use nature of a Stream also implies that a storage framework traversing the stream in order to store the elements had the side-effect of making the stream unusable at the same time.
If you want to store elements, resort to an ordinary Collection.
On the other hand, if you really want to store behavior, you’ll end up storing an object instance whose actual class implements the behavior. This still works with Streams as you can store an instance of a class which has a factory method producing the desired stream. Of course, you are not really storing the behavior but a symbolic reference to it, but this is always the case when you use an OO storage framework to store behavior rather than data.

I had below class having 2 elements one of them was Stream, had to annotate the getterStream method with#JsonSerializer and then override Serialize method, produces stream of JSON in my Response API:
public class DataSetResultBean extends ResultBean
{
private static final long serialVersionUID = 1L;
private final List<ComponentBean> structure;
private final Stream<DataPoint> datapoints;
private static class DataPointSerializer extends JsonSerializer<Stream<DataPoint>>
{
#Override
public void serialize(Stream<DataPoint> stream, JsonGenerator gen, SerializerProvider serializers) throws IOException, JsonProcessingException
{
gen.writeStartArray();
try
{
stream.forEach(dp -> serializeSingle(gen, dp));
}
catch (UncheckedIOException e)
{
throw (IOException) e.getCause();
}
finally
{
stream.close();
}
gen.writeEndArray();
}
public synchronized void serializeSingle(JsonGenerator gen, DataPoint dp) throws UncheckedIOException
{
try
{
gen.writeStartObject();
for (Entry<DataStructureComponent<?, ?, ?>, ScalarValue<?, ?, ?>> entry: dp.entrySet())
{
gen.writeFieldName(entry.getKey().getName());
gen.writeRawValue(entry.getValue().toString());
}
gen.writeEndObject();
}
catch (IOException e)
{
throw new UncheckedIOException(e);
}
}
}
public DataSetResultBean(DataSet dataset)
{
super("DATASET");
structure = dataset.getMetadata().stream().map(ComponentBean::new).collect(toList());
datapoints = dataset.stream();
}
public List<ComponentBean> getStructure()
{
return structure;
}
#JsonSerialize(using = DataPointSerializer.class)
public Stream<DataPoint> getDatapoints()
{
return datapoints;
}
}

Related

Signal R client not raising On even with Json serialized complex objects

I am trying to implement signal r client and server with json serialization.
Currently I am targeting .NET 5 and use Microsoft json serializer implementation.
My messages are represented by complex objects and there is an JsonConverter used for reading and writing.
What I see is that on the client the On event is never raised except if handler parameter specified as an object class.
connection.On("EntityEventAsync", (object obj) =>
{
//obj will be json object here
});
On client side I can clearly see that the messages are received as JsonConverter is called and reads the messages as it should BUT the On event is never raised.
Typed client hub code
public interface IEventsClient
{
Task EntityEventAsync(DetailedMessage message);
}
[Authorize(AuthenticationSchemes = "Basic,Bearer")]
public class EventHub : Hub<Clients.IEventsClient>
{
#region CONSTRUCTOR
public EventHub()
{
}
#endregion
}
What I can be missing here?
If some one else struggles with same problem then it be might the problem with the JsonConverter implementation as it in my case.
Its possible to enable Signal R client logging as mentioned here https://learn.microsoft.com/en-us/aspnet/core/signalr/diagnostics?view=aspnetcore-5.0 and that should make it easier to figure out the problem.

How to implement ClassFileTransformer#transform with byte buddy?

Is there a way to use byte buddy to implement ClassFileTransformer#transform?
At the moment my implementation uses javassist but I want to replace it with byte buddy as it has a better generics support.
So far my implementation looks like this:
public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined,
ProtectionDomain protectionDomain, byte[] classfileBuffer)
{
if (className.startsWith("my.package."))
{
try {
final CtClass ctClass = classPool.makeClass(new ByteArrayInputStream(classfileBuffer));
/* class manipulation */
return ctClass.toBytecode();
// remove class from class pool if it hasn't been modified
ctClass.detach();
} catch(final Exception ex) {
logger.error("failed to analyse/transform class {}", className, ex);
}
}
return classfileBuffer;
}
Is something similar possible with byte buddy? Are there ways to feed byte buddy with the byte code provided in parameter classfileBuffer?
The ClassFileTransformer implementation is configured into the Spring Load Time Weaver. So I already have the "infrastructure" available. Therefore I would rather not install another byte buddy agent to solve this problem.
Yes, look into AgentBuilder.Default. It offers a DSL for implementing Java agents. You do not need to implement your own class file transformer using it, just specify the transformations you want to make.

How to deal with hard to express requirements for dependencies?

When doing IoC, I (think that I) understand its use for getting the desired application level functionality by composing the right parts, and the benefits for testability. But at the microlevel, I don't quite understand how to make sure that an object gets dependencies injected that it can actually work with. My example for this is a BackupMaker for a database.
To make a backup, the database needs to be exported in a specific format, compressed using a specific compression algorithm, and then packed together with some metadata to form the final binary. Doing all of these tasks seems to be far from a single responsibility, so I ended up with two collaborators: a DatabaseExporter and a Compressor.
The BackupMaker doesn't really care how the database is exported (e.g. using IPC to a utility that comes with the database software, or by doing the right API calls) but it does care a lot about the result, i.e. it needs to be a this-kind-of-database backup in the first place, in the transportable (version agnostic) format, either of which I don't really know how to wrap in a contract. Neither does it care if the compressor does the compression in memory or on disk, but it has to be BZip2.
If I give the BackupMaker the wrong kinds of exporter or compressor, it will still produce a result, but it will be corrupt - it'll look like a backup, but it won't have the format that it should have. It feels like no other part of the system can be trusted to give it those collaborators, because the BackupMaker won't be able to guarantee to do the right thing itself; its job (from my perspective) is to produce a valid backup and it won't if the circumstances ain't right, and worse, it won't know about it. At the same time, even when writing this, it seems to me that I'm saying something stupid now, because the whole point of single responsibilities is that every piece should do its job and not worry about the jobs of others. If it were that simple though, there would be no need for contracts - J.B. Rainsberger just taught me there is. (FYI, I sent him this question directly, but I haven't got a reply yet and more opinions on the matter would be great.)
Intuitively, my favorite option would be to make it impossible to combine classes/objects in an invalid way, but I don't see how to do that. Should I write horrendously specific interface names, like IDatabaseExportInSuchAndSuchFormatProducer and ICompressorUsingAlgorithmXAndParametersY and assume that no classes implement these if they don't behave as such, and then call it a day since nothing can be done about outright lying code? Should I go as far as the mundane task of dissecting the binary format of my database's exports and compression algorithms to have contract tests to verify not only syntax but behavior as well, and then be sure (but how?) to use only tested classes? Or can I somehow redistribute the responsibilities to make this issue go away? Should there be another class whose responsibility it is to compose the right lower level elements? Or am I even decomposing too much?
Rewording
I notice that much attention is given to this very particular example. My question is more general than that, however. Therefore, for the final day of the bounty, I will try to summarize is as follows.
When using dependency injection, by definition, an object depends on other objects for what it needs. In many book examples, the way to indicate compatibility - the capability to provide that need - is by using the type system (e.g. implementing an interface). Beyond that, and especially in dynamic languages, contract tests are used. The compiler (if present) checks the syntax, and the contract tests (that the programmer needs to remember about) verify the semantics. So far, so good. However, sometimes the semantics are still too simple to ensure that some class/object is usable as a dependency to another, or too complicated to be described properly in a contract.
In my example, my class with a dependency on a database exporter considers anything that implements IDatabaseExportInSuchAndSuchFormatProducer and returns bytes as valid (since I don't know how to verify the format). Is very specific naming and such a very rough contract the way to go or can I do better than that? Should I turn the contract test into an integration test? Perhaps (integration) test the composition of all three? I'm not really trying to be generic but am trying to keep responsibilities separate and maintain testability.
What you have discovered in your question is that you have 2 classes that have an implicit dependency on one another. So, the most practical solution is to make the dependency explicit.
There are a number of ways you could do this.
Option 1
The simplest option is to make one service depend on the other, and make the dependent service explicit in its abstraction.
Pros
Few types to implement and maintain.
The compression service could be skipped for a particular implementation just by leaving it out of the constructor.
The DI container is in charge of lifetime management.
Cons
May force an unnatural dependency into a type where it is not really needed.
public class MySqlExporter : IExporter
{
private readonly IBZip2Compressor compressor;
public MySqlExporter(IBZip2Compressor compressor)
{
this.compressor = compressor;
}
public void Export(byte[] data)
{
byte[] compressedData = this.compressor.Compress(data);
// Export implementation
}
}
Option 2
Since you want to make an extensible design that doesn't directly depend on a specific compression algorithm or database, you can use an Aggregate Service (which implements the Facade Pattern) to abstract the more specific configuration away from your BackupMaker.
As pointed out in the article, you have an implicit domain concept (coordination of dependencies) that needs to be realized as an explicit service, IBackupCoordinator.
Pros
The DI container is in charge of lifetime management.
Leaving compression out of a particular implementation is as easy as passing the data through the method.
Explicitly implements a domain concept that you are missing, namely coordination of dependencies.
Cons
Many types to build and maintain.
BackupManager must have 3 dependencies instead of 2 registered with the DI container.
Generic Interfaces
public interface IBackupCoordinator
{
void Export(byte[] data);
byte[] Compress(byte[] data);
}
public interface IBackupMaker
{
void Backup();
}
public interface IDatabaseExporter
{
void Export(byte[] data);
}
public interface ICompressor
{
byte[] Compress(byte[] data);
}
Specialized Interfaces
Now, to make sure the pieces only plug together one way, you need to make interfaces that are specific to the algorithm and database used. You can use interface inheritance to achieve this (as shown) or you can just hide the interface differences behind the facade (IBackupCoordinator).
public interface IBZip2Compressor : ICompressor
{}
public interface IGZipCompressor : ICompressor
{}
public interface IMySqlDatabaseExporter : IDatabaseExporter
{}
public interface ISqlServerDatabaseExporter : IDatabaseExporter
{}
Coordinator Implementation
The coordinators are what do the job for you. The subtle difference between implementations is that the interface dependencies are explicitly called out so you cannot inject the wrong type with your DI configuration.
public class BZip2ToMySqlBackupCoordinator : IBackupCoordinator
{
private readonly IMySqlDatabaseExporter exporter;
private readonly IBZip2Compressor compressor;
public BZip2ToMySqlBackupCoordinator(
IMySqlDatabaseExporter exporter,
IBZip2Compressor compressor)
{
this.exporter = exporter;
this.compressor = compressor;
}
public void Export(byte[] data)
{
this.exporter.Export(byte[] data);
}
public byte[] Compress(byte[] data)
{
return this.compressor.Compress(data);
}
}
public class GZipToSqlServerBackupCoordinator : IBackupCoordinator
{
private readonly ISqlServerDatabaseExporter exporter;
private readonly IGZipCompressor compressor;
public BZip2ToMySqlBackupCoordinator(
ISqlServerDatabaseExporter exporter,
IGZipCompressor compressor)
{
this.exporter = exporter;
this.compressor = compressor;
}
public void Export(byte[] data)
{
this.exporter.Export(byte[] data);
}
public byte[] Compress(byte[] data)
{
return this.compressor.Compress(data);
}
}
BackupMaker Implementation
The BackupMaker can now be generic as it accepts any type of IBackupCoordinator to do the heavy lifting.
public class BackupMaker : IBackupMaker
{
private readonly IBackupCoordinator backupCoordinator;
public BackupMaker(IBackupCoordinator backupCoordinator)
{
this.backupCoordinator = backupCoordinator;
}
public void Backup()
{
// Get the data from somewhere
byte[] data = new byte[0];
// Compress the data
byte[] compressedData = this.backupCoordinator.Compress(data);
// Backup the data
this.backupCoordinator.Export(compressedData);
}
}
Note that even if your services are used in other places than BackupMaker, this neatly wraps them into one package that can be passed to other services. You don't necessarily need to use both operations just because you inject the IBackupCoordinator service. The only place where you might run into trouble is if using named instances in the DI configuration across different services.
Option 3
Much like Option 2, you could use a specialized form of Abstract Factory to coordinate the relationship between concrete IDatabaseExporter and IBackupMaker, which will fill the role of the dependency coordinator.
Pros
Few types to maintain.
Only 1 dependency to register in the DI container, making it simpler to deal with.
Moves lifetime management into the BackupMaker service, which makes it impossible to misconfigure DI in a way that will cause a memory leak.
Explicitly implements a domain concept that you are missing, namely coordination of dependencies.
Cons
Leaving compression out of a particular implementation requires you implement the Null object pattern.
The DI container is not in charge of lifetime management and each dependency instance is per request, which may not be ideal.
If your services have many dependencies, it may become unwieldy to inject them through the constructor of the CoordinationFactory implementations.
Interfaces
I am showing the factory implementation with a Release method for each type. This is to follow the Register, Resolve, and Release pattern which makes it effective for cleaning up dependencies. This becomes especially important if 3rd parties could implement the ICompressor or IDatabaseExporter types because it is unknown what kinds of dependencies they may have to clean up.
Do note however, that the use of the Release methods is totally optional with this pattern and excluding them will simplify the design quite a bit.
public interface IBackupCoordinationFactory
{
ICompressor CreateCompressor();
void ReleaseCompressor(ICompressor compressor);
IDatabaseExporter CreateDatabaseExporter();
void ReleaseDatabaseExporter(IDatabaseExporter databaseExporter);
}
public interface IBackupMaker
{
void Backup();
}
public interface IDatabaseExporter
{
void Export(byte[] data);
}
public interface ICompressor
{
byte[] Compress(byte[] data);
}
BackupCoordinationFactory Implementation
public class BZip2ToMySqlBackupCoordinationFactory : IBackupCoordinationFactory
{
public ICompressor CreateCompressor()
{
return new BZip2Compressor();
}
public void ReleaseCompressor(ICompressor compressor)
{
IDisposable disposable = compressor as IDisposable;
if (disposable != null)
{
disposable.Dispose();
}
}
public IDatabaseExporter CreateDatabaseExporter()
{
return new MySqlDatabseExporter();
}
public void ReleaseDatabaseExporter(IDatabaseExporter databaseExporter)
{
IDisposable disposable = databaseExporter as IDisposable;
if (disposable != null)
{
disposable.Dispose();
}
}
}
public class GZipToSqlServerBackupCoordinationFactory : IBackupCoordinationFactory
{
public ICompressor CreateCompressor()
{
return new GZipCompressor();
}
public void ReleaseCompressor(ICompressor compressor)
{
IDisposable disposable = compressor as IDisposable;
if (disposable != null)
{
disposable.Dispose();
}
}
public IDatabaseExporter CreateDatabaseExporter()
{
return new SqlServerDatabseExporter();
}
public void ReleaseDatabaseExporter(IDatabaseExporter databaseExporter)
{
IDisposable disposable = databaseExporter as IDisposable;
if (disposable != null)
{
disposable.Dispose();
}
}
}
BackupMaker Implementation
public class BackupMaker : IBackupMaker
{
private readonly IBackupCoordinationFactory backupCoordinationFactory;
public BackupMaker(IBackupCoordinationFactory backupCoordinationFactory)
{
this.backupCoordinationFactory = backupCoordinationFactory;
}
public void Backup()
{
// Get the data from somewhere
byte[] data = new byte[0];
// Compress the data
byte[] compressedData;
ICompressor compressor = this.backupCoordinationFactory.CreateCompressor();
try
{
compressedData = compressor.Compress(data);
}
finally
{
this.backupCoordinationFactory.ReleaseCompressor(compressor);
}
// Backup the data
IDatabaseExporter exporter = this.backupCoordinationFactory.CreateDatabaseExporter();
try
{
exporter.Export(compressedData);
}
finally
{
this.backupCoordinationFactory.ReleaseDatabaseExporter(exporter);
}
}
}
Option 4
Create a guard clause in your BackupMaker class to prevent non-matching types from being allowed, and throw an exception in the case they are not matched.
In C#, you can do this with attributes (which apply custom metadata to the class). Support for this option may or may not exist in other platforms.
Pros
Seamless - no extra types to configure in DI.
The logic for comparing whether types match could be expanded to include multiple attributes per type, if needed. So a single compressor could be used for multiple databases, for example.
100% of invalid DI configurations will cause an error (although you may wish to make the exception specify how to make the DI configuration work).
Cons
Leaving compression out of a particular backup configuration requires you implement the Null object pattern.
The business logic for comparing types is implemented in a static extension method, which makes it testable but impossible to swap with another implementation.
If the design is refactored so that ICompressor or IDatabaseExporter are not dependencies of the same service, this will no longer work.
Custom Attribute
In .NET, an attribute can be used to attach metadata to a type. We make a custom DatabaseTypeAttribute that we can compare the database type name with two different types to ensure they are compatible.
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
public DatabaseTypeAttribute : Attribute
{
public DatabaseTypeAttribute(string databaseType)
{
this.DatabaseType = databaseType;
}
public string DatabaseType { get; set; }
}
Concrete ICompressor and IDatabaseExporter Implementations
[DatabaseType("MySql")]
public class MySqlDatabaseExporter : IDatabaseExporter
{
public void Export(byte[] data)
{
// implementation
}
}
[DatabaseType("SqlServer")]
public class SqlServerDatabaseExporter : IDatabaseExporter
{
public void Export(byte[] data)
{
// implementation
}
}
[DatabaseType("MySql")]
public class BZip2Compressor : ICompressor
{
public byte[] Compress(byte[] data)
{
// implementation
}
}
[DatabaseType("SqlServer")]
public class GZipCompressor : ICompressor
{
public byte[] Compress(byte[] data)
{
// implementation
}
}
Extension Method
We roll the comparison logic into an extension method so every implementation of IBackupMaker automatically includes it.
public static class BackupMakerExtensions
{
public static bool DatabaseTypeAttributesMatch(
this IBackupMaker backupMaker,
Type compressorType,
Type databaseExporterType)
{
// Use .NET Reflection to get the metadata
DatabaseTypeAttribute compressorAttribute = (DatabaseTypeAttribute)compressorType
.GetCustomAttributes(attributeType: typeof(DatabaseTypeAttribute), inherit: true)
.SingleOrDefault();
DatabaseTypeAttribute databaseExporterAttribute = (DatabaseTypeAttribute)databaseExporterType
.GetCustomAttributes(attributeType: typeof(DatabaseTypeAttribute), inherit: true)
.SingleOrDefault();
// Types with no attribute are considered invalid even if they implement
// the corresponding interface
if (compressorAttribute == null) return false;
if (databaseExporterAttribute == null) return false;
return (compressorAttribute.DatabaseType.Equals(databaseExporterAttribute.DatabaseType);
}
}
BackupMaker Implementation
A guard clause ensures that 2 classes with non-matching metadata are rejected before the type instance is created.
public class BackupMaker : IBackupMaker
{
private readonly ICompressor compressor;
private readonly IDatabaseExporter databaseExporter;
public BackupMaker(ICompressor compressor, IDatabaseExporter databaseExporter)
{
// Guard to prevent against nulls
if (compressor == null)
throw new ArgumentNullException("compressor");
if (databaseExporter == null)
throw new ArgumentNullException("databaseExporter");
// Guard to prevent against non-matching attributes
if (!DatabaseTypeAttributesMatch(compressor.GetType(), databaseExporter.GetType()))
{
throw new ArgumentException(compressor.GetType().FullName +
" cannot be used in conjunction with " +
databaseExporter.GetType().FullName)
}
this.compressor = compressor;
this.databaseExporter = databaseExporter;
}
public void Backup()
{
// Get the data from somewhere
byte[] data = new byte[0];
// Compress the data
byte[] compressedData = this.compressor.Compress(data);
// Backup the data
this.databaseExporter.Export(compressedData);
}
}
If you decide on one of these options, I would appreciate if you left a comment as to which one you go with. I have a similar situation in one of my projects, and I am leaning toward Option 2.
Response to your Update
Is very specific naming and such a very rough contract the way to go or can I do better than that? Should I turn the contract test into an integration test? Perhaps (integration) test the composition of all three? I'm not really trying to be generic but am trying to keep responsibilities separate and maintain testability.
Creating an integration test is a good idea, but only if you are certain that you are testing the production DI configuration. Although it also makes sense to test it all as a unit to verify it works, it doesn't do you much good for this use case if the code that ships is configured differently than the test.
Should you be specific? I believe I have already given you a choice in that matter. If you go with the guard clause, you don't have to be specific at all. If you go with one of the other options, you have a good compromise between specific and generic.
I know you stated that you are not intentionally trying to be generic, and it is good to draw the line somewhere to ensure a solution is not over-engineered. On the other hand, if the solution has to be redesigned because an interface was not generic enough that is not a good thing either. Extensibility is always a requirement whether it is specified up front or not because you never really know how business requirements will change in the future. So, having a generic BackupMaker is definitely the best way to go. The other classes can be more specific - you just need one seam to swap implementations if future requirements change.
My first suggestion would be to critically think if you need to be that generic: You have a concrete problem to solve, you want to backup a very specific database into a specific format. Is there any benefit you get by solving the problem for arbitary databases and arbitary formats? What you surely get of a generic solution is boilerplate code and increased complexity (people understand concrete problems, not generic ones).
If this applies to you, then my suggestion would be to not let your DatabaseExporter accept interfaces, but instead only concrete implementations. There are enough modern tools out there which will also allow you mocking concrete classes, so testability is not an argument for using interfaces here aswell.
on the other hand, if you do have to backup several databases with different strategies, then I would probably introduce something like a
class BackupPlan {
public DatabaseExporter exporter() {/**...*/}
public Compressor compressor() {/** ... */}
}
then your BackupMaker will get passed one BackupPlan, specifying which database to be compressed with which algorithm.
Your question is emphasizing the fact that object composition is very important and that the entity that is responsible for such composition (wiring) has a big responsibility.
Since you already have a generic BackupMaker, I would suggest that you keep it this way, and push the big responsibility of making sure that the right composition of objects (to solve the specific problem) is done in the composition root.
Readers of your application source code (you and your team members), would have a single place (the composition root) to understand how you compose your objects to solve your specific problem by using the generic classes (e.g. BackupMaker).
Put in other words, the composition root is where you decide on the specifics. Its where you use the generic to create the specific.
To reply on the comment:
which should know what about those dependencies?
The composition root needs to know about everything (all the dependencies) since it is creating all the objects in the application and wiring them together. The composition root knows what each piece of the puzzle does and it connects them together to create a meaningful application.
For the BackupMaker, it should only care about just enough to be able to do its single responsibility. In your example, its single (simple) responsibility (as it seems to me) is to orchestrate the consumption of other objects to create a backup.
As long as you are using DI, a class will never be sure that its collaborator will behave correctly, only the composition root will. Consider this simple and extreme example of an IDatabaseExporter implementation (assume that the developer actually gave this class this name, and that he intentionally implemented it this way):
public class StupidDisastrousDatabaseExporter : IDatabaseExporter
{
public ExportedData Export()
{
DoSomethingStupidThatWillDeleteSystemDataAndMakeTheEnterpriseBroke();
...
}
private void DoSomethingStupidThatWillDeleteSystemDataAndMakeTheEnterpriseBroke()
{
//do it
...
}
}
Now, the BackupMaker will never know that it is consuming a stupid and disastrous database exporter, only the composition root does. We can never blame the programmer that wrote the BackupMaker class for this disastrous mistake (or the programmer who designed the IDatabaseExporter contract). But the programmer(s) that are composing the application in the composition root are blamed if they inject a StupidDisastrousDatabaseExporter instance into the constructor of BackupMaker.
Of course, no one should have written the StupidDisastrousDatabaseExporter class in the first place, but I gave you an extreme example to show you that a contract (interface) can never (and should never) guarantee every aspect about its implementors. It should just say enough.
Is there a way to express IDatabaseExporter in such a way that guarantees that implementors of such interface will not make stupid or disastrous actions? No.
Please note that while the BackupMaker is dealing with contracts (no 100% guarantees), the composition root is actually dealing with concrete implementation classes. This gives it the great power (and thus the great responsibility) to guarantee the composition of the correct object graph.
how do I make sure that I'm composing in a sensible way?
You should create automated end-to-end tests for the object graph created by the composition root. Here you are making sure that the composition root has done its big responsibility of composing the objects in a correct way. Here you can test the exact details that you wanted (like that the backup result was in some exact format/details).
Take a look at this article for an approach to automated testing using the Composition Root.
I believe this may be a problem that occurs when focusing too much on object models, at the exclusion of function compositions. Consider the first step in a naive function decomposition (function as in f : a -> b):
exporter: data -> (format, memory), or exception
compressor: memory -> memory, or exception
writer: memory -> side-effect, or exception
backup-maker: (data, exporter, compressor, writer) -> backup-result
So backup-maker, the last function, can be parametized with those three functions, assuming I've considered your use-case correctly, and if the three parameters have the same input and output types, e.g. format, and memory, despite their implementation.
Now, "the guts", or a possible decomposition (read right to left) of backup-maker, with all functions bound, taking data as the argument, and using the composition operator ".":
backup-maker: intermediate-computation . writer . intermediate-computation . compressor . intermediate-computation . exporter
I especially want to note that this model of architecture can be expressed later as either object interfaces, or as first-class functions, e.g. c++ std::function.
Edit: It can also be refined to terms of generics, where memory is a generic type argument, to provide type safety where wanted. E.g.
backup-maker<type M>: (data, exporter<M>, compressor<M>, writer<M>) -> ..
More information about the technique and benefits of Function Decomposition can be found here:
http://jfeltz.com/posts/2015-08-30-cost-decreasing-software-architecture.html
Your requirements seem contradictory:
You want to be specific (allowing only a subset (or only one ?) of combinations)
But you also want to be generic by using interfaces, DI, etc.
My advice is to keep things simple (in your case it means don't try to be generic) until your code evolve.
Only when your code will evolve, refactor in a more generic way. The code below shows a compromise between generic/specific:
public interface ICompressor {
public byte[] compress(byte[] source); //Note: the return type and type argument may not be revelant, just for demonstration purpose
}
public interface IExporter {
public File export(String connectionString); //Note: the return type and type argument may not be revelant, just for demonstration purpose
}
public final class Bzip2 implements ICompressor {
#Override
public final byte[] compress(byte[] source) {
//TODO
}
}
public final class MySQL implements IExporter {
#Override
public final File export(String connnectionString) {
//TODO
}
}
public abstract class ABackupStrategy {
private final ICompressor compressor;
private final IExporter exporter;
public ABackupStrategy(final ICompressor compressor, final IExporter exporter) {
this.compressor = compressor;
this.exporter = exporter;
}
public final void makeBackup() {
//TODO: compose with exporter and compressor to make your backup
}
}
public final class MyFirstBackupStrategy extends ABackupStrategy {
public MyFirstBackupStrategy(final Bzip2 compressor, final MySQL exporter) {
super(compressor, exporter);
}
}
With ICompressor and IExporter, you can easily add other compression algorithm, other database from which to export.
With ABackupStrategy, you can easily define a new allowed combination of concrete compressor/exporter by inheriting it.
Drawback: I had to make ABackupStrategy abstract without declaring any abstract method, which is in contradiction with the OOP-principles.

Java Serialization with RMI

I'm working on a project using Java RMI.
This is the class causing problem:
public class FSFile implements Serializable
{
public static final int READ = 0;
public static final int WRITE = 1;
private int flag;
private String filename;
private transient BufferedWriter writer;
private transient BufferedReader reader;
...
private void writeObject(ObjectOutputStream stream) throws IOException
{
stream.defaultWriteObject();
stream.writeObject(writer);
stream.writeObject(reader);
}
private void readObject(ObjectInputStream stream) throws IOException, ClassNotFoundException
{
stream.defaultReadObject();
writer = (BufferedWriter) stream.readObject();
reader = (BufferedReader) stream.readObject();
}
}
Basically, I use RMI to send that FSFile object to another process locally (for now) and here's the error I get:
java.rmi.UnmarshalException: error unmarshalling return; nested exception is:
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException;
java.io.BufferedReader
To be more precise, there's one class named FileService which use a function fetch() from a FileServer to get a FSFile in return. There is nothing special in the fetch() function, it just creates a FSFile and returns it. All communications between those classes are made via RMI.
How come I have an error like this ?
You can't serialize readers and writers. It makes no sense. It's like trying to send a telephone over a telephone line. If you want to send a file, send the file.
And your code just calls writeObject on these objects as though they were Serializable. They aren't. Otherwise you could have made them non-transient and omitted the custom readObject and writeObject methods altogether. Just re-coding what the system would have done anyway doesn't change anything. It certainly doesn't make classes Serializable that aren't.
If you don't want to re-implement file/stream sending via RMI you could look into RMIIO, it handles such things in concise and effective way.

The Object-Oriented way to separate the model from its representation

Suppose we have an object that represents the configuration of a piece of hardware. For the sake of argument, a temperature controller (TempController). It contains one property, the setpoint temperature.
I need to save this configuration to a file for use in some other device. The file format (FormatA) is set in stone. I don't want the TempController object to know about the file format... it's just not relevant to that object. So I make another object, "FormatAExporter", that transforms the TempController into the desired output.
A year later we make a new temperature controller, let's call it "AdvancedTempController", that not only has a setpoint but also has rate control, meaning one or two more properties. A new file format is also invented to store those properties... let's call it FormatB.
Both file formats are capable of representing both devices ( assume AdvancedTempController has reasonable defaults if it lacks settings ).
So here is the problem: Without using 'isa' or some other "cheating" way to figure out what type of object I have, how can FormatBExporter handle both cases?
My first instinct is to have a method in each temperature controller that can provide a customer exporter for that class, e.g., TempController.getExporter() and AdvancedTempController.getExporter(). This doesn't support multiple file formats well.
The only other approach that springs to mind is to have a method in each temperature controller that returns a list of properties and their values, and then the formatter can decide how to output those. It'd work, but that seems convoluted.
UPDATE: Upon further work, that latter approach doesn't really work well. If all your types are simple it might, but if your properties are Objects then you end up just pushing the problem down a level... you are forced to return a pair of String,Object values, and the exporter will have to know what the Objects actually are to make use of them. So it just pushes the problem to another level.
Are there any suggestions for how I might keep this flexible?
What you can do is let the TempControllers be responsible for persisting itself using a generic archiver.
class TempController
{
private Temperature _setPoint;
public Temperature SetPoint { get; set;}
public ImportFrom(Archive archive)
{
SetPoint = archive.Read("SetPoint");
}
public ExportTo(Archive archive)
{
archive.Write("SetPoint", SetPoint);
}
}
class AdvancedTempController
{
private Temperature _setPoint;
private Rate _rateControl;
public Temperature SetPoint { get; set;}
public Rate RateControl { get; set;}
public ImportFrom(Archive archive)
{
SetPoint = archive.Read("SetPoint");
RateControl = archive.ReadWithDefault("RateControl", Rate.Zero);
}
public ExportTo(Archive archive)
{
archive.Write("SetPoint", SetPoint);
archive.Write("RateControl", RateControl);
}
}
By keeping it this way, the controllers do not care how the actual values are stored but you are still keeping the internals of the object well encapsulated.
Now you can define an abstract Archive class that all archive classes can implement.
abstract class Archive
{
public abstract object Read(string key);
public abstract object ReadWithDefault(string key, object defaultValue);
public abstract void Write(string key);
}
FormatA archiver can do it one way, and FormatB archive can do it another.
class FormatAArchive : Archive
{
public object Read(string key)
{
// read stuff
}
public object ReadWithDefault(string key, object defaultValue)
{
// if store contains key, read stuff
// else return default value
}
public void Write(string key)
{
// write stuff
}
}
class FormatBArchive : Archive
{
public object Read(string key)
{
// read stuff
}
public object ReadWithDefault(string key, object defaultValue)
{
// if store contains key, read stuff
// else return default value
}
public void Write(string key)
{
// write stuff
}
}
You can add in another Controller type and pass it whatever formatter. You can also create another formatter and pass it to whichever controller.
In C# or other languages that support this you can do this:
class TempController {
int SetPoint;
}
class AdvancedTempController : TempController {
int Rate;
}
class FormatAExporter {
void Export(TempController tc) {
Write(tc.SetPoint);
}
}
class FormatBExporter {
void Export(TempController tc) {
if (tc is AdvancedTempController) {
Write((tc as AdvancedTempController).Rate);
}
Write(tc.SetPoint);
}
}
I'd have the "temp controller", through a getState method, return a map (e.g. in Python a dict, in Javascript an object, in C++ a std::map or std::hashmap, etc, etc) of its properties and current values -- what's convoluted about it?! Could hardly be simpler, it's totally extensible, and totally decoupled from the use it's put to (displaying, serializing, &c).
Well, a lot of that depends on the file formats you're talking about.
If they're based on key/value combinations (including nested ones, like xml), then having some kind of intermediate memory object that's loosely typed that can be thrown at the appropriate file format writer is a good way to do it.
If not, then you've got a scenario where you've got four combinations of objects and file formats, with custom logic for each scenario. In that case, it may not be possible to have a single representation for each file format that can deal with either controller. In other words, if you can't generalize the file format writer, you can't generalize it.
I don't really like the idea of the controllers having exporters - I'm just not a fan of objects knowing about storage mechanisms and whatnot (they may know about the concept of storage, and have a specific instance given to them via some DI mechanism). But I think you're in agreement with that, and for pretty much the same reasons.
If FormatBExporter takes an AdvancedTempController, then you can make a bridge class that makes TempController conform to AdvancedTempController. You may need to add some sort of getFormat() function to AdvancedTempController though.
For example:
FormatBExporter exporterB;
TempController tempController;
AdvancedTempController bridged = TempToAdvancedTempBridge(tempController);
exporterB.export(bridged);
There is also the option of using a key-to-value mapping scheme. FormatAExporter exports/imports a value for key "setpoint". FormatBExporter exports/imports a values for keys "setpoint" and "ratecontrol". This way, old FormatAExporter can still read the new file format (it just ignores "ratecontrol") and FormatBExporter can read the old file format (if "ratecontrol" is missing, it uses a default).
In the OO model, the object methods as a collective is the controller. It's more useful to separate your program in to the M and V and not so much the C if you're programming using OO.
I guess this is the where the Factory method pattern would apply