How to correctly configure a global serializer? - serialization

Question

I'm trying to configure a global serializer for Hazelcast 3.12 (My application runs on payara 5 which comes bundled with this version of Hazelcast so I cannot update to 4.x)
According to the manual (https://docs.hazelcast.org/docs/3.12/manual/html-single/index.html#global-serializer) implementing and configuring a global serializer should be simple, here is my Serializer:
public class MyGlobalStreamSerializer implements StreamSerializer<Object> {
private static FSTConfiguration conf = FSTConfiguration.createDefaultConfiguration();
#Override
public int getTypeId() {
return 123;
}
#Override
public void destroy() {
}
#Override
public void write(ObjectDataOutput objectDataOutput, Object o) throws IOException {
objectDataOutput.write(conf.asByteArray(o));
}
#Override
public Object read(ObjectDataInput objectDataInput) throws IOException {
return conf.asObject(objectDataInput.readByteArray());
}
}
I'm using fast-serialization for the actual serialization implementation:
<dependency>
<groupId>de.ruedigermoeller</groupId>
<artifactId>fst</artifactId>
<version>2.57</version>
</dependency>
And the serialization configuration in my hazelcast-config.xml
<serialization>
<portable-version>0</portable-version>
<serializers>
<global-serializer override-java-serialization="true" >de.mycompany.myapp.domain.MyGlobalStreamSerializer</global-serializer>
</serializers>
</serialization>
however at application startup I get a NullPointerException when the configuration is built:
[2020-10-20T12:37:14.348+0200] [Payara 5.2020] [SEVERE] [NCLS-CORE-00014] [javax.enterprise.system.core] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1603190234348] [levelValue: 1000] [[
Shutdown required
MultiException stack 1 of 1
MultiException stack 1 of 4
java.lang.NullPointerException
at fish.payara.nucleus.hazelcast.HazelcastCore.buildConfiguration(HazelcastCore.java:333)
at fish.payara.nucleus.hazelcast.HazelcastCore.bootstrapHazelcast(HazelcastCore.java:480)
at fish.payara.nucleus.hazelcast.HazelcastCore.getInstance(HazelcastCore.java:234)
at fish.payara.nucleus.config.ClusteredConfig.postConstruct(ClusteredConfig.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1268)
at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:309)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:351)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:679)
at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:54)
at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:188)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:211)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:334)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:821)
MultiException stack 2 of 4
java.lang.IllegalStateException: Unable to perform operation: post construct on fish.payara.nucleus.config.ClusteredConfig
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:369)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:679)
at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:54)
at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:188)
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:211)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:334)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:821)
MultiException stack 3 of 4
java.lang.IllegalArgumentException: While attempting to resolve the dependencies of fish.payara.nucleus.requesttracing.RequestTracingService errors were found
at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:224)
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:334)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:821)
MultiException stack 4 of 4
java.lang.IllegalStateException: Unable to perform operation: resolve on fish.payara.nucleus.requesttracing.RequestTracingService
at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:363)
at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:463)
at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281)
at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65)
at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2102)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93)
at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213)
at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:821)
]]
Omitting the package / using just the class name as value for global-serializer in the configuration as seen in the manual didn't work either, my last attempt was with the full path because that's how it's done in the hazelcast-full-example.xml that comes with the release.
I also saw another post here where someone was trying to implement a global serializer using Kryo as implementation for the actual serialization, the code and config looked pretty much the same but he didn't seem to have my problem (Hazelcast with global serializer (Kryo) - There is no suitable de-serializer for type)
Can anyone tell me what I'm doing wrong? Thanks in advance!

Answer 1

Payara Server uses its own serializer which can't be disabled, see the docs: https://docs.payara.fish/community/docs/5.2020.5/documentation/payara-server/hazelcast/configuration.html. The Payara serializer delegates to a Hazelcast global serializer which is specified in the Hazelcast config. This serialized should be in the Payara Server's classpath, it's not enough to add it to your application.
Try adding a JAR file with MyGlobalStreamSerializer and all its dependencies (e.g. the fst JAR and its dependencies) to the lib directory in your domain directory, this will add it to the server's classpath. Here's the docs on the asadmin add-library command, which you can use: https://docs.payara.fish/community/docs/5.2020.5/documentation/payara-server/asadmin-commands/misc-commands.html#add-library-command.
In your case, it seems that the exception is thrown when Payara Server wants to log a message that your global serializer is not a subclass of StreamSesrializer which the Payara serializer expects: https://github.com/payara/Payara/blob/master/nucleus/payara-modules/hazelcast-bootstrap/src/main/java/fish/payara/nucleus/hazelcast/HazelcastCore.java#L333
That code hasn't changed in any version of Payara Server 5.2020.x and the only way it can throw a NullPointerException is if the global serializer is null. I expect this happens because Hazelcast can't find the class of your serializer and returns null for the serializer.