I am writing a Flink application in Kotlin and data classes (as well as other Kotlin classes) are not identified as valid POJO types.
The Flink documentation states that a data type is recognized as a POJO type (and allows "by-name" field referencing) if the following conditions are fulfilled:
The class is public and standalone
The class has a public no-argument constructor
All non-static, non-transient fields in the class are either public (and non-final) or have public getter and setter methods that follow Java beans naming conventions.
I receive the following when implementing a Kotlin data class, which should meet the aforementioned conditions to be recognized as a POJO:
[main] INFO org.apache.flink.api.java.typeutils.TypeExtractor -
Class class <Class> cannot be used as a POJO type because not all
fields are valid POJO fields, and must be processed as GenericType.
Please read the Flink documentation on "Data Types & Serialization"
for details of the effect on performance.
Investigating further, I reviewed Flink's TypeExtractor.isValidPojoField method # https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java
In a separate project, I applied the field checks with java.lang.reflect.Modifier on a simple Kotlin data class in attempt to narrow down the issue.
data class SomeDataClass(
val topic: String = "",
val message: String = ""
)
While Kotlin class fields have public visibility by default, Modifier.isPublic recognizes the fields as private. Additionally, Modifier.isFinal recognizes the fields as final.
val clazz = SomeDataClass::class.java
val fields = clazz.declaredFields
fields.forEach { it ->
println("field: $it")
println(it.genericType)
println("public? " + Modifier.isPublic(it.modifiers))
println("final? " + Modifier.isFinal(it.modifiers))
println("transient? " + Modifier.isTransient(it.modifiers))
println("static? " + Modifier.isStatic(it.modifiers))
}
>
field: private final java.lang.String SomeDataClass.topic
class java.lang.String
public? false
final? true
transient? false
static? false
However, public getter and setter methods are created for these fields, so this object should still meet the POJO criteria.
println(clazz.declaredMethods.toList())
>
[public boolean SomeDataClass.equals(java.lang.Object),
public java.lang.String SomeDataClass.toString(),
public int SomeDataClass.hashCode(),
**public final java.lang.String SomeDataClass.getMessage(),**
public final SomeDataClass SomeDataClass.copy(java.lang.String,java.lang.String),
**public final java.lang.String SomeDataClass.getTopic(),**
public final java.lang.String SomeDataClass.component1(),
public final java.lang.String SomeDataClass.component2(),
public static SomeDataClass SomeDataClass.copy$default(SomeDataClass,java.lang.String,java.lang.String,int,java.lang.Object)]
The getter and setter methods, however, are final, which leads me to believe this is the issue.
I am relatively new to JVM development, so any help would be greatly appreciated. I have reviewed the Flink Jira, Stack Overflow, and Flink mailing list and have not found a similar issue reported.
I see at least two POJO rules violation with provided data class.
1) The class has a public no-argument constructor
By default, Kotlin will not generate overloads to functions with default parameter values (https://kotlinlang.org/docs/reference/java-to-kotlin-interop.html#overloads-generation)
So your compiled class will have only one constructor with two-parameter constructor, and no-argument constructor will not be created. To force Kotlin compiler to generate multiple overloads one should use #JvmOverloads annotation. In your case it will be used on constructor so we also need to add constructor keyword:
data class SomeDataClass #JvmOverloads constructor
2) All non-static, non-transient fields in the class are either public (and non-final) or have public getter and setter methods that follow Java beans naming conventions.
Since you are using val keywords the generated fields will be final, and no setter will be generated for them. So you can change vals to vars and the fields will no longer be final and proper getters and setters will be generated too. (Or you could use another annotation to prevent generating getters and setters and expose a field as it is https://kotlinlang.org/docs/reference/java-to-kotlin-interop.html#instance-fields)
So final code should be like this:
data class SomeDataClass #JvmOverloads constructor(
var topic: String = "",
var message: String = ""
)
If you wish to use kotlin data classes without any modifications to match a Java POJO (i.e: no default/null values required and keep using val).
You can either:
provide a custom Kryo serializer to serialize it using Protobuf or Avro (or the tool and format of your choice).
use a Kotlin friendly type serializer that will serialize your data class similarly to a case class.
Related
(I am a new ByteBuddy user. I'm using ByteBuddy version 1.10.8 and JDK 11 without the module path or any other part of the module system.)
I have a nested class declared like this:
public static class Frob {
protected Frob() {
super();
}
public String sayHello() {
return "Hello!";
}
}
(Its containing class is foo.bar.TestExplorations.)
When I create a dynamic subclass of Frob named foo.bar.Crap like the following, everything works OK as I would expect:
final String className = "foo.bar.Crap";
final DynamicType.Unloaded<?> dynamicTypeUnloaded = new ByteBuddy()
.subclass(Frob.class)
.name(className)
.make();
final Class<?> mySubclass = dynamicTypeUnloaded
.load(this.getClass().getClassLoader(), ClassLoadingStrategy.Default.WRAPPER)
.getLoaded();
assertNotNull(mySubclass);
assertEquals(className, mySubclass.getName());
final Object frobSubclass = mySubclass.newInstance();
assertTrue(frobSubclass instanceof Frob);
But if I change Frob's constructor so that it is package private, I get the following error from the final assertion:
java.lang.IllegalAccessError: class foo.bar.Crap tried to access method 'void foo.bar.TestExplorations$Frob.<init>()' (foo.bar.Crap is in unnamed module of loader net.bytebuddy.dynamic.loading.ByteArrayClassLoader #5e3d57c7; foo.bar.TestExplorations$Frob is in unnamed module of loader 'app')
For some reason, Crap's constructor cannot call super(), even though Crap and Frob are in the same package, and Frob() is defined as package-private.
I have a sense the JDK module system is to blame here, even though I am deliberately (very, very deliberately) not using it. I know the module system does not like split packages, which is what it looks like to me is going on here. Is there a constructor strategy or other mechanism to work around this problem?
In Java, a package is only equal to another package if it has the same name and is loaded by the same class loader (the same as it is with classes). If you are using the WRAPPER strategy, you cannot access package-private members of any super class. Byte Buddy does not forbid the generation as it would be legal to do in javac but you would need to use the INJECTION strategy to do what you want to make sure that classes are loaded by the same class loader. Mind that it uses internal API, therefore, from Java 9, you'd rather use a ForLookup class loading strategy.
I know we can use Jackson MixIn's to rename a property or to ignore a property (see examples here). But is it possible to add a property?
The added property can be:
A constant (such as a version number)
A computed value (e.g. if the source class has properties for getWidth() and getHeight(), but we want to ignore both and export a getArea() instead)
Flattened information from nested members (e.g. a class has a member Information which in turn has a member Description, and we want to have a new property for description and skipping the nesting structure of Information)
From documentation:
"Mix-in" annotations are a way to associate annotations with classes,
without modifying (target) classes themselves, originally intended to
help support 3rd party datatypes where user can not modify sources to
add annotations.
With mix-ins you can:
1. Define that annotations of a '''mix-in class''' (or interface)
2. will be used with a '''target class''' (or interface) such that it
appears
3. as if the ''target class'' had all annotations that the ''mix-in''
class has (for purposes of configuring serialization /
deserialization)
To solve your problems you can:
Create new POJO which has all required fields.
Implement custom serialiser.
Before serialisation convert POJO to Map and add/remove nodes.
Use com.fasterxml.jackson.databind.ser.BeanSerializerModifier to extend custom serialisers. See: Jackson custom serialization and deserialization.
For example, to add a constant version to each object you can wrap it in Verisoned class:
class Versioned {
private final String version;
#JsonUnwrapped
private final Object pojo;
public Versioned(String version, Object pojo) {
this.version = version;
this.pojo = pojo;
}
public String getVersion() {
return version;
}
public Object getPojo() {
return pojo;
}
}
Now, if you wrap an Arae(width, height) object:
Area area = new Area(11, 12);
String json = mapper.writeValueAsString(new Versioned("1.1", area));
output will be:
{
"version" : "1.1",
"width" : 11,
"height" : 12
}
I am using EMF-JSON for serializing an EMF model instance. The problem is, that a subclass of EObject gets treated as such and not as the subclass. Thus, properties in the subclass get lost during serialization, as the EObjectSerializer is selected. How can I make sure that the properties of the subclass are included without changing the class itself (referring to the inclusion annotations)? In the following you find the generated interfaces. In this case, the ref property of the ArithVar class is not serialized.
public interface ArithExpr extends EObject {}
public interface ArithVar extends ArithExpr {
VarType getRef();
void setRef(VarType value);
}
public interface VarType extends EObject {}
Any ideas?
From the documentation:
References are by default serialized as JSON objects that contain two fields. The first field is the type of the referenced object and the second field is the URI of the referenced object. The type field is named eClass and the URI field is named $ref.
So my guess is it should work by default. Have you tried debugging through EObjectSerializer ? My guess is it probably uses the EMF reflective API to iterate through all the features of an EObject and serialize them.
I'm writing a Jax-RS application using Jersey, and Jackson2 under the hood to facilitate JSON i/o. The service itself works fine, but I'd like to improve it by having the Jackson mapper automagically serialize/deserialize date and date-times to JodaTime objects.
I'm following the documentation here and have added the relevant jars, but I'm lost on this instruction:
Registering module
To use Joda datatypes with Jackson, you will first need to register the module first (same as with all Jackson datatype modules):
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JodaModule());
I've tried to do this in the custom class that extends jax.ws.rs.core.Application, but I'm not at all confident in that solution. I'm currently getting this error:
Can not instantiate value of type [simple type, class org.joda.time.DateTime] from String value ('2014-10-22'); no single-String constructor/factory method
at [Source: org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream#3471b6d5; line: 7, column: 25]
Other than the general impression that this module registration needs to happen at application (servlet?) startup, I have no idea what to do with this information. Do I need to annotate a custom class with something in particular to have it picked up ? Should I be extending some class ?
The examples I find on StackOverflow usually stick it in main() and call the mapper directly, but I'm relying on Jackson Databinding so the examples aren't relevant. Any direction is appreciated.
You'll basically want to create/configure/return the ObjectMapper in a ContextResolver. Something like
#Provider
public class ObjectMapperContextResolver implements ContextResolver<ObjectMapper> {
final ObjectMapper mapper = new ObjectMapper();
public ObjectMapperContextResolver() {
mapper.registerModule(new JodaModule());
}
#Override
public ObjectMapper getContext(Class<?> type) {
return mapper;
}
}
If you are using package scanning to discover your resources, then the #Provider annotation should allow this class to be discovered and registered also.
Basically what happens, is the the MessageBodyReader and MessageBodyWriter provided by Jackson, used for unmarshalling and marshalling, respectively, will call the getContext method in the ContextResolver, to determine the ObjectMapper to use. The reader/writer will pass in the class (in a reader it will be the type expected in a method param, in a writer it will be the type returned as-a/in-a response), meaning we are allowed to use differently configured ObjectMapper for different classes, as seen here. In the above solution, it is used for all classes.
I have looked at the Dozer's FAQs and docs, including the SourceForge forum, but I didn't see any good tutorial or even a simple example on how to implement a custom BeanFactory.
Everyone says, "Just implement a BeanFactory". How exactly do you implement it?
I've Googled and all I see are just jars and sources of jars.
Here is one of my BeanFactories, I hope it helps to explain the common pattern:
public class LineBeanFactory implements BeanFactory {
#Override
public Object createBean(final Object source, final Class<?> sourceClass, final String targetBeanId) {
final LineDto dto = (LineDto) source;
return new Line(dto.getCode(), dto.getElectrified(), dto.getName());
}
}
And the corresponding XML mapping:
<mapping>
<class-a bean-factory="com.floyd.nav.web.ws.mapping.dozer.LineBeanFactory">com.floyd.nav.core.model.Line</class-a>
<class-b>com.floyd.nav.web.contract.dto.LineDto</class-b>
</mapping>
This way I declare that when a new instance of Line is needed then it should create it with my BeanFactory. Here is a unit test, that can explain it:
#Test
public void Line_is_created_with_three_arg_constructor_from_LineDto() {
final LineDto dto = createTransientLineDto();
final Line line = (Line) this.lineBeanFactory.createBean(dto, LineDto.class, null);
assertEquals(dto.getCode(), line.getCode());
assertEquals(dto.getElectrified(), line.isElectrified());
assertEquals(dto.getName(), line.getName());
}
So Object source is the source bean that is mapped, Class sourceClass is the class of the source bean (I'm ignoring it, 'cause it will always be a LineDto instance). String targetBeanId is the ID of the destination bean (too ignored).
A custom bean factory is a class that has a method that creates a bean. There are two "flavours"
a) static create method
SomeBean x = SomeBeanFactory.createSomeBean();
b) instance create method
SomeBeanFactory sbf = new SomeBeanFactory();
SomeBean x = sbf.createSomeBean();
You would create a bean factory if creating and setting up your bean requires some tricky logic, like for example initial value of certain properties depend on external configuration file. A bean factory class allows you to centralize "knowledge" about how to create such a tricky bean. Other classes just call create method without worying how to correctly create such bean.
Here is an actual implementation. Obviously it does not make a lot of sense, since Dozer would do the same without the BeanFactory, but instead of just returning an object, you could initialized it somehow differently.
public class ComponentBeanFactory implements BeanFactory {
#Override
public Object createBean(Object source, Class<?> sourceClass,
String targetBeanId) {
return new ComponentDto();
}
}
Why do you need a BeanFactory anyways? Maybe that would help understanding your question.