Jackson ContextualDeserializer cannot get the contextualType when decoding a generic Kotlin class - kotlin

I have a custom implementation of a JsonDeserializer which implements a ContextualDeserializer in order to deserialize a generic class. Everything works fine until the generic class itself has a property which is also a generic class.
For demo purposes, I simplified the code to the minimum
My class is
data class MyObject<out Content>(
val content: Content
)
Deserializing MyObject<String> works:
{
"content": "Hello"
}
but deserializing MyObject<MyObject<String>>> does not:
{
"content": {
"content": "Hello"
}
}
The custom deserializer is the following. The issue seems to be that contextualType.containedType is returning null.
I believe this issue is in the line
val content = ctxt.readTreeAsValue(contentNode, contentType!!.rawClass)
because the .rawClass does not have any additional information about the generic type. So the content is deserialized as just MyObject instead of MyObject<String>.
class MyObjectDeserializer : JsonDeserializer<MyObject<*>?>(), ContextualDeserializer {
private var contentType: JavaType? = null
override fun createContextual(ctxt: DeserializationContext?, property: BeanProperty?): JsonDeserializer<*> {
contentType = if (property == null)
ctxt!!.contextualType.containedType(0)
else
property.type.containedType(0)
if (contentType != null) {
println("${ctxt!!.contextualType} has contained type $contentType")
} else {
// Here is where the issue occurs
println("${ctxt!!.contextualType} does not have any contained types")
}
return this
}
override fun deserialize(p: JsonParser?, ctxt: DeserializationContext): MyObject<*> {
val codec = p?.codec ?: throw NullPointerException()
val node = codec.readTree<JsonNode>(p)
val contentNode = node.get("content")
val content = ctxt.readTreeAsValue(contentNode, contentType!!.rawClass)
return MyObject(content)
}
}
and I register the deserialzers like so
fun ObjectMapper.registerMyDeserializers(): ObjectMapper {
val module = SimpleModule().also {
it.addDeserializer(MyObject::class.java, MyObjectDeserializer())
}
return this.registerModule(module)
}
The unit test the demonstrates the issue is the follow. The first test works fine, the second fails.
class ResponseParsingTests {
private val objectMapper = jacksonObjectMapper().registerMyDeserializers()
#Test
fun `parses String as content`() {
val json = """
{
"content": "Hello"
}
""".trimIndent()
val result = objectMapper.readValue<MyObject<String>>(json)
assertEquals("Hello", result.content)
}
#Test
fun `parses MyObject as content`() {
val json = """
{
"content": {
"content": "Hello"
}
}
""".trimIndent()
val result = objectMapper.readValue<MyObject<MyObject<String>>>(json)
assertEquals("Hello", result.content.content)
}
}
Please note that I'm aware the this example class does not require any custom deserialzer at all. However my real use case is a bit more complex and I need to use a custom deserializer because I'm publishing my code as part of a library which suppoorts multiple serialization frameworks (gson, jackson, kotlinx). So the serialization cannot be part of my actual class but rather in a separate one.

Related

Custom lint rule kotlin object

I'm trying to create a custom lint rule in order to avoid using a kotlin object class.
object AColors {
val white: Color = Color(0xFFFFFFFF)
}
fun main() {
val myColor = AColors.white //Lint error : Do not use AColors
}
How can I manage to get a lint issue when AColors is used?
In my case, AColors can't be private because I need it for a specific case.
I tried to create a custom lint rule to check the import but this is not a bullet-proof solution as you can use AColors without importing anything
class InvalidImportHandler(private val context: JavaContext) : UElementHandler() {
override fun visitImportStatement(node: UImportStatement) {
//check the import
}
}
For this specific case, you may check for USimpleNameReferenceExpressions and then check if the reference is to the AColors class. Like so:
class AColorsReferenceDetector : Detector(), Detector.UastScanner {
override fun getApplicableUastTypes(): List<Class<out UElement>> {
return listOf(USimpleNameReferenceExpression::class.java)
}
override fun createUastHandler(context: JavaContext): UElementHandler {
return object : UElementHandler() {
override fun visitSimpleNameReferenceExpression(
node: USimpleNameReferenceExpression
) {
val element = node.resolve()?.unwrapped
if (element is KtObjectDeclaration && element.name == "AColors") {
context.report(
ISSUE,
node,
context.getLocation(node.uastParent)
)
}
}
}
}
companion object {
val ISSUE = Issue.create(
AColorsReferenceDetector::class.simpleName.orEmpty(),
"Do not use AColors",
"Do not use AColors",
Category.CORRECTNESS,
10,
Severity.ERROR,
Implementation(
AColorsReferenceDetector::class.java,
EnumSet.of(Scope.JAVA_FILE)
)
)
}
}
Example IssueRegistry:
class IssueRegistry : IssueRegistry() {
override val api = CURRENT_API
override val minApi: Int
get() = 8
override val vendor: Vendor = Vendor()
override val issues
get() = listOf(AColorsReferenceDetector.ISSUE)
}
Example result:

Kotlinx Serialization, inlining sealed class/interface [duplicate]

This question already has an answer here:
kotlinx deserialization: different types && scalar && arrays
(1 answer)
Closed 7 months ago.
With a structure similar to the following:
#Serializable
sealed class Parameters
#Serializable
data class StringContainer(val value: String): Parameters()
#Serializable
data class IntContainer(val value: Int): Parameters()
#Serializable
data class MapContainer(val value: Map<String, Parameters>): Parameters()
// more such as list, bool and other fairly (in the context) straight forward types
And the following container class:
#Serializable
data class PluginConfiguration(
// other value
val parameters: Parameters.MapContainer,
)
I want to reach a (de)serialization where the paramters are configured as a flexible json (or other) map, as one would usually expect:
{
"parameters": {
"key1": "String value",
"key2": 12,
"key3": {}
}
}
And so on. Effectively creating a flexible structure that is still structured enough to not be completely uncontrolled as Any would be. There's a fairly clearly defined (de)serialization, but I cannot figure how to do this.
I've tried reading the following
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serialization-guide.md
And I do have a hunch that a polymorphic serializer is needed, but so far I'm bumping in to either generic structures, which I believe is way overkill for my purpose or that it for some reason cannot find the serializer for my subclasses, when writing a custom serializer for Parameters.
Update
So using custom serializers combined with surrogate classes, most things are working. The current problem is when values are put into the map, I get a kotlin.IllegalStateException: Primitives cannot be serialized polymorphically with 'type' parameter. You can use 'JsonBuilder.useArrayPolymorphism' instead. Even when I enable array polymorphism this error arises
The answer with kotlinx deserialization: different types && scalar && arrays is basically the answer, and the one I will accept. However, for future use, the complete code to my solution is as follows:
Class hierarchy
#kotlinx.serialization.Serializable(with = ParametersSerializer::class)
sealed interface Parameters
#kotlinx.serialization.Serializable(with = IntContainerSerializer::class)
data class IntContainer(
val value: Int
) : Parameters
#kotlinx.serialization.Serializable(with = StringContainerSerializer::class)
data class StringContainer(
val value: String
) : Parameters
#kotlinx.serialization.Serializable(with = MapContainerSerializer::class)
data class MapContainer(
val value: Map<String, Parameters>
) : Parameters
#kotlinx.serialization.Serializable
data class PluginConfiguration(
val plugin: String,
val parameters: MenuRunnerTest.MapContainer
)
Serializers:
abstract class BaseParametersSerializer<T : Parameters> : KSerializer<T> {
override val descriptor: SerialDescriptor = JsonElement.serializer().descriptor
override fun serialize(encoder: Encoder, value: T) {
fun toJsonElement(value: Parameters): JsonElement = when (value) {
is IntContainer -> JsonPrimitive(value.value)
is MapContainer -> JsonObject(
value.value.mapValues { toJsonElement(it.value) }
)
is StringContainer -> JsonPrimitive(value.value)
}
val sur = toJsonElement(value)
encoder.encodeSerializableValue(JsonElement.serializer(), sur)
}
override fun deserialize(decoder: Decoder): T {
with(decoder as JsonDecoder) {
val jsonElement = decodeJsonElement()
return deserializeJson(jsonElement)
}
}
abstract fun deserializeJson(jsonElement: JsonElement): T
}
object ParametersSerializer : BaseParametersSerializer<Parameters>() {
override fun deserializeJson(jsonElement: JsonElement): Parameters {
return when(jsonElement) {
is JsonPrimitive -> when {
jsonElement.isString -> StringContainerSerializer.deserializeJson(jsonElement)
else -> IntContainerSerializer.deserializeJson(jsonElement)
}
is JsonObject -> MapContainerSerializer.deserializeJson(jsonElement)
else -> throw IllegalArgumentException("Only ints, strings and strings are allowed here")
}
}
}
object StringContainerSerializer : BaseParametersSerializer<StringContainer>() {
override fun deserializeJson(jsonElement: JsonElement): StringContainer {
return when(jsonElement) {
is JsonPrimitive -> StringContainer(jsonElement.content)
else -> throw IllegalArgumentException("Only strings are allowed here")
}
}
}
object IntContainerSerializer : BaseParametersSerializer<IntContainer>() {
override fun deserializeJson(jsonElement: JsonElement): IntContainer {
return when (jsonElement) {
is JsonPrimitive -> IntContainer(jsonElement.int)
else -> throw IllegalArgumentException("Only ints are allowed here")
}
}
}
object MapContainerSerializer : BaseParametersSerializer<MapContainer>() {
override fun deserializeJson(jsonElement: JsonElement): MapContainer {
return when (jsonElement) {
is JsonObject -> MapContainer(jsonElement.mapValues { ParametersSerializer.deserializeJson(it.value) })
else -> throw IllegalArgumentException("Only maps are allowed here")
}
}
}
This structure should be expandable for lists, doubles and other structures, not included in the example :)

Implementing observable properties that can also serialize in Kotlin

I'm trying to build a class where certain values are Observable but also Serializable.
This obviously works and the serialization works, but it's very boilerplate-heavy having to add a setter for every single field and manually having to call change(...) inside each setter:
interface Observable {
fun change(message: String) {
println("changing $message")
}
}
#Serializable
class BlahVO : Observable {
var value2: String = ""
set(value) {
field = value
change("value2")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value2 = "test2" })
correctly outputs
changing value2
{"value2":"test2"}
I've tried introducing Delegates:
interface Observable {
fun change(message: String) {
println("changing $message")
}
#Suppress("ClassName")
class default<T>(defaultValue: T) {
private var value: T = defaultValue
operator fun getValue(observable: Observable, property: KProperty<*>): T {
return value
}
operator fun setValue(observable: Observable, property: KProperty<*>, value: T) {
this.value = value
observable.change(property.name)
}
}
}
#Serializable
class BlahVO : Observable {
var value1: String by Observable.default("value1")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value1 = "test1" }) correctly triggers change detection, but it doesn't serialize:
changing value1
{}
If I go from Observable to ReadWriteProperty,
interface Observable {
fun change(message: String) {
println("changing $message")
}
fun <T> look(defaultValue: T): ReadWriteProperty<Observable, T> {
return OP(defaultValue, this)
}
class OP<T>(defaultValue: T, val observable: Observable) : ObservableProperty<T>(defaultValue) {
override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
super.setValue(thisRef, property, value)
observable.change("blah!")
}
}
}
#Serializable
class BlahVO : Observable {
var value3: String by this.look("value3")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
the result is the same:
changing blah!
{}
Similarly for Delegates.vetoable
var value4: String by Delegates.vetoable("value4", {
property: KProperty<*>, oldstring: String, newString: String ->
this.change(property.name)
true
})
outputs:
changing value4
{}
Delegates just doesn't seem to work with Kotlin Serialization
What other options are there to observe a property's changes without breaking its serialization that will also work on other platforms (KotlinJS, KotlinJVM, Android, ...)?
Serialization and Deserialization of Kotlin Delegates is not supported by kotlinx.serialization as of now.
There is an open issue #1578 on GitHub regarding this feature.
According to the issue you can create an intermediate data-transfer object, which gets serialized instead of the original object. Also you could write a custom serializer to support the serialization of Kotlin Delegates, which seems to be even more boilerplate, then writing custom getters and setters, as proposed in the question.
Data Transfer Object
By mapping your original object to a simple data transfer object without delegates, you can utilize the default serialization mechanisms.
This also has the nice side effect to cleanse your data model classes from framework specific annotations, such as #Serializable.
class DataModel {
var observedProperty: String by Delegates.observable("initial") { property, before, after ->
println("""Hey, I changed "${property.name}" from "$before" to "$after"!""")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this.toDto())
}
}
fun DataModel.toDto() = DataTransferObject(observedProperty)
#Serializable
class DataTransferObject(val observedProperty: String)
fun main() {
val data = DataModel()
println(data.toJson())
data.observedProperty = "changed"
println(data.toJson())
}
This yields the following result:
{"observedProperty":"initial"}
Hey, I changed "observedProperty" from "initial" to "changed"!
{"observedProperty":"changed"}
Custom data type
If changing the data type is an option, you could write a wrapping class which gets (de)serialized transparently. Something along the lines of the following might work.
#Serializable
class ClassWithMonitoredString(val monitoredProperty: MonitoredString) {
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
fun main() {
val monitoredString = obs("obsDefault") { before, after ->
println("""I changed from "$before" to "$after"!""")
}
val data = ClassWithMonitoredString(monitoredString)
println(data.toJson())
data.monitoredProperty.value = "obsChanged"
println(data.toJson())
}
Which yields the following result:
{"monitoredProperty":"obsDefault"}
I changed from "obsDefault" to "obsChanged"!
{"monitoredProperty":"obsChanged"}
You however lose information about which property changed, as you don't have easy access to the field name. Also you have to change your data structures, as mentioned above and might not be desirable or even possible. In addition, this work only for Strings for now, even though one might make it more generic though.
Also, this requires a lot of boilerplate to start with. On the call site however, you just have to wrap the actual value in an call to obs.
I used the following boilerplate to get it to work.
typealias OnChange = (before: String, after: String) -> Unit
#Serializable(with = MonitoredStringSerializer::class)
class MonitoredString(initialValue: String, var onChange: OnChange?) {
var value: String = initialValue
set(value) {
onChange?.invoke(field, value)
field = value
}
}
fun obs(value: String, onChange: OnChange? = null) = MonitoredString(value, onChange)
object MonitoredStringSerializer : KSerializer<MonitoredString> {
override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("MonitoredString", PrimitiveKind.STRING)
override fun serialize(encoder: Encoder, value: MonitoredString) {
encoder.encodeString(value.value)
}
override fun deserialize(decoder: Decoder): MonitoredString {
return MonitoredString(decoder.decodeString(), null)
}
}

How to validate json in kotlin

There is a kotlin class with the following structure.
data class Person(
#field:Length(max = 5)
val name: String,
val phones: List<Phone>
)
data class Phone(
#field:Length(max = 10)
val number: String
)
When converting the json string through objectMapper, I want to receive all the violation values.
ex) JSON object is not valid. Reasons (3) name length must be 5, number length must be 10, ...
#Test
fun test() {
val json = """
{
"name": "name",
"phones": [
{ "number": "1234567890123456" },
{ "number": "1234567890123456" }
]
}
""".trimIndent()
try {
objectMapper.readValue(json, Person::class.java)
} catch (ex: ConstraintViolationException) {
val violations = ex.constraintViolations
println(violations.size) // expected size = 3
}
}
However, the above code fails to catch the exception and causes the exception below.
com.fasterxml.jackson.databind.JsonMappingException: JSON object is not valid. Reasons (1): {"bean":"Phone","property":"number","value":"1234567890123456","message": "..."}, (through reference chain: Person["phones"]->java.util.ArrayList[0])
Looking at the reason, those wrapped in a list do not throw ConstructionViolationException, but throw JsonMappingException.
below dependencies
plugins {
id("org.springframework.boot") version "2.4.3"
id("io.spring.dependency-management") version "1.0.11.RELEASE"
kotlin("jvm") version "1.4.30"
kotlin("plugin.spring") version "1.4.30"
}
dependencies {
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("org.springframework.boot:spring-boot-starter-validation")
testImplementation("org.springframework.boot:spring-boot-starter-test")
}
class BeanValidationDeserializer(base: BeanDeserializerBase?) : BeanDeserializer(base) {
private val logger = LoggerFactory.getLogger(this::class.java)
private val validator = Validation.buildDefaultValidatorFactory().validator
override fun deserialize(parser: JsonParser?, ctxt: DeserializationContext?): Any {
val instance = super.deserialize(parser, ctxt)
validate(instance)
return instance
}
private fun validate(instance: Any) {
val violations = validator.validate(instance)
if (violations.isNotEmpty()) {
val message = StringBuilder()
message.append("JSON object is not valid. Reasons (").append(violations.size).append("): ")
for (violation in violations) {
message.append("{\"bean\":\"${violation.rootBeanClass.name}\",")
.append("\"property\":\"${violation.propertyPath}\",")
.append("\"value\":\"${violation.invalidValue}\",")
.append("\"message\": \"${violation.message}\"}")
.append(", ")
}
logger.warn(message.toString())
throw ConstraintViolationException(message.toString(), violations)
}
}
}
#Bean
fun objectMapper(): ObjectMapper = Jackson2ObjectMapperBuilder.json()
.featuresToDisable(MapperFeature.DEFAULT_VIEW_INCLUSION)
.featuresToDisable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
.featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
.modules(ParameterNamesModule(), JavaTimeModule(), Jdk8Module(), KotlinModule(), customValidationModule())
.build()
#Bean
fun customValidationModule(): SimpleModule {
val validationModule = SimpleModule()
validationModule.setDeserializerModifier(object : BeanDeserializerModifier() {
override fun modifyDeserializer(
config: DeserializationConfig?,
beanDesc: BeanDescription?,
deserializer: JsonDeserializer<*>?
): JsonDeserializer<*>? {
return if (deserializer is BeanDeserializer) {
BeanValidationDeserializer(deserializer as BeanDeserializer?)
} else deserializer
}
})
return validationModule
}
I'm not sure how to do it. I ask for your help.
I would say an easier and more maintainable way would be to define a JSON Schema.
After that is in place, you can use one of the two json validation libraries mentioned here (https://json-schema.org/implementations.html#validator-kotlin) to validate your json.
The answer by #rbs is good but requires the overhead of creating json schema per json you want to validate.
Seems like object mapper can be configured not to wrap the exceptions it throws with JsonMappingException. - https://github.com/FasterXML/jackson-databind/issues/2033
All you need to do is to disable WRAP_EXCEPTIONS feature for the objectmapper
Note you cant choose a specific exception type, it will not wrap ALL the exceptions.

How can I override logRequest/logResponse to log custom message in Ktor client logging?

Currently, the ktor client logging implementation is as below, and it works as intended but not what I wanted to have.
public class Logging(
public val logger: Logger,
public var level: LogLevel,
public var filters: List<(HttpRequestBuilder) -> Boolean> = emptyList()
)
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
if (level.info) {
logger.log("REQUEST: ${Url(request.url)}")
logger.log("METHOD: ${request.method}")
}
val content = request.body as OutgoingContent
if (level.headers) {
logger.log("COMMON HEADERS")
logHeaders(request.headers.entries())
logger.log("CONTENT HEADERS")
logHeaders(content.headers.entries())
}
return if (level.body) {
logRequestBody(content)
} else null
}
Above creates a nightmare while looking at the logs because it's logging in each line. Since I'm a beginner in Kotlin and Ktor, I'd love to know the way to change the behaviour of this. Since in Kotlin, all classes are final unless opened specifically, I don't know how to approach on modifying the logRequest function behaviour. What I ideally wanted to achieve is something like below for an example.
....
private suspend fun logRequest(request: HttpRequestBuilder): OutgoingContent? {
...
if (level.body) {
val content = request.body as OutgoingContent
return logger.log(value("url", Url(request.url)),
value("method", request.method),
value("body", content))
}
Any help would be appreciative
No way to actually override a private method in a non-open class, but if you just want your logging to work differently, you're better off with a custom interceptor of the same stage in the pipeline:
val client = HttpClient(CIO) {
install("RequestLogging") {
sendPipeline.intercept(HttpSendPipeline.Monitoring) {
logger.info(
"Request: {} {} {} {}",
context.method,
Url(context.url),
context.headers.entries(),
context.body
)
}
}
}
runBlocking {
client.get<String>("https://google.com")
}
This will produce the logging you want. Of course, to properly log POST you will need to do some extra work.
Maybe this will be useful for someone:
HttpClient() {
install("RequestLogging") {
responsePipeline.intercept(HttpResponsePipeline.After) {
val request = context.request
val response = context.response
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}"
}
GlobalScope.launch(Dispatchers.Unconfined) {
val responseBody =
response.content.tryReadText(response.contentType()?.charset() ?: Charsets.UTF_8)
?: "[response body omitted]"
kermit.d(tag = "Network") {
"${request.method} ${request.url} ${response.status}\nBODY START" +
"\n$responseBody" +
"\nBODY END"
}
}
}
}
}
You also need to add a method from the Ktor Logger.kt class to your calss with HttpClient:
internal suspend inline fun ByteReadChannel.tryReadText(charset: Charset): String? = try {
readRemaining().readText(charset = charset)
} catch (cause: Throwable) {
null
}