Trying to deserialize a String to BigDecimal with a different format. The standard format, e.g. "1,000.20" works. However, in my csv the decimal delimiter and group separator are the other way round. So "1.000,20" would be the number one thousand with 20 as the two decimal places.
data class Record(
#field:JsonProperty("Amount")
val amount: BigDecimal,
)
The mapper is created with
val csvMapper = CsvMapper().apply {
registerModule(KotlinModule.Builder().build())
registerModule(JavaTimeModule())
enable(CsvParser.Feature.TRIM_SPACES)
enable(CsvParser.Feature.SKIP_EMPTY_LINES)
}
The file is read with
InputStreamReader(file.inputStream).use { reader ->
csvMapper.readerFor(Record::class.java)
.with(CsvSchema.emptySchema().withHeader().withColumnSeparator(';'))
.readValues<Record>(reader)
.readAll()
.toList();
Exception:
com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.math.BigDecimal` from String "-1,23": not a valid representation
at [Source: (InputStreamReader); line: 2, column: 53] (through reference chain: org.abc.Record["Amount"])
How can the format be specified?
Just found a question on stackoverflow including the solution to my question. The following works for me
class BigDecimalCustomDeserializer : JsonDeserializer<BigDecimal>() {
val dec = DecimalFormat("###,###,##0.0#", DecimalFormatSymbols(Locale.MYLOCALE))
override fun deserialize(parser: JsonParser, context: DeserializationContext?): BigDecimal? {
return if (parser.text != null && parser.text.isNotEmpty())
dec.parse(parser.text).toString().toBigDecimal()
else null
}
}
with
#field:JsonDeserialize(using = BigDecimalCustomDeserializer::class)
val amount: BigDecimal,
Related
I'm new to Kotlin and trying to figure out how I can do the best way. I have an api call that I call and I convert the response to a list of objects:
data class JobAd(
val published: LocalDate?,
val title: String?,
val jobtitle: String?,
val description: String?
)
On the api call, I search for all job ads that are from today and back in time of 6 months. For example I get all objects which is from LocalDate.now() and 6 months back LocalDate).now().minusMonths(6). I want to iterate through all the objects and see if 2 random words (java and kotlin) are contained in the object. I want to check either title, jobtitle or description contain the word java or kotlin. I only need one hit of the word java or kotlin in these properties, if title contain java or kotlin, add it to list and check next object. If not title contain the words and either jobtitle, but description does it, add it to the list and check next object. and add it to a list based on which week it is.
I want the output to be like this:
(2022) Week 12 -> Java: 0, Kotlin: 1
(2022) Week 11 -> Java: 0, Kotlin: 0 (If some weeks does not have hit, i want to show to too)
...
(2021) Week 52 -> Java: 1, Kotlin: 2
This is my code so far:
private fun findAdsBasedOnKeyWords(jobAds: MutableList<JobAd>, keywords: List<String>, from: LocalDate, to: LocalDate): MutableMap<Any, MutableMap<String, Any>> {
val resultMap = mutableMapOf<Any, MutableMap<String, Any>>()
val counter = mutableMapOf<String, Any>() //Meta data
for (jobAd: JobAd in jobAds) {
for (keyword: String in keywords) {
val weekNumber = DateParser.getWeekNumber(jobAd.published!!)
// Initialize placeholder data, to fill even empty weeks
resultMap.putIfAbsent(weekNumber, emptyMapOfKeywords(keywords, jobAd.published))
// Validate keyword exist in job ad
val contains = jobAd.toString().lowercase()
.contains(keyword.lowercase()) //Can be an issue if the toString gets overridden
if (contains) {
counter.putIfAbsent(keyword, 0)
counter.compute(keyword) { _, v -> v.toString().toInt() + 1 }
resultMap[weekNumber]!!.compute(keyword) { _, v -> v.toString().toInt() + 1 }
}
}
}
resultMap["total"] = counter
resultMap["period"] = mutableMapOf("from" to from, "to" to to)
logger.info("[{}] matches found", counter)
return resultMap
}
//Helper method to generate placeholder data
private fun emptyMapOfKeywords(keywords: List<String>, published: LocalDate): MutableMap<String, Any> {
val keywordMap = mutableMapOf<String, Any>()
for (keyword in keywords) {
keywordMap.putIfAbsent(keyword, 0)
}
keywordMap.putIfAbsent("from", DateParser.startOfWeekDate(published))//Monday of the week
keywordMap.putIfAbsent("to", DateParser.endOfWeekDate(published))//Sunday of the week
return keywordMap
}
Is there any way to do it better or optimize it and please add comment for why.
It's a pretty extreme anti-pattern to use Maps to hold various types of data that you need to inspect. That's trying to force a strongly typed language to behave like a weakly typed language, losing all the protection you get from using types.
Maps are appropriate when the keys are something you don't know at compile time and you know you'll need to look up items by their keys at runtime.
So instead of a MutableMap<Any, MutableMap<String, Any>> return value, you should create classes for holding results. From what I can tell, you want to return a series of line items for every week in the input range, so you can create a class like this to represent a line item, and then return a simple list of them from your function. You are currently also returning the range, but I don't see what you're using it for so I left it out.
You're working with a week of a year a lot, so I think it will also be helpful to have a class to represent that, along with a couple of functions to help convert from LocalDate.
data class LocalWeek(val year: Int, val week: Int)
fun LocalDate.toLocalWeek() = LocalWeek(year, get(IsoFields.WEEK_OF_WEEK_BASED_YEAR))
/** Gets every week represented in a range of dates. */
fun ClosedRange<LocalDate>.toLocalWeeks() = sequence {
var date = start
val lastExclusive = endInclusive + Period.ofWeeks(1)
while (date < lastExclusive ) {
yield(date.toLocalWeek())
date += Period.ofWeeks(1)
}
}
data class JobAdsSearchLineItem(
val localWeek: LocalWeek,
val keywordHitCountsByKeyword: Map<String, Int>
) {
fun toReadableString() =
"(${localWeek.year}) Week ${localWeek.week} -> " +
keywordHitCountsByKeyword.entries
.joinToString { (word, count) -> "$word: $count" }
}
Using toString() is fragile, like you mentioned in your code comments. I would create a helper function like this to evaluate whether a term is found:
fun JobAd.containsIgnoreCase(str: String): Boolean {
val value = str.lowercase()
return title.orEmpty().lowercase().contains(value)
|| jobtitle.orEmpty().lowercase().contains(value)
|| description.orEmpty().lowercase().contains(value)
}
Since you're using !! on your published date, I'm assuming these values don't need to be nullable. It would be much easier to work with if you make the property non-nullable:
data class JobAd(
val published: LocalDate,
val title: String?,
val jobtitle: String?,
val description: String?
)
Then your search function can be written like this:
private fun findAdsBasedOnKeyWords(
jobAds: List<JobAd>,
keywords: List<String>,
from: LocalDate,
to: LocalDate
): List<JobAdsSearchLineItem> {
// Initialize empty results holders representing every week in the range
// Use an outer map here because we need to keep retrieving the inner maps by
// the week when iterating the input below.
val results = mutableMapOf<LocalWeek, MutableMap<String, Int>>()
for (localWeek in (from..to).toLocalWeeks()) {
results[localWeek] = mutableMapOf<String, Int>().apply {
for (keyword in keywords) {
put(keyword, 0)
}
}
}
for (jobAd in jobAds) {
val weekResults = results[jobAd.published.toLocalWeek()] ?: continue
for (keyword in keywords) {
if (jobAd.containsIgnoreCase(keyword)) {
weekResults[keyword] = weekResults.getValue(keyword) + 1
}
}
}
return results.entries.map { JobAdsSearchLineItem(it.key, it.value) }
}
And to use it you can call this function and use the toReadableString() function to help generate your output from the list of results.
I'm trying to parse a not very well designed api's json using Moshi + kotlin. For some reasons it parses numbers like 71 as Double.
The 3rd party api has a list of objects that could either look like:
{"foo":[[1234567000,12]]} // long, int
or
{"foo":[[1234567000,"string",0,2]]} // long, string, int, int
Because of the 3rd party api I have the following kotlin class:
#JsonClass(generateAdapter = true)
class D {
var foo: List<Any> // I use Any because it can be either String or Int or Long
}
and in my code I do something like:
val moshi = Moshi.Builder().build()
val adapter = moshi.adapter(D::class.java)
var D d = adapter.fromJson("{\"foo\":[[1234567000,\"string\",0,2]]}")
var index = d.foo[2]
var value : Long = 0
// here I get an error: ClassCastException: java.lang.Double cannot be cast to java.lang.Long
value = d.foo[index]
but for some reason Moshi converts the integers in the json string into Doubles instead of Int or Long. How could I fix it?
I'm not sure if this is the easiest way but it works:
class AnyAdapter {
#FromJson fun fromJson(str: String): Any {
var any: Any
try {
any = Integer.parseInt(str)
} catch (e: NumberFormatException) {
try {
any = java.lang.Long.parseLong(str)
} catch (e: NumberFormatException) {
try {
any = java.lang.Double.parseDouble(str)
} catch (e: NumberFormatException) {
any = str
}
}
}
return any
}
}
val moshi = Moshi.Builder()
.add(AnyAdapter())
.build()
val adapter = moshi.adapter(D::class.java)
var D d = adapter.fromJson("{\"foo\":[[1234567000,\"string\",0,2.0]]}")
var l : Long = d.foo[0] as Long
var s : String = d.foo[1] as String
var i : Int = d.foo[2] as Int
var dd : Double = d.foo[3] as Double
JSON number type makes no distinction between integer and floating-point
Fundamental idea behind any JSON parsing library is to parse JSON into certain type, if that type has properties of type integer then parsing library will try to convert JSON number type to integer, but you are parsing json to Any, which essentially tells moshi to take a guess as to the type of the Object.
Since JSON doesn't distinguish between integer and floating point fields moshi defaults to Float/Double for numeric fields when parsing to Any.
And the issue here is in the API, it should not return different type values for same query. at the very least there should be an indication as to the type of data. What happens if you receive a string value which actually looks like a number?
Hi I am a newbie to kotlinx serialization and I am using KMP, my requirement is a little different
my data class
#Serializable data class Student(val name : String , val age : Int)
and my simple JSON would be "['Avinash', 22]",
which should be deserialized to Student("Avinash", 22)
I'm not able to deserialize it can anyone help me
While input such as [Avinash, 22] is not well formed Json, you can still
work with it by parsing it into a JsonElement:
import kotlinx.serialization.json.*
data class Student(val name: String, val age: Int)
fun decode(stringData: String, parser: Json): List<Student> {
val element: JsonArray = parser.parseToJsonElement(stringData).jsonArray
return element.windowed(2, 2).map {
Student(
it[0].toString(),
it[1].toString().toInt()
)
}
}
fun main() {
val parser = Json { isLenient = true }
val students = decode("[A, 22, B, 33, C, 44]", parser)
println(students)
// [Student(name=A, age=22), Student(name=B, age=33), Student(name=C, age=44)]
}
Try this:
val student: Student = Json.decodeFromString("{\"name\": \"Avinash\", \"age\": \"22\"}")
Pay attention how to format your JSON string.
[] square brackets are for arrays
{} curly brackets are for objects
And you must provide your fields name, and use double quotes for fields and values, or use a less strict Json deserialization:
val json = Json {
isLenient = true
}
val student: Student = json.decodeFromString("{name: Avinash, age: 22}")
If you want a deep view on json schema, you can read here.
I have multiple variables that can be nullable and i need to check them ( Strings and Dates ) .
I need a method where i pass it X number of variables and it returns me a list of the variables that are null.
I was thinking something that i can call like this :
internal fun checkNullVariables ( var x, var y , ..... ) : MutableList<String>{
// yada yada
return listOfNamesOfNullVariables
}
This definitely requires reflection, since you want parameter names. You need to add reflection as a dependency as explained in the documentation to use the below code.
private fun listNullProperties (vararg props: KProperty0<Any?>) : List<String> {
val list = mutableListOf<String>()
for (prop in props)
if (param.get() == null)
list.add(param.name)
return list
}
Usage:
val nullPropertiesByName = listNullParameters(
::myProperty,
::myOtherProperty,
::myDateProperty
)
println(nullPropertiesByName.joinToString())
If it is just about logging, you could:
fun <T> T?.logNull(name: String) {
when(this) {
null -> //log '$name' was null
else -> //do nothing
}
}
and call it like
var a: String? = null
a.logNull("my a variable") // "'my a variable' was null"
I still recommend the Map-approach. You may want to use properties stored in a map to overcome the use of reflection.
Here is an example using a type with 2 dates and 2 strings, both having a nullable and a non-null variant:
class YourData(internal val backedMap : Map<String, Any?>) {
val beginDate : Date by backedMap
val endDate : Date? by backedMap
val maybeString : String? by backedMap
val string : String by backedMap
constructor(beginDate : Date, string : String, endDate: Date? = null, maybeString : String? = null) : this(mapOf(
"beginDate" to beginDate,
"endDate" to endDate,
"maybeString" to maybeString,
"string" to string
))
}
While it may seem more complicated having that additional constructor in place, it just helps to easily create new objects the way you are most comfortable with.
Now you can either supply that function I placed in the comment or any variant of it. I now use an extension function for YourData instead:
fun YourData.getKeysWithNullValues() = backedMap.filterValues { it == null }.keys
Usage then may look as follows:
YourData(Date(), "test string")
.getKeysWithNullValues()
.forEach(::println)
which for this example would print:
endDate
maybeString
I am trying to insert some data into the database, and am getting the following error:
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.joda.time.DateTime` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('2019-04-19')
My content negotiation
install(ContentNegotiation) {
jackson {
enable(SerializationFeature.INDENT_OUTPUT)
}
}
And my model:
data class User(
//some codes
val registrationDate = DateTime // org.joda.time.DateTime
)
And when will I send by json:
{
//some other data
"registrationDate" : "2019-07-15"
}
Can someone help me, please?
You have to install the Joda module for Jackson https://github.com/FasterXML/jackson-datatype-joda and add it to your jackson configuration in ktor :
install(ContentNegotiation) {
jackson {
registerModule(JodaModule())
enable(SerializationFeature.INDENT_OUTPUT)
}
}
You can also control serialization/deserialization behavior with annotations on your data class properties :
data class Account(
val uid: String? = null,
val firstName: String,
val lastName: String,
val email: String,
#get:JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd HH:mm")
val createdTime: DateTime? = null
)