I'm trying to parse a not very well designed api's json using Moshi + kotlin. For some reasons it parses numbers like 71 as Double.
The 3rd party api has a list of objects that could either look like:
{"foo":[[1234567000,12]]} // long, int
or
{"foo":[[1234567000,"string",0,2]]} // long, string, int, int
Because of the 3rd party api I have the following kotlin class:
#JsonClass(generateAdapter = true)
class D {
var foo: List<Any> // I use Any because it can be either String or Int or Long
}
and in my code I do something like:
val moshi = Moshi.Builder().build()
val adapter = moshi.adapter(D::class.java)
var D d = adapter.fromJson("{\"foo\":[[1234567000,\"string\",0,2]]}")
var index = d.foo[2]
var value : Long = 0
// here I get an error: ClassCastException: java.lang.Double cannot be cast to java.lang.Long
value = d.foo[index]
but for some reason Moshi converts the integers in the json string into Doubles instead of Int or Long. How could I fix it?
I'm not sure if this is the easiest way but it works:
class AnyAdapter {
#FromJson fun fromJson(str: String): Any {
var any: Any
try {
any = Integer.parseInt(str)
} catch (e: NumberFormatException) {
try {
any = java.lang.Long.parseLong(str)
} catch (e: NumberFormatException) {
try {
any = java.lang.Double.parseDouble(str)
} catch (e: NumberFormatException) {
any = str
}
}
}
return any
}
}
val moshi = Moshi.Builder()
.add(AnyAdapter())
.build()
val adapter = moshi.adapter(D::class.java)
var D d = adapter.fromJson("{\"foo\":[[1234567000,\"string\",0,2.0]]}")
var l : Long = d.foo[0] as Long
var s : String = d.foo[1] as String
var i : Int = d.foo[2] as Int
var dd : Double = d.foo[3] as Double
JSON number type makes no distinction between integer and floating-point
Fundamental idea behind any JSON parsing library is to parse JSON into certain type, if that type has properties of type integer then parsing library will try to convert JSON number type to integer, but you are parsing json to Any, which essentially tells moshi to take a guess as to the type of the Object.
Since JSON doesn't distinguish between integer and floating point fields moshi defaults to Float/Double for numeric fields when parsing to Any.
And the issue here is in the API, it should not return different type values for same query. at the very least there should be an indication as to the type of data. What happens if you receive a string value which actually looks like a number?
Related
Trying to deserialize a String to BigDecimal with a different format. The standard format, e.g. "1,000.20" works. However, in my csv the decimal delimiter and group separator are the other way round. So "1.000,20" would be the number one thousand with 20 as the two decimal places.
data class Record(
#field:JsonProperty("Amount")
val amount: BigDecimal,
)
The mapper is created with
val csvMapper = CsvMapper().apply {
registerModule(KotlinModule.Builder().build())
registerModule(JavaTimeModule())
enable(CsvParser.Feature.TRIM_SPACES)
enable(CsvParser.Feature.SKIP_EMPTY_LINES)
}
The file is read with
InputStreamReader(file.inputStream).use { reader ->
csvMapper.readerFor(Record::class.java)
.with(CsvSchema.emptySchema().withHeader().withColumnSeparator(';'))
.readValues<Record>(reader)
.readAll()
.toList();
Exception:
com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.math.BigDecimal` from String "-1,23": not a valid representation
at [Source: (InputStreamReader); line: 2, column: 53] (through reference chain: org.abc.Record["Amount"])
How can the format be specified?
Just found a question on stackoverflow including the solution to my question. The following works for me
class BigDecimalCustomDeserializer : JsonDeserializer<BigDecimal>() {
val dec = DecimalFormat("###,###,##0.0#", DecimalFormatSymbols(Locale.MYLOCALE))
override fun deserialize(parser: JsonParser, context: DeserializationContext?): BigDecimal? {
return if (parser.text != null && parser.text.isNotEmpty())
dec.parse(parser.text).toString().toBigDecimal()
else null
}
}
with
#field:JsonDeserialize(using = BigDecimalCustomDeserializer::class)
val amount: BigDecimal,
I'm new to Kotlin and try to convert a project from Java to Kotlin
I just need one last step and I don't understand what's going on :(
I'm getting a Not enough information to infer type variable T on model.predict call
override fun link(word: String): LinkSuggestion {
val input: DoubleArray = gramToInt.toArray(word)
val output: Array<Any> = model.predict(input)
// ~~~~~~~ Not enough information to infer type variable T
val maxPredictionIndex: Int = (output[output.size - 1] as Long).toInt()
val maxPredictionProbability: Double = output[maxPredictionIndex] as Double
return LinkSuggestion(word, intToLink.fromInt(maxPredictionIndex), maxPredictionProbability)
}
where model is import org.pmml4s.model.Model
The previous Java code:
#Override
public LinkSuggestion link(String word) {
double[] input = gramToInt.toArray(word);
Object[] output = model.predict(input);
int maxPredictionIndex = ((Long) output[output.length - 1]).intValue();
double maxPredictionProbability = (Double) output[maxPredictionIndex];
return new LinkSuggestion(word, intToLink.fromInt(maxPredictionIndex), maxPredictionProbability);
}
I needed to write
val output: Array<Any> = model.predict<Any>(input)
I have a kotlin data class described as:
data class Credentials(val p: String, val id: String, val key: String, val pass: String, val out: String)
I am trying to read from 2 text files located in a directory, and put them into this data class.
How the data looks:
config file
[user1]
out = specialk
id = mike
[user2]
out = specialk
id = mike
[user3]
out = specialk
id = mike
credentials file
[user1]
key = qwer1
pass = 3452
[user2]
key = qwer3
pass = 345232
[user3]
key = qwer5
pass = 3452gfd
Setting it up:
val homepath = System.getProperty("user.home")
val config = "$homepath/foobar"
val cred= "$homepath/credbar"
val configStream: InputStream = File(config).inputStream()
val credStream: InputStream = File(cred).inputStream()
This next part is something I am unsure of. What I think is that I should be reading each stream and putting it into a list of data class grouped by the user. However, I'm not sure how that should be accomplished.
configStream.bufferedReader().forEachLine {
// put to data class here.
}
I can't write single comment cause i am new at Stackowerflow. Sorry about that. If this file holds your app backup you can take backup as JSON file. This is much easier than this.
Firstly you need to add Gson dependency in your project
implementation 'com.google.code.gson:gson:2.8.6'
Secondly you need to two parser object that is for make parse to data class to JSON and JSON to data class.
You need to declare your export and import type as below
val exportType = object : TypeToken<List<Credentials>>() {}.type
And after that for convert your data to JSON String you can use this
private fun List<Credentials>.toJson() = Gson().toJson(this, exportType)
this code returns String.
and if you want to convert JSON to String you can use this code
private fun fromJson(str: String): List<Credentials> {
return try {
Gson().fromJson(str, exportType)
} catch (e: Exception) {
Log.e("From Json Exception", "$e")
emptyList()
}
}
this code returns list of your data class.
I hope this can help you. I did not ask is this you want to do because my Stacowerflow account is new.
Basically I would try to convert your streams into two strings
val configInputFileString = convertStreamToString(configStream)
val credentialsInputFileString = convertStreamToString(credStream)
You can easily convert a stream by using this function:
fun convertStreamToString(stream: InputStream?): String? {
val reader = BufferedReader(InputStreamReader(stream))
val sb = StringBuilder()
var line: String? = null
while (reader.readLine().also { line = it } != null) {
sb.append(line).append("\n")
}
reader.close()
return sb.toString()
}
Once you have the two streams, I would define to different data classes to get the elements of the streams:
data class UserConfig(val placeholder: String, val out: String, val id: String)
data class UserCredentials(val placeholder: String, val key: String, val pass: String)
for each stream you need to get each row, splitting by new line character \n and clearing unuseful parts:
private fun getUserConfigs(elements: List<String>): ArrayList<UserConfig> {
val configs = arrayListOf<UserConfig>()
for (element in elements) {
val splittedConfig = element
.replace("out = ", "")
.replace("id = ", "")
.split("\n")
val config = UserConfig(splittedConfig[0], splittedConfig[1], splittedConfig[2])
configs.add(config)
}
return configs
}
private fun getUserCredentials(elements: List<String>): ArrayList<UserCredentials> {
val credentials = arrayListOf<UserCredentials>()
for (element in elements) {
val splittedCredentials = element
.replace("key = ", "")
.replace("pass = ", "")
.split("\n")
val config = UserCredentials(splittedCredentials[0], splittedCredentials[1], splittedCredentials[2])
credentials.add(config)
}
return credentials
}
Now you can map userConfig and credentialConfigs in a usersMap
val userConfigs = getUserConfigs(configInputFileString!!.split("\n\n"))
val credentialConfigs = getUserCredentials(credentialsInputFileString!!.split("\n\n"))
val usersMap = userConfigs.map { userConfig ->
userConfig to credentialConfigs.find { it.placeholder == userConfig.placeholder }
}
Finally you can transform the usersMap to match your data class by doing:
val credentials = usersMap.map { Credentials(it.first.placeholder, it.first.id, it.second!!.key, it.second!!.pass, it.first.out) }
As long as your files seem to be of Windows INI format you can use third-party library Ini4J to parse them.
Parse both files and merge two Inis into list of Credentials
val configs = Ini(File(config))
val credentials = Ini(File(cred))
val result: List<Credentials> = configs.keySet().map { user ->
Credentials(p = user, // or whatever 'p' is
id = configs.get(user, "id")!!,
key = credentials.get(user, "key")!!,
pass = credentials.get(user, "pass")!!,
out = configs.get(user, "out")!!)
}
I am trying to use Kinesis, which expects data in byte buffer format. All the examples I have seen so far are in Java and pass simple strings.
Can anybody give an idea of how to convert a kotlin data class to bytebuffer?
e.g.
data class abc (
var a: Long,
var b: String,
var c: Double
)
Check the below method
fun toByteArray(): ByteArray? {
val size: Int = 8 + 8 + string.Size
val byteBuffer = ByteBuffer.allocate(size)
.put(long) //long veriable
.put(double) // double veriable
.put(string)
return byteBuffer.array()
}
You can allocate the size based on dataType size like Int 4 bytes, Double and Long 8 bytes
for reading back to dataType
val byteBuffer = ByteBuffer.wrap(byteArray)
byteBuffer.get(Int) //Int variable
byteBuffer.get(Double) //Double variable
byteBuffer.get(nonce)
You might want to have a look at kotlinx.serialization. It is an official Kotlin project and supports several formats out-of-the-box. You can use the output and wrap it in with ByteBuffer.wrap
Thanks for all the suggestions.
Solved the problem using ObjectMapper() of Jackson library (jackson-databind) and annotations.
Following code used for serialization:
val objectMapper = ObjectMapper()
objectMapper.registerModule(JavaTimeModule())
val buf = ByteBuffer.wrap(objectMapper.writeValueAsString(className).toByteArray(Charsets.UTF_8))
code for deserialization:
val objectMapper = ObjectMapper()
objectMapper.registerModule(JavaTimeModule())
val obj = objectMapper.readValue(Charsets.UTF_8.decode(record.data()).toString(), ClassName::class.java)
Apart from this, I had to add constructors of all the data classes and had to add the following annotation to all the LocalDateTime attributes:
#DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME)
#JsonFormat(pattern = "YYYY-MM-dd HH:mm")
var edd: LocalDateTime?,
A simple solution with no additional libraries
Note: must be tailored for each data class
data class TimerConfig(val startTime: Long, val repeatCount: Int, val sequenceDuration: Int)
Converting the data class to a ByteArray
private fun TimerConfig.toByteArray(): ByteArray {
val byteBuffer = ByteBuffer.allocate(Long.SIZE_BYTES + Int.SIZE_BYTES + Int.SIZE_BYTES)
byteBuffer.putLong(this.startTime)
byteBuffer.putInt(this.repeatCount)
byteBuffer.putInt(this.sequenceDuration)
return byteBuffer.array()
}
Recovering the data class from the received ByteArray
private fun ByteArray.toTimerConfig(): TimerConfig {
val byteBuffer = ByteBuffer.wrap(this)
return TimerConfig(byteBuffer.long, byteBuffer.int, byteBuffer.int)
}
I am working on a console application in Kotlin where I accept multiple arguments in main() function
fun main(args: Array<String>) {
// validation & String to Integer conversion
}
I want to check whether the String is a valid integer and convert the same or else I have to throw some exception.
How can I resolve this?
You could call toInt() on your String instances:
fun main(args: Array<String>) {
for (str in args) {
try {
val parsedInt = str.toInt()
println("The parsed int is $parsedInt")
} catch (nfe: NumberFormatException) {
// not a valid int
}
}
}
Or toIntOrNull() as an alternative:
for (str in args) {
val parsedInt = str.toIntOrNull()
if (parsedInt != null) {
println("The parsed int is $parsedInt")
} else {
// not a valid int
}
}
If you don't care about the invalid values, then you could combine toIntOrNull() with the safe call operator and a scope function, for example:
for (str in args) {
str.toIntOrNull()?.let {
println("The parsed int is $it")
}
}
Actually, there are several ways:
Given:
// aString is the string that we want to convert to number
// defaultValue is the backup value (integer) we'll have in case of conversion failed
var aString: String = "aString"
var defaultValue : Int = defaultValue
Then we have:
Operation
Successful operation
Unsuccessful Operation
aString.toInt()
Numeric value
NumberFormatException
aString.toIntOrNull()
Numeric value
null
aString.toIntOrNull() ?: defaultValue
Numeric value
defaultValue
If aString is a valid integer, then we will get is numeric value, else, based on the function used, see a result in column Unsuccessful Operation.
val i = "42".toIntOrNull()
Keep in mind that the result is nullable as the name suggests.
As suggested above, use toIntOrNull().
Parses the string as an [Int] number and returns the result
or null if the string is not a valid representation of a number.
val a = "11".toIntOrNull() // 11
val b = "-11".toIntOrNull() // -11
val c = "11.7".toIntOrNull() // null
val d = "11.0".toIntOrNull() // null
val e = "abc".toIntOrNull() // null
val f = null?.toIntOrNull() // null
I use this util function:
fun safeInt(text: String, fallback: Int): Int {
return text.toIntOrNull() ?: fallback
}
In Kotlin:
Simply do that
val abc = try {stringNumber.toInt()}catch (e:Exception){0}
In catch block you can set default value for any case string is not converted to "Int".
string_name.toString().toInt()
converts string_name to String and then the resulting String is converted to int.
i would go with something like this.
import java.util.*
fun String?.asOptionalInt() = Optional.ofNullable(this).map { it.toIntOrNull() }
fun main(args: Array<String>) {
val intArgs = args.map {
it.asOptionalInt().orElseThrow {
IllegalArgumentException("cannot parse to int $it")
}
}
println(intArgs)
}
this is quite a nice way to do this, without introducing unsafe nullable values.
add (?) before fun toInt()
val number_int = str?.toInt()
You can Direct Change by using readLine()!!.toInt()
Example:
fun main(){
print("Enter the radius = ")
var r1 = readLine()!!.toInt()
var area = (3.14*r1*r1)
println("Area is $area")
}
fun getIntValueFromString(value : String): Int {
var returnValue = ""
value.forEach {
val item = it.toString().toIntOrNull()
if(item is Int){
returnValue += item.toString()
}
}
return returnValue.toInt()
}