I have a library that sends me results that include tuples. I need to process some of the data, serialize it and then it goes on its way to another system.
the tuples are ALWAYS made of 2 values but they are extremely wasteful when serialized:
(3, 4)
will serialize as:
{"Item1":3,"Item2":4}
whereas
[3; 4]
will serialize as:
[3,4]
I would like to avoid rebuilding the whole data structure and copying all the data to change this part.
Is there a way, at the serializer level, to convert the tuples into list?
the next process' parser can be easily changed to accommodate a list instead of tuples, so it seems like the best scenario.
the ugly option would be to fix the serialized string with a regex, but I would really like to avoid doing this.
You can override the default serialization behaviour by specifying your own JsonConverter. The following example shows a formatter that writes int * int tuples as two-element JSON arrays.
open Newtonsoft.Json
type IntIntConverter() =
inherit JsonConverter<int * int>()
override x.WriteJson(writer:JsonWriter, (a:int,b:int), serializer:JsonSerializer) =
writer.WriteStartArray()
writer.WriteValue(a)
writer.WriteValue(b)
writer.WriteEndArray()
override x.ReadJson(reader, objectType, existingValue, hasExistingValue, serializer) =
(0, 0)
let sample = [ (1,2); (3,4) ]
let json = JsonConvert.SerializeObject(sample, Formatting.None, IntIntConverter())
The result of running this will be [[1,2],[3,4]]. Note that I have not implemented the ReadJson method, so you cannot yet parse the tuples. This will involve some extra work, but you can look at existing JsonConverters to see how this should be done.
Also note that this is for a specific tuple type containing two integers. If you need to support other tuples, you will probably need to provide several variants of the converter.
Related
I have Set<FlagFilter> and I need to convert it to Map<Class<out FlagFilter>, FlagFilter>.
I tried doing it like this:
val result: Map<Class<out FlagFilter>, FlagFilter> =
target
.takeIf { !it.isEmpty() }
?.map { mapOf(it.javaClass to it) }
?: emptyMap<>()
but instead of a Map it turns out to be a List and I get a compilation error:
Type mismatch.
Required: Map<Class<out FlagFilter>, FlagFilter>
Found: List<Map<Class<FlagFilter>, FlagFilter>>
What am I doing wrong? As if there is not enough operation, but I do not understand yet which one
map isn't anything to do with the Map type - it's a functional programming term (coming from a broader mathematical concept) that basically means a function that maps each input value to an output value.
So it's a transformation or conversion that takes a collection of items, transforms each one, and results in another collection with the same number of items. In Kotlin, you get a List of items (unless you're working with a Sequence, in which case you get another Sequence that yields the same number of items).
It's worth getting familiar with the kotlin.collections package - there's lots of useful stuff in there! But each function has a specific purpose, in terms of how they process the collection and what they return:
map - returns a new value for each item
onEach - returns the original items (allows you to do something with each, then continue processing the collection)
forEach - returns nothing (allows you to do something with each, but as a final operation - you can't chain another operation, it's terminal)
filter - returns a subset of the original items, matching a predicate
first - returns a single item, matching a predicate
reduce - returns a single item, transforming the values to produce a single result
count - returns a single item, based on an attribute of the collection (not the values themselves)
There's more but you get the idea - some things transform, some things pass-through, some things give you an identically sized collection, or a potentially smaller one, or a single value. map is just the one that takes a collection, and gives you a collection of the same size where every item has been (potentially) altered.
Like Vadik says, use one of the associate* functions to turn values into a Map object, effectively transforming each item in the collection into a Map.Entry. Which is technically mapping it to a mapping, so I wasn't totally accurate earlier when I said it's nothing to do with Maps, but I figured I'd save this thought til the end ;)
Just use associateBy extension function:
val result: Map<Class<out FlagFilter>, FlagFilter> =
target.associateBy { it.javaClass }
Or if you want to fix your code, remove the excessive call of mapOf(). Just convert your Set to List of Pairs, and then call toMap() to create a map:
val result: Map<Class<out FlagFilter>, FlagFilter> =
target.map { it.javaClass to it }.toMap()
I'd like to create a JSON strategy using rust proptest library. However, I do not want to create an arbitrary JSON. I'd like to create it according to a schema (more specifically, OpenAPI schema). This means that keys of the JSON are known and I do not want to create them using any strategy, but I'd like to create the values using the strategy (pretty-much recursively).
I already implemented the strategy for primitive types, but I do not how to create a JSON object strategy.
I would like the strategy to have the type BoxedStratedy<serde_json::Value> or be able to map the strategy to this type because the JSON objects can contain other objects, and thus I need to be able to compose the strategies.
I found a HashMapStrategy strategy, however, it can be only created by a hash_map function that takes two strategies - one for generating keys and one for values. I thought that I could use Just strategy for the keys, but it did not lead anywhere. Maybe prop_filter_map could be used.
Here is the code. There are tests too. One is passing because it tests only primitive type and the other is failing since I did not find a way to implement generate_json_object function.
I tried this but the types do not match. Instead of a strategy of map from string to JSON value, it is a strategy of a map from string to BoxedStrategy.
fn generate_json_object(object: &ObjectType) -> BoxedStrategy<serde_json::Value> {
let mut json_object = serde_json::Map::with_capacity(object.properties.len());
for (name, schema) in &object.properties {
let schema_kind = &schema.to_item_ref().schema_kind;
json_object.insert(name.clone(), schema_kind_to_json(schema_kind));
}
Just(serde_json::Value::Object(json_object)).boxed()
}
One can create a vector of strategies, which implements a Strategy trait and can be boxed. So to create a serde_json::Value::Object, we create a vector of tuples. The first element will be a Just of key and the second element will be a boxed strategy of value. The boxed strategy of value can be created by schema_kind_to_json function. After we have a vector of tuples which implement a Strategy, we can use .prop_map to transform it to a serde_json::Value::Object.
fn generate_json_object(object: &ObjectType) -> BoxedStrategy<serde_json::Value> {
let mut vec = Vec::with_capacity(object.properties.len());
for (name, schema) in &object.properties {
let schema_kind = &schema.to_item_ref().schema_kind;
vec.push((Just(name.clone()), schema_kind_to_json(schema_kind)));
}
vec.prop_map(|vec| serde_json::Value::Object(serde_json::Map::from_iter(vec)))
.boxed()
}
I want to use the Scan() function from the sql package for executing a select statement that might (or not) return multiple rows, and return these results in my function.
I´m new to Golang generics, and am confused about how to achieve this.
Usually, we would use the Scan function on a *sql.Rows and provide the references to all fields of our expected 'result type' we want to read the rows into, e.g.:
var alb Album
rows.Scan(&alb.ID, &alb.Title, &alb.Artist,
&alb.Price, &alb.Quantity)
where Album is a struct type with those five fields shown.
Now, for the purpose of not writing a similar function N times for every SQL table I have, I want to use a generic type R instead. R is of generic interface type Result, and I will define this type as one of N different structs:
type Result interface {
StructA | StructB | StructC
}
func ExecSelect[R Result](conn *sql.DB, cmd Command, template R) []R
How can I now write rows.Scan(...) to apply the Scan operation on all fields of my struct of R´s concrete type? e.g. I would want to have rows.Scan(&res.Field1, &res.Field2, ...) where res is of type R, and Scan should receive all fields of my current concrete type R. And do I actually need to provide a 'template' as argument of R´s concrete type, so that at runtime it becomes clear which struct is now relevant?
Please correct me on any mistake I´m making considering the generics.
This is a poor use case for generics.
The arguments to the function sql.Rows.Scan are supposed to be the scan destinations, i.e. your struct fields, one for each column in the result set, and within the generic function body you do not have access to the fields of R type parameter.
Even if you did, the structs in your Result constraint likely have different fields...? So how do you envision writing generic code that works with different fields?
You might accomplish what you want with a package that provides arbitrary struct scanning like sqlx with facilities like StructScan, but that uses reflection under the hood to map the struct fields into sql.Rows.Scan arguments, so you are not getting any benefit at all with generics.
If anything, you are making it worse, because now you have the additional performance overheads of using type parameters.
I have am trying to create a recursive data class like so:
data class AttributeId (
val name: String,
val id: Int,
val children: List<AttributeId>?
)
The thing I'm struggling with now is building the data class by iterating over a source object.
How do I recursively build this object?? Is a data class the wrong solution here?
EDIT: Some more information about the Source object from which I want to construct my data class instance
The source object is a Java Stream that essentially* has the following shape:
public Category(final String value,
final Integer id,
final List<Category> children) {
this.value = value;
this.id = id;
this.children = children;
}
(For brevity the fields I don't care about have been removed from example)
I think I need to map over this stream and call a recursive function in order to construct the AttributeId data class, but my attempts seem to end in a stack overflow and a lot of confusion!
I don't think there's anything necessarily wrong with a data class that contains references to others.
There are certainly some gotchas. For example:
If the list were mutable, or if its field was mutable (i.e. var rather than val), then you'd have to take care because its hashcode &c could change.
And if the chain of links could form a loop (i.e. you could follow the links and end up back at the original class), that could be very dangerous. (E.g. calling a method such as toString() or hashCode() might either get stuck in an endless loop or crash the thread with a StackOverflowError. You'd have to prevent that by overriding those methods to prevent them recursing.) But that couldn't happen if the list and field were both immutable.
None of these issues are specific to data classes, though; a normal class could suffer the same issues (especially if you overrode methods like toString() or hashCode() without taking care). So whether you make this a data class comes down to whether it feels like one: whether its primary purpose is to hold data, and/or whether the automatically-generated methods match how you want it to behave.
As Tenfour04 says, it depends what you're constructing these from. If it naturally forms a tree structure, then this could be a good representation for it.
Obviously, you wouldn't be able to construct a parent before any of its children. (In particular, the first instance you create would have to have either null or an empty list for its children.) This would probably mean traversing the source in post-order. The rest should fall out naturally from that.
I have a list of Longs in Kotlin and I want to make them strings for UI purposes with maybe some prefix or altered in some way. For example, adding "$" in the front or the word "dollars" at the end.
I know I can simply iterate over them all like:
val myNewStrings = ArrayList<String>()
longValues.forEach { myNewStrings.add("$it dollars") }
I guess I'm just getting nitpicky, but I feel like there is a way to inline this or change the original long list without creating a new string list?
EDIT/UPDATE: Sorry for the initial confusion of my terms. I meant writing the code in one line and not inlining a function. I knew it was possible, but couldn't remember kotlin's map function feature at the time of writing. Thank you all for the useful information though. I learned a lot, thanks.
You are looking for a map, a map takes a lambda, and creates a list based on the result of the lambda
val myNewStrings = longValues.map { "$it dollars" }
map is an extension that has 2 generic types, the first is for knowing what type is iterating and the second what type is going to return. The lambda we pass as argument is actually transform: (T) -> R so you can see it has to be a function that receives a T which is the source type and then returns an R which is the lambda result. Lambdas doesn't need to specify return because the last line is the return by default.
You can use the map-function on List. It creates a new list where every element has been applied a function.
Like this:
val myNewStrings = longValues.map { "$it dollars" }
In Kotlin inline is a keyword that refers to the compiler substituting a function call with the contents of the function directly. I don't think that's what you're asking about here. Maybe you meant you want to write the code on one line.
You might want to read over the Collections documentation, specifically the Mapping section.
The mapping transformation creates a collection from the results of a
function on the elements of another collection. The basic mapping
function is
map().
It applies the given lambda function to each subsequent element and
returns the list of the lambda results. The order of results is the
same as the original order of elements.
val numbers = setOf(1, 2, 3)
println(numbers.map { it * 3 })
For your example, this would look as the others said:
val myNewStrings = longValues.map { "$it dollars" }
I feel like there is a way to inline this or change the original long list without creating a new string list?
No. You have Longs, and you want Strings. The only way is to create new Strings. You could avoid creating a new List by changing the type of the original list from List<Long> to List<Any> and editing it in place, but that would be overkill and make the code overly complex, harder to follow, and more error-prone.
Like people have said, unless there's a performance issue here (like a billion strings where you're only using a handful) just creating the list you want is probably the way to go. You have a few options though!
Sequences are lazily evaluated, when there's a long chain of operations they complete the chain on each item in turn, instead of creating an intermediate full list for every operation in the chain. So that can mean less memory use, and more efficiency if you only need certain items, or you want to stop early. They have overhead though so you need to be sure it's worth it, and for your use-case (turning a list into another list) there are no intermediate lists to avoid, and I'm guessing you're using the whole thing. Probably better to just make the String list, once, and then use it?
Your other option is to make a function that takes a Long and makes a String - whatever function you're passing to map, basically, except use it when you need it. If you have a very large number of Longs and you really don't want every possible String version in memory, just generate them whenever you display them. You could make it an extension function or property if you like, so you can just go
fun Long.display() = "$this dollars"
val Long.dollaridoos: String get() = "$this.dollars"
print(number.display())
print(number.dollaridoos)
or make a wrapper object holding your list and giving access to a stringified version of the values. Whatever's your jam
Also the map approach is more efficient than creating an ArrayList and adding to it, because it can allocate a list with the correct capacity from the get-go - arbitrarily adding to an unsized list will keep growing it when it gets too big, then it has to copy to another (larger) array... until that one fills up, then it happens again...