I'd like to create a JSON strategy using rust proptest library. However, I do not want to create an arbitrary JSON. I'd like to create it according to a schema (more specifically, OpenAPI schema). This means that keys of the JSON are known and I do not want to create them using any strategy, but I'd like to create the values using the strategy (pretty-much recursively).
I already implemented the strategy for primitive types, but I do not how to create a JSON object strategy.
I would like the strategy to have the type BoxedStratedy<serde_json::Value> or be able to map the strategy to this type because the JSON objects can contain other objects, and thus I need to be able to compose the strategies.
I found a HashMapStrategy strategy, however, it can be only created by a hash_map function that takes two strategies - one for generating keys and one for values. I thought that I could use Just strategy for the keys, but it did not lead anywhere. Maybe prop_filter_map could be used.
Here is the code. There are tests too. One is passing because it tests only primitive type and the other is failing since I did not find a way to implement generate_json_object function.
I tried this but the types do not match. Instead of a strategy of map from string to JSON value, it is a strategy of a map from string to BoxedStrategy.
fn generate_json_object(object: &ObjectType) -> BoxedStrategy<serde_json::Value> {
let mut json_object = serde_json::Map::with_capacity(object.properties.len());
for (name, schema) in &object.properties {
let schema_kind = &schema.to_item_ref().schema_kind;
json_object.insert(name.clone(), schema_kind_to_json(schema_kind));
}
Just(serde_json::Value::Object(json_object)).boxed()
}
One can create a vector of strategies, which implements a Strategy trait and can be boxed. So to create a serde_json::Value::Object, we create a vector of tuples. The first element will be a Just of key and the second element will be a boxed strategy of value. The boxed strategy of value can be created by schema_kind_to_json function. After we have a vector of tuples which implement a Strategy, we can use .prop_map to transform it to a serde_json::Value::Object.
fn generate_json_object(object: &ObjectType) -> BoxedStrategy<serde_json::Value> {
let mut vec = Vec::with_capacity(object.properties.len());
for (name, schema) in &object.properties {
let schema_kind = &schema.to_item_ref().schema_kind;
vec.push((Just(name.clone()), schema_kind_to_json(schema_kind)));
}
vec.prop_map(|vec| serde_json::Value::Object(serde_json::Map::from_iter(vec)))
.boxed()
}
Related
We already have flatbuffer library embedded in our software code for simple schemas with JSON output data generation.
More update: We are generating the header files using flatc compiler against the schema and integrate these files inside of our code along with FB library for further serialization/deserialization.
Now we also need to have the following schema tree to be supported.
namespace SampleNS;
/// user defined key value pairs to add custom metadata
/// key namespacing is the responsibility of the user
table KeyValue {
key:string (key, required);
value:string (required);
}
enum SchemaVersion:byte {
V1,
V2
}
table Sometable {
value1:ubyte;
value2:ushort (key);
}
table ComponentData {
inputs: [Sometable];
outputs: [Sometable];
}
table Node {
name:string (key);
/// IO definition
data:ComponentData;
/// nested child
child:[Components];
}
table Components {
type:ubyte;
index:ubyte;
nodes:[Node];
}
table GroupMasterData {
schemaversion:SchemaVersion = sampleNS::SchemaVersion::V1;
metainfo:[KeyValue];
/// List of expected components in the system
components:[Components];
}
root_type GroupMasterData;
As from above, table Components is nested recursively. The intention is components may have childs that have the same fields.
I have few queries:
Flatc didnt gave me any error during schema compilation for such
recursive nested tables. But is this supported during the field
access for such tables?
I tried to generate a sample json data file based on above data but I
could not see the field for schemaversion. I learned FB doesn't
serialize the default values. so, I removed the default value that I
assigned in the schema. But, it still doesnt write into the json data
file. On this I also learned we can forcefully write into the file
using force_defaults option. I don't know where is this is to be
put: in the attribute or elsewhere?
Can I create a struct of enum field?
Is their any API to set Flatbuffer options that we otherwise pass to the compiler arguments? or if not, may be I think we have to tinker with the FB library code. Please suggest.
Method 1:
In our serialization method, we do this:
flatbuffers::Parser* parser = new flatbuffers::Parser();
parser->opts.output_default_scalars_in_json = true;
Is this the right method or should I use any other API?
Yes, trees (and even DAG) structures are fully supported. The type definition is recursive, but the data will eventually have leaf nodes with an empty vector of children, presumably.
The integer value for V1 is 0, and that is also the default value for all fields with no explicit default assigned. Use --defaults-json to see this field when converting. Note that explicit versions in a schema is an anti-pattern, since schemas are naturally evolvable without breaking backwards compatibility.
You can put enum fields in structs, yes. Is that what you mean?
I seem to recall reading that manually checking types in an object-oriented language (e.g., Kotlin) usually means you are doing something "bad." I'm wondering if there is a better pattern for handling my situation than what I am currently doing ; currently I use when to check the type of the generic parameter, and then call the correct function based on type.
I have a set of data classes that store arrays of different types:
interface Data<T>
AData: Data<Int>
BData: Data<Double>
CData: Data<ComplexType>
DData: Data<Foo>
I can use these classes for doing various kinds of math, and sometimes I want to display them in a table. In order to do that (using JavaFX) I need to create a TableColumn<Int, *>, but I want to create a differently styled table column type based on type contained in the Data class.
I could think of two solutions. What I currently do is:
fun makeCol(data: Data<*>): TableColumn<Int, *> {
return when {
data[0] is Int -> makeBasicCol(data)
data[0] is Double-> makeBasicCol(data)
data[0] is ComplexType -> makeComplexCol(data)
data[0] is Int -> makeFooCol(data)
}
}
The other option I considered was creating a subclass of Data, something like FxData, which would have a function like fun <TableColumn<Int, *>> provideFxCol(). The problem I see with that is that now I am mixing UI functionality with the math functionality.
I would like to know if there is a design pattern to cover making multiple objects representing mutiple permutations of a string. For example:
I have a database table containing item names.
Each item has a sort of "signature" (can't think of a better word for it), which is all the letters of the item name, sorted in alphabetical order with the spaces removed.
Given a "soup string" of jumbled up letters, I would like to sort those letters in alphabetical order to match it to a signature in the database, but here's the catch...
Each soup string may contain a few extra letters. So what I'm looking for is a design pattern which would be suitable for taking a string and returning a list of objects, each representing a permutation of that soup string, which I can then fire at the database.
I was thinking about just using a factory, but isn't this outside of the scope of a factory? It does contain logic, (am I right in saying this is not business logic?), but perhaps this is acceptable for a factory or factory method? Then again, perhaps this is an perfect usecase for a factory.
Ultimately, I will probably just go with the factory method. I just wanted to see if there was a better option.
Thanks in advance.
Let's start with an object-oriented way of creating n objects from a given item. First, let's assume that the item is of type String; you can create a class Permutations which implements the interface Iterable<String> (basically, an object that acts as a list of elements of type String)
data class Permutations(val strings: Iterable<String>): Iterable<String> {
constructor(string: String): this(...) {
# transform string to permutations here (bonus: with lazy computations)
}
override fun iterator(): Iterator<String> = strings.iterator()
}
Now, any object of type Permutations can replace a list of type String. Note that this class has two constructors, one takes a list of strings (the primary basic constructor) and one takes just one string and transforms it. This is not a design pattern; it's just a nice way to write objects out of objects without using static methods on util classes.
You can encapsulate the computation that transforms your string into permutations in (1) a different object (such as a strategy class), (2) a lambda function or (3) write our logic into the constructor (not recommended). The way you encapsulate the computation depends on how much flexibility you need. :)
Edit: Small improvement for the primary constructor.
Is there way to declare a Map that would accept both String and Int in Kotlin? Like:
var map: Map<String, String | Int>
There is no union type in Kotlin, so you can't directly express this.
A few options:
use TreeMap<String, Any>, which is the simplest of all, but it allows more types than just Int and String, and requires additional conversion boilerplate on the consumer side when accessing values.
use a custom sealed class to encapsulate your "Int or String" type, and use it in a regular TreeMap<String, IntOrString>. This will have some runtime overhead for wrapping/unwrapping the int/string objects for each insertion/access. Also, this puts the burden of (un)wrapping on the consumer side. That being said, maybe your domain can be better modeled with clearer names than Int or String, so it might actually improve the consumer's experience to define such types.
use a custom map type, with independent accessors for ints and strings, but backed by a single TreeMap<String, Any> behind the scenes. This has the advantage of encapsulating the logic and exposing a neat API for the consumers. If you extend TreeMap you will save some overhead as opposed to composition, but it shouldn't matter much.
I am wondering about the actual use case at hand, though. Usually the need for union types stems from using plain generic primitive types to represent meaningful domain data that could maybe better be expressed using more specific custom domain types.
Option 2 would therefore usually be my choice, but I would use a sealed class that actually represents my domain data instead of the generic "string or int".
You can use Any here, see this example:
import java.util.TreeMap
fun main() {
var map = TreeMap<String, Any>()
map.put("One", 1)
map.put("Two", "two")
println(map)
}
which outputs
{One=1, Two=two}
Please note:
This will allow more types than just the desired String and Int, because the Kotlin Docs state (...). Every Kotlin class has Any as a superclass..
I have a library that sends me results that include tuples. I need to process some of the data, serialize it and then it goes on its way to another system.
the tuples are ALWAYS made of 2 values but they are extremely wasteful when serialized:
(3, 4)
will serialize as:
{"Item1":3,"Item2":4}
whereas
[3; 4]
will serialize as:
[3,4]
I would like to avoid rebuilding the whole data structure and copying all the data to change this part.
Is there a way, at the serializer level, to convert the tuples into list?
the next process' parser can be easily changed to accommodate a list instead of tuples, so it seems like the best scenario.
the ugly option would be to fix the serialized string with a regex, but I would really like to avoid doing this.
You can override the default serialization behaviour by specifying your own JsonConverter. The following example shows a formatter that writes int * int tuples as two-element JSON arrays.
open Newtonsoft.Json
type IntIntConverter() =
inherit JsonConverter<int * int>()
override x.WriteJson(writer:JsonWriter, (a:int,b:int), serializer:JsonSerializer) =
writer.WriteStartArray()
writer.WriteValue(a)
writer.WriteValue(b)
writer.WriteEndArray()
override x.ReadJson(reader, objectType, existingValue, hasExistingValue, serializer) =
(0, 0)
let sample = [ (1,2); (3,4) ]
let json = JsonConvert.SerializeObject(sample, Formatting.None, IntIntConverter())
The result of running this will be [[1,2],[3,4]]. Note that I have not implemented the ReadJson method, so you cannot yet parse the tuples. This will involve some extra work, but you can look at existing JsonConverters to see how this should be done.
Also note that this is for a specific tuple type containing two integers. If you need to support other tuples, you will probably need to provide several variants of the converter.