Through a json schema validator (like z-schema), I would like to validate an integer formatted in a string, e.g.:
{
"myvalue": "45"
}
Currently, the following validation schema is:
{
"type": "string",
"pattern": "^[0-9]+$"
}
However, now it would be great to be able to validate a minimum and maximum value, like:
{
"type": "integer",
"minimum": 0,
"maximum": 32
}
However the above json value "45" is not an integer.
Without changing the type to integer, the best you can do is use the pattern keyword to enforce the range using a regular expression. Here is an example of a regular expression to match integers from 0..32.
/^[1-2]?[0-9]$|^3[0-2]$/
Related
Is there a supported way (without using the anyOf keyword) to specify multiple ranges for a number in JsonSchema?
E.g., a value can either be in the range of 0-10 or 50-100 (the range of 10 < x < 50 is considered invalid).
The anyOf keyword can be used as follows:
{
"anyOf": [
{
"type": "number",
"minimum": 0,
"maximum": 10
},
{
"type": "number",
"minimum": 5,
"maximum": 100
}
]
}
Additionally, if the only allowed values were whole integers, I could use an enum and actually hand-specify each allowed number, but obviously that's less ideal than specifying ranges.
So, just wondering if there is a way to accomplish this with something like a "restrictions" keyword:
//NOTE: the below is not actually supported (I don't think), just using it as an example of what I'm interested in
{
"type": "number",
"restrictions": [
{
"minimum": 0,
"maximum": 10
},
{
"minimum": 50,
"maximum": 100
}
]
}
Also, for those wondering why if anyOf is available, it's that I have some custom tooling to maintain and supporting anyOf would be more of a lift than something that is specific to numeric validation.
The scenario you describe is exactly why anyOf exists. So no, if you want to express a logical OR, you need to implement the keyword that implements that. I don't see why adding a new custom keyword would make things any easier.
I created two Avro schemas, one had short keys, and the other hand long keys. I put the same data on them, and the serialised them as byte array.
Then, I checked the length, and it was the same.
If it does not effect the size of the data, then how are keys saved?
Schema I tested on:
{
"namespace": "namespace",
"type": "record",
"name": "TestA",
"fields": [
{"name": "dataFieldIsVeryVeryVeryLong", "type": "string"},
{"name": "dataField2IsVeryVeryVeryVerylong", "type": ["null", "string"], "default": null}
]
}
AND
{
"namespace": "namespace",
"type": "record",
"name": "TestB",
"fields": [
{"name": "s", "type": "string"},
{"name": "ss", "type": ["null", "string"], "default": null}
]
}
Keys are not saved, avro serialisation works based on types and order of fields. In your case you use:
record - it's a concatenation of fields
string is long for length and then UTF-8 encoding of characters
union of two types, it has int to specify which type it is and then it's using this type encoding
For this value:
{
"s": "qwe",
"ss": null
}
Encoding in hex will be 06 71 77 65 00 - We know it's a record, so we go for fields encoding. First is a string, for a string first is a length, 06 is an avro encoding of long 3, so next 3 values are a string content. If you search for ASCII table with hex values you will see that 71 is q, 77 is w and 65 is e.
Next field is union, so encoding has int for which type, 00 is avro encoding for int 0, so first type in the union. This is "null" in your case, null is encoded as no bytes, so that's the end of the data. If it was 02, that would mean type on position 1 - string in your schema and it would follow with encoding of string value.
You can read more about it in documentation
If you want to see more examples I wrote a post about it
I have been struggling with this problem for a long time. I need to create a new JSON flowfile using QueryRecord by taking an array (field ref) from input JSON field refs and skip the object field as shown in example below:
Input JSON flowfile
{
"name": "name1",
"desc": "full1",
"refs": {
"ref": [
{
"source": "source1",
"url": "url1"
},
{
"source": "source2",
"url": "url2"
}
]
}
}
QueryRecord configuration
JSONTreeReader setup as Infer Schema and JSONRecordSetWriter
select name, description, (array[rpath(refs, '//ref[*]')]) as sources from flowfile
Output JSON (need)
{
"name": "name1",
"desc": "full1",
"references": [
{
"source": "source1",
"url": "url1"
},
{
"source": "source2",
"url": "url2"
}
]
}
But got error:
QueryRecord Failed to write MapRecord[{references=[Ljava.lang.Object;#27fd935f, description=full1, name=name1}] with schema ["name" : "STRING", "description" : "STRING", "references" : "ARRAY[STRING]"] as a JSON Object due to java.lang.ClassCastException: null
Try the following approach, in your case it shoud work:
1) Read your JSON field fully (I imitated it with GenerateFlowFile processor with your example)
2) Add EvaluateJsonPath processor which will put 2 header fileds (name, desc) into the attributes:
3) Add SplitJson processor which will split your JSON byt refs/ref/ groups (split by "$.refs.ref"):
4) Add ReplaceText processor which will add you header fields (name, desc) to the split lines (replace "[{]" value with "{"name":"${json.name}","desc":"${json.desc}","):
5) It`s done:
Full process in my demo case:
Hope this helps.
Solution!: use JoltTransformJSON to transform JSON by Jolt specification. About this specification.
I have the following response:
[
{
"id": 53,
"fileUri": "abc",
"filename": "abc.jpg",
"fileSizeBytes": 578466,
"createdDate": "2018-10-15",
"updatedDate": "2018-10-15"
},
{
"id": 54,
"fileUri": "xyz",
"filename": "xyz.pdf",
"fileSizeBytes": 88170994,
"createdDate": "2018-10-15",
"updatedDate": "2018-10-15"
}
]
and I am trying to match the id value to the object in JUnit like so:
RestAssured.given() //
.expect() //
.statusCode(HttpStatus.SC_OK) //
.when() //
.get(String.format("%s/%s/file", URL_BASE, id)) //
.then() //
.log().all() //
.body("", hasSize(2)) //
.body("id", hasItems(file1.getId(), file2.getId()));
But when the match occurs it tries to match an int to a long. Instead I get this output:
java.lang.AssertionError: 1 expectation failed.
JSON path id doesn't match.
Expected: (a collection containing <53L> and a collection containing <54L>)
Actual: [53, 54]
How does one tell Rest Assured that the value is indeed a long even though it might be short enough to fit in an int? I can cast the file's id to an int and it works, but that seems sloppy.
The problem is that when converting from json to java type, int type selected,
one solution is to compare int values.
instead of
.body("id", hasItems(file1.getId(), file2.getId()));
use
.body("id", hasItems(new Long(file1.getId()).intValue(), new Long(file2.getId()).intValue()));
Hi when working in Backand I try to run the following query:
{
"object": "dr_persons",
"q": {
"person_type" : "4"
},
"fields": ["first_name", "last_name"]
}
person_type is a table in mysql db with "4" as a value.
When I run it I get this error:
Errors in Query
Please fix the following errors in the query:
not a valid constant for field person_type of object dr_persons
The only thing I can see is that when I sync my db it makes it a "float" which I can't change. Can anyone give me some direction on this?
The error message is due to the constant "4" being a string. According to the field type, float, it should be a number. Hence your query should be:
{
"object": "dr_persons",
"q": {
"person_type" : 4
},
"fields": ["first_name", "last_name"]
}