I have a situation where I'll need to import a bunch of different data that may end up having conflicting data types. I have decided to convert everything to a string and then convert back later if the data is needed. I can't figure out how to do this with Elasticsearches (ES) dynamic mapping using the javascript client.
What ES says in their docs:
{
"mappings": {
"my_type": {
"dynamic_templates": [
{ "es": {
"match": "*_es",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"analyzer": "spanish"
}
}},
{ "en": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
"type": "string",
"analyzer": "english"
}
}}
]
}}}
in their docs it says " Match string fields whose name ends in _es".
"Match all other string fields": https://www.elastic.co/guide/en/elasticsearch/guide/current/custom-dynamic-mapping.html
This is what I've tried, but doesn't convert all to string (also tried without quotes around wildcard):
event.mappings = {
"mytype": {
"match": "*",
"mapping": {
"type": "string"
}
}
}
I've also tried "match_mapping_type" : "*".
I've tried: esClient.indices.putMapping({index:"myindex", type:"mytype", body:mybody})
in the response and outside of the .create function.
Any tips?
Your mapping should look like this
PUT /test
{
"mappings": {
"test": {
"dynamic_templates": [
{
"en": {
"match": "*",
"mapping": {
"type": "string"
}
}
}
]
}
}
}
Test data:
POST /test/test/1
{
"nr": 1,
"jsonDate":"2015-06-08T03:41:12-05:00",
"bool": true
}
The resulting mapping, as seen by ES:
{
"test": {
"mappings": {
"test": {
"dynamic_templates": [
{
"en": {
"mapping": {
"type": "string"
},
"match": "*"
}
}
],
"properties": {
"bool": {
"type": "string"
},
"jsonDate": {
"type": "string"
},
"nr": {
"type": "string"
}
}
}
}
}
}
Related
I could not find any examples of this so I'm assuming it's not possible, but want to confirm.
I have a main schema that references other schemas:
https://www.jsonschemavalidator.net/s/4aLvXa4I
{
"$defs": {
"mainSchema": {
"$id": "https://example.com/person.schema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"type": "object",
"properties": {
"objectOne": {
"$ref": "#/$defs/objectOne"
},
"objectTwo": {
"$ref": "#/$defs/objectTwo"
}
}
},
"objectOne": {
"type": "object",
"properties": {
"checkThisValue": {
"type": "string",
"enum": [
"one",
"two",
"three"
]
}
}
},
"objectTwo": {
"type": "object",
"properties": {
"whenSettingThisValue": {
"type": "string",
"enum": [
"A",
"B",
"C"
]
}
}
}
},
"$ref": "#/$defs/mainSchema"
}
I want to define this rule:
IF objectOne.checkThisValue == one
THEN objectTwo.whenSettingThisValue MUST == A
Same for two=>B and three=>C
Is this possible somehow? How do I reference objectOne properties inside of objectTwo?
Edit
I tried to create an if rule for objectTwo that references objectOne here, but my syntax is wrong because it's not working. whenSettingThisValue is set to C and it's saying valid when it should be invalid: https://www.jsonschemavalidator.net/s/NNjEIhWW
"objectTwo": {
"type": "object",
"properties": {
"whenSettingThisValue": {
"type": "string",
"enum": [
"A",
"B",
"C"
]
}
},
"if": {
"properties": {
"objectOne": {
"checkThisValue": {
"const": "one"
}
}
}
},
"then": {
"properties": {
"objectTwo": {
"whenSettingThisValue": {
"const": "A"
}
}
}
}
}
},
I also tried using a oneOf rule in mainSchema where both the subschemas are used, but it is not working either (this example should be invalid): https://www.jsonschemavalidator.net/s/WkmMasDC
"mainSchema": {
"$id": "https://example.com/person.schema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"type": "object",
"properties": {
"objectOne": {
"$ref": "#/$defs/objectOne"
},
"objectTwo": {
"$ref": "#/$defs/objectTwo"
}
},
"oneOf": [
{
"type": "object",
"properties": {
"objectOne": {
"checkThisValue": {
"const": "one"
}
},
"objectTwo": {
"whenSettingThisValue": {
"const": "A"
}
}
}
}
]
},
Solution
Per answer my OneOf rule was malformed. This correctly checks a value in one object is set in response to another. I thought because I was referencing them in another schema with $ref I would have to do something special, but I don't.
https://www.jsonschemavalidator.net/s/HcVhrShk
"oneOf": [
{
"type": "object",
"properties": {
"objectOne": {
"properties": {
"checkThisValue": {
"const": "one"
}
}
},
"objectTwo": {
"properties": {
"whenSettingThisValue": {
"const": "A"
}
}
}
}
},
{
"type": "object",
"properties": {
"objectOne": {
"properties": {
"checkThisValue": {
"const": "two"
}
}
},
"objectTwo": {
"properties": {
"whenSettingThisValue": {
"const": "B"
}
}
}
}
}
]
},
Yes you can. There is an if/then/else construct which takes schemas as its arguments, so you can define a rule "if property A exists with value X, then property B must exist with value Y" etc.
There are some examples here:
https://json-schema.org/understanding-json-schema/reference/conditionals.html#if-then-else
edit:
in your re-edit, change the oneOf clause to this:
"oneOf": [
{
"type": "object",
"properties": {
"objectOne": {
"properties": {
"checkThisValue": {
"const": "one"
}
}
}
}
},
{
"type": "object",
"properties": {
"objectTwo": {
"properties": {
"whenSettingThisValue": {
"const": "A"
}
}
}
}
}
]
I want to conditionally render an input field depending on a checkbox checked.
This checkbox is nested and I dont know how to access it.
I got this to work:
{
"type": "object",
"properties": {
"firstName": {
"type": "boolean"
}
},
"dependencies": {
"firstName": {
"oneOf": [
{
"properties": {
"firstName": {
"enum": [true]
},
"lastName": {
"type": "string"
}
}
}
]
}
}
}
example here
Now what if the dependency is nested like so?:
{
"type": "object",
"properties": {
"test": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
}
}
}
},
"dependencies": {
"test": {
"enabled": {
"oneOf": [
{
"properties": {
"enabled": {
"enum": [true]
},
"lastName": {
"type": "string"
}
}
}
]
}
}
}
}
as you can see: I have tried to access it but it is not recognized correctly.
How would I approach this problem? Is this even possible?
JSON Scheme Validator says its valid
You are missing a "properties" under "dependencies" -> "test". Everything under that point is not recognized as a schema.
"dependencies": {
"test": {
"properties": {
"enabled": {
...
"dependencies" is documented here: https://json-schema.org/understanding-json-schema/reference/conditionals.html#id5.
It looks like you have to have the dependencies in the same schema object as the property... like this
{
"type": "object",
"properties": {
"test": {
"type": "object",
"properties": {
"enabled": {
"type": "boolean"
}
},
"dependencies": {
"enabled": {
"oneOf": [
{
"properties": {
"enabled": {
"const": true
},
"lastName": {
"type": "string"
}
}
}
]
}
}
}
}
}
This works, but I don't know if the resulting data is what you would expect.
I'm an expert with JSON Schema, but I've never used the form generation tooling.
I have a REST API that support for most calls a with query parameter, allowing the user to specify which additional fields they want to load along with the resource’s basic data.
I can describe the query parameter with OpenAPI, and I can describe the format of the response with JSON-Schema. But how can I describe the relation between the parameter and the response?
What I want to describe is:
/api/resource/{id} returns:
{"id": 123, … }
/api/resource/{id}?with=stuff returns:
{"id": 123, …, "stuff": { … }}
I would like this stuff property to be defined as required. Is there a way of doing this?
(What makes this probably even more impossible is that the value of with can be a comma-separated list of properties, but at this point I’m not hoping)
You can't do anything in JSON Schema using out of context data. So, the only way to do what you want is to put the query data into your response.
{
"id": 123,
"query": {},
...
}
{
"id": 123,
"with": ["stuff"],
...
"stuff": { ... }
}
With pure JSON Schema or OpenAPI 3.1, you can use if/then and contains to express the conditional requirement.
{
"type": "object",
"properties": {
"id": { "type": "object" },
"with": { "type": "array", "items": { "type": "string" } },
...
"stuff": { ... }
},
"required": ["id"],
"allOf": [
{
"if": {
"properties": {
"with": { "contains": { "const": "stuff" } }
},
"required": ["with"]
},
"then": { "required": ["stuff"] }
},
... additional conditionals ...
]
}
However, assuming you are on OpenAPI 3.0, you don't have if, then, or contains. It's still possible, but it gets a bit ugly because you need to use a bunch of confusing boolean logic concepts instead.
{
"type": "object",
"properties": {
"id": { "type": "object" },
"with": { "type": "array", "items": { "type": "string" } },
...
"stuff": { ... }
},
"required": ["id"],
"allOf": [
{
"anyOf": [
{
"not": {
"properties": {
"with": {
"not": {
"items": { "not": { "enum": ["stuff"] } }
}
}
},
"required": ["with"]
}
},
{ "required": ["stuff"] }
]
},
... additional conditionals ...
]
}
JSON schema: query string required
schema: {
tags: ["sample"],
summary: "Set query string values to required",
querystring: {
type: "object",
properties: {
name: {
type: "string",
},
address: {
type: "string",
},
},
required: ["name"],
},
},
A year ago I've asked how to set the type of a schema object based on the value of another property? which I've got a great answer for, and I've been using that schema ever since.
Now the source data have changed - and the schema is failing under the following circumstances:
The source data contains many properties, however only two of them are relevant for this question: "key" and "value" - the type of "value" depends on the value of "key" -
For instance:
If the key is "comment", the type of value {"Text":"commentValue"}.
If the key is "offset", the type of value is {"seconds":int}.
If the key is "weather", the type of value is {"value": Enum["sun", "clouds", "rain"...]}
Some of the keys do not have the value property, so the schema should forbid it from appearing with these keys - for instance, if the key is "standby" the value property should not appear at all - which is what my current schema is doing good.
However, now the data source have changed and I get "value" :{} as a part of my Json where once it was omitted - and the current schema does not allow it.
So my question is - how do I allow one of these two options? I've tried any combination of anyOf I could think of, but failed miserably - the Newtonsoft.Json.Schema.JSchema failed to parse the text.
Here's a simplified version of the schema I'm currently using:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "TestOptionalObject",
"type": "object",
"additionalProperties": false,
"required": [
"test"
],
"properties": {
"test": {
"$ref": "#/definitions/test"
}
},
"definitions": {
"test": {
"type": "object",
"required": [
"key"
],
"properties": {
"key": {
"type": "string",
"enum": [
"comment",
"offset",
"standby",
"status_unsure",
"status_ok"
]
}
},
"allOf": [
{
"if": {
"properties": {
"event": {
"enum": [
"standby",
"status_unsure",
"status_ok"
]
}
}
},
"then": {
"properties": {
"value": false
}
}
},
{
"if": {
"properties": {
"key": {
"const": "comment"
}
}
},
"then": {
"properties": {
"value": {
"$ref": "#/definitions/commentValue"
}
}
}
},
{
"if": {
"properties": {
"key": {
"const": "offset"
}
}
},
"then": {
"properties": {
"value": {
"$ref": "#/definitions/offsetValue"
}
}
}
}
]
},
"commentValue": {
"type": "object",
"additionalProperties": false,
"required": [
"text"
],
"properties": {
"text": {
"type": "string"
}
}
},
"offsetValue": {
"type": "object",
"additionalProperties": false,
"required": [
"seconds"
],
"properties": {
"seconds": {
"type": "integer",
"format": "int32"
}
}
}
}
}
Here are some of the things I've tried:
"then": {
"properties": {
"anyOf": [
{ "value": false },
{ "value": null }
]
}
}
"then": {
"anyOf": [
{
"properties": {
{ "value": false }
},
"properties": {
{ "value": null }
}
]
}
"then": {
"properties": {
"value":
"anyOf": [false, null ]
}
}
Json examples to validate:
Should fail:
{
"test": {
"key": "comment",
"value": {"seconds":12}
}
}
{
"test": {
"key": "standby",
"value": {"asdf":12}
}
}
Should pass:
{
"test": {
"key": "comment",
"value": {"text":"comment text"}
}
}
{
"test": {
"key": "offset",
"value": {"seconds":12}
}
}
{
"test": {
"key": "standby"
}
}
{
"test": {
"key": "standby",
"value": {}
}
}
Note the last example - the value property is an empty object - that also should pass but fails with the current schema since the value property should not exist at all for this key.
You're close, but not quite. You have to remember that the allOf is an array of subschemas (JSON Schemas). (Null isn't a valid schema, so you might have got some "not a valid schema" errors.)
As such, consider this modified subschema from allOf[0]...
{
"if": {
"properties": {
"key": {
"enum": [
"standby",
"status_unsure",
"status_ok"
]
}
}
},
"then": {
"properties": {
"value": {
"type": [
"object"
],
"additionalProperties": false
}
}
}
}
You can test it here: https://jsonschema.dev/s/EfNI1
The if block remains the same. (Although I've corrected what I assume was a mistake in using event rather than key in your simplification from the real schema.)
The then block needs to define that the object (already checked by definitions.test) has a key of value, where the value of value is an object, and has no properties (aka an empty object).
To achive "an empty object", you need to use additionalProperties.
additionalProperties applies its value subschema to all properties that exist that haven't been defined in properties or that match the keys (regexes) from patternProperties.
false as a schema always fails validation.
additionalProperties without properites applies to ALL properties, and therefore with a value of false, validates "is an empty object".
I am trying to create an index in elasticsearch with the following settings. I am facing the following error
error:
RemoteTransportException[[ys2order-stg2-01][inet[/64.101.206.15:9300]][indices:admin/mapping/put]];
nested: MapperParsingException[Analyzer [autocomplete] not found for
field [DESCRIPTION_AUTO]]; status: 400
{
"settings": {
"index": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
},
"analyzer": {
" autocomplete": {
"type": "custom",
"filter": [
"lowercase",
"autocomplete_filter"
],
"tokenizer": "standard"
}
}
},
"refresh_interval": -1,
"number_of_replicas": 1,
"number_of_shards": 4,
"index_analyzer": "default_index",
"search_analyzer": "default_search"
}
},
"mappings": {
"itemsnew": {
"properties": {
"DESCRIPTION_AUTO": {
"index_analyzer": "autocomplete",
"search_analyzer": "standard",
"type": "string"
}
}
}
}
}
You need to remove a space in your analyzer name:
"analyzer": {
" autocomplete": { <--- remove initial space here
"type": "custom",
"filter": [
"lowercase",
"autocomplete_filter"
],
"tokenizer": "standard"
}
....
"DESCRIPTION_AUTO": {
"index_analyzer": "autocomplete", <-- because there is no initial space here !!!
"search_analyzer": "standard",
"type": "string"
}