I have the data in the following structure
"mappings" : {
"PERSON" : {
"properties" : {
"ADDRESS" : {
"type": "nested",
"properties" : {
"STREET" : {
"type": "nested",
"properties" : {
"street": {
"type": "string"
},
"number": {
"type": "integer"
}
}
},
"CITY" : {
"type": "nested",
"properties" : {
"name": {
"type": "string"
},
"size": {
"type": "integer"
}
}
}
,
"country": {
"type": "string"
}
}
},
"INFORMATION" : {
"type": "nested",
"properties" : {
"age": {
"type": "integer"
},
"sex": {
"type": "string"
}
}
}
"name" : {
"type": "string",
}
}
}
}
I want to aggregate nested object dynamically in form:
["object type": count of records with this type].
e.g. for the PERSON I want to get something like:
[ADDRESS: 1000, INFORMATION: 1230]
and for ADDRESS:
[STREET: 200, CITY: 100]
Is it possible?
You can first filter based on PERSON or ADDRESS and then use cardinality aggregation to get the count
Related
Consider this following example,
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Animal",
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "Type of animal."
},
"data": {
"$ref": "#/$definations/cat"
}
},
"$definations":{
"cat" : {
"properties" : {
"meow" : {
"type": "string"
}
}
}
},
"required": ["type"]
}
and the correct JSON is ,
{
"type" : "cat",
"data" : {
"meow" : "OK"
}
}
Now I am having enum of Animals, and the data ref will vary based on type of Animal.
I have have tried if else but it seems not efficient as the condition will keep on growing.
Also used anyOf but how will I make sure that meow will always belong to animal type cat and not dog.
Can we have something like,
cat : { "$ref" : "#/$definations/cat" },
dog : { "$ref" : "#/$definations/dog" }
EDIT : Or dynamic value in ref like #/$definations/{type-value} ?
Thanks in advance.
I have have tried if else but it seems not efficient as the condition
will keep on growing.
Can we have something like...
No. JSON Schema (2019-09 and previous) doesn't have a "switch".
You'll need to use allOf to create multiple if then conditions.
After #Relequestual's response and some more digging I found there is no such way. At least in this version, fingers crossed for the future releases.
Here is my solution, feel free to suggest improvements.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Animal",
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "Type of animal."
}
},
"$definations": {
"cat": {
"properties": {
"meow": {
"type": "string"
}
},
"required": [
"meow"
]
},
"dog": {
"properties": {
"bhow": {
"type": "string"
}
},
"required": [
"bhow"
]
}
},
"oneOf": [
{
"properties": {
"type": {
"const": "cat"
},
"data": {
"$ref": "#/$definations/cat"
}
}
},
{
"properties": {
"type": {
"const": "dog"
},
"data": {
"$ref": "#/$definations/dog"
}
}
}
],
"required": [
"type"
]
}
I am trying to create index in Elasticsearch using API using the following mapping in kibana dev tools. Once I create the index, I want to use reindex API to copy documents from an already existing index.
PUT /ipflow-logs
{
"ipflow-logs" : {
"mappings" : {
"properties" : {
"conn_state" : {
"type" : "keyword"
},
"content_length" : {
"type" : "long"
},
"content_type" : {
"type" : "keyword"
},
"createdDate" : {
"type" : "keyword"
},
"dst_ip" : {
"type" : "ip"
},
"dst_port" : {
"type" : "long"
},
"duration" : {
"type" : "long"
},
"history" : {
"type" : "keyword"
},
"local_orig" : {
"type" : "keyword"
},
"missed_bytes" : {
"type" : "long"
},
"orig_bytes" : {
"type" : "long"
},
"orig_ip_bytes" : {
"type" : "long"
},
"orig_pkts" : {
"type" : "long"
},
"protocol" : {
"type" : "keyword"
},
"resp_bytes" : {
"type" : "long"
},
"resp_ip_bytes" : {
"type" : "long"
},
"resp_pkts" : {
"type" : "long"
},
"service" : {
"type" : "keyword"
},
"src_ip" : {
"type" : "ip"
},
"src_port" : {
"type" : "long"
},
"timestamp" : {
"type" : "date",
"format" : "yyyy-MM-dd 'T' HH:mm:ss.SSS"
},
"uid" : {
"type" : "keyword"
}
}
}
}
}
I am getting the below error when I try to create the index.
"type": "parse_exception", "reason": "unknown key [ipflow-logs] for create index", "status": 400
Any help is appreciated. Thanks
You need to do it this way (i.e. mappings should be at the top):
PUT /ipflow-logs
{
"mappings": {
"properties": {
"conn_state": {
"type": "keyword"
},
"content_length": {
"type": "long"
},
"content_type": {
"type": "keyword"
},
"createdDate": {
"type": "keyword"
},
"dst_ip": {
"type": "ip"
},
"dst_port": {
"type": "long"
},
"duration": {
"type": "long"
},
"history": {
"type": "keyword"
},
"local_orig": {
"type": "keyword"
},
"missed_bytes": {
"type": "long"
},
"orig_bytes": {
"type": "long"
},
"orig_ip_bytes": {
"type": "long"
},
"orig_pkts": {
"type": "long"
},
"protocol": {
"type": "keyword"
},
"resp_bytes": {
"type": "long"
},
"resp_ip_bytes": {
"type": "long"
},
"resp_pkts": {
"type": "long"
},
"service": {
"type": "keyword"
},
"src_ip": {
"type": "ip"
},
"src_port": {
"type": "long"
},
"timestamp": {
"type": "date",
"format": "yyyy-MM-dd 'T' HH:mm:ss.SSS"
},
"uid": {
"type": "keyword"
}
}
}
}
I have been trying to validate my request parameters using x-amazon-apigateway-request-validator, but unfortunately it is not working. Below is swagger file-
{
"swagger": "2.0",
"info": {
"title": "API Gateway - Request Validation Demo"
},
"schemes": [
"https"
],
"produces": [
"application/json"
],
"x-amazon-apigateway-request-validators" : {
"full" : {
"validateRequestBody" : true,
"validateRequestParameters" : true
},
"body-only" : {
"validateRequestBody" : true,
"validateRequestParameters" : false
}
},
"x-amazon-apigateway-request-validator" : "full",
"paths": {
"/orders": {
"post": {
"x-amazon-apigateway-request-validator": "body-only",
"parameters": [
{
"in": "body",
"name": "CreateOrders",
"required": true,
"schema": {
"$ref": "#/definitions/CreateOrders"
}
}
],
"responses": {
"200": {
"schema": {
"$ref": "#/definitions/Message"
}
},
"400" : {
"schema": {
"$ref": "#/definitions/Message"
}
}
},
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200",
"responseTemplates": {
"application/json": "{\"message\" : \"Orders successfully created\"}"
}
}
},
"requestTemplates": {
"application/json": "{\"statusCode\": 200}"
},
"passthroughBehavior": "never",
"type": "mock"
}
},
"get": {
"x-amazon-apigateway-request-validator": "full",
"parameters": [
{
"in": "header",
"name": "Account-Id",
"required": true
},
{
"in": "query",
"name": "type",
"required": false,
"schema": {
"$ref": "#/definitions/InputOrders"
}
}
],
"responses": {
"200" : {
"schema": {
"$ref": "#/definitions/Orders"
}
},
"400" : {
"schema": {
"$ref": "#/definitions/Message"
}
}
},
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200",
"responseTemplates": {
"application/json": "[{\"order-id\" : \"qrx987\",\n \"type\" : \"STOCK\",\n \"symbol\" : \"AMZN\",\n \"shares\" : 100,\n \"time\" : \"1488217405\",\n \"state\" : \"COMPLETED\"\n},\n{\n \"order-id\" : \"foo123\",\n \"type\" : \"STOCK\",\n \"symbol\" : \"BA\",\n \"shares\" : 100,\n \"time\" : \"1488213043\",\n \"state\" : \"COMPLETED\"\n}\n]"
}
}
},
"requestTemplates": {
"application/json": "{\"statusCode\": 200}"
},
"passthroughBehavior": "never",
"type": "mock"
}
}
}
},
"definitions": {
"CreateOrders": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Create Orders Schema",
"type": "array",
"minItems" : 1,
"items": {
"type": "object",
"$ref" : "#/definitions/Order"
}
},
"Orders" : {
"type": "array",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Get Orders Schema",
"items": {
"type": "object",
"properties": {
"order_id": { "type": "string" },
"time" : { "type": "string" },
"state" : {
"type": "string",
"enum": [
"PENDING",
"COMPLETED"
]
},
"order" : {
"$ref" : "#/definitions/Order"
}
}
}
},
"Order" : {
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Schema for a single Order",
"required": [
"account-id",
"type",
"symbol",
"shares",
"details"
],
"properties" : {
"account-id": {
"type": "string",
"pattern": "[A-Za-z]{6}[0-9]{6}"
},
"type": {
"type" : "string",
"enum" : [
"STOCK",
"BOND",
"CASH"]
},
"symbol" : {
"type": "string",
"minLength": 1,
"maxLength": 4
},
"shares": {
"type": "number",
"minimum": 1,
"maximum": 1000
},
"details": {
"type": "object",
"required": [
"limit"
],
"properties": {
"limit": {
"type": "number"
}
}
}
}
},
"InputOrder" : {
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Schema for a Input Order",
"required": [
"type"
],
"properties" : {
"type": {
"type" : "string",
"enum" : [
"STOCK",
"BOND",
"CASH"]
}
}
},
"Message": {
"type": "object",
"properties": {
"message" : {
"type" : "string"
}
}
}
}
}
I am trying to validate my request parameters against some regex and enum values.
I am not sure if this is even possible or not. Can anybody please help me with this?
For HTTP parameter validation, API Gateway only supports marking one as 'required'. It does not support regex/enum values for parameters.
I'm working on creating a JsonSchema(v4).
I'm trying to make one property required based off the value of another property from it's parent.
Parent
User
subtype
address
Child
Address
line1
line2
companyName (required if user subtype is company)
How could this be done?
I have something like this now...
{
"User": {
"title": "User",
"type": "object",
"id": "#User",
"properties": {
"subtype": {
"type": "string"
},
"address": {
"$ref": "Address"
}
}
}
"Address": {
"title": "Address",
"type": "object",
"id": "#Address",
"properties": {
"line1": {
"type": "string"
},
"line2": {
"type": "string"
},
"companyName": {
"type": "string"
}
},
"required": ["line1", "line2"]
}
}
Subtype is a arbitrary string, so a full list of the different subtypes is not possible.
Add this to your User schema. Essentially it reads as: either "subtype" is not "company" or "address" requires "companyName".
"anyOf": [
{
"not": {
"properties": {
"subtype": { "enum": ["company"] }
}
}
},
{
"properties": {
"address": {
"required": ["companyName"]
}
}
}
]
Need help to find the bug with this schema. It has oneOf operator.
Schema is here :
`{
"type": "object",
"required": [
"type",
"body"
],
"properties": {
"type": {
"description": "type of the document to post",
"type": "string",
"enum": [
"123",
"456"
]
},
"body": {
"type": "object",
"description": "body",
"oneOf": [{
"$ref": "#/definitions/abc",
"$ref": "#/definitions/def"
}]
}
},
"definitions": {
"abc": {
"type": "array",
"description": "abc",
"properties" : {
"name" : { "type" : "string" }
}
},
"def": {
"type": "array",
"description": "users","properties" : {
"name" : { "type" : "string" }
}
}
}
}`
My Json is this :
`{
"type": "123",
"body": {
"abc": [{
"name": "test"
}]
}
}`
It does not validate with tv4 and I also tried this online tool. It works without oneOf operator. Otherwise it does not validate it any tool.
Edit :
After reading the answers I modified the schema. New schema is :
{
"type": "object",
"properties": {
"type": {
"description": "type of the document to post",
"type": "string",
},
"body": {
"type": "object",
"description": "body",
"properties": {
"customers": {
"type": "array"
}
},
"anyOf": [
{
"title": "customers prop",
"properties": {
"customers": {
"type": "array",
"description": "customers",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
}
}
}
}
}
}
]
}
}
}
And json is here
{
"type": "customer",
"body": {
"none": [
{
"name": "test"
}
]
}
}
But it validates. I want to enforce one of "customers" or "users" in the body. To test I have removed users from the body.
Pl help.
The issue is that the data is passing both of your sub-schemas. oneOf means "match exactly one" - if you want "match at least one", then use anyOf.
In fact, both of your sub-schemas will pass all data. The reason is that properties is ignored when dealing with arrays.
What you presumably wanted to do instead is specify properties for the items in the array. For this, you need the items keyword:
"definitions": {
"abc": {
"type": "array",
"items": {
"type": "object",
"properties" : {
"name" : { "type" : "string" }
}
}
}
}
(You'll also need to add some distinct constraints - at the moment, both the "abc" and "def" definitions are identical apart from description, which makes the oneOf impossible because it will always match both or neither.)
Since you have the type at root level, you probably want the oneOf statement to check that an object with type "customer" has customers in the body (even though I would suggest skipping the body and placing customers and users directly in root object).
This works with your example, will require that an object with type "customer" has a body with "customers", and to clarify the matching, I let customer have the property "name" while the user has "username":
{
"type": "object",
"properties": {
"type": { "type": "string" },
"body": {
"type": "object",
"properties": {
"customers": {
"type": "array",
"items": { "$ref": "#/definitions/customer" }
},
"users": {
"type": "array",
"items": { "$ref": "#/definitions/user" }
}
}
}
},
"definitions": {
"customer": {
"type": "object",
"properties": { "name": { "type": "string" } },
"required": [ "name" ]
},
"user": {
"type": "object",
"properties": { "username": { "type": "string" } },
"required": [ "username" ]
}
},
"oneOf": [
{
"properties": {
"type": {
"pattern": "customer"
},
"body": {
"required": [ "customers" ]
}
}
},
{
"properties": {
"type": {
"pattern": "user"
},
"body": {
"required": [ "users" ]
}
}
}
]
}
When using "type": "array" then the item type is defined in the "items" property not "properties" property... Also both types in oneOf are same, but only one must match.
Try
...
"definitions": {
"abc": {
"type": "array",
"description": "abc",
"items" : {
"name" : { "type" : "string" }
}
},
"def": {
"type": "array",
"description": "users",
"items" : {
"username" : { "type" : "string" }
}
}
}