Rest Assured Exception - invalid JSON Schema, cannot continue - jsonschema

I am using TestNG along with Rest Assured.
We have json schema file. Inside the Json schema it refers to other schema using $ref attribute.
When I use matchesJsonSchemaInClasspath for Hamcrest assertion it is failing with an validation error.
The error is basically the Validator is unable to read the nested JSON schema.
Can someone help me to resolve the issue.
Error and the Json schemas are attached.
Error:
com.github.fge.jsonschema.core.exceptions.InvalidSchemaException:
fatal: invalid JSON Schema, cannot continue
Syntax errors:
[ {
"level" : "warning",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/response.json#",
"pointer" : ""
},
"domain" : "syntax",
"message" : "the following keywords are unknown and will be ignored: [javaType, name]",
"ignored" : [ "javaType", "name" ]
}, {
"level" : "error",
"message" : "value has incorrect type (found boolean, expected one of [array])",
"domain" : "syntax",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/service_header.json#",
"pointer" : ""
},
"keyword" : "required",
"found" : "boolean",
"expected" : [ "array" ]
}, {
"level" : "error",
"message" : "URI \"./service_header.json\" is not normalized",
"domain" : "syntax",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/service_header.json#",
"pointer" : "/properties/service_header"
},
"keyword" : "$ref",
"value" : "./service_header.json"
}, {
"level" : "error",
"message" : "value has incorrect type (found boolean, expected one of [array])",
"domain" : "syntax",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/service_header.json#",
"pointer" : "/properties/service_header"
},
"keyword" : "required",
"found" : "boolean",
"expected" : [ "array" ]
}, {
"level" : "error",
"message" : "value has incorrect type (found boolean, expected one of [array])",
"domain" : "syntax",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/service_item.json#",
"pointer" : "/properties/items"
},
"keyword" : "required",
"found" : "boolean",
"expected" : [ "array" ]
}, {
"level" : "error",
"message" : "URI \"./setvice_item.json\" is not normalized",
"domain" : "syntax",
"schema" : {
"loadingURI" : "file:/C:/GitWorkspace/OM/service_OM/functionalTest/target/test-classes/specs/specification/v1/schema/response/service_item.json#",
"pointer" : "/properties/items/items"
},
"keyword" : "$ref",
"value" : "./service_item.json"
} ]
level: "fatal"
JSON Schema:
main.json
{
"type":"object",
"$schema":"http://json-schema.org/draft-04/hyper-schema",
"name": "Response",
"title": "Response",
"javaType": "com.ruthresh.Response",
"description": "Enables you to make a POST response.",
"required":true,
"properties":{
"service_header":{
"type":"object",
"description": "The service header for a response.",
"required":true,
"$ref":"./service_header.json"
},
"items":{
"type":"array",
"description": "An array of individual items.",
"required":true,
"maxLength":5000,
"minLength":1,
"items":{
"$ref":"./service_item.json"
}
}
}
}
service_header.json
{
"type":"object",
"$schema":"http://json-schema.org/draft-04/hyper-schema",
"name":"service_header",
"title":"Service response Header",
"description":"The service header for a response.",
"id":"service_header:v1",
"javaType": "com.ruthresh.Header",
"required":true,
"properties":{
"email_subject":{
"type":"string",
"description":"Email subject.",
"maxLength":255
},
"email_message":{
"type":"string",
"description":"Email Message.",
"maxLength":4000
}
}
}
serivice_item.json
{
"type":"object",
"$schema":"http://json-schema.org/draft-04/hyper-schema",
"name":"serivice_item",
"title":"Service response Header",
"description":"The service header for a response.",
"id":"serivice_item:v1",
"javaType": "com.ruthresh.Item",
"required":true,
"properties":{
"item_subject":{
"type":"string",
"description":"Item subject.",
"maxLength":255
},
"item_message":{
"type":"string",
"description":"Item Message.",
"maxLength":4000
}
}
}

As the error message says, "required" should be an array of required properties and not boolean.

Related

How to use springdoc with #PostMapping(consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE) and #RequestParam

I'm upgrading a project from SpringFox to SpringDoc v1.6.12 and I struggle to make the new code work for the following method of my RestController:
#PostMapping(path = TASK_MAPPING_PATH, consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
public ResponseEntity<String> loadTask(
#RequestParam String applicationId,
#RequestParam String businessId,
#RequestParam boolean directLink
) {[...]}
The particularity of this method is that it should encode its parameters in the body since the Content-Type application/x-www-form-urlencoded is used.
But when I browse the url https://localhost:8443/v3/api-docs, the generated code is the following:
"/api/enrolment/task" : {
"post" : {
"operationId" : "loadTask",
"parameters" : [ {
"in" : "query",
"name" : "applicationId",
"required" : true,
"schema" : {
"type" : "string"
}
}, {
"in" : "query",
"name" : "businessId",
"required" : true,
"schema" : {
"type" : "string"
}
}, {
"in" : "query",
"name" : "directLink",
"required" : true,
"schema" : {
"type" : "boolean"
}
} ],
"responses" : {
[...]
},
"summary" : [...],
"tags" : [...]
}
},
All of the applicationId, businessId and directLink parameters are passed in the URL instead of the request body as expected.
I would have expected the following openApi definition instead:
"/api/enrolment/task" : {
"post" : {
"operationId" : "loadTask",
"requestBody" : {
"content" : {
"application/x-www-form-urlencoded" : {
"schema" : {
"type" : "object",
"properties" : {
"applicationId" : {
"type" : "string"
},
"businessId" : {
"type" : "string"
},
"directLink" : {
"type" : "boolean"
}
},
"required" : [ "applicationId", "businessId", "directLink" ]
}
}
}
},
"responses" : {
[...]
},
"summary" : [...],
"tags" : [...]
}
},
Does anyone ever had the same issue ?
Does anyone knows the solution to my problem ?
Thanks.

BQ load job failing when trying to create table from AVRO file

I am trying to create a BQ Table from AVRO file. I am getting this error when i run the BQ load job:
"Error while reading data, error message: The Apache Avro library
failed to parse the header with the following error: Unexpected type
for default value. Expected long, but found null: null"
The Schema of the AVRO file is:
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
I am not sure what is wrong with the schema or anything else, because of which I am unable to load the data.
The problem is most likely the fields that have type int but you have null as the default value. For example:
"name" : "type_id",
"type" : "int",
"default" : null
The default should either be changed to be an integer or the type should be changed to be a union that includes null (like many of the other fields).

mapper_parsing_exception in Elasticsearch(Reason: No type specified for field [X])

I wanted to provide explicit mapping to the fields in my document, So I defined a mapping for my index demo and It looks like this below:
PUT /demo
{
"mappings": {
"properties": {
"X" : {
"X" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"Sub_X" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
After running the query , I am getting error as :
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "No type specified for field [X]"
}
],
"type" : "mapper_parsing_exception",
"reason" : "Failed to parse mapping [_doc]: No type specified for field [X]",
"caused_by" : {
"type" : "mapper_parsing_exception",
"reason" : "No type specified for field [X]"
}
},
"status" : 400
}
The field X in json document looks like :
"X" : {
"X" : [
"a"
],
"Sub_X" : [
[
"b"
]
]
},
Please help me out with this elastic search mapper_parse_exception error.
What you have is called nested data type
You have X which in turn contains X and Sub_X.
Mapping:
{
"properties": {
"X": {
"type": "nested"
}
}
}
Data:
{
"X": {
"X": [
"a"
],
"Sub_X": [
[
"b"
]
]
}
}
Query:
{
"query": {
"nested": {
"path": "X",
"query": {
"bool": {
"must": [
{ "match": { "X.X": "a" }},
{ "match": { "X.Sub_X": "b" }}
]
}
}
}
}
}
It outputs the document.

How to load Nested json data into a single column in druid

I am trying to load nested json data in Apache druid:
Data-->
{
"a": "a_data",
"b": "b_data",
"c_blob_Column": {"aaaa"{"k":"sample"{"c":"sample2"}}}}
Spec -->
{ "type" : "kafka", "dataSchema" : { "dataSource" : "blob", "parser" : { "type" : "string", "parseSpec" : { "format" : "json", "dimensionsSpec" : { "dimensions" : [ "a", "b", "c_blob_Column"
]
},
"timestampSpec": {
"column": "timestamp",
"format": "iso"
}
}
},
"metricsSpec" : [],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "DAY",
"queryGranularity" : "none",
"rollup" : false
}
},
"ioConfig" : {
"topic":"blob_topic",
"consumerProperties":{
"bootstrap.servers":"<local server>"
},
"appendToExisting" : false,
"useEarliestOffset": true,
"taskDuration": "PT15M"
},
"tuningConfig" : {
"type" : "kafka",
"maxRowsPerSegment" : 5000000,
"maxRowsInMemory" : 25000
}
}
Output columns-->
a,b,c_blob_Column,__time
I am able to load the data but the issue is in the column c_blob_Column the data is not coming as in json form data Could someone please help me to find how to load the json blob data?
you can use jq expression:
"flattenSpec": {
"fields": [
{
"type": "jq",
"name": "c_blob_Column",
"expr": ".c_blob_Column | tojson"
}
]
}

Setting API Key required to true using serverless.template in AWS API Gateway

I am deploying an ASP.Net Core project on AWS Lambda and I am struggling with making the API Key required.
Here is my Json template:
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Transform" : "AWS::Serverless-2016-10-31",
"Description" : "An AWS Serverless Application that uses the ASP.NET Core framework running in Amazon Lambda.",
"Parameters" : {
"ShouldCreateBucket" : {
"Type" : "String",
"AllowedValues" : ["true", "false"],
"Description" : "If true then the S3 bucket that will be proxied will be created with the CloudFormation stack."
},
"BucketName" : {
"Type" : "String",
"Description" : "Name of S3 bucket that will be proxied. If left blank a name will be generated.",
"MinLength" : "0"
}
},
"Conditions" : {
"CreateS3Bucket" : {"Fn::Equals" : [{"Ref" : "ShouldCreateBucket"}, "true"]},
"BucketNameGenerated" : {"Fn::Equals" : [{"Ref" : "BucketName"}, ""]}
},
"Resources" : {
"AspNetCoreFunction" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Handler": "SmartClockAPI::SmartClockAPI.LambdaEntryPoint::FunctionHandlerAsync",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [ "AWSLambdaFullAccess","AmazonCognitoPowerUser","AmazonAPIGatewayAdministrator"],
"Environment" : {
"Variables" : {
"AppS3Bucket" : { "Fn::If" : ["CreateS3Bucket", {"Ref":"Bucket"}, { "Ref" : "BucketName" } ] }
}
},
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/{proxy+}",
"Method": "ANY"
}
}
}
}
},
"BasicUsagePlan" : {
"Type" : "AWS::ApiGateway::UsagePlan",
"Properties" : {
"UsagePlanName" : "Basic plan",
"Quota" : {
"Limit" : 100,
"Period" : "MONTH"
},
"Throttle" : {
"RateLimit" : 10,
"BurstLimit" : 10
}
}
},
"Bucket" : {
"Type" : "AWS::S3::Bucket",
"Condition" : "CreateS3Bucket",
"Properties" : {
"BucketName" : { "Fn::If" : ["BucketNameGenerated", {"Ref" : "AWS::NoValue" }, { "Ref" : "BucketName" } ] }
}
}
},
"Outputs" : {
"ApiURL" : {
"Description" : "API endpoint URL for Prod environment",
"Value" : { "Fn::Sub" : "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" }
},
"S3ProxyBucket" : {
"Value" : { "Fn::If" : ["CreateS3Bucket", {"Ref":"Bucket"}, { "Ref" : "BucketName" } ] }
}
}
}
What I am trying to achieve is setting this value to true from the Json template.
I was expecting some extra property for Proxy where I can specify this value.
Any ideas?
Use AWS::ApiGateway::UsagePlanKey
{
"Type" : "AWS::ApiGateway::UsagePlanKey",
"Properties" : {
"KeyId" : String,
"KeyType" : String,
"UsagePlanId" : String
}
}
Add a DependsOn to ApiKey, ApiUsagePlan, and ApiUsagePlanKey to ensure they were created in the correct order. This is a nice working example