I want to create a cloudformation template to deploy a redis parameter group. The problem lies in the fact that each of my parameters looks like this:
{
"Parameters": [
{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
},
{
"ParameterName": "active-defrag-cycle-max",
"ParameterValue": "75",
"Description": "Maximal effort for defrag in CPU percentage",
"Source": "user",
"DataType": "integer",
"AllowedValues": "1-75",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}
}
Now I understand the basic format of the template but can't figure out how to pass in parameters as seen above.
Failing template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"Parameters": [{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}]
}
}
}
}
}
Welcome to StackOverflow!
You need to incorporate those parameters into the Parameters section of the template, and change the syntax so that it conforms to the Parameters format.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
},
"activedefrag": {
"Description": "Enabled active memory defragmentation",
"Type": "String",
"AllowedValues": ["yes", "no"],
"Default": "no"
},
"activedefragcyclemax": {
"Description": "Maximal effort for defrag in CPU percentage",
"Type": "Number",
"Default": 75,
"MinValue": 1,
"MaxValue": 75
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"activedefrag": {"Ref": "activedefrag"},
"active-defrag-cycle-max": {"Ref": "activedefragcyclemax"}
}
}
}
}
}
Related
I am trying to get proper validation error from oneof or anyof pattern. I have json schema with two or more oneof/anyof condition as mentioned below:
json_schema = {
"type": "object",
"properties": {
"comment": {
"description": "Server Pool Policy Qualification Comments",
"type": "string",
"default": ""
},
"name": {
"description": "Server Pool Policy Qualification Name",
"type": "string",
"default": "",
"pattern": "^[\\-\\.:_a-zA-Z0-9]{1,16}$"
},
"qualifications": {
"description": "Qualifications of Server Pool Policy Qualification",
"type": "array",
"items": {
"description": "Qualification of Server Pool Policy Qualification",
"type": "object",
"oneOf": [
{
"properties": {
"type": {
"description": "Qualification Type",
"type": "string",
"enum": [
"adapter"
]
},
"adapter_qualification":{
"description": "Adapter Qualifications - Adapter Type",
"type": "array",
"properties": {
"adapter_type": {
"description": "Adapter Qualifications - Adapter Type",
"type": "string",
"enum": [
"virtualized-scsi-if"
]
},
"adapter_pid": {
"description": "Adapter Qualifications - Adapter PID (RegEx)",
"type": "string",
"default": "",
"pattern": "[ !#$%\\(\\)\\*\\+,\\-\\./:;\\?#\\[\\\\\\]\\^_\\{\\|\\}~a-zA-Z0-9]{0,256}"
},
"adapter_maximum_capacity": {
"description": "Adapter Qualifications - Maximum Capacity",
"type": "string",
"default": "unspecified",
"pattern": "^unspecified$|^[0-9]$|^[0-9][0-9]$|^[0-9][0-9][0-9]$|^[0-9][0-9][0-9][0-9]$|^[0-5][0-9][0-9][0-9][0-9]$|^6[0-4][0-9][0-9][0-9]$|^65[0-4][0-9][0-9]$|^655[0-2][0-9]$|^6553[0-5]$"
}
},
"additionalProperties": False,
"required": [
"type",
"adapter_type"
]
}
}
},
{
"properties": {
"type": {
"description": "Qualification Type",
"type": "string",
"enum": [
"server_pid"
]
},
"server_pid": {
"description": "Server PID Qualifications - Server PID",
"type": "string",
"default": "",
"pattern": "^123$"
}
},
"additionalProperties": False,
"required": [
"type",
"server_pid"
]
}
]
}
}
},
"additionalProperties": False,
"required": [
"name"
]
}
I have data which has additional element first_rack_id but best matches 2nd element from oneof.
data = {
"descr": "description",
"name": "domainGroup",
"qualifications": [
{
"server_pid": "B200M5",
"type": "server_pid",
"first_rack_id": "10"
}
]
}
validator = Draft7Validator(json_schema)
best = best_match(validator.iter_errors(data))
My expectation is that the error message thrown by validation will find 2nd element from oneof and throw error saying additional property is not allowed. but i get match for 1st element as mentioned below:
'server_pid' is not one of ['adapter']
Failed validating 'enum' in schema[0]['properties']['type']:
{'description': 'Qualification Type',
'enum': ['adapter'],
'type': 'string'}
On instance['type']:
'server_pid'
how do i specify validator to best match with property "type" which will match with enum "server_pid" instead of enum "adapter"
You can specify which schema to validate against with the if/then keywords. It's a bit verbose and can be error prone, but it's the best way to express this sort of thing. Although popular, oneOf is almost never the right choice.
"allOf": [
{
"if": {
"type": "object",
"properties": {
"type": { "const": "adapter" }
},
"required": ["type"]
},
"then": { "$ref": "#/definitions/adapter" }
},
{
"if": {
"type": "object",
"properties": {
"type": { "const": "server_pid" }
},
"required": ["type"]
},
"then": { "$ref": "#/definitions/server-pid" }
}
],
Since best_match need a sort key to help match errors and the default key is to use most depth errors key, see:
best_match
relevance
So maybe you can use a less depth key to match the errors.(below function is just a draft test, you can use it as reference)
def match_less_path(error):
return len(error.path)
best = best_match(validator.iter_errors(data), match_less_path)
And I test the output like this:
Additional properties are not allowed ('first_rack_id' was unexpected)
Failed validating 'additionalProperties' in schema[1]:
{'additionalProperties': False,
'properties': {'server_pid': {'default': '',
'description': 'Server PID '
'Qualifications - Server '
'PID',
'pattern': '^123$',
'type': 'string'},
'type': {'description': 'Qualification Type',
'enum': ['server_pid'],
'type': 'string'}},
'required': ['type', 'server_pid']}
On instance:
{'first_rack_id': '10', 'server_pid': 'B200M5', 'type': 'server_pid'}
Depending on the salaryRange the user selects I need to validate differently by requiring some fields and rejecting others. I feel like its a combination of allOf and not but I can't seem to quite get it.
Scenario #1
User selects salaryRange(Hourly)
Require hourlyRate
Prevent the submission of fields feeOne and feeTwo
Scenario #2
User selects salaryRange(0-50k OR 50-100k)
Require feeOne and feeTwo
Prevent the submission of field hourlyRate
Here is my schema
{
"schema": "http://json-schema.org/draft-04/schema#",
"$id": "http://mysite/schemas/job.json#",
"title": "Job",
"description": "Create job",
"type": "object",
"properties": {
"title": { "type": "string" },
"description": { "type": "string" },
"salaryRange": { "enum": ["0-50k", "50-100k", "100-150k", "150-200k", "200-300k", "300k+", "nonExempt", "Hourly"] },
"hourlyRate": {
"type": "number",
"minimum": 0,
"maximum": 300
},
"feeOne": {
"type": "number",
"minimum": 0
},
"feeTwo": {
"type": "number",
"minimum": 0
}
} ,
"additionalProperties": false,
"required": [
"title",
"description",
"salaryRange"
]
}
You can use oneOf and not required to model all possible combinations.
Here is an example in js:
https://runkit.com/embed/cf8cra1mwvx3/
{
"schema": "http://json-schema.org/draft-04/schema#",
"$id": "http://mysite/schemas/job.json#",
"title": "Job",
"description": "Create job",
"type": "object",
"properties": {
"title": { "type": "string" },
"description": { "type": "string" },
"salaryRange": { "enum": ["0-50k", "50-100k", "100-150k", "150-200k", "200-300k", "300k+", "nonExempt", "Hourly"] },
"hourlyRate": {
"type": "number",
"minimum": 0,
"maximum": 300
},
"feeOne": {
"type": "number",
"minimum": 0
},
"feeTwo": {
"type": "number",
"minimum": 0
}
},
"oneOf": [
{
"description": "Disallow fees for hourly salary",
"properties": {
"salaryRange": { "enum": ["Hourly"] }
},
"required": ["hourlyRate"],
"allOf": [
{"not":{"required":["feeOne"]}},
{"not":{"required":["feeTwo"]}}
]
},
{
"description": "Disallow hourly rate for 0-50k, 50-100k salaries",
"properties": {
"salaryRange": { "enum": ["0-50k", "50-100k"] }
},
"required": ["feeOne", "feeTwo"],
"not":{"required":["hourlyRate"]}
},
{
"description": "Allow other cases",
"properties": {
"salaryRange": { "not" : {"enum": ["Hourly", "0-50k", "50-100k"] } }
}
}
],
"additionalProperties": false,
"required": [
"title",
"description",
"salaryRange"
]
}
Take the following schema:
{
"properties": {
"StageHEP": {
"description": "The stage of hepatitis",
"type": "string",
"enum": ["ACUTE", "CHRONIC", "UNK"]
},
"complications": {
"description": "Disease complications",
"type": "string",
"enum: ["CIRR", "CANCER", "NONE", "UNK"]
}
}
}
I want to create a validation rule (within the schema) that states that:
if StageHEP = ACUTE, complications property cannot be CIRR
Is it possible with json-schema draft v4?
You can do it using "oneOf":
{
"oneOf": [
{
"properties": {
"StageHEP": {
"description": "The stage of hepatitis",
"type": "string",
"enum": [
"CHRONIC",
"UNK"
]
},
"complications": {
"description": "Disease complications",
"type": "string",
"enum": [
"CIRR",
"CANCER",
"NONE",
"UNK"
]
},
"additionalProperties": false
}
},
{
"properties": {
"StageHEP": {
"description": "The stage of hepatitis",
"type": "string",
"enum": [
"ACUTE"
]
},
"complications": {
"description": "Disease complications",
"type": "string",
"enum": [
"CANCER",
"NONE",
"UNK"
]
},
"additionalProperties": false
}
}
]
}
I have the below JSONSchema, I want the root tag envelope to be mandatory.
Any help would be appreciated.
{
"id": "envelope",
"$schema": "http://json-schema.org/schema#",
"tittle": "Root schema",
"description": "Root schema for all services",
"apiVersion": "1.0",
"type": "object",
"required": [
"metadata",
"data"
],
"properties": {
"metadata": {
"description": "The meta data of the data field",
"type": "object",
"required": [
"sourceSystem",
"deliveryCount",
"retryPeriod",
"correlationId"
],
"properties": {
"sourceSystem": {
"description": "The source System ",
"type": "string"
},
"deliveryCount": {
"description": "Number of times the request tried",
"type": "number",
"default": 0,
"maxLength": 5
},
"retryPeriod": {
"description": "Time set to retry",
"type": "number"
},
"correlationId": {
"description": "Unique id for reference",
"type": "string"
}
}
},
"data": {
"description": "The actual content",
"type": "object"
},
"response": {
"description": "Response",
"type": "string"
}
}
}
The output is
{
"metadata": {
"sourceSystem": "",
"deliveryCount": 1,
"retryPeriod": 30,
"correlationId": ""
},
"data": {}
}
expected output is
{ "envelope" : {
"metadata": {
"sourceSystem": "",
"deliveryCount": 1,
"retryPeriod": 30,
"correlationId": ""
},
"data": {} } }
The "id" attribute does not define any root element, it is used for different purposes.
All you need to do is to define your root schema as an object with a single "envelope" property:
{
"type" : "object"
"properties" : {
"envelope" : {
// here comes your (current) schema
}
}
}
I'm testing bigquery platform with real traffic of my site (more than 80M of events by day).
I'm uploading gz files using java api, using insert jobs.
In some cases, i've receive this message: Input contained no data
{
"kind": "bigquery#job",
"etag": "\"******************\"",
"id": "*********",
"selfLink": "********",
"jobReference": {
"projectId": "********",
"jobId": "**************"
},
"configuration": {
"load": {
"schema": {
"fields": [
{
"name": "tms",
"type": "TIMESTAMP"
},
{
"name": "page",
"type": "STRING"
},
{
"name": "user_agent",
"type": "STRING"
},
{
"name": "print_id",
"type": "STRING"
},
{
"name": "referer",
"type": "STRING"
},
{
"name": "gtms",
"type": "TIMESTAMP"
},
{
"name": "cookies",
"type": "STRING"
},
{
"name": "ip",
"type": "STRING"
},
{
"name": "site",
"type": "STRING"
},
{
"name": "call_params",
"type": "STRING"
},
{
"name": "domains",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "name",
"type": "STRING"
},
{
"name": "ads",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "type",
"type": "STRING"
},
{
"name": "position",
"type": "STRING"
},
{
"name": "strategy",
"type": "STRING"
},
{
"name": "score",
"type": "STRING"
},
{
"name": "cpc",
"type": "STRING"
},
{
"name": "site",
"type": "STRING"
},
{
"name": "categ",
"type": "STRING"
},
{
"name": "cust",
"type": "STRING"
},
{
"name": "campaign",
"type": "STRING"
}
]
}
]
}
]
},
"destinationTable": {
"projectId": "**********",
"datasetId": "*******",
"tableId": "********"
},
"createDisposition": "CREATE_IF_NEEDED",
"writeDisposition": "WRITE_APPEND",
"sourceFormat": "NEWLINE_DELIMITED_JSON"
}
},
"status": {
"state": "DONE",
"errors": [
{
"reason": "invalid",
"message": "Input contained no data"
}
]
},
"statistics": {
"creationTime": "1416491042309",
"startTime": "1416491061440",
"endTime": "1416491076876",
"load": {
"inputFiles": "1",
"inputFileBytes": "0",
"outputRows": "0",
"outputBytes": "0"
}
}
}
And then of this, all my jobs return the same response.
Can anybody tell me what is the reason of this behaviour?
Thanks!!!!
Your job succeeded: there is no "errorResult" field in the status.
First, I understand this mistake: the return of errors and warnings in the job api is, frankly, as clear as mud.
Here's the quick overview:
status.errorResult is where job error is reported. If no errorResult is reported, the job succeeded.
status.errors is where individual errors and warnings are reported.
Please reference the documentation https://cloud.google.com/bigquery/docs/reference/v2/jobs and search for status.errorResult and status.errors.
Most people don't hit this problem since a job only encountering a warning is pretty rare.
Ok, the problem was very simple: the gz file.
Thanks!