Azure Policy (deployifnotexists) not behaving as expected - azure-storage

This is my first post here. What I'm trying to do in Azure is deployifnotexists for storage accounts if certain settings are not enabled. I've attached my code. What I want to do is this:
Check for secure transfer being enabled
Check for TLS1_2 only
Check the FW
On the FW, have the Azure Services accepted (e.g. nsg flow logs etc)
If any of those conditions are not met, then deploy them through the ARM template. What is catching me is that I have intentionally put in bad settings to see it work and it will not say that they are non-compliant.
{
"mode": "All",
"policyRule": {
"if": {
"field": "type",
"equals": "Microsoft.Storage/storageAccounts"
},
"then": {
"effect": "deployIfNotExists",
"details": {
"type": "Microsoft.Storage/storageAccounts",
"roleDefinitionIds": [
"/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
],
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
"equals": true
},
{
"field": "Microsoft.Storage/storageAccounts/minimumTlsVersion",
"equals": "TLS1_2"
},
{
"field": "Microsoft.Storage/storageAccounts/networkAcls.defaultAction",
"equals": "deny"
},
{
"field": "Microsoft.Storage/storageAccounts/networkAcls.bypass",
"contains": "AzureServices"
}
]
},
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"type": "String",
"metadata": {
"description": "storageAccountName"
}
},
"location": {
"type": "String",
"metadata": {
"description": "location"
}
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"name": "[parameters('storageAccountName')]",
"location": "[parameters('location')]",
"properties": {
"minimumTlsVersion": "TLS1_2",
"networkAcls": {
"bypass": "AzureServices",
"defaultAction": "Deny"
},
"supportsHttpsTrafficOnly": true
}
}
],
"outputs": {}
},
"parameters": {
"storageAccountName": {
"value": "[field('Name')]"
},
"location": {
"value": "[field('location')]"
}
}
}
}
}
}
},
"parameters": {}
}
Thanks everyone

So through further reading and talking with more experienced colleagues I've determined that "deployIfNotExists" conditions are not to be used for a resources own settings.
By that I mean I cannot "deployIfNotExists" to a storage accounts storage account settings (as above) but i could deploy diagnostic logging to a SA. I am closing this question. I will try append and if I do anything good I'll loop it back in to this question for keen eyes.

Related

OpenAPI Example multipart form data

I have a multipart/form-data POST in an API endpoint which takes some key/value strings, and a file upload via the key files.
I believe I have defined it correctly in OpenAPI;
"requestBody": {
"required": true,
"content": {
"multipart/form-data": {
"schema": {
"properties": {
"file": {
"type": "array",
"items": {
"type": "string",
"format": "binary"
}
},
"myKey1": {
"type": "string"
},
"myKey2": {
"type": "string"
}
}
},
"examples": {
"value": ?
}
}
}
},
However, I am unsure how I can describe an example for a multipart/form-data in the examples field.
I assume I don't need to represent the file (although that would be a bonus) but just the myKey1 and myKey2 keys and values.
Your OAS definition seems to be correct. You can define the examples as shown below:
"requestBody": {
"required": true,
"content": {
"multipart/form-data": {
"schema": {
"properties": {
"file": {
"type": "array",
"items": {
"type": "string",
"format": "binary"
},
"example": [
{
"externalValue": "http://www.africau.edu/images/default/sample.pdf"
}
]
},
"myKey1": {
"type": "string",
"example": "myKey1Example"
},
"myKey2": {
"type": "string",
"example": "myKey2Example"
}
}
}
}
}
},
externalValue can be added to point the sample file URL in Open API Specification. This is only for the document purpose.
However, it will not be displayed in the swagger-ui as swagger-ui does not support it yet. It is tracked in [1].
[1] https://github.com/swagger-api/swagger-ui/issues/5433

Cloudformation for redis parameter group

I want to create a cloudformation template to deploy a redis parameter group. The problem lies in the fact that each of my parameters looks like this:
{
"Parameters": [
{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
},
{
"ParameterName": "active-defrag-cycle-max",
"ParameterValue": "75",
"Description": "Maximal effort for defrag in CPU percentage",
"Source": "user",
"DataType": "integer",
"AllowedValues": "1-75",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}
}
Now I understand the basic format of the template but can't figure out how to pass in parameters as seen above.
Failing template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"Parameters": [{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}]
}
}
}
}
}
Welcome to StackOverflow!
You need to incorporate those parameters into the Parameters section of the template, and change the syntax so that it conforms to the Parameters format.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
},
"activedefrag": {
"Description": "Enabled active memory defragmentation",
"Type": "String",
"AllowedValues": ["yes", "no"],
"Default": "no"
},
"activedefragcyclemax": {
"Description": "Maximal effort for defrag in CPU percentage",
"Type": "Number",
"Default": 75,
"MinValue": 1,
"MaxValue": 75
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"activedefrag": {"Ref": "activedefrag"},
"active-defrag-cycle-max": {"Ref": "activedefragcyclemax"}
}
}
}
}
}

How to extent an object with 1 parameter in json-schema

I wrote one JSON schema before, but now, as I am trying to make it a bit more advanced I get stuck.
(I am open to 'good practice' tips in the comments)
(Is the $id optional? should I remove it for simplicity in the example code?)
Goal:
I am trying to make a schema with an object definition (example_obj) that is recursively used. This object may only have 1 argument (or or and or value). But in the root of the json, I want to add 1 additional property.
json-schema
{
"definitions": {
"example_obj": {
"$id": "#/definitions/example_obj",
"type": "object",
"maxProperties": 1,
"properties": {
"or": {
"$id": "#/definitions/example_obj/properties/or",
"type": "array",
"items": {
"$id": "#/definitions/example_obj/properties/or/items",
"$ref": "#/definitions/example_obj"
}
},
"and": {
"$id": "#/definitions/example_obj/properties/and",
"type": "array",
"items": {
"$id": "#/definitions/example_obj/properties/and/items",
"$ref": "#/definitions/example_obj"
}
},
"value": {
"$id": "#/definitions/example_obj/properties/value",
"type": "string"
}
}
}
},
"type": "object",
"title": "The Root Schema",
"required": ["filter_version"],
"allOf": [
{
"$ref": "#/definitions/example_obj"
},
{
"properties": {
"filter_version": {
"$id": "#/properties/filter_version",
"type": "string",
"pattern": "^([0-9]+\\.[0-9]+)$"
}
}
}
]
}
json which I want to pass validation:
{
"filter_version": "1.0",
"or": [
{
"and": [
{
"value": "subject"
}
]
},
{
"or": [
{
"value": "another subject"
}
]
}
]
}
Issue:
When I try to extend example_obj for the root definition it seems to fail because the example_obj object does not allow more then 1 property by design.
In other words, it appears that every check for the number of argument that I add to example_obj is also performed on the additional property (i.e. filter_version).
Does anyone know where to place this check for 'exactly 1 argument' so that it is not evaluated on the root object?
Attempts:
I tried working with different ways of determining the requirements of example_obj, but with no success. Like with replacing "maxProperties": 1 with:
"oneOf": [
{
"required": [
"or"
]
},
{
"required": [
"and"
]
},
{
"required": [
"where"
]
},
{
"required": [
"where not"
]
}
],
Thanks in advance for any help!!
Checking my schema with the online schema validator.
(In the end I need to validate it in Python, in case it matters)
You can use oneOf instead of maxProperties to get around this.
{
"type": "object",
"properties": {
"filter_version": {
"type": "string",
"pattern": "^([0-9]+\\.[0-9]+)$"
}
},
"required": ["filter_version"],
"allOf": [{ "$ref": "#/definitions/example_obj" }],
"definitions": {
"example_obj": {
"type": "object",
"properties": {
"or": { "$ref": "#/definitions/example-obj-array" },
"and": { "$ref": "#/definitions/example-obj-array" },
"value": { "type": "string" }
},
"oneOf": [
{ "required": ["or"] },
{ "required": ["and"] },
{ "required": ["value"] }
]
},
"example-obj-array": {
"type": "array",
"items": { "$ref": "#/definitions/example_obj" }
}
}
}
P.S. You are using $id wrong. I know there is a tool out there that generates schemas like this and causes this confusion. The way $id is used here is a no-op. It doesn't hurt, but it doesn't do anything other than bloating your schema.

can I do a patternProperties with $ref validation

I am trying to match a patternProperties with a schema, like, here is the jsonschema text:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"fabric_id": {
"enum": [
"ADMIN",
"COPPER",
"NETWORK",
"STORAGE",
"STORAGE2",
"TENANT"
]
},
"fabrics": {
"additionalProperties": false,
"patternProperties": {
"[A-Z0-9-]*": {
"additionalProperties": false,
"properties": {
"description": {
"type": "string"
},
"fabric_id": {
"$ref": "#/definitions/fabric_id",
"type": "string"
}
},
"required": [
"description",
"fabric_id"
],
"type": "object"
}
},
"type": "object"
}
},
"description": "fabrics spec",
"properties": {
"fabrics": {
"$ref": "#/definitions/fabrics"
}
},
"required": [
"fabrics"
],
"title": "network fabric",
"type": "object"
}
and here is my input json file:
{
"fabrics": {
"ADMIN": {
"description": "Primary bonded pair on the bigswitches.",
"fabric_id": "ADMIN"
},
"COPPER": {
"description": "Primary IPMI fabric on the tor switches.",
"fabric_id": "COPPER"
}
}
}
I can't figure out how to validate the patternProperty against the fabric_id enum? The pattern object has fabric_id in it, and that is able to reference the fabric_id enum in the definitions section. I'd like to have that same $ref for the "[A-z0-9-]*" pattern, but I just can't make it work. Is this possible?
This schema does is the best you can do. The only thing it can't do is constrain the property name to match the value of "fabric_id". Unfortunately, this is not possible with JSON Schema.
{
"$schema": "http://json-schema.org/draft-06/schema#",
"type": "object",
"properties": {
"fabrics": { "$ref": "#/definitions/fabrics" }
},
"required": ["fabrics"],
"definitions": {
"fabric_id": {
"enum": ["ADMIN", "COPPER", "NETWORK"]
},
"fabrics": {
"type": "object",
"propertyNames": { "$ref": "#/definitions/fabric_id" },
"patternProperties": {
".*": {
"type": "object",
"properties": {
"description": { "type": "string" },
"fabric_id": { "$ref": "#/definitions/fabric_id" }
},
"required": ["description", "fabric_id"]
}
}
}
}
}
Sadly, I don't believe this is possible with draft-4 of JSON Schema.
If you can upgrade to 6 or 7 (+), you can make this possible.
propertyNames: https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-validation-01#section-6.5.8
If the instance is an object, this keyword validates if every property name in the instance validates against the provided schema. Note the property name that the schema is testing will always be a string.
An example of how this can be used can be found at https://github.com/json-schema-org/json-schema-org.github.io/issues/77
...
"fooProperties": {
"propertyNames": {
"$comment": "Need to anyOf these or else the enum and pattern conflict",
"anyOf": [
{"enum": ["foo1", "foo2"]},
{"pattern": "foo[A-Z][a-z0-9]*"}
]
}
},
...
Sorry I don't have time to update your schema to follow this, but hopefully I sufficiently explained this for you to adapt it.
if you're unable to migrate beyond draft-4... well you'll have to do that validation aspect manually outside of JSON Schema.

Json property structure dependant of another property

I've been working on a json schema to validate the answers from one of my webservices.
The answer is splitted in two properties: data and status. If status.code is set to 0, then data will have to respect a specific schema. Else, if status.code is set to -1, data won't be read, so I don't want to check if it respects the schema.
Here is the schema :
{
"$schema": "http://json-schema.org/schema#",
"id": "file://registration.json",
"type": "object",
"properties": {
"status": {
"$ref": "#/definitions/classes/status"
}
},
"anyOf": [
{
"$ref": "#/definitions/conditions/status-is-ok"
},
{
"$ref": "#/definitions/conditions/status-is-nok"
}
],
"definitions": {
"classes": {
"status": {
"type": "object",
"properties": {
"code": {
"type": "integer"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
]
},
"data": {
"type": "object",
"properties": {
"propertyA": {
"type": "#/definitions/classes/metadatauser"
},
"propertyB": {
"type": "#/definitions/classes/membreinfo"
}
},
"required": ["propertyA", "propertyB"]
}
},
"conditions": {
"status-is-ok": {
"status": {
"properties": {
"code": 0
}
},
"data": {
"$ref": "#/definitions/classes/data"
}
},
"status-is-nok": {
"status": {
"properties": {
"code": -1
}
},
"data": {
"type": "object"
}
}
}
}
}
And here's an example of what should not be validated:
{
"data": {},
"status": {
"code": 0,
"message": "OK"
}
}
At the moment, this portion of code passes, and I don't know why.
You've got a few things wrong here, so I'll try to explain all of them. You were on the right track!
"properties": {
"code": 0
}
The value of "properties" MUST be an object. Each value of this object
MUST be a valid JSON Schema.
http://json-schema.org/latest/json-schema-validation.html#rfc.section.6.5.4
You can't put the value you expect as the value for a property key.
You CAN however use the [const]1 keyword to achive a specific value validation.
"$ref": "#/definitions/conditions/status-is-ok"
...
"conditions": {
"status-is-ok": {
"status": {
"properties": {
[The definitions] keyword's value MUST be an object. Each member value of this
object MUST be a valid JSON Schema.
https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-validation-01#section-9
This means that you need to treat each value of each key in a defintions as a JSON Schema. If you had a JSON Schema where you did not nest "status" inside a properties object, no validation would take place. The same is true for "data".
(Strictly, according to the definitions section of the spec, you MUST NOT nest schemas deeply in the definitions object, but this seems to be supported by some implementations anyway, and resolves using the correct resolution rules. Prefixing may be better.)
The complete fixed schema is as follows.
{
"$schema": "http://json-schema.org/schema#",
"id": "file://registration.json",
"type": "object",
"properties": {
"status": {
"$ref": "#/definitions/classes/status"
}
},
"anyOf": [
{
"$ref": "#/definitions/conditions/status-is-ok"
},
{
"$ref": "#/definitions/conditions/status-is-nok"
}
],
"definitions": {
"classes": {
"status": {
"type": "object",
"properties": {
"code": {
"type": "integer"
},
"message": {
"type": "string"
}
},
"required": [
"code",
"message"
]
},
"data": {
"type": "object",
"properties": {
},
"required": [
"propertyA",
"propertyB"
]
}
},
"conditions": {
"status-is-ok": {
"properties": {
"status": {
"properties": {
"code": {
"const": 0
}
}
},
"data": {
"$ref": "#/definitions/classes/data"
},
},
"additionalProperties": false
},
"status-is-nok": {
"properties": {
"status": {
"properties": {
"code": {
"const": -1
}
}
},
"data": {
"type": "object"
},
},
"additionalProperties": false
}
}
}
}
Please do let me know if any of this doesn't make sense.
Feel free to join the JSON Schema slack server should you want to discuss any aspects further! Happy to also comment here.