ADF V2 failure when using bool variable - azure-data-factory-2

Very simple issue. I am trying to set up a pipeline that has a variable of type bool. As soon as I add it, the pipeline fails with:
{
"code":"BadRequest",
"message":"Invalid value for property 'type'",
"target":"pipeline/pipeline2/runid/66b9c7be-9894-494a-abd9-34fd92bbd972",
"details":null,
"error":null
}
simple pipeline with a string variable and a wait activity succeeds.
{
"name": "pipeline2",
"properties": {
"activities": [
{
"name": "Wait1",
"type": "Wait",
"typeProperties": {
"waitTimeInSeconds": 1
}
}
],
"variables": {
"Test": {
"type": "String",
"defaultValue": "\"Hello\""
}
}
}
}
When I add a bool and nothing else, it fails to debug.
{
"name": "pipeline2",
"properties": {
"activities": [
{
"name": "Wait1",
"type": "Wait",
"typeProperties": {
"waitTimeInSeconds": 1
}
}
],
"variables": {
"Test": {
"type": "String",
"defaultValue": "\"Hello\""
},
"TestBool": {
"type": "Bool",
"defaultValue": false
}
}
}
}
Any clue how to get this to work? I am trying to use this variable as a condition for an Until loop.
Many thanks.

ok, I experimented.
If I go into the code and set the type as boolean rather than Bool, then the above pipeline runs.
Looks like a UI bug in the designer that sets the type to Bool. I'll file a bug report.
Mark.
Update ok it runs but I can't set a default value (it disappears) and anything that references the value causes an Internal Server Error (presumably because it is null which is invalid for a Boolean). Definitely something for the engineers to look at.
Update 2 It appears you can set the variable with SetVariable without error but it appears not to do anything. The value is always true in my test case.
Update 3 Microsoft has a fix coming next week.

Related

Can JSON Schema if statements handle nested $refs?

I have a JSON Schema using draft 2020-12 and I am trying to use an if-else subschema to check that a particular property exists based on the value of another property. Below is the if statement I am currently using. There are more properties but I have have omitted them for the sake of brevity. They are identical except the type of the property in the then statement is different. They are all wrapped in an allOf array:
{
"AValue": {
"allOf": [
{
"if": {
"$ref": "#/$defs/ValueItem/properties/dt",
"const": "type1"
},
"then": {
"properties": {
"string": { "type": "string" }
},
"required": ["string"]
}
}
]
}
}
The #/$defs/ValueItem/properties/dt being referred to is here:
{
"ValueItem": {
"properties": {
"value": {
"$ref": "#/$defs/AValue"
},
"dt": {
"$ref": "#/$defs/DT"
}
},
"additionalProperties": false
}
}
#/$defs/DT is here:
{
"DT": {
"type": "string",
"enum": [
"type1",
"type2",
"type3",
"type4"
]
}
}
I expected that when dt is encountered in a JSON instance document, the validator will check if the value of dt is type1 and then validate that an additional property called string is also present and is of type string. However, what actually happens is the validator complains that "Property 'string' has not been defined and the schema does not allow additional properties".
I assume that this is because the condition in the if statement evaluates to false so the subschema is never applied. However, I am unsure why that would be as I followed the example here when creating the if-then-else block. The only thing I can think of that is different is the use of $ref which I have in my schema but it is not in the example.
I found this answer which makes me think that it is possible to use $ref in an if statement but is it possible to use a ref that points to another ref or am I thinking about it incorrectly?
I have also tried removing the $ref from the if statement like below but it still doesn't work. Is it because of the $ref in the properties?
{
"AValue": {
"properties": {
"dt": {
"$ref": "#/$defs/DT"
}
},
"required": [
"dt"
],
"allOf": [
{
"if": {
"properties": {
"dt": {
"const": "type1"
}
}
},
"then": {
"properties": {
"string": {
"type": "string"
}
}
}
}
]
}
}
The problem is not cascading the $ref keywords. The const keyword at the if statement is not applied to the target of the $ref, but to the current location in the JSON input data. In this case to "AValue". To check if the property "dt" is of value "type1" at this point, you would need an if like this (simple solution with no $ref):
"if": {
"properties": {
"dt": {
"const": "type1"
}
},
"required": [ "dt" ]
}
Edit: Showing complete JSON Schema and error in JSONBuddy with $ref:

Put validation of two array fields in JSON Schema using oneOf

Can I put check on two fields in JSON schema ? Both field are of type array of objects. Conditions:
Either one of them can contain value at a time (i.e. other should be empty).
Both can be empty.
Any leads ?
// The schema
var schema = {
"id": "https://kitoutapi.lrsdedicated.com/v1/json_schemas/login-request#",
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Login request schema",
"type": "object",
"oneOf": [
{ "categories": {
"maxItems": 0
},
"positionedOffers": {
"minItems": 1
}},
{ "categories": {
"minItems": 1
},
"positionedOffers": {
"maxItems": 0
}}
],
"properties": {
"categories": {
"type": "array"
},
"positionedOffers": {
"type": "array"
}
},
"additionalProperties": false
};
// Test data 1
// This test should return a good result
var data1 = {
"positionedOffers":['hello'],
"categories":[],
}
For your requirement, I think I'd come at this from the other direction. Rather than saying
If one contains a value, the other must be empty, but both may be empty.
I'd say
At least one must be empty.
That leads you to use an anyOf with subschemas checking that each property is an empty array.
{
"id": "https://kitoutapi.lrsdedicated.com/v1/json_schemas/login-request#",
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Login request schema",
"type": "object",
"anyOf": [
{
"properties": {
"categories": {
"maxItems": 0
}
}
},
{
"properties": {
"positionedOffers": {
"maxItems": 0
}
}
}
],
"properties": {
"categories": {
"type": "array"
},
"positionedOffers": {
"type": "array"
}
},
"additionalProperties": false
}
Bonus Material
In your original post, you omitted the properties keywords under the oneOf. This may have been the cause of the schema's failure to validate. I've added it in the above.
Secondly, draft 4 is quite old at this point. You may be limited by the implementation you're using, but if you can, you should consider using a more recent version of JSON Schema. You can view available implementations and what versions they support on the JSON Schema implementations page.

ajv-cli always says bad data is valid

Running ajv-cli as part of my automated testing scripts to make sure my mock data is up to date.
./node_modules/.bin/ajv -s ./test-data/manifest.schema.json -d ./test-data/fleet.manifest.json
./test-data/fleet.manifest.json valid
But the data isn't valid.
manifest.schema.json:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"ManifestHistoryItem": {
"properties": {
"id": {
"default": [
"assetCatalog",
"Roster"
],
"items": {
"type": "string"
},
"type": "array"
},
"name": {
"default": "",
"type": "string"
}
},
"required": [
"id",
"name"
],
"type": "object"
}
}
}
fleet.manifest.json:
{
"namee": "Epic Space Battles"
}
(it's missing the required "id" property, and "name" is misspelled)
Schema is generated from "typescript-json-schema": "^0.54.0" from a typescript model and evaluated via "ajv-cli": "^5.0.0".
Your schema declares definitions, but it doesn't reference them anywhere. You need to add a "$ref": "#/definitions/ManifestHistoryItem" at the root.
{
"definitions": {
"ManifestHistoryItem": { ... }
},
"$ref": "#/definitions/ManifestHistoryItem"
}
Either that or you can just get rid of the definitions wrapper altogether and just have the { ... } part from above.
Effectively what's happening is you've defined an empty schema, which applies no constraints, meaning all instances (data) pass.

Alexa timer API not working when changing {continutewithskillname} to something else

I am trying to call subsequent task once the timer's is over via y timer API call. Per the doc, I have to similar formated code:
timer_request = {
"duration": "PT10S",
"timerLabel": "My Task Timer",
"creationBehavior": {
"displayExperience": {
"visibility": "VISIBLE"
}
},
"triggeringBehavior": {
"operation": {
"type": "LAUNCH_TASK",
"textToConfirm": [{
"locale": "en-US",
"text": "Timer elapsed. Would you like to launch {continueWithSkillName}?"
}],
"task": {
"name": "CountDown",
"version": "1",
}
},
"notificationConfig": {
"playAudible": True
}
}
}
However, as soon as, I change the {continueWithSkillName}to any other name or simply take the brackets out, I run into bad request error. Does anybody why or what shall I do?
If you read farther down in the docs you linked, it says it's mandatory.
** {continueWithSkillName} is mandatory somewhere. This is replaced with "continue with ". **
While it appears you can move it within the string where it appears, it must be part of the string.

How to specify an analyzer while creating an index in ElasticSearch

I'd like to specify an analyzer, name it, and use that name in a mapping while creating an index. I'm lost, my ES instance always returns me an error message.
This is, roughly, what I'd like to do:
"settings": {
"mappings": {
"alfedoc": {
"properties": {
"id": { "type": "string" },
"alfefield": { "type": "string", "analyzer": "alfeanalyzer" }
}
}
},
"analysis": {
"analyzer": {
"alfeanalyzer": {
"type": "pattern",
"pattern":"\\s+"
}
}
}
}
But this does not seem to work; the ES instance always returns me an error like
MapperParsingException[mapping [alfedoc]]; nested: MapperParsingException[Analyzer [alfeanalyzer] not found for field [alfefield]];
I tried putting the "analysis" branch of the dictionary at several places (inside the mapping etc.) but to no avail. I guess a working complete example (which I couldn't find up to now) would help me along as well. Probably I'm missing something rather basic.
"analysis" goes in the "settings" block, which goes either before or after the "mappings" block when creating an index.
"settings": {
"analysis": {
"analyzer": {
"alfeanalyzer": {
"type": "pattern",
"pattern": "\\s+"
}
}
}
},
"mappings": {
"alfedoc": { ... }
}
Here's a good complete, example: Example 1