Related
I want to create a JSON schema for an object in which one of the attributes is restricted to multiple sets of enums.
For example:
{
"data": {
"type": "myObject",
"attributes": {
"states": [
"Washington",
"Oregon",
"California"
]
}
}
}
is a valid JSON object against the schema.
And
{
"data": {
"type": "myObject",
"attributes": {
"states": [
"British Columbia",
"Alberta",
"Ontario"
]
}
}
}
is also a valid JSON object agains the schema
BUT,
{
"data": {
"type": "myObject",
"attributes": {
"states": [
"Washington",
"Oregon",
"Alberta"
]
}
}
}
is NOT a valid JSON object against the schema.
I have tried the following schema definition:
{
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"type": {
"type": "string"
},
"attributes": {
"type": "object",
"properties": {
"states": {
"type": "array",
"items": {
"oneOf": [
{
"enum": ["Washington","Oregon","California"],
"description": "United States"
},
{
"enum": ["British Columbia","Alberta", "Ontario"],
"description": "Canada"
}
]
},
"description": "Filter by states"
}
}
}
}
}
}
}
But with this schema above this is still considered valid:
{
"data": {
"type": "myObject",
"attributes": {
"states": [
"Washington",
"Oregon",
"Alberta"
]
}
}
}
BTW, you can use this for testing whether a JSON object conforms to a schema: https://www.jsonschemavalidator.net/
Thank you!
You need to invert the order of the oneOf and the items keywords, so that the same oneOf clause is used for all items:
...
"states": {
"type": "array",
"oneOf": [
{
"items": {
"enum": ["Washington","Oregon","California"],
"description": "United States"
}
},
{
"items": {
"enum": ["British Columbia","Alberta", "Ontario"],
"description": "Canada"
}
}
]
},
...
I'm trying to connect to a database through a data gateway (SQL Server Connector) with ARM templates. But I'm not sure if I miss something because I'm getting connection error with the gateway.
This is what I have so far in my api connection:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"connections_sql_name": {
"defaultValue": "sql",
"type": "String"
},
"connections_sql_displayName": {
"defaultValue": "displaynameDB",
"type": "String"
},
"server": {
"defaultValue": "SERV01",
"type": "String"
},
"database": {
"defaultValue": "DB01",
"type": "String"
},
"authType": {
"defaultValue": "windows",
"type": "String"
},
"username": {
"defaultValue": "USER01",
"type": "String"
},
"password": {
"defaultValue": "PASS123",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_sql_name')]",
"location": "northeurope",
"properties": {
"displayName": "[parameters('connections_sql_displayName')]",
"customParameterValues": {},
"parameterValues": {
"server": "[parameters('server')]",
"database": "[parameters('database')]",
"authType": "[parameters('authType')]",
"username": "[parameters('username')]",
"password": "[parameters('password')]"
},
"api": {
"id": "[concat('/subscriptions/{sub-id}/providers/Microsoft.Web/locations/northeurope/managedApis/', parameters('connections_sql_name'))]"
}
}
}
]
}
And this is a part of my logic app under inputs:
"gateway": {
"gatewaySettings": {
"connectionDetails": [
"[parameters('gatewayServer')]",
"[parameters('gatewayDatabase')]"
],
"credentialType": "Windows",
"dataSourceType": "sql"
},
"type": "gatewaySetting"
},
Any help is appreciated! :)
Try removing the gateway block from the Logic App definition and changing the connection definition to this:
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"name": "[parameters('connections_sql_name')]",
"location": "northeurope",
"properties": {
"displayName": "[parameters('connections_sql_displayName')]",
"customParameterValues": {
},
"parameterValues": {
"server": "[parameters('server')]",
"database": "[parameters('database')]",
"authType": "[parameters('authType')]",
"username": "[parameters('username')]",
"password": "[parameters('password')]",
"gateway": {
"id": "/subscriptions/{sub-id}/resourceGroups/{gateway-resource-group-name}/providers/Microsoft.Web/connectionGateways/{gateway-name}"
}
},
"api": {
"id": "[concat('/subscriptions/{sub-id}/providers/Microsoft.Web/locations/northeurope/managedApis/', parameters('connections_sql_name'))]"
}
}
}
The documentation isn't very helpful regarding the gateway property in connection resources.
I have been trying to validate my request parameters using x-amazon-apigateway-request-validator, but unfortunately it is not working. Below is swagger file-
{
"swagger": "2.0",
"info": {
"title": "API Gateway - Request Validation Demo"
},
"schemes": [
"https"
],
"produces": [
"application/json"
],
"x-amazon-apigateway-request-validators" : {
"full" : {
"validateRequestBody" : true,
"validateRequestParameters" : true
},
"body-only" : {
"validateRequestBody" : true,
"validateRequestParameters" : false
}
},
"x-amazon-apigateway-request-validator" : "full",
"paths": {
"/orders": {
"post": {
"x-amazon-apigateway-request-validator": "body-only",
"parameters": [
{
"in": "body",
"name": "CreateOrders",
"required": true,
"schema": {
"$ref": "#/definitions/CreateOrders"
}
}
],
"responses": {
"200": {
"schema": {
"$ref": "#/definitions/Message"
}
},
"400" : {
"schema": {
"$ref": "#/definitions/Message"
}
}
},
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200",
"responseTemplates": {
"application/json": "{\"message\" : \"Orders successfully created\"}"
}
}
},
"requestTemplates": {
"application/json": "{\"statusCode\": 200}"
},
"passthroughBehavior": "never",
"type": "mock"
}
},
"get": {
"x-amazon-apigateway-request-validator": "full",
"parameters": [
{
"in": "header",
"name": "Account-Id",
"required": true
},
{
"in": "query",
"name": "type",
"required": false,
"schema": {
"$ref": "#/definitions/InputOrders"
}
}
],
"responses": {
"200" : {
"schema": {
"$ref": "#/definitions/Orders"
}
},
"400" : {
"schema": {
"$ref": "#/definitions/Message"
}
}
},
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200",
"responseTemplates": {
"application/json": "[{\"order-id\" : \"qrx987\",\n \"type\" : \"STOCK\",\n \"symbol\" : \"AMZN\",\n \"shares\" : 100,\n \"time\" : \"1488217405\",\n \"state\" : \"COMPLETED\"\n},\n{\n \"order-id\" : \"foo123\",\n \"type\" : \"STOCK\",\n \"symbol\" : \"BA\",\n \"shares\" : 100,\n \"time\" : \"1488213043\",\n \"state\" : \"COMPLETED\"\n}\n]"
}
}
},
"requestTemplates": {
"application/json": "{\"statusCode\": 200}"
},
"passthroughBehavior": "never",
"type": "mock"
}
}
}
},
"definitions": {
"CreateOrders": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Create Orders Schema",
"type": "array",
"minItems" : 1,
"items": {
"type": "object",
"$ref" : "#/definitions/Order"
}
},
"Orders" : {
"type": "array",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Get Orders Schema",
"items": {
"type": "object",
"properties": {
"order_id": { "type": "string" },
"time" : { "type": "string" },
"state" : {
"type": "string",
"enum": [
"PENDING",
"COMPLETED"
]
},
"order" : {
"$ref" : "#/definitions/Order"
}
}
}
},
"Order" : {
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Schema for a single Order",
"required": [
"account-id",
"type",
"symbol",
"shares",
"details"
],
"properties" : {
"account-id": {
"type": "string",
"pattern": "[A-Za-z]{6}[0-9]{6}"
},
"type": {
"type" : "string",
"enum" : [
"STOCK",
"BOND",
"CASH"]
},
"symbol" : {
"type": "string",
"minLength": 1,
"maxLength": 4
},
"shares": {
"type": "number",
"minimum": 1,
"maximum": 1000
},
"details": {
"type": "object",
"required": [
"limit"
],
"properties": {
"limit": {
"type": "number"
}
}
}
}
},
"InputOrder" : {
"type": "object",
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Schema for a Input Order",
"required": [
"type"
],
"properties" : {
"type": {
"type" : "string",
"enum" : [
"STOCK",
"BOND",
"CASH"]
}
}
},
"Message": {
"type": "object",
"properties": {
"message" : {
"type" : "string"
}
}
}
}
}
I am trying to validate my request parameters against some regex and enum values.
I am not sure if this is even possible or not. Can anybody please help me with this?
For HTTP parameter validation, API Gateway only supports marking one as 'required'. It does not support regex/enum values for parameters.
I'm using the Azure Resource Group project template in Visual studio to deploy two API Apps and a Logic App. I want one of those API Apps to be a Blob Connector from the marketplace. What I need is the uri of the .zip package for the connector, as shown here:
{
"apiVersion": "2014-06-01",
"name": "MSDeploy",
"type": "Extensions",
"dependsOn": [
//........
],
"properties": {
"packageUri": "https://auxmktplceprod.blob.core.windows.net/packages/UmbracoCms.WebPI.7.2.5.zip",
"dbType": "SQL",
(source)
I tried this solution, but that cmdlet is now deprecated. Is there any way to get these URIs?
-Thanks!
I found a way of deploying custom api app with Marketplace apps.
Below is a sample script to just guide you
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"blobConnectorName": {
"type": "string",
"minLength": 1,
"defaultValue" : "mytestblobconnector"
},
"blobStorageAccount": {
"type": "string",
"minLength": 1,
"defaultValue" : "mystorage.blob.core.windows.net"
},
"blobStorageKey": {
"type": "securestring",
"minLength": 1,
"defaultValue" : "storgekey"
},
"blobContainerName": {
"type": "string",
"minLength": 1,
"defaultValue" : "mycontainer"
},
"gatewayName": {
"type": "string",
"minLength": 1,
"defaultValue" : "myblobconnectorgateway"
},
"logicAppName": {
"type": "string",
"minLength": 1,
"defaultValue" : "testinglogicapp"
},
"svcPlanName": {
"type": "string",
"minLength": 1,
"defaultValue" : "myresourcegrpserviceplan"
},
"sku": {
"type": "string",
"defaultValue": "Basic",
"allowedValues": [
"Free",
"Basic",
"Standard",
"Premium"
]
},
"svcPlanSize": {
"defaultValue": "0",
"type": "string",
"allowedValues": [
"0",
"1",
"2"
]
},
"gatewayToApiAppSecret": {
"defaultValue": "0000000000000000000000000000000000000000000000000000000000000000",
"type": "securestring"
}
},
"variables": {
"$packageId": "Microsoft.ApiApp",
"$nugetFeed": "http://apiapps-preview.nuget.org/api/v2/"
},
"resources": [
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2015-04-01",
"name": "[parameters('svcPlanName')]",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "AppServicePlan"
},
"properties": {
"name": "[parameters('svcPlanName')]",
"sku": "[parameters('sku')]",
"workerSize": "[parameters('svcPlanSize')]",
"numberOfWorkers": 1
}
},
{
"type": "Microsoft.Web/sites",
"apiVersion": "2015-04-01",
"name": "[parameters('gatewayName')]",
"location": "[resourceGroup().location]",
"kind": "gateway",
"tags": {
"displayName": "GatewayHost"
},
"resources": [
{
"type": "providers/links",
"apiVersion": "2015-01-01",
"name": "Microsoft.Resources/gateway",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('gatewayName'))]"
],
"properties": {
"targetId": "[resourceId('Microsoft.AppService/gateways', parameters('gatewayName'))]"
}
}
],
"dependsOn": [
"[concat(resourceGroup().id, '/providers/Microsoft.Web/serverfarms/', parameters('svcPlanName'))]"
],
"properties": {
"name": "[parameters('gatewayName')]",
"gatewaySiteName": "[parameters('gatewayName')]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('svcPlanName'))]",
"siteConfig": {
"appSettings": [
{
"name": "ApiAppsGateway_EXTENSION_VERSION",
"value": "latest"
},
{
"name": "EmaStorage",
"value": "D:\\home\\data\\apiapps"
},
{
"name": "WEBSITE_START_SCM_ON_SITE_CREATION",
"value": "1"
}
]
}
}
},
{
"type": "Microsoft.AppService/gateways",
"apiVersion": "2015-03-01-preview",
"name": "[parameters('gatewayName')]",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "Gateway"
},
"resources": [
{
"type": "providers/links",
"apiVersion": "2015-01-01",
"name": "Microsoft.Resources/gatewaySite",
"dependsOn": [
"[resourceId('Microsoft.AppService/gateways', parameters('gatewayName'))]"
],
"properties": {
"targetId": "[resourceId('Microsoft.Web/sites', parameters('gatewayName'))]"
}
},
{
"type": "tokens",
"apiVersion": "2015-03-01-preview",
"location": "[resourceGroup().location]",
"name": "[parameters('logicAppName')]",
"tags": {
"displayName": "AuthenticationToken"
},
"dependsOn": [
"[resourceId('Microsoft.AppService/gateways', parameters('gatewayName'))]"
]
}
],
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('gatewayName'))]"
],
"properties": {
"host": {
"resourceName": "[parameters('gatewayName')]"
}
}
},
{
"type": "Microsoft.Web/sites",
"apiVersion": "2015-04-01",
"name": "[parameters('blobConnectorName')]",
"location": "[resourceGroup().location]",
"kind": "apiApp",
"tags": {
"displayName": "APIAppHost",
"packageId": "AzureStorageBlobConnector"
},
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', parameters('svcPlanName'))]",
"[resourceId('Microsoft.AppService/gateways', parameters('gatewayName'))]"
],
"resources": [
{
"type": "siteextensions",
"tags": {
"displayName": "APIAppExtension"
},
"apiVersion": "2015-02-01",
"name": "AzureStorageBlobConnector",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('blobConnectorName'))]"
],
"properties": {
"type": "WebRoot",
"feed_url": "[variables('$nugetFeed')]"
}
},
{
"type": "providers/links",
"apiVersion": "2015-01-01",
"name": "Microsoft.Resources/apiApp",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', parameters('blobConnectorName'))]"
],
"properties": {
"targetId": "[resourceId('Microsoft.AppService/apiapps', parameters('blobConnectorName'))]"
}
}
],
"properties": {
"gatewaySiteName": "[parameters('gatewayName')]",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('svcPlanName'))]",
"siteConfig": {
"appSettings": [
{
"name": "EMA_MicroserviceId",
"value": "[parameters('blobConnectorName')]"
},
{
"name": "EMA_Secret",
"value": "[parameters('gatewayToAPIappSecret')]"
},
{
"name": "EMA_RuntimeUrl",
"value": "[concat('https://', reference(resourceId('Microsoft.Web/sites', parameters('gatewayName'))).hostNames[0])]"
},
{
"name": "WEBSITE_START_SCM_ON_SITE_CREATION",
"value": "1"
},
{
"name": "BlobConnector_ContainerUrl",
"value": "[concat('https://', parameters('blobStorageAccount'),'/',parameters('blobContainerName'))]"
},
{
"name": "BlobConnector_AccessKey",
"value": "[parameters('blobStorageKey')]"
}
],
"applicationLogs": {
"filesystem": {
"level": "Verbose"
},
"azureTableStorage": {
"level": "Off",
"sasUrl": null
},
"azureBlobStorage": {
"level": "Off",
"sasUrl": null,
"retentionInDays": null
}
}
}
}
},
{
"type": "Microsoft.AppService/apiapps",
"apiVersion": "2015-03-01-preview",
"name": "[parameters('blobConnectorName')]",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "BlobConnector"
},
"resources": [
{
"type": "providers/links",
"apiVersion": "2015-01-01",
"name": "Microsoft.Resources/apiAppSite",
"dependsOn": [
"[resourceId('Microsoft.AppService/apiapps', parameters('blobConnectorName'))]"
],
"properties": {
"targetId": "[resourceId('Microsoft.Web/sites', parameters('blobConnectorName'))]"
}
}
],
"dependsOn": [
"[resourceId('Microsoft.Web/sites/siteextensions', parameters('blobConnectorName'), 'AzureStorageBlobConnector')]"
],
"properties": {
"package": {
"id": "AzureStorageBlobConnector"
},
"host": {
"resourceName": "[parameters('blobConnectorName')]"
},
"gateway": {
"resourceName": "[parameters('gatewayName')]"
},
"dependencies": [ ]
}
},
{
"type": "Microsoft.Logic/workflows",
"apiVersion": "2015-02-01-preview",
"name": "[parameters('logicAppName')]",
"location": "[resourceGroup().location]",
"tags": {
"displayName": "LogicApp"
},
"dependsOn": [
"[resourceId('Microsoft.AppService/apiApps', parameters('blobConnectorName'))]"
],
"properties": {
"sku": {
"name": "[parameters('sku')]",
"plan": {
"id": "[concat(resourceGroup().id, '/providers/Microsoft.Web/serverfarms/',parameters('svcPlanName'))]"
}
},
"definition": {
"$schema": "http://schema.management.azure.com/providers/Microsoft.Logic/schemas/2014-12-01-preview/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"token": {
"defaultValue": "[reference(resourceId('Microsoft.AppService/gateways/tokens', parameters('gatewayName'), parameters('logicAppName'))).token]",
"type": "String",
"metadata": {
"token": {
"name": "token"
}
}
},
"runworkflowmanually": {
"defaultValue": true,
"type": "Bool"
}
},
"triggers": { },
"actions": {
"azurestorageblobconnector": {
"type": "ApiApp",
"inputs": {
"apiVersion": "2015-01-14",
"host": {
"id": "[concat(resourceGroup().id, '/providers/Microsoft.AppService/apiApps/',parameters('blobConnectorName'))]",
"gateway": "[concat('https://', reference(resourceId('Microsoft.Web/sites', parameters('gatewayName'))).hostNames[0])]"
},
"operation": "UploadBlob",
"parameters": {
"BlobPath": "myfolder/test.txt",
"BlobContent": {
"Content": "TestMessage",
"ContentTransferEncoding": "None"
},
"Overwrite": true
},
"authentication": {
"type": "Raw",
"scheme": "Zumo",
"parameter": "#parameters('token')"
}
},
"conditions": [ ]
}
},
"outputs": {
}
},
"parameters": { }
}
}
]
}
Just search for "AzureStorageBlobConnector" in the above json to observe the usage which is the package id of the blob connector from marketplace. I found the package id for the blob connector from azure portal by deploying one manually and then checking its settings. Please feel free to post a comment for package id for other market place apps if you face any difficulty.
I'm testing bigquery platform with real traffic of my site (more than 80M of events by day).
I'm uploading gz files using java api, using insert jobs.
In some cases, i've receive this message: Input contained no data
{
"kind": "bigquery#job",
"etag": "\"******************\"",
"id": "*********",
"selfLink": "********",
"jobReference": {
"projectId": "********",
"jobId": "**************"
},
"configuration": {
"load": {
"schema": {
"fields": [
{
"name": "tms",
"type": "TIMESTAMP"
},
{
"name": "page",
"type": "STRING"
},
{
"name": "user_agent",
"type": "STRING"
},
{
"name": "print_id",
"type": "STRING"
},
{
"name": "referer",
"type": "STRING"
},
{
"name": "gtms",
"type": "TIMESTAMP"
},
{
"name": "cookies",
"type": "STRING"
},
{
"name": "ip",
"type": "STRING"
},
{
"name": "site",
"type": "STRING"
},
{
"name": "call_params",
"type": "STRING"
},
{
"name": "domains",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "name",
"type": "STRING"
},
{
"name": "ads",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "type",
"type": "STRING"
},
{
"name": "position",
"type": "STRING"
},
{
"name": "strategy",
"type": "STRING"
},
{
"name": "score",
"type": "STRING"
},
{
"name": "cpc",
"type": "STRING"
},
{
"name": "site",
"type": "STRING"
},
{
"name": "categ",
"type": "STRING"
},
{
"name": "cust",
"type": "STRING"
},
{
"name": "campaign",
"type": "STRING"
}
]
}
]
}
]
},
"destinationTable": {
"projectId": "**********",
"datasetId": "*******",
"tableId": "********"
},
"createDisposition": "CREATE_IF_NEEDED",
"writeDisposition": "WRITE_APPEND",
"sourceFormat": "NEWLINE_DELIMITED_JSON"
}
},
"status": {
"state": "DONE",
"errors": [
{
"reason": "invalid",
"message": "Input contained no data"
}
]
},
"statistics": {
"creationTime": "1416491042309",
"startTime": "1416491061440",
"endTime": "1416491076876",
"load": {
"inputFiles": "1",
"inputFileBytes": "0",
"outputRows": "0",
"outputBytes": "0"
}
}
}
And then of this, all my jobs return the same response.
Can anybody tell me what is the reason of this behaviour?
Thanks!!!!
Your job succeeded: there is no "errorResult" field in the status.
First, I understand this mistake: the return of errors and warnings in the job api is, frankly, as clear as mud.
Here's the quick overview:
status.errorResult is where job error is reported. If no errorResult is reported, the job succeeded.
status.errors is where individual errors and warnings are reported.
Please reference the documentation https://cloud.google.com/bigquery/docs/reference/v2/jobs and search for status.errorResult and status.errors.
Most people don't hit this problem since a job only encountering a warning is pretty rare.
Ok, the problem was very simple: the gz file.
Thanks!