I am trying to build a template for SQL IaaS deployments including the VM, the SQL IaaS extension, and the SQL IaaS machine. I am getting an error when the SQL IaaS machine deploys because the VM has not fully finished deploying I think.
My current template looks like this:
{
"name": "[parameters('virtualMachineName')]",
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-10-01",
"location": "[variables('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('virtualMachineSize')]"
},
"osProfile": {
"computerName": "[parameters('virtualMachineName')]",
"adminUsername": "",
"adminPassword": "",
"windowsConfiguration": {
"provisionVMAgent": true
}
},
"storageProfile": {
"imageReference": {
"publisher": "MicrosoftSQLServer",
"offer": "[variables('sqlOffer')]",
"sku": "[parameters('sqlEdition')]",
"version": "latest"
},
"osDisk": {
"createOption": "FromImage",
"osType": "Windows",
"caching": "ReadWrite",
"name": "[variables('OSDiskName')]",
"managedDisk": {
"storageAccountType": "Premium_LRS"
}
},
"dataDisks": [
{
"name": "[concat(parameters('virtualMachineName'),'-DATA1')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "StandardSSD_LRS"
},
"lun": 0,
"diskSizeGB": "[parameters('sizeOfUserDBDiskInGB')]",
"caching": "ReadOnly"
},
{
"name": "[concat(parameters('virtualMachineName'),'-DATA2')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "Standard_LRS"
},
"lun": 1,
"diskSizeGB": 128,
"caching": "ReadOnly"
},
{
"name": "[concat(parameters('virtualMachineName'),'-DATA3')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "Standard_LRS"
},
"lun": 2,
"diskSizeGB": 1023,
"caching": "ReadOnly"
},
{
"name": "[concat(parameters('virtualMachineName'),'-DATA4')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "Standard_LRS"
},
"lun": 4,
"diskSizeGB": 512,
"caching": "ReadOnly"
},
{
"name": "[concat(parameters('virtualMachineName'),'-DATA5')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "Standard_LRS"
},
"lun": 5,
"diskSizeGB": 512,
"caching": "None"
},
{
"name": "[concat(parameters('virtualMachineName'),'-DATA6')]",
"createOption": "Empty",
"managedDisk": {
"storageAccountType": "Standard_LRS"
},
"lun": 6,
"diskSizeGB": 512,
"caching": "None"
}
]
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[reference(resourceId(resourceGroup().name, 'Microsoft.Storage/storageAccounts', parameters('diagnosticsStorageAccountName')), '2015-06-15').primaryEndpoints['blob']]"
}
}
},
"tags": {
"Application Stack": "[parameters('Application Stack')]",
"Business Contact": "[parameters('Business Contact')]",
"Associated to": "[resourceGroup().name]",
"Business Owner": "[parameters('Business Owner')]",
"BusinessUnit": "[parameters('Business Unit')]",
"Department": "[parameters('Department')]",
"Region": "[parameters('Region')]",
"Type": "[variables('VMType')]"
}
},
{
"apiVersion": "2015-06-15",
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(parameters('virtualMachineName'), '/SqlIaasExtension')]",
"location": "[variables('location')]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/', parameters('virtualMachineName'))]"
],
"properties": {
"type": "SqlIaaSAgent",
"publisher": "Microsoft.SqlServer.Management",
"typeHandlerVersion": "1.2",
"autoUpgradeMinorVersion": "true",
"settings": {
"AutoTelemetrySettings": {
"Region": "[variables('location')]"
},
"AutoPatchingSettings": {
"PatchCategory": "WindowsMandatoryUpdates",
"Enable": true,
"DayOfWeek": "Saturday",
"MaintenanceWindowStartingHour": "2",
"MaintenanceWindowDuration": "180"
},
"ServerConfigurationsManagementSettings": {
"SQLConnectivityUpdateSettings": {
"ConnectivityType": "Private",
"Port": "1433"
},
"SQLWorkloadTypeUpdateSettings": {
"SQLWorkloadType": "General"
},
"SQLStorageUpdateSettings": {
"DiskCount": 1,
"NumberOfColumns": 1,
"StartingDeviceID": "2",
"DiskConfigurationType": "NEW"
},
"AdditionalFeaturesServerConfigurations": {
"IsRServicesEnabled": "false"
}
}
},
"protectedSettings": {
"SQLAuthUpdateUserName": "",
"SQLAuthUpdatePassword": ""
}
}
},
{
"type": "Microsoft.SqlVirtualMachine/sqlVirtualMachines",
"apiVersion": "2017-03-01-preview",
"name": "[parameters('virtualMachineName')]",
"location": "[variables('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName'))]"
],
"properties": {
"virtualMachineResourceId": "[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachineName'))]",
"sqlImageOffer": "[variables('sqlOffer')]",
"sqlServerLicenseType": "AHUB",
"sqlManagement": "Full",
"sqlImageSku": "[parameters('sqlEdition')]"
}
}
This last portion deploys the Azure SQL IaaS machine, which is dependent on the VM Agent running. I am getting an error saying,
"error": {
"code": "VmAgentNotRunning",
"message": "The VM agent in Virtual Machine: '/subscriptions/{GUID}/resourceGroups/{ResourceGroup}/providers/Microsoft.Compute/virtualMachines/{VmName}' is not in running state. Please make sure it is installed and in running state and try again later."
}
Is there a way I can add a waitfor or add a dependency on the agent running before deploying the SQL IaaS machine? Without deploying this, the template works great. After the machine is deployed, i can run a powershell script to create the object, but i'm trying to avoid that. Thanks in advance.
Randomly discovered this article, which explains my issue.
https://learn.microsoft.com/en-us/azure/virtual-machines/windows/sql/virtual-machines-windows-sql-register-with-resource-provider?tabs=azure-cli%2Cpowershell#management-modes
The trick was to change "sqlManagement": "Full", to "sqlManagement": "LightWeight",
Related
I'm trying to use the following Square API endpoint to update an item, but I'm getting some errors. Here is the item that I'm trying to update.
{
"type": "ITEM",
"id": "7HXK6NVBDKSOK64CCAYZJ2KW",
"updated_at": "2020-08-08T21:43:41.849Z",
"version": 1596923021849,
"is_deleted": false,
"present_at_all_locations": false,
"present_at_location_ids": [
"1JZ5JTPBW0EXY"
],
"item_data": {
"name": "Hummus Plate",
"description": "Olives, feta, tomatoes, tzatziki and warm pita.",
"visibility": "PRIVATE",
"category_id": "ARB5LTTDUSER6T66ZWYO65FL",
"variations": [
{
"type": "ITEM_VARIATION",
"id": "HBIHPLROOJUUNUSW3BZUDQ5J",
"updated_at": "2020-08-08T21:40:14.581Z",
"version": 1596922814581,
"is_deleted": false,
"present_at_all_locations": false,
"present_at_location_ids": [
"1JZ5JTPBW0EXY"
],
"item_variation_data": {
"item_id": "7HXK6NVBDKSOK64CCAYZJ2KW",
"name": "Regular",
"ordinal": 0,
"pricing_type": "FIXED_PRICING",
"price_money": {
"amount": 1000,
"currency": "USD"
}
}
}
],
"product_type": "REGULAR",
"skip_modifier_screen": true,
"ecom_available": false,
"ecom_visibility": "UNINDEXED"
}
}
Here the body content.
{
"batches": [{
"objects": [{
"type": "ITEM_VARIATION",
"id": "T6KKCATDIU2VV4BDBMVGML4W",
"item_variation_data": {
"item_id": "FNWWGFTBQWFAYK6JGBD3LDLD",
"pricing_type": "FIXED_PRICING",
"price_money": {
"amount": 150,
"currency": "USD"
}
}
}]
}],
"idempotency_key": "61994762-3a6s-4b75-sf81-fdfaf"
}
Here is the error I'm getting:
**{
"errors": [
{
"category": "INVALID_REQUEST_ERROR",
"code": "VERSION_MISMATCH",
"detail": "Object version does not match latest database version.",
"field": "version"
}
]
}**
I'm not sure what I'm doing wrong, but I appreciate any help I can get. Thank you in advance!
When updating a catalog object in Square, you need to provide the latest version number in the request.
{
"batches": [{
"objects": [{
"type": "ITEM_VARIATION",
"id": "T6KKCATDIU2VV4BDBMVGML4W",
"version": "12345",
"item_variation_data": {
"item_id": "FNWWGFTBQWFAYK6JGBD3LDLD",
"pricing_type": "FIXED_PRICING",
"price_money": {
"amount": 150,
"currency": "USD"
}
}
}]
}],
"idempotency_key": "61994762-3a6s-4b75-sf81-fdfaf"
}
The following ARM template deploys: Virtual Network, Network Security Group, Storage Account, App Service Plan, Function App
When the settings for WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE are omitted (commented out) the deployment succeeds but the function app configuration shows a warning.
When enabling the two settings, the deployment fails with a 403 Forbidden message.
New-AzResourceGroupDeployment : 17:04:05 - The deployment '20201209-170356' failed with error(s). Showing 1 out of 1 error(s).
Status Message: There was a conflict. The remote server returned an error: (403) Forbidden. (Code: BadRequest)
- There was a conflict. The remote server returned an error: (403) Forbidden. (Code:)
- (Code:BadRequest)
- (Code:)
CorrelationId: ec11767b-9f8f-4722-acca-e751e5c1bbe8
I have tried numerous settings on the NSG, adding service tags, allowing IPs associated with the function app. I have also tried allowing IPRules on the storage account firewall. The only setting that worked was to entirely disable the storage account firewall with 'Allow access from all networks', which is not an acceptable setting for the network.
The ARM template to demonstrate the error:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"vnetName": "vnet1a",
"addressPrefixVnet": "10.17.0.0/20",
"addressPrefixSubnet": "10.17.4.0/24",
"nsgName_sb_functionapp": "[concat(variables('vnetName'), '-sb-functionapp-nsg')]",
"storageAccountName": "[concat(uniquestring(resourceGroup().id), 'sa1a')]",
"appServicePlanName": "[concat(uniquestring(resourceGroup().id), 'asp1a')]",
"functionAppName": "[concat(uniquestring(resourceGroup().id), 'asp1a')]"
},
"resources": [
{
"type": "Microsoft.Network/networkSecurityGroups",
"apiVersion": "2019-11-01",
"name": "[variables('nsgName_sb_functionapp')]",
"location": "[resourceGroup().location]",
"tags": {
"Purpose": "Function App"
},
"properties": {
"securityRules": []
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2019-11-01",
"name": "[variables('vnetName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName_sb_functionapp'))]"
],
"tags": {
"Purpose": "Debug Function App and Storage Account Connectivity"
},
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefixVnet')]"
]
},
"subnets": [
{
"name": "sb-functionapp",
"properties": {
"addressPrefix": "[variables('addressPrefixSubnet')]",
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('nsgName_sb_functionapp'))]"
},
"serviceEndpoints": [
{
"service": "Microsoft.Storage",
"locations": [
"*"
]
}
],
"delegations": [
{
"name": "delegation",
"properties": {
"serviceName": "Microsoft.Web/serverFarms"
}
}
],
"privateEndpointNetworkPolicies": "Enabled",
"privateLinkServiceNetworkPolicies": "Enabled"
}
}
],
"enableDdosProtection": false,
"enableVmProtection": false
}
},
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-04-01",
"name": "[variables('storageAccountName')]",
"location": "[resourceGroup().location]",
"tags": {
"Purpose": "Debug Function App and Storage Account Connectivity"
},
"kind": "StorageV2",
"sku": {
"name": "Standard_GRS",
"tier": "Standard"
},
"properties": {
"networkAcls": {
"defaultAction": "Deny",
"bypass": "AzureServices",
"supportsHttpsTrafficOnly": true,
"ipRules": [],
"encryption": {
"keySource": "Microsoft.Storage",
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
}
},
"accessTier": "Hot",
"virtualNetworkRules": [
{
"id": "[concat(resourceId('Microsoft.Network/virtualNetworks', variables('vnetName')), '/subnets/sb-functionapp')]",
"ignoreMissingVNetServiceEndpoint": false
}
]
}
}
},
{
"type": "Microsoft.Web/serverfarms",
"apiVersion": "2018-02-01",
"name": "[variables('appServicePlanName')]",
"location": "[resourceGroup().location]",
"tags": {
"Purpose": "Debug Function App and Storage Account Connectivity"
},
"sku": {
"name": "EP1",
"tier": "ElasticPremium",
"size": "EP1",
"family": "EP",
"capacity": 1
},
"kind": "elastic",
"properties": {
"perSiteScaling": false,
"maximumElasticWorkerCount": 20,
"isSpot": false,
"reserved": false,
"isXenon": false,
"hyperV": false,
"targetWorkerCount": 0,
"targetWorkerSizeId": 0
}
},
{
"type": "Microsoft.Web/sites",
"apiVersion": "2018-11-01",
"name": "[variables('functionAppName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]"
],
"tags": {
"Purpose": "Debug Function App and Storage Account Connectivity"
},
"kind": "functionapp",
"properties": {
"enabled": true,
"hostNameSslStates": [
{
"name": "[concat(variables('functionAppName'), '.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Standard"
},
{
"name": "[concat(variables('functionAppName'), '.scm.azurewebsites.net')]",
"sslState": "Disabled",
"hostType": "Repository"
}
],
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]",
"reserved": false,
"isXenon": false,
"hyperV": false,
"scmSiteAlsoStopped": false,
"clientAffinityEnabled": true,
"clientCertEnabled": false,
"hostNamesDisabled": false,
"containerSize": 1536,
"dailyMemoryTimeQuota": 0,
"httpsOnly": true,
"redundancyMode": "None",
"siteConfig": {
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~1"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageAccountName'), ';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-04-01').keys[0].value)]"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[variables('functionAppName')]"
},
{
"name": "WEBSITE_DNS_SERVER",
"value": "168.63.129.16"
},
{
"name": "WEBSITE_VNET_ROUTE_ALL",
"value": "1"
}
]
}
},
"resources": [
{
"type": "networkConfig",
"apiVersion": "2018-11-01",
"name": "virtualNetwork",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('functionAppName'))]"
],
"properties": {
"subnetResourceId": "[concat(resourceId('Microsoft.Network/virtualNetworks', variables('vnetName')), '/subnets/sb-functionapp')]",
"swiftSupported": true
}
}
]
},
{
"type": "Microsoft.Web/sites/config",
"apiVersion": "2018-11-01",
"name": "[concat(variables('functionAppName'), '/web')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('functionAppName'))]"
],
"tags": {
"Purpose": "Debug Function App and Storage Account Connectivity"
},
"properties": {
"numberOfWorkers": 1,
"defaultDocuments": [
"Default.htm",
"Default.html",
"Default.asp",
"index.htm",
"index.html",
"iisstart.htm",
"default.aspx",
"index.php"
],
"netFrameworkVersion": "v4.0",
"phpVersion": "5.6",
"requestTracingEnabled": false,
"remoteDebuggingEnabled": false,
"remoteDebuggingVersion": "VS2019",
"httpLoggingEnabled": false,
"logsDirectorySizeLimit": 35,
"detailedErrorLoggingEnabled": false,
"publishingUsername": "[concat('$', variables('functionAppName'))]",
"scmType": "VSTSRM",
"use32BitWorkerProcess": true,
"webSocketsEnabled": false,
"alwaysOn": false,
"managedPipelineMode": "Integrated",
"virtualApplications": [
{
"virtualPath": "/",
"physicalPath": "site\\wwwroot",
"preloadEnabled": true
}
],
"loadBalancing": "LeastRequests",
"experiments": {
"rampUpRules": [
]
},
"autoHealEnabled": false,
"cors": {
"allowedOrigins": [],
"supportCredentials": false
},
"localMySqlEnabled": false,
"ipSecurityRestrictions": [],
"scmIpSecurityRestrictions": [
{
"ipAddress": "Any",
"action": "Allow",
"priority": 1,
"name": "Allow all",
"description": "Allow all access"
}
],
"scmIpSecurityRestrictionsUseMain": false,
"http20Enabled": false,
"minTlsVersion": "1.2",
"ftpsState": "AllAllowed",
"reservedInstanceCount": 1
}
}
]
}
Command to deploy to existing resource group:
New-AzResourceGroupDeployment -Name (Get-Date).ToString('yyyyMMdd-HHmmss') -ResourceGroupName 'Test-FunctionApp-Storage-VNet' -TemplateFile .\DebugFunctionApp.json -Verbose
I have seen the question/answer at Function App Deployment Failed - The remote server returned an error: (403) Forbidden but it doesn't solve the problem I see.
The solution is to add another setting named WEBSITE_CONTENTOVERVNET and to set the value to "1".
The updated appSettings section looks like:
"siteConfig": {
"appSettings": [
{
"name": "FUNCTIONS_EXTENSION_VERSION",
"value": "~1"
},
{
"name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageAccountName'), ';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName')), '2019-04-01').keys[0].value)]"
},
{
"name": "WEBSITE_CONTENTOVERVNET",
"value": "1"
},
{
"name": "WEBSITE_CONTENTSHARE",
"value": "[variables('functionAppName')]"
},
{
"name": "WEBSITE_DNS_SERVER",
"value": "168.63.129.16"
},
{
"name": "WEBSITE_VNET_ROUTE_ALL",
"value": "1"
}
]
}
The setting is document at https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_contentovervnet
For Premium plans only. A value of 1 enables your function app to scale when you have your storage account restricted to a virtual network. You should enable this setting when restricting your storage account to a virtual network.
I have a relatively simple process set up in Azure Data Factory to copy, cleanse and process some log files from a chatbot which has been running fine until I recently started getting the following errorCode:
"errorCode": "InvalidTemplate",
"message": "Unable to process expressions for action 'EvaluatefinaliseTSCRPTS': 'The function 'bool' was invoked with a parameter that is not valid. The value cannot be converted to the target type",
"failureType": "UserError",
"target": "finaliseTSCRPTS",
"details": ""
I can't seem to identify the error in the ADF process despite going through the code for my process below:
"name": "SearchBot dailyTranscripts",
"properties": {
"activities": [
{
"name": "MST Validation",
"type": "Validation",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"dataset": {
"referenceName": "teamsLogs",
"type": "DatasetReference"
},
"timeout": "0.00:00:30",
"sleep": 10,
"childItems": true
}
},
{
"name": "Get MST-TSCRPTS",
"type": "Copy",
"dependsOn": [
{
"activity": "MST Validation",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true,
"wildcardFileName": "*.json",
"enablePartitionDiscovery": false
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobStorageWriteSettings",
"copyBehavior": "MergeFiles"
},
"formatSettings": {
"type": "JsonWriteSettings",
"quoteAllText": true
}
},
"enableStaging": false,
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"path": "$['type']"
},
"sink": {
"path": "$['type']"
}
},
{
"source": {
"path": "$['timestamp']"
},
"sink": {
"path": "$['timestamp']"
}
},
{
"source": {
"path": "$['id']"
},
"sink": {
"path": "$['id']"
}
},
{
"source": {
"path": "$['channelId']"
},
"sink": {
"path": "$['channelId']"
}
},
{
"source": {
"path": "$['serviceUrl']"
},
"sink": {
"path": "$['serviceUrl']"
}
},
{
"source": {
"path": "$['from']['id']"
},
"sink": {
"path": "$['from']['id']"
}
},
{
"source": {
"path": "$['from']['aadObjectId']"
},
"sink": {
"path": "$['from']['aadObjectId']"
}
},
{
"source": {
"path": "$['from']['role']"
},
"sink": {
"path": "$['from']['role']"
}
},
{
"source": {
"path": "$['from']['name']"
},
"sink": {
"path": "$['from']['name']"
}
},
{
"source": {
"path": "$['conversation']['conversationType']"
},
"sink": {
"path": "$['conversation']['conversationType']"
}
},
{
"source": {
"path": "$['conversation']['tenantId']"
},
"sink": {
"path": "$['conversation']['tenantId']"
}
},
{
"source": {
"path": "$['conversation']['id']"
},
"sink": {
"path": "$['conversation']['id']"
}
},
{
"source": {
"path": "$['recipient']['id']"
},
"sink": {
"path": "$['recipient']['id']"
}
},
{
"source": {
"path": "$['recipient']['name']"
},
"sink": {
"path": "$['recipient']['name']"
}
},
{
"source": {
"path": "$['recipient']['aadObjectId']"
},
"sink": {
"path": "$['recipient']['aadObjectId']"
}
},
{
"source": {
"path": "$['recipient']['role']"
},
"sink": {
"path": "$['recipient']['role']"
}
},
{
"source": {
"path": "$['channelData']['tenant']['id']"
},
"sink": {
"path": "$['channelData']['tenant']['id']"
}
},
{
"source": {
"path": "$['text']"
},
"sink": {
"path": "$['text']"
}
},
{
"source": {
"path": "$['inputHint']"
},
"sink": {
"path": "$['inputHint']"
}
},
{
"source": {
"path": "$['replyToId']"
},
"sink": {
"path": "$['replyToId']"
}
},
{
"source": {
"path": "$['textFormat']"
},
"sink": {
"path": "$['textFormat']"
}
},
{
"source": {
"path": "$['localTimestamp']"
},
"sink": {
"path": "$['localTimestamp']"
}
},
{
"source": {
"path": "$['locale']"
},
"sink": {
"path": "$['locale']"
}
},
{
"source": {
"path": "$['value']"
},
"sink": {
"path": "$['value']"
}
},
{
"source": {
"path": "$['valueType']"
},
"sink": {
"path": "$['valueType']"
}
},
{
"source": {
"path": "$['name']"
},
"sink": {
"path": "$['name']"
}
},
{
"source": {
"path": "$['label']"
},
"sink": {
"path": "$['label']"
}
}
]
}
},
"inputs": [
{
"referenceName": "teamsLogs",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "transcriptsStaging",
"type": "DatasetReference"
}
]
},
{
"name": "finaliseTSCRPTS",
"type": "IfCondition",
"dependsOn": [
{
"activity": "MST Validation",
"dependencyConditions": [
"Completed"
]
},
{
"activity": "Get MST-TSCRPTS",
"dependencyConditions": [
"Succeeded",
"Skipped"
]
}
],
"userProperties": [],
"typeProperties": {
"expression": {
"value": "activity('MST Validation').output.exists",
"type": "Expression"
},
"ifTrueActivities": [
{
"name": "Combine TSCRPTS",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true,
"wildcardFileName": "*.json",
"enablePartitionDiscovery": false
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobStorageWriteSettings",
"copyBehavior": "MergeFiles"
},
"formatSettings": {
"type": "JsonWriteSettings",
"quoteAllText": true
}
},
"enableStaging": false
},
"inputs": [
{
"referenceName": "transcriptsStaging",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "SearchBotDailyTranscripts",
"type": "DatasetReference",
"parameters": {
"sourceFileName": "#concat(formatDateTime(utcnow(), 'yyyy-MM-dd'),'.json')"
}
}
]
},
{
"name": "Delete Staging",
"type": "Delete",
"dependsOn": [
{
"activity": "Combine TSCRPTS",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"dataset": {
"referenceName": "transcriptsStaging",
"type": "DatasetReference"
},
"enableLogging": false,
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true
}
}
},
{
"name": "Get monthlyTSCRPTS",
"type": "Copy",
"dependsOn": [
{
"activity": "Combine TSCRPTS",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobStorageWriteSettings"
},
"formatSettings": {
"type": "JsonWriteSettings",
"quoteAllText": true
}
},
"enableStaging": false
},
"inputs": [
{
"referenceName": "SearchBotDailyTranscripts",
"type": "DatasetReference",
"parameters": {
"sourceFileName": "#concat('2019-',formatDateTime(utcnow(), 'MM'),'-??.json')"
}
}
],
"outputs": [
{
"referenceName": "SearchBotMonthlyTranscripts",
"type": "DatasetReference",
"parameters": {
"sourceFileName": "#dataset().sourceFileName"
}
}
]
},
{
"name": "Get yearlyTSCRPTS",
"type": "Copy",
"dependsOn": [
{
"activity": "Get monthlyTSCRPTS",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobStorageWriteSettings",
"copyBehavior": "MergeFiles"
},
"formatSettings": {
"type": "JsonWriteSettings",
"quoteAllText": true
}
},
"enableStaging": false
},
"inputs": [
{
"referenceName": "SearchBotMonthlyTranscripts",
"type": "DatasetReference",
"parameters": {
"sourceFileName": {
"value": "#concat(formatDateTime(utcnow(), 'yyyy'),'-??.json')",
"type": "Expression"
}
}
}
],
"outputs": [
{
"referenceName": "SearchBotYearlyTranscripts",
"type": "DatasetReference",
"parameters": {
"sourceFileName": "#dataset().sourceFileName"
}
}
]
},
{
"name": "Copy MST-TSCRPTS",
"type": "Copy",
"dependsOn": [
{
"activity": "Delete Staging",
"dependencyConditions": [
"Succeeded"
]
}
],
"policy": {
"timeout": "0.00:01:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true,
"wildcardFileName": "*.json",
"enablePartitionDiscovery": false
}
},
"sink": {
"type": "JsonSink",
"storeSettings": {
"type": "AzureBlobStorageWriteSettings",
"copyBehavior": "MergeFiles"
},
"formatSettings": {
"type": "JsonWriteSettings",
"quoteAllText": true
}
},
"enableStaging": false
},
"inputs": [
{
"referenceName": "teamsLogs",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "transcriptsHistory",
"type": "DatasetReference"
}
]
},
{
"name": "Delete MST-TSCRPTS",
"type": "Delete",
"dependsOn": [
{
"activity": "Copy MST-TSCRPTS",
"dependencyConditions": [
"Succeeded",
"Failed"
]
}
],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"dataset": {
"referenceName": "teamsLogs",
"type": "DatasetReference"
},
"enableLogging": false,
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true
}
}
},
{
"name": "run learningList",
"type": "ExecutePipeline",
"dependsOn": [
{
"activity": "Delete MST-TSCRPTS",
"dependencyConditions": [
"Succeeded",
"Failed",
"Skipped"
]
}
],
"userProperties": [],
"typeProperties": {
"pipeline": {
"referenceName": "runLearningList",
"type": "PipelineReference"
},
"waitOnCompletion": true
}
}
]
}
}
],
"parameters": {
"sourceFileName": {
"type": "string",
"defaultValue": "#concat(formatDateTime(utcnow(),'yyyy-MM-dd'),'.json')"
}
},
"annotations": []
},
"type": "Microsoft.DataFactory/factories/pipelines"
There is no expression 'EvaluatefinaliseTSCRPTS' and I can't find a 'bool' function either. The only hint I've found was a previous question here
You have an IF activity named "finaliseTSCRPTS", so "EvaluatefinaliseTSCRPTS" is most likely the internal name of the function that performs the IF condition check. The message seems to indicate that it cannot evaluate your expression "activity('MST Validation').output.exists".
When you deploy an azure container group ("Microsoft.ContainerInstance/containerGroups"), can you replace just one of the containers at a later time?
Or does the creation of the container_group have to have all the containers at the time of creation (of the container group) ?
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-multi-container-group
"resources": [
{
"name": "[parameters('resourceGroupName')]",
"type": "Microsoft.ContainerInstance/containerGroups",
"apiVersion": "2018-06-01",
"location": "[resourceGroup().location]",
"properties": {
"containers": [
{
"name": "[parameters('loggingContainerName')]",
"properties": {
"image": "[parameters('loggingContainerImage')]",
"resources": {
"requests": {
"cpu": 1,
"memoryInGb": 1
}
},
"volumeMounts": [
{
"name": "[parameters('volumeName')]",
"mountPath": "/aci/logs/"
}
],
"ports": [
{
"port": 8080
}
]
}
},
{
"name": "[parameters('jobGeneratorContainerName')]",
"properties": {
"image": "[parameters('jobGeneratorContainerImage')]",
"resources": {
"requests": {
"cpu": 1,
"memoryInGb": 1
}
},
"ports": [
{
"port": 80
}
],
"volumeMounts": [
{
"name": "[parameters('volumeName')]",
"mountPath": "/aci/logs/"
}
],
"environmentVariables": [
{
"name": "ServiceBusConnectionString",
"value": "[parameters('serviceBusConnectionStringSend')]"
},
{
"name": "LoggingServiceUrl",
"value": "[parameters('loggingServiceUrl')]"
}
]
}
},
{
"name": "[parameters('jobProcessingContainerName')]",
"properties": {
"image": "[parameters('jobProcessingContainerImage')]",
"resources": {
"requests": {
"cpu": 1,
"memoryInGb": 1
}
},
"ports": [
{
"port": 8000
}
],
"environmentVariables": [
{
"name": "ServiceBusConnectionString",
"value": "[parameters('serviceBusConnectionStringListen')]"
},
{
"name": "LoggingServiceUrl",
"value": "[parameters('loggingServiceUrl')]"
}
]
}
}
],
"osType": "Linux",
"ipAddress": {
"type": "Public",
"ports": [
{
"protocol": "tcp",
"port": "80"
},
{
"protocol": "TCP",
"port": 443
}
],
"dnsNameLabel": "[uniqueString( resourceGroup().id )]"
},
As I know, what you said is right. The top level in Azure container instances is container group. No matter one or more than one container instances you want to create in a container group, you should create it or them in one time.
If the container group is created, you cannot change it, such as adding containers or changing container images. If you really want, you just can create a new one.
By the way, the multi-container group only support for Linux containers.
I'm using latest Windows 2016 AMI from AWS - Windows_Server-2016-English-Full-Base-2017.01.11
IAM Role assigned to instance contains AdministratorAccess policy for testing purposes.
I've added AWS.EC2.Windows.CloudWatch.json file to C:\Program Files\Amazon\SSM\Plugins\awsCloudWatch directory.
Config file content:
{
"schemaVersion": "1.2",
"description": "Example CloudWatch Logs tasks",
"runtimeConfig": {
"aws:cloudWatch": {
"settings": {
"startType": "Enabled"
},
"properties": {
"IsEnabled": true,
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
{
"Id": "ApplicationEventLog",
"FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogName": "Application",
"Levels": "7"
}
},
{
"Id": "SecurityEventLog",
"FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogName": "Security",
"Levels": "7"
}
},
{
"Id": "MemoryCounter",
"FullName": "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"CategoryName": "Memory",
"CounterName": "Available MBytes",
"InstanceName": "",
"MetricName": "Available Memory",
"Unit": "Megabytes",
"DimensionName": "instance_id",
"DimensionValue": "{instance_id}"
}
},
{
"Id": "CloudWatchLogs",
"FullName": "AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"AccessKey": "",
"SecretKey": "",
"Region": "eu-west-1",
"LogGroup": "test-2016",
"LogStream": "{instance_id}"
}
},
{
"Id": "CloudWatch",
"FullName": "AWS.EC2.Windows.CloudWatch.CloudWatch.CloudWatchOutputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"AccessKey": "",
"SecretKey": "",
"Region": "eu-west-1",
"NameSpace": "Windows/Default"
}
}
],
"Flows": {
"Flows": [
"(ApplicationEventLog, SecurityEventLog),CloudWatchLogs",
"(MemoryCounter),CloudWatch"
]
}
}
}
}
}
}
And it doesn't work, both CloudWatch Metrics and Logs are unavailable.
Any ideas how it can be fixed?
Thanks in advance!