How can I call directly CloudWatch from a step function? - amazon-cloudwatch

I have a step function that catches an eventual failure from a Glue function (fallback):
{
"Comment": "A Hello World example demonstrating various state types of the Amazon States Language",
"StartAt": "Parallel State",
"States": {
"Parallel State": {
"Comment": "A Parallel state can be used to create parallel branches of execution in your state machine.",
"Type": "Parallel",
"Branches": [
{
"StartAt": "Wait",
"States": {
"Wait": {
"Type": "Wait",
"Seconds": 500,
"Next": "Glue StartJobRun SUCCESS"
},
"Glue StartJobRun SUCCESS": {
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Parameters": {
"JobName": "job_success"
},
"End": true,
"Credentials": {
"RoleArn": "arn:aws:iam::094815468753:role/glue_test"
}
}
}
},
{
"StartAt": "Glue StartJobRun FAIL",
"States": {
"Glue StartJobRun FAIL": {
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Catch": [ {
"ErrorEquals": ["States.TaskFailed"],
"Next": "fallback"
} ],
"Parameters": {
"JobName": "job_fail"
},
"End": true,
"Credentials": {
"RoleArn": "arn:aws:iam::094815468753:role/glue_test"
}
},
"fallback": {
"Type": "Pass",
"Result": "ERROR HERE",
"End": true
}
}
}
],
"Next": "Pass"
},
"Pass": {
"Type": "Pass",
"End": true
}
}
}
when the fallback is called, how can I send from there a metric data to CloudWatch?

You can call the CloudWatch PutMetricData API from a Step Functions Task with the so-called AWS SDK service integration. The request parameters go in the task's Parameters key.
{
"PutMetricData": {
"Next": "Success",
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:cloudwatch:putMetricData",
"Parameters": {
"MetricData": [
{ "MetricName": "FooMetric", "Value": 1 },
{ "MetricName": "FooMetric", "Value.$": "$.foo" }
],
"Namespace": "Dummy"
}
}
}

You can not put metric from Step Function to Cloudwatch metrics.
But you can do it in this way.
Create a lambda function
Call lambda function on failure
Lambda function put metric to cloudwatch

Related

Run dynamic stored procedure from azure logic app

I want to run multiple stored procedures from logic app for Azure SQL database. I want names of the stored procedure to be calculated based on a variable name.
I have a variable with values (API_test1_SP1, API_test2_SP1, API_test3_SP1).
In a for loop, I want to run these stored procedures API_test1, API_test2 and API_test3.
I want to remove _SPI from the variable names and run the stored procedures (API_test1, API_test2, API_test3) for Azure SQL database.
I tried following expression without luck
#{concat(API_,slice(#{variables('variable_name')},1,lastIndexOf('_')))}
Is it possible to run stored procedure like this in logic app?
You can use the below expression to achieve your requirement.
first(split(variables('Array')?[iterationIndexes('Until')],'_SP1'))
To reproduce the issue, I have used the below flow in my logic app.
RESULTS:
Below is the codeview of my logic app
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Initialize_variable_-_array": {
"inputs": {
"variables": [
{
"name": "Array",
"type": "array",
"value": [
"API_test1_SP1",
"API_test2_SP1",
"API_test3_SP1"
]
}
]
},
"runAfter": {},
"type": "InitializeVariable"
},
"Initialize_variable_-_loop": {
"inputs": {
"variables": [
{
"name": "loop",
"type": "integer",
"value": 0
}
]
},
"runAfter": {
"Initialize_variable_-_array": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Until": {
"actions": {
"Compose": {
"inputs": "#first(split(variables('Array')?[iterationIndexes('Until')],'_SP1'))",
"runAfter": {},
"type": "Compose"
},
"Increment_variable": {
"inputs": {
"name": "loop",
"value": 1
},
"runAfter": {
"Compose": [
"Succeeded"
]
},
"type": "IncrementVariable"
}
},
"expression": "#equals(variables('loop'), length(variables('Array')))",
"limit": {
"count": 60,
"timeout": "PT1H"
},
"runAfter": {
"Initialize_variable_-_loop": [
"Succeeded"
]
},
"type": "Until"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {},
"triggers": {
"manual": {
"inputs": {
"schema": {}
},
"kind": "Http",
"type": "Request"
}
}
},
"parameters": {}
}

Logic App with azure monitor and conditions

I create a workflow with logicAPP. The goal is to notify a team when patch is missing for VM. I use azure monitor in the logic app to set the query. I decided to put after the Azure Monitor , a condition to know if the query table is empty or have data. if the table is empty, the logix is true , so it does'nt send notification, and when its false , it sends notification.
When I run , I got a logic errors. Normally , the table has not data but after condition , the function empty([my_table]) returns false and sends me notification with the result ("The query yielded no data")
what is the problem ??
Thanks
Based on the above shared requirement we have created the logic app & tested it our local environment , it is working fine.
Below is the complete logic code :
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Condition_2": {
"actions": {
"Terminate_2": {
"inputs": {
"runStatus": "Cancelled"
},
"runAfter": {},
"type": "Terminate"
}
},
"else": {
"actions": {
"Send_an_email_(V2)_2": {
"inputs": {
"body": {
"Body": "<p>#{base64ToString(body('Run_query_and_visualize_results')?['body'])}</p>",
"Subject": "list of vm from update management ",
"To": "<UserEmailId>"
},
"host": {
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "post",
"path": "/v2/Mail"
},
"runAfter": {},
"type": "ApiConnection"
}
}
},
"expression": {
"and": [
{
"equals": [
"#length(body('Run_query_and_visualize_results')?['body'])",
0
]
}
]
},
"runAfter": {
"Run_query_and_visualize_results": [
"Succeeded"
]
},
"type": "If"
},
"Run_query_and_visualize_results": {
"inputs": {
"body": "Update\n| where Classification == 'Security Updates' or Classification == 'Critical Updates'\n| where UpdateState == 'Needed'\n| summarize by Computer,ResourceGroup,Classification,UpdateState\n|sort by Computer",
"host": {
"connection": {
"name": "#parameters('$connections')['azuremonitorlogs']['connectionId']"
}
},
"method": "post",
"path": "/visualizeQuery",
"queries": {
"resourcegroups": "<Resource_group_Name",
"resourcename": "<log analytics workspacename",
"resourcetype": "Log Analytics Workspace",
"subscriptions": "<subcription_id>",
"timerange": "Last 12 hours",
"visType": "Html Table"
}
},
"runAfter": {},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"Recurrence": {
"evaluatedRecurrence": {
"frequency": "Hour",
"interval": 3
},
"recurrence": {
"frequency": "Hour",
"interval": 3
},
"type": "Recurrence"
}
}
},
"parameters": {
"$connections": {
"value": {
"azuremonitorlogs": {
"connectionId": "/subscriptions/<subcription-id>/resourceGroups/<resource-group>/providers/Microsoft.Web/connections/azuremonitorlogs",
"connectionName": "azuremonitorlogs",
"id": "/subscriptions/<subcription-id>/providers/Microsoft.Web/locations/northcentralus/managedApis/azuremonitorlogs"
},
"office365": {
"connectionId": "/subscriptions/<subcription-id>/resourceGroups/<resource-group>/providers/Microsoft.Web/connections/office365",
"connectionName": "office365",
"id": "/subscriptions/<subcription-id>/providers/Microsoft.Web/locations/northcentralus/managedApis/office365"
}
}
}
}
}
please find the reference output of the above logic sample run :

Azure Policy (deployifnotexists) not behaving as expected

This is my first post here. What I'm trying to do in Azure is deployifnotexists for storage accounts if certain settings are not enabled. I've attached my code. What I want to do is this:
Check for secure transfer being enabled
Check for TLS1_2 only
Check the FW
On the FW, have the Azure Services accepted (e.g. nsg flow logs etc)
If any of those conditions are not met, then deploy them through the ARM template. What is catching me is that I have intentionally put in bad settings to see it work and it will not say that they are non-compliant.
{
"mode": "All",
"policyRule": {
"if": {
"field": "type",
"equals": "Microsoft.Storage/storageAccounts"
},
"then": {
"effect": "deployIfNotExists",
"details": {
"type": "Microsoft.Storage/storageAccounts",
"roleDefinitionIds": [
"/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
],
"existenceCondition": {
"allOf": [
{
"field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
"equals": true
},
{
"field": "Microsoft.Storage/storageAccounts/minimumTlsVersion",
"equals": "TLS1_2"
},
{
"field": "Microsoft.Storage/storageAccounts/networkAcls.defaultAction",
"equals": "deny"
},
{
"field": "Microsoft.Storage/storageAccounts/networkAcls.bypass",
"contains": "AzureServices"
}
]
},
"deployment": {
"properties": {
"mode": "incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"type": "String",
"metadata": {
"description": "storageAccountName"
}
},
"location": {
"type": "String",
"metadata": {
"description": "location"
}
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-06-01",
"name": "[parameters('storageAccountName')]",
"location": "[parameters('location')]",
"properties": {
"minimumTlsVersion": "TLS1_2",
"networkAcls": {
"bypass": "AzureServices",
"defaultAction": "Deny"
},
"supportsHttpsTrafficOnly": true
}
}
],
"outputs": {}
},
"parameters": {
"storageAccountName": {
"value": "[field('Name')]"
},
"location": {
"value": "[field('location')]"
}
}
}
}
}
}
},
"parameters": {}
}
Thanks everyone
So through further reading and talking with more experienced colleagues I've determined that "deployIfNotExists" conditions are not to be used for a resources own settings.
By that I mean I cannot "deployIfNotExists" to a storage accounts storage account settings (as above) but i could deploy diagnostic logging to a SA. I am closing this question. I will try append and if I do anything good I'll loop it back in to this question for keen eyes.

Cloudformation for redis parameter group

I want to create a cloudformation template to deploy a redis parameter group. The problem lies in the fact that each of my parameters looks like this:
{
"Parameters": [
{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
},
{
"ParameterName": "active-defrag-cycle-max",
"ParameterValue": "75",
"Description": "Maximal effort for defrag in CPU percentage",
"Source": "user",
"DataType": "integer",
"AllowedValues": "1-75",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}
}
Now I understand the basic format of the template but can't figure out how to pass in parameters as seen above.
Failing template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"Parameters": [{
"ParameterName": "activedefrag",
"ParameterValue": "no",
"Description": "Enabled active memory defragmentation",
"Source": "user",
"DataType": "string",
"AllowedValues": "yes,no",
"IsModifiable": true,
"MinimumEngineVersion": "4.0.9",
"ChangeType": "immediate"
}]
}
}
}
}
}
Welcome to StackOverflow!
You need to incorporate those parameters into the Parameters section of the template, and change the syntax so that it conforms to the Parameters format.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Template to create a consul cluster",
"Parameters": {
"CacheParameterGroupFamily": {
"Description": "The name of the cache parameter group family that this cache parameter group is compatible with.",
"Type": "String",
"Default": "redis4.0",
"AllowedValues": ["memcached1.4", "memcached1.5", "redis2.6", "redis2.8", "redis3.2", "redis4.0", "redis5.0"]
},
"ParameterGroupDescription": {
"Description": "What this parameter group will be used for",
"Type": "String"
},
"activedefrag": {
"Description": "Enabled active memory defragmentation",
"Type": "String",
"AllowedValues": ["yes", "no"],
"Default": "no"
},
"activedefragcyclemax": {
"Description": "Maximal effort for defrag in CPU percentage",
"Type": "Number",
"Default": 75,
"MinValue": 1,
"MaxValue": 75
}
},
"Resources": {
"RedisParameterGroup": {
"Type": "AWS::ElastiCache::ParameterGroup",
"Properties": {
"CacheParameterGroupFamily" : { "Ref": "CacheParameterGroupFamily" },
"Description" : { "Ref": "ParameterGroupDescription" },
"Properties" : {
"activedefrag": {"Ref": "activedefrag"},
"active-defrag-cycle-max": {"Ref": "activedefragcyclemax"}
}
}
}
}
}

LoopbackJS - Tokens for user invites

I'm using loopback as the API server for my application. I'm building a social like network, where it's required to invite users via email. In order to associate the invitee with the inviter, I want the inviter to create a 'request token' associated with his userId, which then is sent via email in a format like this: domain.com/register?token=XXXXXX
The built-in Access Token model seems perfect for this purpose as a base model used so the idea is to create a new model "RequestToken" inheriting from the AccessToken model, however, the new model is then used for authentication purposes as well, which I don't want.
Following are my config files. It's worth mentioning that the below seen "Customer" model is extending Loopbacks "User" Model.
/server/model-config.json:
"_meta": {
"sources": [
"loopback/common/models",
"loopback/server/models",
"../common/models",
"./models"
],
"mixins": [
"loopback/common/mixins",
"loopback/server/mixins",
"../node_modules/loopback-ds-timestamp-mixin",
"../common/mixins",
"./mixins"
]
},
"User": {
"dataSource": "db",
"public": false
},
"AccessToken": {
"dataSource": "db",
"public": false,
"relations": {
"user": {
"type": "belongsTo",
"model": "Customer",
"foreignKey": "userId"
}
}
},
"ACL": {
"dataSource": "db",
"public": false
},
"RoleMapping": {
"dataSource": "db",
"public": false,
"options": {
"strictObjectIDCoercion": true
}
},
"Role": {
"dataSource": "db",
"public": false
},
"Email": {
"dataSource": "mail",
"public": false
},
"Customer": {
"dataSource": "db",
"public": true
},
"Friend": {
"dataSource": "db",
"public": true
},
"Memory": {
"dataSource": "db",
"public": true
},
"RequestToken": {
"dataSource": "db",
"public": true
}
}
Under "Customer" I've also tried to include:
"relations": {
"accessTokens": {
"type": "hasMany",
"model": "AccessToken",
"foreignKey": "userId",
"options": {
"disableInclude": true
}
}
}
common/customer.json
{
"name": "Customer",
"base": "User",
"idInjection": true,
"options": {
"validateUpsert": true
},
"mixins": {
"TimeStamp": true
},
"properties": {
"firstName": {
"type": "string",
"required": true
},
"lastName": {
"type": "string",
"required": true
},
"dob": {
"type": "date"
},
"country": {
"type": "string"
}
},
"validations": [],
"relations": {
"accessTokens": {
"type": "hasMany",
"model": "AccessToken",
"foreignKey": "userId",
"options": {
"disableInclude": true
}
},
"requestTokens": {
"type": "hasMany",
"model": "RequestToken",
"foreignKey": "userId",
"options": {
"disableInclude": true
}
}
},
"acls": [
{
"accessType": "EXECUTE",
"principalType": "ROLE",
"principalId": "$owner",
"permission": "ALLOW"
},
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "$everyone",
"permission": "DENY"
}
],
"methods": {}
}
common/request-token.json
{
"name": "RequestToken",
"base": "AccessToken",
"idInjection": true,
"options": {
"validateUpsert": true
},
"properties": {},
"validations": [],
"relations": {
"user": {
"type": "belongsTo",
"model": "Customer",
"foreignKey": "ownerId"
}
},
"acls": [],
"methods": {}
}
Summary:
How can I create a new "RequestToken" model, extending Loopbacks "AccessToken" model, but keep using the built-in AccessToken model for authentication etc.? Is it possible at all? As soon as I take the line '"base": "AccessToken"' out of the request-token.json file, all authentication method work again.
Thanks a lot in advance!
It seems I found a solution for this. Within server.js I needed to tell the app to use the AccessToken model.
server.js
...
app.use(loopback.token({
model: app.models.accessToken,
}));
...
I've added it just after
const app = loopback();
The docs refer to it to use it for authentication via cookies in the LB2 docs.
https://loopback.io/doc/en/lb2/Making-authenticated-requests.html
I'm using Loopback3. Within the LB3 docs, they do not mention this way anymore, so if there is another solution, happy to change the accepted answer.
Cheers