Filter CloudWatch logs with a property selector for a JSON property containing a period character "." - amazon-cloudwatch

How would I write a CloudWatch filter with a property selector for a JSON property containing a period character "."?
The CloudWatch documentation describes using a property selector to filter log data (https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#metric-filters-extract-json):
Documentation Syntax:
{ PropertySelector EqualityOperator String }
Documentation Example:
{ $.eventType = "UpdateTrail" }
If a log event contains a JSON property containing a period character ".", is it possible to describe it with a property selector?
I have attempted the following property selectors:
Filter encounters error { $.attributes.http\.method = "POST" }
Filter encounters error { $.attributes["http.method"] = "POST" }
Filter produces zero results { $.attributes.http.method = "POST" }
Example Event
{
"traceId": "75e07edf6f50ddf0d4a8239cbe91d60a",
"parentId": "f8875623e2e77de4",
"name": "request handler - /subscriptions",
"id": "7f8b38dcbbb15983",
"kind": 0,
"timestamp": 1673621283065,
"duration": 3,
"attributes": {
"http.route": "/subscriptions",
"express.name": "/subscriptions",
"express.type": "request_handler",
"http.method": "POST",
"http.url": "/subscriptions",
"http.body": "{\"foo\":\"bar\"}"
},
"status": {
"code": 0
},
"events": [],
"links": []
}

Related

Having issues reusing a stored variable in Graphql query in Karate framework

I have an issue with the variable usage. Tried different options(storing variable differently, declaring, using text for defining the query, storing the query as a variable).
Still have the below error:
"errors": [
{
"message": "invalid input syntax for type uuid: \"#(queueID)\"",
"locations": [
{
"line": 1,
"column": 11
}
],
"path": [
"deleteQueue"
],
"extensions": {
"code": "INTERNAL_SERVER_ERROR",
"exception": {
"name": "SequelizeDatabaseError",
"parent": {
"length": 109,
"name": "error",
"severity": "ERROR",
"code": "22P02",
"position": "34",
"file": "uuid.c",
"line": "137",
"routine": "string_to_uuid",
"sql": "DELETE FROM \"Queue\" WHERE \"id\" = '#(queueID)'"
This are my Gherkin steps:
Given request { query: 'mutation {createQueue(input: {name: "BDD-delete" }) {id} }'}
When method POST
Then status 200
And match response.data.createQueue.name == "BDD-delete"
* def queueID = response.data.createQueue.id
* print queueID
Given request { query: 'mutation {deleteQueue (id:"#(queueID)")} '}
And this is the output, when I print the queueID:
13:14:16.745 [main] INFO com.intuit.karate - [print] 758c0524-b18d-41f6-96aa-9db5eb8a7ac8
Tried using variable for the query
Given text payload =
"""
mutation {
createQueue(input: {name: "BDD-delete" })
{id, name}
}
"""
And the same tried for the deleteQueue
Feels like the issue is related with str and uuid. I must pass a uuid between the brackets in "#(queueID)"
First read this to get a sense of why this is happening: https://github.com/karatelabs/karate#rules-for-embedded-expressions
So try this:
Given request `{ query: 'mutation {deleteQueue (id:"${queueID}")} '}`
The good thing is that Karate supports JS-style placeholder replacement in strings within back-ticks.
Also refer: https://stackoverflow.com/a/69349118/143475

Loopback 4 auto generated model with required id failing validation

I'm using an automated script that runs an auto-generation model using lb4 cli.
Looks like validation expects id to be provided, but swagger missing it in its schema. Why I can't see the id property in swagger?
PLEASE NOTE! I don't want to modify manually my models
lb4 model activity --dataSource DS --table activity
Created model:
export class Activity extends Entity {
#property({
type: 'string',
required: true,
id: 1,
postgresql: {
columnName: 'id',
dataType: 'uuid',
dataLength: null,
dataPrecision: null,
dataScale: null,
nullable: 'NO',
},
})
id: string;
...
}
When I run the swagger tool and try to POST new activity, it missing the id field and returns the following error:
{
"error": {
"statusCode": 422,
"name": "ValidationError",
"message": "The `Activity` instance is not valid. Details: `id` can't be blank (value: undefined).",
"details": {
"context": "Activity",
"codes": {
"id": [
"presence"
]
},
"messages": {
"id": [
"can't be blank"
]
}
}
}
}
If I add a property id manually, then it throws a validation error:
{
"error": {
"statusCode": 422,
"name": "UnprocessableEntityError",
"message": "The request body is invalid. See error object `details` property for more info.",
"code": "VALIDATION_FAILED",
"details": [
{
"path": "",
"code": "additionalProperties",
"message": "must NOT have additional properties",
"info": {
"additionalProperty": "id"
}
}
]
}
}
Change your #model() by #model({settings: {strict: false}}) and add this line [prop: string]: any; into your model
#model({settings: {strict: false}})
[prop: string]: any;

AppSync request mapping template errors not logged in CloudWatch

Crosspost from: https://repost.aws/questions/QUp5jDZ6bsRkeXhIwHgQaWkg/app-sync-request-mapping-template-errors-not-logged-in-cloud-watch
I have a simple resolver that has a simple Lambda function as a data source. This function always throws an error (to test out logging).
The resolver has request mapping template enabled and it is configured as follows:
$util.error("request mapping error 1")
The API has logging configured to be as verbose as possible yet I cannot see this request mapping error 1 from my CloudWatch logs in RequestMapping log type:
{
"logType": "RequestMapping",
"path": [
"singlePost"
],
"fieldName": "singlePost",
"resolverArn": "xxx",
"requestId": "bab942c6-9ae7-4771-ba45-7911afd262ac",
"context": {
"arguments": {
"id": "123"
},
"stash": {},
"outErrors": []
},
"fieldInError": false,
"errors": [],
"parentType": "Query",
"graphQLAPIId": "xxx"
}
The error is not completely lost because I can see this error in the query response:
{
"data": {
"singlePost": null
},
"errors": [
{
"path": [
"singlePost"
],
"data": null,
"errorType": null,
"errorInfo": null,
"locations": [
{
"line": 2,
"column": 3,
"sourceName": null
}
],
"message": "request mapping error 1"
}
]
}
When I add $util.appendError("append request mapping error 1") to the request mapping template so it looks like this:
$util.appendError("append request mapping error 1")
$util.error("request mapping error 1")
Then the appended error appears in the RequestMapping log type but the errors array is still empty:
{
"logType": "RequestMapping",
"path": [
"singlePost"
],
"fieldName": "singlePost",
"resolverArn": "xxx",
"requestId": "f8eecff9-b211-44b7-8753-6cc6e269c938",
"context": {
"arguments": {
"id": "123"
},
"stash": {},
"outErrors": [
{
"message": "append request mapping error 1"
}
]
},
"fieldInError": false,
"errors": [],
"parentType": "Query",
"graphQLAPIId": "xxx"
}
When I do the same thing with response mapping template then everything works as expected (errors array contains $util.error(message) and outErrors array contains $util.appendError(message) messages.
Is this working as expected so the $util.error(message) will never show up in RequestMapping type CloudWatch logs?
Under what conditions will errors array in RequestMapping log type be populated?
Bonus question: can the errors array contain more than 1 item for either RequestMapping or ResponseMapping log types?

Only structured queries are supported FIRESTORE API REST

I am trying to run a query with the rest api but I canĀ“t get it to work, I am sending this body in json:
"structuredQuery": {
"where": {
"fieldFilter": {
"field": {
"fieldPath": "total"
},
"op": "EQUAL",
"value": {
"integerValue": "10",
}
}
},
"from": [{
"collectionId": "Total"
}]
}
I am just testing if querys work, I have a collection called Total with documents, and these documents have a field called total that is an integer value. I am using a POST request with the following URL:
"https://firestore.googleapis.com/v1/projects/MY_PROJECT_NAME/databases/(default)/documents:runQuery"
and I am getting this error:
[{
"error": {
"code": 400,
"message": "only structured queries are supported",
"status": "INVALID_ARGUMENT"
}
}]
The CollectionSelector in your structured query is missing the allDescendants field.
If you check this documentation you can see that this field is a flag which can either be true or false but not null, so you have to set this in the query otherwise it will not work.
Also you need to add the select clause to add all fields you want to get as a result of the query or keep it empty to return all fields.
Finally in the said documentation you can check that there is a proper order that should be respected, in which the from must be declared before where. So if you change your structured query to the following:
"structuredQuery": {
"from": [{
"collectionId": "Total"
"allDescendants": false
}],
"select": { "fields": [] },
"where": {
"fieldFilter": {
"field": {
"fieldPath": "total"
},
"op": "EQUAL",
"value": {
"integerValue": "10",
}
}
}
}
It will work as expected.

How do you get all the email body parts? And how do you know how many parts exist?

I'm trying to read emails responded by the Gmail API.
I have trouble accessing all the "parts". And don't have great ways to traverse through the response. I'm also lost as to how many parts can exist so that I can make sure I read the different email responses properly. I've shortened the response below...
{ "payload": { "mimeType": "multipart/mixed", "filename": "",
], "body": { "size": 0 }, "parts": [ {
"body": {
"size": 0
},
"parts": [
{
"partId": "0.0",
"mimeType": "text/plain",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/plain; charset=\"us-ascii\""
},
{
"name": "Content-Transfer-Encoding",
"value": "quoted-printable"
}
],
"body": {
"size": 2317,
"data": "RGVhciBNSVQgQ2x1YiBWb2x1bnRlZXJzIGluIEFzaWEsDQoNCkJ5IG5vdyBlYWNoIG9mIHlvdSBzaG91bGQgaGF2ZSByZWNlaXZlZCBpbnZpdGF0aW9ucyB0byB0aGUgcmVjZXB0aW9ucyBpbiBib3RoIFNpbmdhcG9yZSBhbmQgSG9uZyBLb25nIHdpdGggUHJlc2lkZW50IFJlaWYgb24gTm92ZW1iZXIgNyBhbmQgTm92ZW1iZXIg"
}
},
{
"partId": "0.1",
"mimeType": "text/html",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/html; charset=\"us-ascii\""
},
{
"name": "Content-Transfer-Encoding",
"value": "quoted-printable"
}
],
"body": {
"size": 9116,
"data": "PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVybjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWljcm9zb2Z0LmNvbS9vZmZpY2UvMjA"
}
}
] }, {
"partId": "1",
"mimeType": "text/plain",
"filename": "",
"body": {
"size": 411,
"data": "X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NClRoYW5rIHlvdSBmb3IgYWxsb3dpbmcgdXMgdG8gcmVhY2ggeW91IGJ5IGVtYWlsLCB0aGUgbW9zdCBpbW1lZGlhdGUgbWVhbnMgZm9yIHNoYXJpbmcgaW5mb3JtYXRpb24gd2l0aCBNSVQgYWx1bW5pLiANCklmIHlvdSB3b3VsZCBsaWtlIHRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBtYWlsaW5nIGxpc3Qgc2VuZCBhIGJsYW5rIGVtYWlsIHRvIGxpc3RfdW5zdWJzY3JpYmVAYWx1bS5taXQuZWR1IGFuZCBwdXQgdGhlIGxpc3QgbmFtZSBpbiB0aGUgc3ViamVjdCBsaW5lLg0KRm9yIGV4YW1wbGU6DQpUbzogbGlzdF91bnN1YnNjcmliZUBhbHVtLm1pdC5lZHUNCkNjOg0KU3ViamVjdDogYXNpYW9mZg0K"
} } ] } }
Is there something I'm missing?
A MIME message is not just an array it's a full blown tree structure. So you'll have to traverse it to correctly handle it. Luckily JSON parsers are plentiful and the problem can easily be handled with recursion. In many languages there exist very useful email parsing libraries that can make accessing traditional parts (e.g. the text/plain or text/html displayable part, or attachments) not too laborious.
You'll have to set up walker functions to traverse through the json and pick out the bits you are after. Here is a part of what I wrote. This may help you jumpstart your code. NOTE: this is used inside of wordpress...hence the special jQuery call. Not needed if you do not need to use jquery inside wordpress.
function makeApiCall() {
gapi.client.load('gmail', 'v1', function() {
//console.log('inside call: '+myquery);
var request = gapi.client.gmail.users.messages.list({
'userId': 'me',
'q': myquery
});
request.execute(function(resp) {
jQuery(document).ready(function($) {
//console.log(resp);
//$('.ASAP-emailhouse').height(300);
$.each(resp.messages, function(index, value){
messageId = value.id;
var messagerequest = gapi.client.gmail.users.messages.get({
'userId': 'me',
'id': messageId
});//end var message request
messagerequest.execute(function(messageresp) {
//console.log(messageresp);
$.each(messageresp, responsewalker);
function responsewalker(key, response){
messagedeets={};
$.each(messageresp.payload.headers, headerwalker);
function headerwalker(headerkey, header){
if(header.name =='Date'){
d = new Date(header.value);
var curr_date = d.getDate();
var curr_month = d.getMonth() + 1; //Months are zero based
var curr_year = d.getFullYear();
var formatteddate = curr_month+'/'+curr_date+'/'+curr_year;
messagedeets['date']=formatteddate;
//$('.ASAP-emailhouse').append('<p>'+header.value+'</p>');
}
if(header.name =='Subject'){
//console.log(header.value);
messagedeets.subject=header.value;
}
}
messagedeets.body = {};
$.each(messageresp.payload.parts, walker);
function walker(partskey, value) {
//console.log(value.body);
if (value.body.data !== "undefined") {
//console.log(value.body);
var messagebody = atob(value.body.data);
messagedeets.body.partskey = messagebody;
}
console.log(messagedeets);
$('.ASAP-emailhouse').append('<div class="messagedeets"><p class="message-date">'+messagedeets.date+': <span class="message-subject">'+messagedeets.subject+'</span></p><p>'+messagedeets.body.partskey+'</p></div>');
}//end responsewalker
//$('.ASAP-emailhouse').append('</li>');
}
//$('.ASAP-emailhouse').append('</ul>');
});//end message request
});//end each message id
});//end jquery wrapper for wordpress
});//end request execute list messages
});//end gapi client load gmail
}
The MIME parts you are looking for are in an array. JSON does not tell you up front how many items are in an array. Even MIME itself does not provide a way of knowing how many parts are present without looking at the entire message. You will just have to traverse the entire array to know how many parts are in it, and process each part as you encounter it.
To know how much parts exists, you can just use the Length property.
Example :
json.payload.parts.length
For your example, this property is 2 because there are 2 parts.