Adding XRAY Tracing to non-rest functions e.g., SQS, Cognito Triggers etc - serverless-framework

Question

Using the Serverless framework, I have functions that aren’t attached to an API Gateway Endpoint, such as:
Cognito Triggers
S3 Event
DynamoDB Stream
SQS Events
I am also using XRAY tracing, which I have set as tracing: true in my serverless.yml file. It seems that these functions are not being traced, the debug message I receive is:
Ignoring flush on subsegment 20dcd559aa2ab487. Associated segment is marked as not sampled.
Is there any way to have these functions added, either via serverless or cloudformation?
Thanks in advance.

Answer 1

To enable X-Ray tracing for all your Service’s Lambda functions you just need to set the corresponding tracing configuration on the provider level:
provider:
tracing:
lambda: true
If you want to setup tracing on a per-function level you can use the tracing config in your function definition:
functions:
myFunction:
handler: index.handler
tracing: true
Setting tracing to true translates to the Active tracing configuration. You can overwrite this behavior by providing the desired configuration as a string:
functions:
myFunction:
handler: index.handler
tracing: PassThrough
Also note that you can mix the provider- and function-level configurations. All functions will inherit the provider-level configuration which can then be overwritten on an individual function basis:
service:
name: my-tracing-service
provider:
name: aws
stage: dev
runtime: nodejs8.10
tracing:
lambda: true
functions:
myFunc1: # this function will inherit the provider-level tracing configuration
handler: index.func1
myFunc2:
handler: handler.func2
tracing: PassThrough # here we're overwriting the provider-level configuration
It's recommended to setup X-Ray tracing for Lambda with the aforementioned tracing configuration since doing so will ensure that the X-Ray setup is managed by the Serverless Framework core via CloudFormation.
You can get more granular and specify which resources you want traced as well:
Open your serverless.yml and add a tracing config inside the provider section:
provider:
...
tracing:
apiGateway: true
lambda: true
IMPORTANT: Due to CloudFormation limitations it's not possible to enable AWS X-Ray Tracing on existing deployments which don’t use tracing right now.
Please remove the old Serverless Deployments and re-deploy your lambdas with tracing enabled if you want to use AWS X-Ray Tracing for Lambda.
Lastly, don't forget to have the right IAM permission policies configured:
provider:
...
iamRoleStatements:
- Effect: Allow
Action:
...
- xray:PutTraceSegments
- xray:PutTelemetryRecords
Resource: "*"
To enable X-Ray tracing for other AWS services invoked by AWS Lambda, you MUST Install the AWS X-Ray SDK. In your project directory, run:
$ npm install -s aws-xray-sdk
Update your Lambda code and wrap AWS SDK with the X-Ray SDK. Change:
const AWS = require('aws-sdk');
To:
const AWSXRay = require('aws-xray-sdk-core');
const AWS = AWSXRay.captureAWS(require('aws-sdk'));
As of Release Serverless v140

Answer 2

At the moment Lambda doesn't support continuing traces from triggers other than REST APIs or direct invocation
The upstream service can be an instrumented web application or another Lambda function. Your service can invoke the function directly with an instrumented AWS SDK client, or by calling an API Gateway API with an instrumented HTTP client.
https://docs.aws.amazon.com/xray/latest/devguide/xray-services-lambda.html
In every other case it will create its own, new Trace ID and use that instead.
You can work around this yourself by creating a new AWS-Xray segment inside the Lambda Function and using the incoming TraceID from the SQS message. This will result in two Segments for your lambda invocation. One which Lambda itself creates, and one which you create to extend the existing trace. Whether that's acceptable or not for your use case is something you'll have to decide for yourself!
If you're working with Python you can do it with aws-xray-lambda-segment-shim.
If you're working with NodeJS you can follow this guide on dev.to.
If you're working with .NET there are some examples on this GitHub issue.