I am using serverless framework. I have set the stage as the dev and deploying using sls deploy --stage dev command. The dev stage is not added to the end point. The endpoints are not like the one given below:
https://****.execute-api.us-east-1.amazonaws.com/users
One of my lambda functions needs an endpoint to submit a post request to a third-party API to post back the result. I need to be sure that the endpoint is rightly sent from the production/dev stage.
postback_url = `https://${process.env.RestApiId}.execute-api.${
process.env.REGION
}.amazonaws.com/${process.env.stage}/dfs-pingback?id=$id&tag=$tag`;
As you can see the above postback url is wrong if I do not get the stage (process.env.stage) added to the endpoint.
serverless.yml
service: lytoolsApi
frameworkVersion: '2 || 3'
configValidationMode: error
provider:
name: aws
runtime: nodejs12.x
region: us-east-1
stage: dev
Serverless Framework does things a bit differently and instead of using stages of APIGW, it creates a totally new APIGW for each stage, that's why you don't see the prefix in your path with the stage name, but if you observe the url, you'll see that the base url will be different across stages. That's how you can differentiate between them.
Related
I am using Gitlab CI with docker executor and services.
During test I'm starting a server in the main script, and I need the service to make a request back to the main script.
Is there address/alias I can use to connect back to the main build script? Something like host.docker.internal.
Pseudo-example:
test:
services:
name: ping-pong-service
variables:
CALLBACK_ADDRESS: 'http://host.docker.internal:8090/pong'
script:
- "Start a server at 0.0.0.0:8090"
- curl http://ping-pong-service:80/ping
Supose that ping-pong-service is a service that when receiving any http request on :80, performs new request to CALLBACK_ADDRESS. What should I enter into CALLBACK_ADDRESS to connect back to main container?
I tried looking into what containers get started on the runner, but the main container doesn't seem to have predictable name or alias in the docker network.
Env:
Docker: 20.10.12
Gitlab Runner: 14.8.0, self-hosted, FF_NETWORK_PER_BUILD=1
Gitlab: 14.9.2-ee, self-hosted
When using the FF_NETWORK_PER_BUILD feature flag for networks per job, containers started using services: can reach the main job container using the network alias build
Assuming your service is configured as you describe, you would use:
variables:
CALLBACK_ADDRESS: 'http://build:8090/pong'
Note: this does not apply to containers started using docker run in the job container for this scenario.
So I have an API that's the gateway for two other API's.
Using docker in wsl 2 (ubuntu), when I build my Gateway API.
docker run -d -p 8080:8080 -e A_API_URL=$A_API_URL B_API_URL=$B_API_URL registry:$(somePort)//gateway
I have 2 environnement variables that are the API URI of the two API'S. I just dont know how to make this work in the config.
env:
- name: A_API_URL
value: <need help>
- name: B_API_URL
value: <need help>
I get 500 or 502 errors when accessing then in the network.
I tried specifyng the value of the env var as:
their respective service's name.
the complete URI (http://$(addr):$(port)
the relative path : /something/anotherSomething
Each API is deployed with a Deployment controller and a service
I'm at a lost, any help is appreciated
You just have to hardwire them. Kubernetes doesn't know anything about your local machine. There are templating tools like Helm that could inject things like Bash is in your docker run example but generally not a good idea since if anyone other than you runs the same command, they could see different results. The values should look like http://servicename.namespacename.svc.cluster.local:port/whatever. So if the service is named foo in namespace default with port 8000 and path /api, http://foo.default.svc.cluster.local:8000/api.
Using the Serverless framework, I have functions that aren’t attached to an API Gateway Endpoint, such as:
Cognito Triggers
S3 Event
DynamoDB Stream
SQS Events
I am also using XRAY tracing, which I have set as tracing: true in my serverless.yml file. It seems that these functions are not being traced, the debug message I receive is:
Ignoring flush on subsegment 20dcd559aa2ab487. Associated segment is marked as not sampled.
Is there any way to have these functions added, either via serverless or cloudformation?
Thanks in advance.
To enable X-Ray tracing for all your Service’s Lambda functions you just need to set the corresponding tracing configuration on the provider level:
provider:
tracing:
lambda: true
If you want to setup tracing on a per-function level you can use the tracing config in your function definition:
functions:
myFunction:
handler: index.handler
tracing: true
Setting tracing to true translates to the Active tracing configuration. You can overwrite this behavior by providing the desired configuration as a string:
functions:
myFunction:
handler: index.handler
tracing: PassThrough
Also note that you can mix the provider- and function-level configurations. All functions will inherit the provider-level configuration which can then be overwritten on an individual function basis:
service:
name: my-tracing-service
provider:
name: aws
stage: dev
runtime: nodejs8.10
tracing:
lambda: true
functions:
myFunc1: # this function will inherit the provider-level tracing configuration
handler: index.func1
myFunc2:
handler: handler.func2
tracing: PassThrough # here we're overwriting the provider-level configuration
It's recommended to setup X-Ray tracing for Lambda with the aforementioned tracing configuration since doing so will ensure that the X-Ray setup is managed by the Serverless Framework core via CloudFormation.
You can get more granular and specify which resources you want traced as well:
Open your serverless.yml and add a tracing config inside the provider section:
provider:
...
tracing:
apiGateway: true
lambda: true
IMPORTANT: Due to CloudFormation limitations it's not possible to enable AWS X-Ray Tracing on existing deployments which don’t use tracing right now.
Please remove the old Serverless Deployments and re-deploy your lambdas with tracing enabled if you want to use AWS X-Ray Tracing for Lambda.
Lastly, don't forget to have the right IAM permission policies configured:
provider:
...
iamRoleStatements:
- Effect: Allow
Action:
...
- xray:PutTraceSegments
- xray:PutTelemetryRecords
Resource: "*"
To enable X-Ray tracing for other AWS services invoked by AWS Lambda, you MUST Install the AWS X-Ray SDK. In your project directory, run:
$ npm install -s aws-xray-sdk
Update your Lambda code and wrap AWS SDK with the X-Ray SDK. Change:
const AWS = require('aws-sdk');
To:
const AWSXRay = require('aws-xray-sdk-core');
const AWS = AWSXRay.captureAWS(require('aws-sdk'));
As of Release Serverless v140
At the moment Lambda doesn't support continuing traces from triggers other than REST APIs or direct invocation
The upstream service can be an instrumented web application or another Lambda function. Your service can invoke the function directly with an instrumented AWS SDK client, or by calling an API Gateway API with an instrumented HTTP client.
https://docs.aws.amazon.com/xray/latest/devguide/xray-services-lambda.html
In every other case it will create its own, new Trace ID and use that instead.
You can work around this yourself by creating a new AWS-Xray segment inside the Lambda Function and using the incoming TraceID from the SQS message. This will result in two Segments for your lambda invocation. One which Lambda itself creates, and one which you create to extend the existing trace. Whether that's acceptable or not for your use case is something you'll have to decide for yourself!
If you're working with Python you can do it with aws-xray-lambda-segment-shim.
If you're working with NodeJS you can follow this guide on dev.to.
If you're working with .NET there are some examples on this GitHub issue.
I've got a web app deployed to AKS and it's generally up and running fine. I am now trying to extend its functionality by adding access to Azure sql.
We are using VSTS/Azure DevOps for deployment.
I've deployed a secret to the cluster using the command:
kubectl secret generic sampleapp-appsettings --from-literal=DBConnectionString="$(var_DBConnectionString)"
I've checked the cluster from the kubernetes dashboard and can see it has been deployed as expected. The secret is a connection string to the database
However, I'm struggling to retrieve the secret from the deployed pods. I've created an environment variable for ASPNETCORE_ENVIRONMENT, with a value for Kubernetes.
Here's part of my deployment yaml:
spec:
containers:
- name: sampleapp-services
image: sampleapp.azurecr.io/sampleapp-services:latest
imagePullPolicy: Always
env:
- name: "ASPNETCORE_ENVIRONMENT"
value: "Kubernetes"
- name: services-appsettings
valueFrom:
secretKeyRef:
name: services-appsettings
key: DBConnectionString
ports:
- containerPort: 80
I've added an api endpoint to my app for the purposes of debugging and can see that the ASPNETCORE_ENVIRONMENT value is being pull correctly.
however, the DBConnectionString value isn't being correctly pull from the kubernetes secret. instead it's being retrieved from the appsettings.json file. I've got some code in my app which is just outputting the values:
[HttpGet("settings")]
public ActionResult<string> GetAppSettings()
{
var message = $"Host: {Environment.MachineName}\n" +
$"EnvironmentName: {_env.EnvironmentName}\n" +
$"Secret value: {_dataSettings.ConnectionString}";
return message;
}
in my DataSettings class I've got code like this:
var value = Environment.GetEnvironmentVariable("DBConnectionString");
However, this isn't pulling back the secret value from the kubernetes cluster that I'm expecting.
I've followed some examples, like this - but they don't help.
blog
Has anyone got some simple step by step instructions/samples that might help?
thanks
The command you have specified to create secret is missing create and creates a secret by name 'sampleapp-appsettings' however in deployment.yaml you have specified 'services-appsettings' instead. I assume the snippets you listed are just for reference and in actual code these values match.
Secondly, environment variable - name: services-appsettings should match to the name you have specified in code. As per your snippets, Environment.GetEnvironmentVariable("DBConnectionString") is having 'DBConnectionString' however your yaml is having 'services-appsettings'
Lastly, I hope in Web API, you are calling .AddEnvironmentVariables() while building config.
I am trying to figure out how to manage secrets for different environements while creating serverless applications
my serverless.yml file looks something like this:
provider:
name: aws
runtime: nodejs6.10
stage: ${opt:stage}
region: us-east-1
environment:
NODE_ENV: ${opt:stage}
SOME_API_KEY: // this is different depending upon dev or prod
functions:
....
when deploying i use the following command
serverless deploy --stage prod
I want the configuration information to be picked up from AWS parameter store as described here:
https://serverless.com/blog/serverless-secrets-api-keys/
However I do not see a way to provide different keys for development or prod environment.
Any suggestions ?
I put prefixes in my Parameter Store variables.
e.g.
common/SLACK_WEBHOOK
development/MYSQL_PASSWORD
production/MYSQL_PASSWORD
Then, in my serverless.yml, I can do...
...
environment:
SLACK_WEBHOOK: ${ssm:common/SLACK_WEBHOOK}
MYSQL_PASSWORD: ${ssm:${opt:stage}/MYSQL_PASSWORD}
You should be able to do something similar.