Is it possible to call lambda function from other lambda functions in AWS serverless API? - asp.net-core

I am creating a serverless API using AWS SAM template and ASP.Net Core.
I wanted to know if it is possible to call a common lamnda function from multiple lambda functions?
I have 2 APIs for user authentication.
/user/authenticate
/admin/authenticate
Now when the user calls these API endpoints I want to call a common lambda function which will look like following:
public AuthResponse Authenticate(AuthInfo info, int role);
I get a user role based on which API endpoint is called. For example if /user/authetication is called then role=1 otherwise role=0.
And then I want Authenticate() lambda to perform user authentication based on the AuthInfo + Role.
I want to do this because all my users are stored in the same table and I would like to cross verify if user has the correct role to access the feature.
I will also share a portion of serverless.template used for above APIs.
/admin/authenticate
"Handler": "Server::API.Admin::Authenticate",
"Description" : "Allows admin to authenticate",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout" : 300,
"Role": {"Fn::GetAtt" : [ "LambdaExecutionRole", "Arn"]},
"FunctionName" : "AdminAuthenticate",
"Events":
{
"PutResource":
{
"Type": "Api",
"Properties":
{
"Path": "/v1/admin/authenticate",
"Method": "POST"
}
}
}
}
}
/user/authenticate
"Handler": "Server::API.User::Authenticate",
"Description" : "Allows user to authenticate",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout" : 300,
"Role": {"Fn::GetAtt" : [ "LambdaExecutionRole", "Arn"]},
"FunctionName" : "UserAuthenticate",
"Events":
{
"PutResource":
{
"Type": "Api",
"Properties":
{
"Path": "/v1/user/authenticate",
"Method": "GET"
}
}
}
}
}
As you can see above, 2 lambda functions are created AdminAuthenticate and UserAuthentication. I want these lambda functions to share the common code.
Does anyone has any idea how to do it?
Thanks and Regards.

I can think about 2 options to achieve your goal. In the first option, you use multiple Lambda functions, one for each endpoint, both pointing to your same codebase. In the second option, you have a single Lambda function that handles all authentication needs.
Single codebase, multiple functions
In this case, you can define your template file with 2 functions but use the CodeUri property to point to the same codebase.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Resources": {
"AdminFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "Server::API.Admin::Authenticate",
"Description": "Allows admin to authenticate",
"Runtime": "dotnetcore2.1",
"CodeUri": "./codebase_path/",
"MemorySize": 256,
"Timeout": 300,
"FunctionName": "AdminAuthenticate",
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/v1/admin/authenticate",
"Method": "POST"
}
}
}
}
},
"UserFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "Server::API.User::Authenticate",
"Description": "Allows user to authenticate",
"Runtime": "dotnetcore2.1",
"CodeUri": "./codebase_path/",
"MemorySize": 256,
"Timeout": 300,
"FunctionName": "UserAuthenticate",
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/v1/user/authenticate",
"Method": "POST"
}
}
}
}
}
}
}
Single codebase, single function
In this case, you will expose 2 endpoints on API Gateway, but they will be directed to the same handler on your function. Therefore, you will need to write some logic on your code to handle the login properly. The event object passed to your Lambda function will have information on the original URL in the path property (reference -- even though this is for Lambda proxy, still applies).
The template file in this case would be similar to (note I replaced Admin/User terms to "Any", since this will handle any form of authentication):
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Resources": {
"AnyFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "Server::API.Any::Authenticate",
"Description": "Allows any to authenticate",
"Runtime": "dotnetcore2.1",
"CodeUri": "./hello_world/",
"MemorySize": 256,
"Timeout": 300,
"Events": {
"UserEndpoint": {
"Type": "Api",
"Properties": {
"Path": "/v1/user/authenticate",
"Method": "POST"
}
},
"AdminEndpoint": {
"Type": "Api",
"Properties": {
"Path": "/v1/admin/authenticate",
"Method": "POST"
}
}
}
}
}
}
}

You can invoke lambda functions through any lambda function, using aws sdk in your chosen language and there you have invoke function defined.
For a reference here is the link to boto3 invoke definition.
Also
The approach you are using for authentication using common codebase is not the right one.
If you need a lambda function to check or authentication particular request, you can setup custom authorizer for that, which in your terms i can say, share lambda code or call it first before invoking the lambda you setup for the particular endpoint with the possibility to pass the custom data if you want.
A Lambda authorizer (or as a custom authorizer) is an API Gateway feature that uses a Lambda function to control access to your API.
If still this doesn't solve your problem and you want common codebase, you can point out as many as api endpoint's to same lambda function.
Then you have to handle event['resources'] inside your codebase.
This is not the recommended way, but you can use it.
You can refer aws samples to setup custom authorizer or the documentation is fair enough to clear all your doubts.

Related

Chaostoolkit istio extension hangs when playing experiment

I'm trying to use the chaos toolkit istio extension, my problem is as follows:
I have a experiment.json file which contains a single probe to retrieve a virtual service. The file looks similar to the following:
{
"version": "1.0.0",
"title": "test",
"description": "N/A",
"tags": []
"secrets": {
"istio": {
"KUBERNETES_CONTEXT": {
"type": "env",
"key": "KUBERNETES_CONTEXT"
}
}
},
"method": [
{
"type": "probe",
"name": get_virtual_service:,
"provider": {
"type": "python",
"module": "chaosistio.fault.probes",
"func": "get_virtual_service",
"arguments": {
"virtual_service_name": "test"
"ns": "test-ns"
}
}
}
}
I have set KUBERNETES_CONTEXT and http/https proxy as env vars. My authorisation is using $HOME/.kube/config.
When playing the experiment it validates the file fine and tries to perform the action but becomes stuck and just hangs until it times out.
The error I see in the logs is a HTTPSConnectionPool error (failed to establish a new connection, operation timed out).
Am I missing any settings? All help appreciated.

HTTP request in Azure Data Factory

In Azure Data Factory, I need to tap into a HTTP requests via URL using the HTTP connector. I was able to do this as well as setup the dataset. Where I'm having issues is on the pipeline. Here's what I need to do. What is the best way to accomplish this?
Call out to the service base URL and retrieve the header returned of TotalPages.
Using the value for TotalPages, make subsequent requests to the URL with the parameter page (e.g., page=1, page=2, etc.) using the value from TotalPages to form those requests.
Thanks.
Ok. So the issue here is that you cannot nest control structures in Data Factory more than 1 time. The solution is to create two or more pipelines (aka Master and Child).
From the Master pipeline retrieve the number of tasks you will need to execute, and pass them to a for loop. Within the for loop launch for each activity pair a new Child pipeline which will then execute the second activity.
If the Activity is simple enough you can skip the Child Pipeline altogether and do it directly inside the first for loop.
As a Json representation of pipelines in question it should look along these lines:
{
"name": "generic_master",
"properties": {
"activities": [
{
"name": "Web1",
"type": "WebActivity",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"url": "https://jsonplaceholder.typicode.com/posts/1",
"method": "GET"
}
},
{
"name": "ForEach1",
"type": "ForEach",
"dependsOn": [
{
"activity": "Web1",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"items": {
"value": "#activity('Web1').output",
"type": "Expression"
},
"activities": [
{
"name": "Execute Pipeline1",
"type": "ExecutePipeline",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"pipeline": {
"referenceName": "generic_child",
"type": "PipelineReference"
},
"waitOnCompletion": true
}
}
]
}
}
],
"annotations": []
}
}
{
"name": "generic_child",
"properties": {
"activities": [
{
"name": "Web1",
"type": "WebActivity",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"url": "https://jsonplaceholder.typicode.com/posts/1",
"method": "POST"
}
}
],
"annotations": []
}
}
In order to read the TotalPages values from the HTTP Request's response, you can use a "Lookup" activity to submit the HTTP request and store the TotalPages value in a variable with the "Set variable" activity.
Actions:
Pipeline level:
create a variable called TotalPages
Lookup activity:
tick the first row only box on the Settings tab
As a source dataset, use the data set defined for your HTTP request
Select the GET method.
Set variable activity:
Select the TotalPages variable on the Variables tab
In the value box, click on "Add dynamic content" and enter something like this: #{activity('GetTotalPages').output.firstRow.RegisterSearch['#TotalPages']}
In my case, the lookup activity is called GetTotalPages, and my HTTP request returns the total number of pages in a RegisterSearch array, under a column name #TotalPages

How to pass a variable to EMR addStep in AWS StepFunctions

AWS Stepfunctions recently added EMR integration, which is cool, but i couldn't find a way to pass a variable from step functions into the addstep args.
For example i would like to pass "$.dayid" variable into "Parameters">"Step">"HadoopJarStep">Args. Similar to "ClusterId.$": "$.ClusterId" (this cluster id variable works).
{
"Step_One": {
"Type": "Task",
"Resource": "arn:aws:states:::elasticmapreduce:addStep.sync",
"Parameters": {
"ClusterId.$": "$.ClusterId",
"Step": {
"Name": "The first step",
"ActionOnFailure": "CONTINUE",
"HadoopJarStep": {
"Jar": "command-runner.jar",
"Args": [
"hive-script",
"--run-hive-script",
"--args",
"-f",
"s3://<region>.elasticmapreduce.samples/cloudfront/code/Hive_CloudFront.q",
"-d",
"INPUT=s3://<region>.elasticmapreduce.samples",
"-d",
"OUTPUT=s3://<mybucket>/MyHiveQueryResults/$.dayid"
]
}
}
},
"End": true
}
Parameters allow you to define key-value pairs, so as the value for the "Args" key is an array, you won't be able to dynamically reference a specific element in the array, you would need to reference the whole array instead. For example "Args.$": "$.Input.ArgsArray". With that said, you also won't be able to reference substitute a value inside a string like you are trying to do in "OUTPUT=s3:///MyHiveQueryResults/$.dayid"
So for your use-case the best way to achieve this would be to add a pre-processing state, before calling this state. In the pre-processing state I would recommend you call a Lambda function to construct the string "OUTPUT=s3:///MyHiveQueryResults/$.dayid" as well as the full Array you send to Args.
{
"StartAt": "Pre-Process",
"States": {
"Pre-Process": {
"Type": "Task",
"Resource": "<Lambda function to generate the string OUTPUT=s3://<mybucket>/MyHiveQueryResults/$.dayid and output the Args array>",
"Next": "Step_One"
},
"Step_One": {
"Type": "Task",
"Resource": "arn:aws:states:::elasticmapreduce:addStep.sync",
"Parameters": {
"ClusterId.$": "$.ClusterId",
"Step": {
"Name": "The first step",
"ActionOnFailure": "CONTINUE",
"HadoopJarStep": {
"Jar": "command-runner.jar",
"Args.$": "$.ArgsGeneratedByPreProcessingState"
}
}
},
"End": true
}
}
}
Step Functions now has intrinsic functions that can help this situation.
"PayloadString.$": "States.Format('[[{}]]', States.JsonToString($.in.summary))",
"CmdLine.$": "States.Array('--maxp', $.params.maxpr, '--minp', $.params.minpr)"
Can't believe it took this long for these functions to become available.
See documentation

GraphJSON serialization in Gremlin.Net

I'm trying to query the TinkerPop server (hosted inside docker container) via CosmosDB client library, which uses under the hood Gremlin.Net. So I managed to connect it and insert the data, here's intercepted WebSocket request:
!application/vnd.gremlin-v1.0+json{
"requestId": "b64bd2eb-46c3-4095-9eef-768bca2a14ed",
"op": "eval",
"processor": "",
"args": {
"gremlin": "g.addV(\"User\").property(\"UserId\",2).property(\"CustomerId\",1)"
}
}
The response:
{
"requestId": "b64bd2eb-46c3-4095-9eef-768bca2a14ed",
"status": {
"message": "",
"code": 200,
"attributes": {
"host": "/172.19.0.1:38848"
}
},
"result": {
"data": [
{
"id": 0,
"label": "User",
"type": "vertex",
"properties": {}
}
],
"meta": {}
}
}
Problem is that I see those properties when I'm connected via gremlin console
gremlin> g.V().hasLabel("User").has("CustomerId",1).has("UserId",2).limit(1).valueMap()
==>{UserId=[2], CustomerId=[1]}
Also, I'm able to query the TinkerPop server with Gremlin.Net:
!application/vnd.gremlin-v1.0+json{
"requestId": "de35909f-4bc1-4aae-aa5f-28361b3c0933",
"op": "eval",
"processor": "",
"args": {
"gremlin": "g.V().hasLabel(\"User\").has(\"CustomerId\",1).has(\"UserId\",2).limit(1)"
}
}
But it returns a payload with zero-valued ID and without any properties included:
{
"requestId": "de35909f-4bc1-4aae-aa5f-28361b3c0933",
"status": {
"message": "",
"code": 200,
"attributes": {
"host": "/172.19.0.1:38858"
}
},
"result": {
"data": [
{
"id": 0,
"label": "User",
"type": "vertex",
"properties": {}
}
],
"meta": {}
}
}
Tried to swap between GraphSON v1, v2, v3 with no luck. Documentation says that script serializers should include all the properties. Do I have to tweak the config somehow to make this work and return properties?
So it seems that with a version of 3.4 of the Gremlin server ReferenceElementStrategy
was added by default to traversals, to preserve compatibility between binary and script serializers. In our case we wanted to mimic the behavior of the CosmosDB, so to adjust and receive desired behavior just remove the strategy from init script (in our case it was empty-sample.groovy
globals << [g : graph.traversal().withStrategies(ReferenceElementStrategy.instance())]
to
globals << [g : graph.traversal()]

AWS API Gateway: How to combine multiple Method Request params into a single Integration Request param

I'd like to use API Gateway as a proxy to S3. The bucket is keyed by a composite key made up of two parts like this: [userId]-[documentId].
UserId comes to API Gateway as a path parameter, documentId comes in as a request parameter, for example: [gateway-url]/user1?documentId=doc1
How can I combine the two so that the s3 lookup URL has the following format: https://[bucket-url]/user1-doc1?
Thank you.
Setup your Method Request to accept the path param {userid} and query param {docid}.
Setup your Integration Request to accept both method.request.querystring.docid and method.request.path.userid as URL path params.
Finally, setup your integration endpoint URL as https://your-url/{userid}-{docid}.
A swagger snippet for this is as follows-
"paths": {
"/concat-params/{userid}": {
"get": {
"parameters": [
{
"name": "userid",
"in": "path",
"required": true,
"type": "string"
},
{
"name": "docid",
"in": "query",
"required": false,
"type": "string"
}
],
"responses": {...},
"x-amazon-apigateway-integration": {
"responses": {...},
"requestParameters": {
"integration.request.path.userid":"method.request.path.userid",
"integration.request.path.docid":"method.request.querystring.docid"
},
"uri": "https:.../{userid}-{docid}",
...
}
}
}
Hope this helps,
Ritisha.