Write putObject to S3 directly from HTTP API in API Gateway - amazon-s3

My intention is to create an HTTP API on Amazon API Gateway that writes a file to S3 using the PutObject action via the S3 API (without calling Lambda in between). This is the PutObject request syntax: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_RequestSyntax
I'm not sure if this is technically possible and I'm getting a 403 Forbidden: ForbiddenException response in Postman.
So far I have:
Created the S3 bucket (with CORS configured)
Created the HTTP API in API Gateway (with CORS configured), with a 'putObject' POST action
Configured an integration on the HTTP API to https://[s3-bucket-name].s3.us-east-1.amazonaws.com
Create a Postman request to the HTTP API 'invoke URL', with 'Host' and 'x-apigw-api-id' set on the headers
The ForbiddenException obviously indicates a permission issue, either on the HTTP API or the S3 API behind it. I did configure a Cloudwatch Log Group on the HTTP API, which is showing no entries, so it seems that it's an HTTP API access issue.
I also suspect that I need to add Parameter Mappings to the HTTP API to pass in all of the necessary headers to the S3 putObject action.
My questions are:
Is this type of HTTP API integration direct to S3 possible?
What is the likely cause of the 403 Forbidden response from the service?
Would I use 'Append' Parameter Mappings in the HTTP API integration configuration to add the standard S3 API parameters (and avoid exposing them to the client)?

I managed to solve this myself. Answers to my own questions:
Is this type of HTTP API integration direct to S3 possible?
Yes. On my HTTP API I used an HTTP PUT integration that points to the S3 service endpoint (including the bucket name in the endpoint is incorrect).
What is the likely cause of the 403 Forbidden response from the service?
I didn't get the request working from Postman, however, when I made the request from the browser it worked. I had to create a Blob in Javascript before sending it as a request via navigator.beacon() to the HTTP API endpoint URL.
Would I use 'Append' Parameter Mappings in the HTTP API integration configuration to add the standard S3 API parameters (and avoid exposing them to the client)?
I did have to use Parameter Mappings to get the S3 PutObject request to work from API Gateway. My configuration is shown below.
Screenshot of Parameter Mapping configuration in my HTTP API
Edit: I have discovered a problem here with this approach: the HTTP API doesn't allow certain security-related headers to be added on the Parameter Mappings. I was trying to set header.x-amz-acl: 'bucket-owner-full-control' but I got the error message below:
Invalid mapping expression specified: Validation Result: warnings : [], errors : [Operations on header x-amz-acl are restricted]
It seems that modifying any security-related S3 API header isn't possible in the HTTP API. This is a major problem for calling the S3 API directly as it means that in order to function, the S3 bucket needs to be public.

Related

Internal Server Error - S3 integration with API Gateway via POST method

I'm trying to upload a file into the S3 bucket via AWS API Gateway integration with S3.
I have created an API gateway and integrated it with Amazon S3. I have created both PUT and POST methods. When trying to upload via POST method, I observe internal server error. It works well via PUT method. But I have requirment only for POST method.
I have attached both my Gateway configuration API Gateway configuration and Test results via postman postman test results.

How to block API call if not from my website?

I have an application built around AWS:
a lambda function
an API gateway calling the lambda, must be called with an API key
an S3 bucket as static website, that calls the API gateway
How can I secure the calls to the API gateway so that it cannot be called from anywhere but my S3 bucket ?
Some solutions have already come in my mind like:
proxy : helps hiding the API key, but anyone accessing the proxy can call the API, right ?
IP whitelisting : I can't know the IP range the bucket is using, so I can't do that
Thanks

AWS API Gateway make single endpoint publicly available

I have successfully set up multiple API Gateways on AWS and they work perfectly with client API Keys.
Is it possible to make a specific endpoint in an API publicly available, but all other endpoints protected with the client API key required?
Simply set the apiKeyRequired field to false on whichever Methods you want to open to the public.
This is in the Method Request page in the console. Here is an example using the AWS CLI:
aws apigateway update-method --rest-api-id 1234123412 --resource-id a1b2c3 --http-method GET --patch-operations op="replace",path="/apiKeyRequired",value="false"
Please note that if you remove the requirement for an API Key, any rate limit or quota you have set up on a Usage Plan will not be applied.
You can use AWS CloudFront to forward the request to API Gateway for the public endpoint and define the Client API Key in CloudFront headers forwarded to the Origin.
Note: When including API Gateway as a origin to CloudFront you need to do the following.
Whitelist header (Except Host Header)
Make HTTPS only
Make TTL values 0

AWS API Gateway CORS issue

i've made an api gateway with a lambda function, the api works fine over rest clients, and over browser.
when i created a api key and made it obligatory for the post request, it works fine over rest client but no more over the browser, it sais
No 'Access-Control-Allow-Origin' header
this by knowing that all CORS are activated, and "X-Api-Key" is in the cors headers
Enable CORS in AWS API Gateway
from API Gateway choose the API resource or method and click on the Actions button, it will display a list of options. Select Enable CORS options from the options and deploy the API by choosing Deploy API option. It will enable CORS to the selected Resource or Method.
note: I don't have enough reputation to embed the image in answer that's why the image link is provided.

What is an Endpoint?

I have been reading about OAuth and it keeps talking about endpoints. What is exactly an endpoint?
Come on guys :) We could do it simpler, by examples:
/this-is-an-endpoint
/another/endpoint
/some/other/endpoint
/login
/accounts
/cart/items
and when put under a domain, it would look like:
https://example.com/this-is-an-endpoint
https://example.com/another/endpoint
https://example.com/some/other/endpoint
https://example.com/login
https://example.com/accounts
https://example.com/cart/items
Can be either http or https, we use https in the example.
Also endpoint can be different for different HTTP methods, for example:
GET /item/{id}
PUT /item/{id}
would be two different endpoints - one for retrieving (as in "cRud" abbreviation), and the other for updating (as in "crUd")
And that's all, really that simple!
All of the answers posted so far are correct, an endpoint is simply one end of a communication channel. In the case of OAuth, there are three endpoints you need to be concerned with:
Temporary Credential Request URI (called the Request Token URL in the OAuth 1.0a community spec). This is a URI that you send a request to in order to obtain an unauthorized Request Token from the server / service provider.
Resource Owner Authorization URI (called the User Authorization URL in the OAuth 1.0a community spec). This is a URI that you direct the user to to authorize a Request Token obtained from the Temporary Credential Request URI.
Token Request URI (called the Access Token URL in the OAuth 1.0a community spec). This is a URI that you send a request to in order to exchange an authorized Request Token for an Access Token which can then be used to obtain access to a Protected Resource.
It's one end of a communication channel, so often this would be represented as the URL of a server or service.
An endpoint is a URL pattern used to communicate with an API.
An endpoint is the 'connection point' of a service, tool, or application accessed over a network. In the world of software, any software application that is running and "listening" for connections uses an endpoint as the "front door." When you want to connect to the application/service/tool to exchange data you connect to its endpoint
Endpoint, in the OpenID authentication lingo, is the URL to which you send (POST) the authentication request.
Excerpts from Google authentication API
To get the Google OpenID endpoint, perform discovery by sending either a GET or HEAD HTTP request to https://www.google.com/accounts/o8/id. When using a GET, we recommend setting the Accept header to "application/xrds+xml". Google returns an XRDS document containing an OpenID provider endpoint URL.The endpoint address is annotated as:
<Service priority="0">
<Type>http://specs.openid.net/auth/2.0/server</Type>
<URI>{Google's login endpoint URI}</URI>
</Service>
Once you've acquired the Google endpoint, you can send authentication requests to it, specifying the appropriate parameters (available at the linked page). You connect to the endpoint by sending a request to the URL or by making an HTTP POST request.
Short answer: "an endpoint is an abstraction that models the end of a message channel through which a system can send or receive messages" (Ibsen, 2010).
Endpoint vs URI (disambiguation)
The endpoint is not the same as a URI. One reason is because a URI can drive to different endpoints like an endpoint to GET, another to POST, and so on. Example:
#GET /api/agents/{agent_id} //Returns data from the agent identified by *agent_id*
#PUT /api/agents/{agent_id} //Update data of the agent identified by *agent_id*
Endpoint vs resource (disambiguation)
The endpoint is not the same as a resource. One reason is because different endpoints can drive to the same resource. Example:
#GET /api/agents/{agent_id} #Produces("application/xml") //Returns data in XML format
#GET /api/agents/{agent_id} #Produces("application/json") //Returns data in JSON format
The term Endpoint was initially used for WCF services. Later even though this word is being used synonymous to API resources, REST recommends to call these URI (URI[s] which understand HTTP verbs and follow REST architecture) as "Resource".
In a nutshell, a Resource or Endpoint is kind of an entry point to a remotely hosted application which lets the users to communicate to it via HTTP protocol.
The endpoint of the term is the URL that is focused on creating a request.
Take a look at the following examples from different points:
/api/groups/6/workings/1
/api/v2/groups/5/workings/2
/api/workings/3
They can clearly access the same source in a given API.
API stands for Application Programming Interface. It is a way for your application to interact with other applications via an endpoint. Conversely, you can build out an API for your application that is available for other developers to utilize/connect to via HTTP methods, which are RESTful. Representational State Transfer (REST):
GET: Retrieve data from an API endpoint.
PUT: Update data via an API - similar to POST but more about updating info.
POST: Send data to an API.
DELETE: Remove data from given API.
PATCH: Update data.