I am trying to upload a file to aws s3 and there is a proxy server in between and the proxy servers uses a token.
I am requested to send the credentials on header as “Proxy-Authorization”. Here is the code I am using.
com.amazonaws.ClientConfiguration config = new ClientConfiguration();
config.addHeader(“Proxy-Authorization”, “Basic $$$$$$$$$“);
config.setProtocol(Protocol.HTTPS);
config.setProxyHost(proxyHost);
config.setProxyPort(proxyPort);
BasicAWSCredentials awsCreds = new BasicAWSCredentials(accessKey, secret);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withClientConfiguration(config)
.withCredentials(new
AWSStaticCredentialsProvider(awsCreds)).withRegion(Regions.US_EAST_1).build();
s3Client.putObject(name,key,file)
I am getting HTTP 403.Forbidden error, when we try to upload to s3 (using s3Client putObject )
com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: null; S3 Extended Request ID: null;
I tried the same using a curl command with -H "Proxy-Authorization:Basic $$$$" and it worked fine. Appreciate any help on how can I pass the header in the request from a java code.
Related
I am using S3 transfer manager V2 to upload a file to AWS S3 bucket and its failing with following exception.
software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: Failed to write to TLS handler
Following is the code snippet
StaticCredentialsProvider credentials = StaticCredentialsProvider.create(AWSSessionCredentials.create(access_key_id, secret_access_key);
S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder().credentialsProvider(credentials).region(region).build();
S3TransferManager s3tm = S3TransferManager.builder().s3Client(s3AsyncClient).build();
UploadFileRequest uploadFileRequest = UploadFileRequest.builder().putObjectRequest(req -> req.bucket("bucket").key("key")).source(Paths.get("/file_to_upload.txt")).build();
FileUpload upload = transferManager.uploadFile(uploadFileRequest);
upload.completionFuture().join();
What am I missing here? Please advise
I am trying to use minio as a local Amazon S3 server. I started minio server on my computer, created a test bucket, and uploaded one file - Completed.jpg. Now, I have this file in the minio and I can download it via link http://localhost:9000/minio/testbucket/Completed.jpg. But when I try to read this file from java, I get an exception. I wrote this test:
#Test
public void readObject() {
ClientConfiguration clientConf = PredefinedClientConfigurations.defaultConfig().withProtocol(Protocol.HTTPS).withMaxErrorRetry(1);
BasicAWSCredentials awsCredentials = new BasicAWSCredentials("minioadmin", "minioadmin");
AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard()
.withClientConfiguration(clientConf)
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("http://127.0.0.1:9000/minio", "us-east-1"));
AmazonS3 amazonS3 = builder.build();
S3Object object = amazonS3.getObject(new GetObjectRequest("testbucket", "Completed.jpg"));
assertNotNull(object);
}
And It is the exception:
com.amazonaws.services.s3.model.AmazonS3Exception: All access to this bucket has been disabled. (Service: Amazon S3; Status Code: 403; Error Code: AllAccessDisabled; Request ID: /minio/testbucket/Completed.jpg; S3 Extended Request ID: 4a46a947-6473-4d53-bbb3-a4f908d444ce)
, S3 Extended Request ID: 4a46a947-6473-4d53-bbb3-a4f908d444ce
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1383)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1359)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5052)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4998)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1486)
I'm setting up a vuejs / DropzoneJS - app loosely based on kfei's vue-s3-dropzone app. It's designed to upload files (by using a PUT method) to AWS S3 serverlessly using a AWS Lambda function and a AWS S3 bucket.I'm basically getting a XMLHttpRequest at 'https://xxxxxxxxxxxxxxxxxxx' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status and a 403 error code when I try to upload an image to the S3 bucket. Is there anything I can do to
This is what I've did:
created an S3 bucket
Set up a bucket policy and a CORS configuration in the S3 bucket settings:
enter image description here
enter image description here
Created a lambda function that is supposed to sign a URL which allows a PUT upload for each file to S3, with the Role executing the Lambda having a PutObject and PutObjectAcl permission on the S3 bucket:
enter image description here
Set up an Api Gateway API with an OPTIONS method (to pass the preflight check) and a PUT method with these CORS settings:
b. The OPTIONS method has a Mock backend integration with the Integration Response returning the following:
Access-Control-Allow-Headers 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,x-requested-with'
Access-Control-Allow-Methods ‘PUT,OPTIONS'
Access-Control-Allow-Origin '*'
c. The PUT method has:
"Access-Control-Allow-Origin": "*"
In AWS Api Gateway: Setup a api-key and a usage plan
The lambda code:
var AWS = require('aws-sdk');
var s3 = new AWS.S3();
var bucketName = process.env.AWS_BUCKET_NAME;
exports.handler = (event, context) => {
if (!event.hasOwnProperty('contentType')) {
context.fail({ err: 'Missing contentType' });
}
if (!event.hasOwnProperty('filePath')) {
context.fail({ err: 'Missing filePath' });
}
var params = {
Bucket: bucketName,
Key: event.filePath,
Expires: 3600,
ContentType: event.contentType
};
s3.getSignedUrl('putObject', params, (err, url) => {
if (err) {
context.fail({ err });
} else {
context.succeed({ url });
}
});
};
Expected: Successful upload of files
Actual: Possible CORS issues.
getSignedUrl will work fine if you were uploading the file from an API client like Postman or a Node.js server, but as you state you are seeing a preflight check fail, I'm assuming you are using some kind of HTML form & frontend js.
From the AWS JavaScript SDK Docs regarding getSignedUrl:
Note: Not all operation parameters are supported when using pre-signed
URLs. Certain parameters, such as SSECustomerKey, ACL, Expires,
ContentLength, or Tagging must be provided as headers when sending a
request. If you are using pre-signed URLs to upload from a browser and
need to use these fields, see createPresignedPost().
As you are setting the 'Expires' param when calling getSignedUrl and are sending from the browser, you need to use createPresignedPost instead of getSignedUrl in your Lambda code.
You will then need to POST instead of PUT from the browser to S3.
NB: Remember to update your CORS rules for S3 with POST
I am getting a 405 method not allowed while trying to retrieve metadata of older versions of an S3 object using a Lambda Java function.
AmazonS3 amazonS3 = getAmazonS3();
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, templateKey, versionId);
ObjectMetadata objectMetadata = amazonS3.getObject(getObjectRequest).getObjectMetadata(); // Exception thrown at this line
public AmazonS3 getAmazonS3() {
String region = PropertyManager.getValue(PropertyKey.AWS_REGION.getKey(stage));
return AmazonS3ClientBuilder.standard().withRegion(region)
.withCredentials(new EnvironmentVariableCredentialsProvider()).build();
}
Stack trace in Lambda:
The specified method is not allowed against this resource. (Service: Amazon S3; Status Code: 405; Error Code: MethodNotAllowed; Request ID: 1D12DDA5F0493282): com.amazonaws.services.s3.model.AmazonS3Exception
com.amazonaws.services.s3.model.AmazonS3Exception: The specified method is not allowed against this resource. (Service: Amazon S3; Status Code: 405; Error Code: MethodNotAllowed; Request ID: 1D12DDA5F0493282), S3 Extended Request ID: jTNnAl8ifgsUlPMV0GEHAEVBtWwjTprEJy45C9BMJ5kTk/Qn8Pne8/ZM/tH27ZoeUtHrd1NeuyQ=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4187)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4134)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1385)
at com.ghx.templateengine.template.GetTemplateVersions.handleRequest(GetTemplateVersions.java:66)
A few of the older versions of the S3 object had delete marker. AWS support conveyed that trying to head object a version that is a delete marker results in a 405 error.
I'm trying to upload a file to Amazon bucket using S3 service. Getting below exception when connecting through company's "proxy server".
<code>
AWSCredentials credentials = new BasicSessionCredentials(
uploadLocation.getAccessKeyId(), uploadLocation.getSecretAccessKey(), uploadLocation.getSessionToken());
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setProxyHost(proxyAddress);
clientConfiguration.setProxyPort(Integer.parseInt(proxyPort));
clientConfiguration.setProtocol(Protocol.HTTP);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withClientConfiguration(clientConfiguration)
.withRegion("us-east-1")
.build();
PutObjectResult putObjectResult = s3Client.putObject(new PutObjectRequest(uploadLocation.getBucket(),
uploadLocation.getObject_key(), file));
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: null; S3 Extended Request ID: null), S3 Extended Request ID: null
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1592)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1257)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1029)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:741)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:715)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:697)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:665)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:647)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:511)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4227)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4174)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1722)
</code>
Please note that it works perfectly when proxy is not used and hence ClientConfiguration object is not required.
Don't have much visibility into S3 bucket configuration as a different vendor company is using Amazon S3 as their storage system.
Any suggestions, please?