I am trying to use minio as a local Amazon S3 server. I started minio server on my computer, created a test bucket, and uploaded one file - Completed.jpg. Now, I have this file in the minio and I can download it via link http://localhost:9000/minio/testbucket/Completed.jpg. But when I try to read this file from java, I get an exception. I wrote this test:
#Test
public void readObject() {
ClientConfiguration clientConf = PredefinedClientConfigurations.defaultConfig().withProtocol(Protocol.HTTPS).withMaxErrorRetry(1);
BasicAWSCredentials awsCredentials = new BasicAWSCredentials("minioadmin", "minioadmin");
AmazonS3ClientBuilder builder = AmazonS3ClientBuilder.standard()
.withClientConfiguration(clientConf)
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("http://127.0.0.1:9000/minio", "us-east-1"));
AmazonS3 amazonS3 = builder.build();
S3Object object = amazonS3.getObject(new GetObjectRequest("testbucket", "Completed.jpg"));
assertNotNull(object);
}
And It is the exception:
com.amazonaws.services.s3.model.AmazonS3Exception: All access to this bucket has been disabled. (Service: Amazon S3; Status Code: 403; Error Code: AllAccessDisabled; Request ID: /minio/testbucket/Completed.jpg; S3 Extended Request ID: 4a46a947-6473-4d53-bbb3-a4f908d444ce)
, S3 Extended Request ID: 4a46a947-6473-4d53-bbb3-a4f908d444ce
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1383)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1359)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5052)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4998)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1486)
Related
I am using S3 transfer manager V2 to upload a file to AWS S3 bucket and its failing with following exception.
software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: Failed to write to TLS handler
Following is the code snippet
StaticCredentialsProvider credentials = StaticCredentialsProvider.create(AWSSessionCredentials.create(access_key_id, secret_access_key);
S3AsyncClient s3AsyncClient = S3AsyncClient.crtBuilder().credentialsProvider(credentials).region(region).build();
S3TransferManager s3tm = S3TransferManager.builder().s3Client(s3AsyncClient).build();
UploadFileRequest uploadFileRequest = UploadFileRequest.builder().putObjectRequest(req -> req.bucket("bucket").key("key")).source(Paths.get("/file_to_upload.txt")).build();
FileUpload upload = transferManager.uploadFile(uploadFileRequest);
upload.completionFuture().join();
What am I missing here? Please advise
I am trying to upload a file to aws s3 and there is a proxy server in between and the proxy servers uses a token.
I am requested to send the credentials on header as “Proxy-Authorization”. Here is the code I am using.
com.amazonaws.ClientConfiguration config = new ClientConfiguration();
config.addHeader(“Proxy-Authorization”, “Basic $$$$$$$$$“);
config.setProtocol(Protocol.HTTPS);
config.setProxyHost(proxyHost);
config.setProxyPort(proxyPort);
BasicAWSCredentials awsCreds = new BasicAWSCredentials(accessKey, secret);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withClientConfiguration(config)
.withCredentials(new
AWSStaticCredentialsProvider(awsCreds)).withRegion(Regions.US_EAST_1).build();
s3Client.putObject(name,key,file)
I am getting HTTP 403.Forbidden error, when we try to upload to s3 (using s3Client putObject )
com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: null; S3 Extended Request ID: null;
I tried the same using a curl command with -H "Proxy-Authorization:Basic $$$$" and it worked fine. Appreciate any help on how can I pass the header in the request from a java code.
I am getting a 405 method not allowed while trying to retrieve metadata of older versions of an S3 object using a Lambda Java function.
AmazonS3 amazonS3 = getAmazonS3();
GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, templateKey, versionId);
ObjectMetadata objectMetadata = amazonS3.getObject(getObjectRequest).getObjectMetadata(); // Exception thrown at this line
public AmazonS3 getAmazonS3() {
String region = PropertyManager.getValue(PropertyKey.AWS_REGION.getKey(stage));
return AmazonS3ClientBuilder.standard().withRegion(region)
.withCredentials(new EnvironmentVariableCredentialsProvider()).build();
}
Stack trace in Lambda:
The specified method is not allowed against this resource. (Service: Amazon S3; Status Code: 405; Error Code: MethodNotAllowed; Request ID: 1D12DDA5F0493282): com.amazonaws.services.s3.model.AmazonS3Exception
com.amazonaws.services.s3.model.AmazonS3Exception: The specified method is not allowed against this resource. (Service: Amazon S3; Status Code: 405; Error Code: MethodNotAllowed; Request ID: 1D12DDA5F0493282), S3 Extended Request ID: jTNnAl8ifgsUlPMV0GEHAEVBtWwjTprEJy45C9BMJ5kTk/Qn8Pne8/ZM/tH27ZoeUtHrd1NeuyQ=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4187)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4134)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1385)
at com.ghx.templateengine.template.GetTemplateVersions.handleRequest(GetTemplateVersions.java:66)
A few of the older versions of the S3 object had delete marker. AWS support conveyed that trying to head object a version that is a delete marker results in a 405 error.
I'm trying to upload a file to Amazon bucket using S3 service. Getting below exception when connecting through company's "proxy server".
<code>
AWSCredentials credentials = new BasicSessionCredentials(
uploadLocation.getAccessKeyId(), uploadLocation.getSecretAccessKey(), uploadLocation.getSessionToken());
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setProxyHost(proxyAddress);
clientConfiguration.setProxyPort(Integer.parseInt(proxyPort));
clientConfiguration.setProtocol(Protocol.HTTP);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withClientConfiguration(clientConfiguration)
.withRegion("us-east-1")
.build();
PutObjectResult putObjectResult = s3Client.putObject(new PutObjectRequest(uploadLocation.getBucket(),
uploadLocation.getObject_key(), file));
Exception in thread "main" com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: null; S3 Extended Request ID: null), S3 Extended Request ID: null
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1592)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1257)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1029)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:741)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:715)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:697)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:665)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:647)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:511)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4227)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4174)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1722)
</code>
Please note that it works perfectly when proxy is not used and hence ClientConfiguration object is not required.
Don't have much visibility into S3 bucket configuration as a different vendor company is using Amazon S3 as their storage system.
Any suggestions, please?
Using Java AWS SDK I've created a lambda function to read a csv file from an s3 bucket. I've made the bucket public and can access it and the file easily from any browser.
To test it, I'm using the test button on the lambda console. I'm just using the hello world test config input template.
It fails with:
Error Message: The specified bucket is not valid. (Service: Amazon S3; Status Code: 400; Error Code: InvalidBucketName; Request ID: XXXXXXXXXXXXXXX)
Lambda function and s3 bucket are in the same region (us-east-1).
I've added AmazonS3FullAccess to lambda_basic_execution role.
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().build();
also tried
AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();
then the call
S3Object s3object = s3Client.getObject(new GetObjectRequest(
bucketName, keyName));
bucketName is:
https://s3.amazonaws.com/<allAlphaLowerCaseBucketName>
keyName is:
<allAlphaLowerCaseKeyName>.csv
Any help is appreciated.
The bucket name is not the URL to the bucket, but only the actual name of your bucket.
S3Object s3object = s3Client.getObject(
new GetObjectRequest(
<allAlphaLowerCaseBucketName>,
<allAlphaLowerCaseKeyName>.csv
)
);