ASP.Net Core - EC2 to S3 file upload with Access Denied - asp.net-core

Question

I have developed a .NET Core 3.1 Web API which allows the users to upload their documents to S3 bucket. When I deploy the API to AWS ElasticBeansTalk EC2 instance and call the endpoint which uploads the file to S3, I get an error "Access Denied".
By the way, I have created IAM policy and role to give full access to S3 from my EC2 instance. I have also copied the .aws folder which contains credentials file onto the EC2 instance.
API Controller Action
public async Task<ApiResponse> UpdateProfilePic([FromBody]UploadProfilePicRequest model)
{
using (Stream stream = model.profilePicData.Base64StringToStream(out string header))
{
var tags = new List<KeyValuePair<string, string>>();
var metaData = new List<KeyValuePair<string, string>>();
metaData.Add(new KeyValuePair<string, string>("Content-Disposition", $"attachment; filename=\"{model.filename}\""));
if (_host.IsDevelopment())
{
tags.Add(new KeyValuePair<string, string>("public", "yes"));
}
await AmazonS3Uploader.UploadFileAsync(stream, "myDir/", model.fileId, tags, metaData);
}
}
The AmazonS3Helper class shown below:
using Amazon;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Transfer;
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
namespace UploderApp.Services
{
public static class AmazonS3Uploader
{
private static readonly RegionEndpoint bucketRegion = RegionEndpoint.APSouth1;
private static readonly IAmazonS3 s3Client = new AmazonS3Client(GetAwsCredentials(), bucketRegion);
private static readonly string S3Bucket = "abc-test";
private static AWSCredentials GetAwsCredentials()
{
var chain = new CredentialProfileStoreChain();
if (chain.TryGetAWSCredentials("MYPROFILE", out AWSCredentials awsCredentials))
{
return awsCredentials;
}
return null;
}
public static async Task UploadFileAsync(Stream fileStream, string virtualDirectory, string keyName)
{
try
{
using (var fileTransferUtility = new TransferUtility(s3Client))
{
//Upload data from a type of System.IO.Stream.
await fileTransferUtility.UploadAsync(fileStream, S3Bucket, virtualDirectory + keyName).ConfigureAwait(true);
}
}
catch (AmazonS3Exception e)
{
throw new Exception($"Error encountered on server. Message:'{e.Message}' when writing an object");
}
}
public static async Task UploadFileAsync(Stream stream, string virtualDirectory, string keyName, List<KeyValuePair<string, string>> tags = null, List<KeyValuePair<string, string>> metadata = null)
{
try
{
// Specify advanced settings.
var fileTransferUtilityRequest = new TransferUtilityUploadRequest
{
BucketName = S3Bucket,
InputStream = stream,
StorageClass = S3StorageClass.Standard,
Key = virtualDirectory + keyName
};
if (metadata != null)
{
foreach (var item in metadata)
{
fileTransferUtilityRequest.Metadata.Add(item.Key, item.Value);
}
}
if (tags != null)
{
fileTransferUtilityRequest.TagSet = new List<Tag>();
foreach (var tag in tags)
{
fileTransferUtilityRequest.TagSet.Add(new Tag { Key = tag.Key, Value = tag.Value });
}
}
using (var fileTransferUtility = new TransferUtility(s3Client))
{
await fileTransferUtility.UploadAsync(fileTransferUtilityRequest).ConfigureAwait(true);
}
}
catch (AmazonS3Exception e)
{
throw new Exception($"Error encountered on server. Message:'{e.Message}' when writing an object");
}
}
}
}
However, if I create a console application and use the above class without any modifications, it uploads the file from the same EC2 instance.
Code from the Main function of my Console Application.
public static void Main()
{
var file = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + "/Screenshot.png";
try
{
var tags = new List<KeyValuePair<string, string>>();
var metaData = new List<KeyValuePair<string, string>>();
metaData.Add(new KeyValuePair<string, string>("Content-Disposition", $"attachment; filename=\"profile-pic.png\""));
using (var stream = new FileStream(file, FileMode.Open))
{
AmazonS3Uploader.UploadFileAsync(stream, "mydir/", "screenshot.png", tags, metaData).GetAwaiter().GetResult();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
This is very strange. Can anybody help me to understand the root cause, please?
Edit:1
Output of the aws s3 ls s3://abc-test is shown below
Edit:2
Uploading the EC2 folder to S3