Upload File to Amazon S3 - amazon-s3

Question

Let me explain my requirement..
I want to store files uploaded to my app (by clients) to Amazon S3..
File Size ~ 1-10 MB
However, the client interface has to be a REST API
provided by my application. Consequently, after parsing file upload (HTTP POST) request, my application must store the file in S3.
As a result, I have to store file temporarily on disk before uploading to S3..
Is there a workaround? Can I do away with temporary file store on my server.. Please let me know if I am not clear..
EDIT - Is it OK to get byte array from FileItem object and store it rather than the file itself..?

Answer 1

Your whole idea is to avoid I/O right ? you don't need to save the file before doing the upload, you could simple send the array of bytes to amazon REST API.
Here is my sample VB.NET code that do both upload and download:
Imports System.Collections.Generic
Imports System.IO
Imports System.Linq
Imports System.Net
Imports System.Security.Cryptography
Imports System.Text
Imports System.Threading.Tasks
Module Module1
Sub Main()
Dim obj As New Program
obj.UploadFile()
'obj.DownloadFile() 'Download Example
End Sub
End Module
Class Program
Private Const KeyId As String = "yourkey"
Private Const AccessKey As String = "your/access"
Private Const S3Url As String = "https://s3.amazonaws.com/"
Public Sub DownloadFile()
Dim bucketName As String = "yourbucket"
Dim FileName As String = "file.png"
Dim timeStamp As String = String.Format("{0:r}", DateTime.UtcNow)
Dim stringToConvert As String = Convert.ToString((Convert.ToString((Convert.ToString("GET" & vbLf + vbLf + vbLf + vbLf + "x-amz-date:") & timeStamp) + vbLf + "/") & bucketName) + "/") & FileName
Dim ae = New UTF8Encoding()
Dim signature = New HMACSHA1() With { _
.Key = ae.GetBytes(AccessKey) _
}
Dim bytes = ae.GetBytes(stringToConvert)
Dim moreBytes = signature.ComputeHash(bytes)
Dim encodedCanonical = Convert.ToBase64String(moreBytes)
' Send the request
Dim request As HttpWebRequest = DirectCast(HttpWebRequest.Create(Convert.ToString((Convert.ToString("https://") & bucketName) + ".s3.amazonaws.com" + "/") & FileName), HttpWebRequest)
'request.ContentType = "application/octet-stream";
request.Headers.Add("x-amz-date", timeStamp)
request.Headers.Add("Authorization", "AWS " + KeyId + ":" + encodedCanonical)
request.Method = "GET"
' Get the response
Dim response As HttpWebResponse = DirectCast(request.GetResponse(), HttpWebResponse)
Dim ReceiveStream As Stream = response.GetResponseStream()
Console.WriteLine(response.StatusCode)
End Sub
Public Sub UploadFile()
Dim fileData = File.ReadAllBytes("C:\file.png")
Dim timeStamp As String = String.Format("{0:r}", DateTime.UtcNow)
Dim stringToConvert As String = (Convert.ToString("PUT" & vbLf + vbLf + "application/octet-stream" & vbLf + vbLf + "x-amz-acl:public-read" + vbLf + "x-amz-date:") & timeStamp) + vbLf + "/celso711/file.png"
'resource
Dim ae = New UTF8Encoding()
Dim signature = New HMACSHA1() With { _
.Key = ae.GetBytes(AccessKey) _
}
Dim bytes = ae.GetBytes(stringToConvert)
Dim moreBytes = signature.ComputeHash(bytes)
Dim encodedCanonical = Convert.ToBase64String(moreBytes)
Dim url = "https://bucket.s3.amazonaws.com/file.png"
Dim request = TryCast(WebRequest.Create(url), HttpWebRequest)
request.Method = "PUT"
request.Headers("x-amz-date") = timeStamp
request.Headers("x-amz-acl") = "public-read"
request.ContentType = "application/octet-stream"
request.ContentLength = fileData.Length
request.Headers("Authorization") = (Convert.ToString("AWS ") & KeyId) + ":" + encodedCanonical
Dim requestStream = request.GetRequestStream()
requestStream.Write(fileData, 0, fileData.Length)
requestStream.Close()
Using response = TryCast(request.GetResponse(), HttpWebResponse)
Dim reader = New StreamReader(response.GetResponseStream())
Dim data = reader.ReadToEnd()
End Using
End Sub
End Class