save jpg image in s3 with signed URL - amazon-s3

I am using a signed URL returned by AWS Lambda to upload an jpg image with Angular HTTPClient to S3 bucket. I can see the image.jpg file but when I open it, it says it is invalid format. Please help!
I am expecting I need these HTTP request headers:
'Content-Type': 'image/jpeg'
'Content-Encoding': 'base64'
and I tried with and without the
"data:image/jpeg;base64,"
followed by the based64 encoded data as string.
My lambda is:
var AWS = require('aws-sdk');
var s3 = new AWS.S3({
signatureVersion: 'v4',
});
exports.handler = (event, context, callback) => {
const url = s3.getSignedUrl('putObject', {
Bucket: 'landlord-bucket',
Key: 'image' + '.jpg',
Expires: 20,
ContentEncoding: 'base64',
ContentType: 'image/jpeg',
});
callback(null, url);
};
I upload it using:
#Effect() postToS3$: Observable<Action> = this.actions$
.ofType(PropertyActions.UPLOAD_FILE_PUTS3)
.switchMap((action: PropertyActions.UploadFilePutS3) => {
return this.httpClient.put(action.payload, "data:image/jpeg;base64," + this.filesToUpload)
.pipe(map(res => {
return new PropertyActions.OpenAllProperties(res);},
err => {
console.log("Error occured in get signed url");
}))
})
this.filesToUpload is the string of the base64 encoded jpg. This is correctly encoded since I can see the image at any decoder site.
HTTP interceptor is:
#Injectable()
export class AuthInterceptor implements HttpInterceptor {
constructor(private authService: AuthService) {}
intercept (req: HttpRequest<any>, next: HttpHandler): Observable<HttpEvent<any>> {
return this.authService.getAuthenticatedUser().getSession((err, session) => {
if (err) {
console.log("Error getSession")
return next.handle(req);
}
let authReq = null;
if (req.url.indexOf('X-Amz-Algorithm') === -1) {
authReq = req.clone({
headers: req.headers.set('Authorization', session.getIdToken().getJwtToken())
});
} else {
authReq = req.clone({
setHeaders: {'Content-Type': 'image/jpeg', 'Content-Encoding': 'base64'}
});
}
return next.handle(authReq);
})
}
}

Related

uploading files to endpoint from a static webpage

I am trying to upload files to an S3 endpoint from a static HTML page but the files are always malformed when I download them from the bucket. The relevant code is below - what am I doing wrong with fetch?
const onSubmitForm = function (e) {
const file = this.files[0];
const reader = new FileReader();
// reader.readAsText(file); // didn't work
reader.readAsDataURL(file); // also didn't work
reader.onload = async function () {
const bodyData = {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
file: {
// the backend endpoint expects a base64 encoded img
// the upload completes but the
"data": reader.result.toString()
},
"name": file.name
})
}
const response = await fetch(uploadUrl, bodyData).then(
res => res.json()
);
};
reader.onerror = function (error) {
console.log('Error: ', error);
};
//
}

nativescript image upload to s3 encoding issue

I'm having issues uploading an image from Nativescript to AWS, and I'm pretty sure it's a configuration issue.
select an image
const context = imagepicker.create({
mode: 'single' // use "multiple" for multiple selection
});
await context.authorize();
const selection: Array<ImageAsset> = await context.present();
const imageAsset = selection[0];
const source: ImageSource = await new ImageSource().fromAsset(imageAsset);
const fileLocation = imageAsset.android ? imageAsset.android : imageAsset.ios;
const fileType = mime.extension(mime.lookup(fileLocation));
const image = source.toBase64String(fileType);
console.log(image);
image at this point is: iVBORw0KGgoAAAANSUhEUgAAB4AAAASwCAIAAACVUsChAAAAA3NCSVQI...
image location at this point is: /storage/emulated/0/DCIM/Screenshots/Screenshot_20181106-150854.png
const fileLocation = imageAsset.android ? imageAsset.android : imageAsset.ios;
const signedUrl = await this.getSignedUrl(fileLocation);
Backend Code to get signedURL
const getSignedUrlPromise = (operation, params) => {
return new Promise((resolve, reject) => {
s3.getSignedUrl(operation, params, (err, url) => {
err ? reject(err) : resolve(url);
});
});
}
const params = {
Bucket: BUCKET_NAME,
Key: `abc123/456/3/${fileName}`,
ContentType: contentType,
ContentEncoding: 'base64'
}
const url = await getSignedUrlPromise('putObject', params).catch(err => {
console.log('error', JSON.stringify(err))
return {
statusCode: 400,
body: JSON.stringify(err)
}
});
console.log('success', url);
return {
statusCode: 200,
body: JSON.stringify({ url: url })
}
signedUrl at this point is:
https://myproject.s3.amazonaws.com/abc123/456/3/Screenshot_20181106-150854.png?AWSAccessKeyId=xxxxxxxxxxxxxx&Content-Encoding=base64&Content-Type=image%2Fpng&Expires=1555517358&Signature=yyyyyyyy&x-amz-security-token=long_token
Then, using the signedURL, i upload the image:
const mimeType = mime.lookup(fileLocation);
this.http.put(signedUrl, image, {
headers: {
'Content-Type': mimeType,
'Content-Encoding': 'base64'
}
}).subscribe((resp) => {
console.log('resp2', resp);
});
}
When I open the file, this is what I see
and the meta-data on the S3 object looks correct
When I download the file and open it in NP++, I see the base64 value.
iVBORw0KGgoAAAANSUhEUgAAB4AAAASwCAIAAACVUsChAAAAA3NCSVQI...
I also cannot open the downloaded image
ATTEMPT 2
I saw where some people were using buffers, so I changed my image code to
const image = Buffer.from(source.toBase64String(fileType).replace(/^data:image\/\w+;base64,/, ''), 'base64');
which the image is still broken, and when I download and open the file using NP++ I see
{"type":"Buffer","data":[137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,7,128,0,0,4,176,8,2,0,0,0

In Nodejs/Express js i am unable to return response to Angular 6

I am using Angular 6 and Nodejs/Expressjs to avoid cross domain issue. So here is my code
In Angular i am calling:
this.httpClient.post('/uploadFile', formData, {params: params})
.pipe(map((res: any) => {return res}),
catchError((error: HttpErrorResponse) => {})
Nodejs/Expressjs:
app.post('/uploadFile', (req, res) => {
let formData
const form = new IncomingForm();
let readStream;
form.on('file', (field, file) => {
console.log('file.path>>>',file.path);
readStream = fs.createReadStream(file.path);
});
form.on ('fileBegin', function(name, file) {
//rename the incoming file to the file's name
let fileName = file.path.split("\\");
fileName[fileName.length-1] = file.name.split('.')[0];
fileName = fileName.join("\\");
file.path = fileName;
console.log('file.path', file.path);
console.log('file.name', file.name);
})
form.parse(req, function (err, fields, files) {
formData = new FormData();
formData.append("file", readStream);
formData.append('package_name', req.query.packagename);
formData.append('type', req.query.type);
formData.append('version', req.query.version);
formData.append('descr', req.query.descr);
console.log('req.query.packagename',req.query.packagename);
const axiosConfig = {
headers: {
'Content-Type': 'multipart/form-data'
}
};
let uploadRequest = request.post("WebAPiCallURL", requestCallback);
uploadRequest._form = formData;
uploadRequest.setHeader('Content-Type', 'multipart/form-data');
function requestCallback(err, res, body) {
return JSON.parse(body);
}
});
});
From requestCallback i am unable to send response to Angular6
Your not sending response from to the client. To send you can use res.send or any of the response function from Expressjs.
function requestCallback(err, response, body) { //Rename res to response to avoid conflict
res.send(body); // Send Response to Client
}
Note : As you used same variable res for request, Rename to some other name

How to upload images from react native to an express backend?

I followed this example to upload images to my backend. https://github.com/expo/image-upload-example
The code I use in my RN application is:
let formData = new FormData();
formData.append('name',this.state.name);
formData.append('description',this.state.description);
formData.append('price',this.state.price);
formData.append('oprice',this.state.oprice);
let fileArr = (this.state.image).split('.');
let type = fileArr[fileArr.length -1]
let uri = this.state.image
formData.append('photo',{uri, name: `${this.state.name}.${type}`, type: `image/${type}`});
console.log(formData);
AsyncStorage.getItem('jwt', (err, token) => {
fetch('http://192.168.1.83:8000/ShopRouter/deal', {
method: 'POST',
headers: {
Accept: 'application/json',
Authorization: token,
'Content-type': false
},
body: formData
})
The code I use for my express backend is:
app.post('/ShopRouter/deal', passport.authenticate('jwt2', {session:false}), function(req,res) {
upload(req, res, function(err) {
if (err) {
console.log(err);
}
console.log(req.file);
console.log(req.files);
console.log(req.photo);
console.log(req.body.name);
console.log(req.body.description);
My Multer configuration is:
var storage = multer.diskStorage({
destination: function (req, file, cb, err){
console.log(file)
if (err){
console.log(err)
}
cb(null, './uploads/')
},
filename: function (req, file, cb) {
console.log(file)
cb(null, file.originalname + '-' +Date.now())
}
});
var upload = multer({ storage: storage }).single('photo');
The line console.log(file) prints
{ fieldname: 'photo',
originalname: 'An empty bottle .jpg',
encoding: '7bit',
mimetype: 'image/jpeg' }
I;m not sure why the backend receives this without the uri of the image, nothing gets saved in the upload folder
req.file comes back undefined.
I sorted this out by just uploading the images to s3

upload an image to amazon s3 in react-native

I am trying to upload image to amazon s3,If possible can any one provide links /docs for how to upload to amazon s3, any help much appreciated
S3 options:
// this.state.s3options in YourComponent
{
"url": "https://yourapp.s3.eu-central-1.amazonaws.com",
"fields": {
"key": "cache/22d65141b48c5c44eaf93a0f6b0abc30.jpeg",
"policy": "eyJleHBpcm...1VDE0Mzc1OVoifV19",
"x-amz-credential": "AK...25/eu-central-1/s3/aws4_request",
"x-amz-algorithm": "AWS4-HMAC-SHA256",
"x-amz-date": "20161125T143759Z",
"x-amz-signature": "87863c360...b9b304bfe650"
}
}
Component:
class YourComponent extends Component {
// ...
// fileSource looks like: {uri: "content://media/external/images/media/13", isStatic: true}
async uploadFileToS3(fileSource) {
try {
var formData = new FormData();
// Prepare the formData by the S3 options
Object.keys(this.state.s3options.fields).forEach((key) => {
formData.append(key, this.state.s3options.fields[key]);
});
formData.append('file', {
uri: fileSource.uri,
type: 'image/jpeg',
});
formData.append('Content-Type', 'image/jpeg')
var request = new XMLHttpRequest();
request.onload = function(e) {
if (e.target.status === 204) {
// Result in e.target.responseHeaders.Location
this.setState({avatarSourceRemote: {uri: e.target.responseHeaders.Location}})
}
}.bind(this)
request.open('POST', this.state.s3options.url, true);
request.setRequestHeader('Content-type', 'multipart/form-data');
request.send(formData);
} catch(error) {
console.error(error);
}
}
// Example display the uploaded image
render() {
if (this.state.avatarSourceRemote) {
return (
<Image source={this.state.avatarSourceRemote} style={{width: 100, height: 100}} />
);
} else {
return (
<Text>No Image</Text>
);
}
}
}
This works for me
import fs from 'react-native-fs';
import {decode} from 'base64-arraybuffer';
import AWS from 'aws-sdk';
export const uploadFileToS3 = async (file) => {
const BUCKET_NAME = 'XXXXXXXXXX';
const IAM_USER_KEY = 'XXXXXXXXXX';
const IAM_USER_SECRET = 'XXXXXXXXXXXXXXX';
const s3bucket = new AWS.S3({
accessKeyId: IAM_USER_KEY,
secretAccessKey: IAM_USER_SECRET,
Bucket: BUCKET_NAME,
signatureVersion: 'v4',
});
const contentType = file.type;
const contentDeposition = `inline;filename="${file.name}"`;
const fPath = file.uri;
const base64 = await fs.readFile(fPath, 'base64');
const arrayBuffer = decode(base64);
return new Promise((resolve, reject) => {
s3bucket.createBucket(() => {
const params = {
Bucket: BUCKET_NAME,
Key: file.name,
Body: arrayBuffer,
ContentDisposition: contentDeposition,
ContentType: contentType,
};
s3bucket.upload(params, (error, data) => {
utils.stopLoader();
if (error) {
reject(getApiError(error));
} else {
console.log(JSON.stringify(data));
resolve(data);
}
});
});
});
};
This worked for me after a significant amount of trying over and over again...
I am also using a lambda function to serve me the link to post with.
The lambda function is just using getSignedUrl.
// Lambda Function
const AWS = require('aws-sdk')
AWS.config.update({
accessKeyId: {bucket_access},
secretAccessKey: {bucket_secret},
signatureVersion: 'v4',
region: {bucket_region}
})
const s3 = new AWS.S3()
exports.handler = async (event) => {
const URL = s3.getSignedUrl('putObject', {Bucket: {bucket_name},
// name of file name being placed in S3 Bucket
// event === metaData object
Key: `${event.{key}}/photo00`})
return URL
};
// React Native
const imagePreview = '{image_uri}'
const handleURL = async () => {
// metaData object
const obj = {
key: "meta_data"
}
const response = await fetch{{lambda_func_endpoint}, {
method: 'POST',
body: JSON.stringify(obj)
})
const json = await response.json();
return json
}
const handleUpload = async () => {
const URL = await handleURL()
const imageExt = imagePreview.split('.').pop()
// I have no idea why you are supposed to fetch before fetching...
// makes no sense. But it works. Lots of trying as I said.
let image = await fetch(imagePreview)
// I have no idea why it needs to be a blob in order
// to upload... makes no sense.
image = await image.blob()
await fetch(URL, {
method: 'PUT',
body: image,
headers: {
Accept: `image/${imageExt}`,
'Content-Type': `image/${imageExt}`
}
})
.then((res) => console.log(JSON.parse(JSON.stringify(res)).status))
.catch((err) => console.error(err))
}
Let me know what you guys think!