I tried searching a lot for this issue but did not get anything that helps me.
I am trying to download a bunch of pdf files from Strapi using URL and after downloading every pdf I am pushing it into a folder to create a zip.
My issue is while downloading, few of the files are not able to push in zip folder because of large file size. Basically, zip creation is not waiting to complete the download.
Here is my code, I am using map() to get URLs and using blob() to download it and then used zip to save it in the zip folder.
const downloadPdf = async (newData: any) => {
return new Promise(async (resolve) => {
let PolicyDocData: any;
[...some operation...]
if (newData.laws) {
if (newData.laws.description[0]) {
[...some operation to create an array for URL...]
let lawDoc = Array.from(new Set(lawDocArray));
lawDoc.map(async (val: any, index: any) => {
const lawUrl = val.url.slice(1);
const DocUrl = serverUrl + lawUrl;
let download = await fetch(DocUrl, {
method: "GET",
}).then((response) => response.blob()).then((response) => {
let fileType = ".pdf";
const timeStamp = Math.floor(Date.now() / 1000);
const fileName = val.name;
let docFileName = fileName + "_" + timeStamp;
zip.folder("Compliance/Laws").file(docFileName + fileType, response);
})
if (lawDoc.length - 1 === index) {
resolve(Contents);
}
})
}
}
}).then((Contents: any) => {
if (Contents) {
Packer.toBlob(PolicyDocumentCreator(Contents)).then((blob) => {
zip.generateAsync({ type: "blob" }).then(function (content: any) {
saveAs(content, folderName ? folderName : complianceName + ".zip");
setLoading(false);
})
});
}
});
};
I tried to solve it using setTimeout() but this is not a feasible solution.
Finally, I got the solution and I am posting here if anyone looking for it.
I break my whole function into two different functions and used Async/await and Promise.all()
Like this:
const downloadPdf = async (newData: any) => {
if (newData.laws) {
if (newData.laws.description[0]) {
[...Few operation to create an array of URLs...]
let lawDoc = Array.from(new Set(lawDocArray));
const val: any = await downloadAllLawPdf(lawDoc);
}
}
if (Contents) {
const blob = await Packer.toBlob(PolicyDocumentCreator(Contents));
const content: any = await zip.generateAsync({ type: "blob" });
saveAs(content, folderName ? folderName : complianceName + ".zip");
setLoading(false);
}
};
and then use a function downloadAllLawPdf() to download and wait till download gets completed and return the response.
const downloadAllLawPdf = async (lawDoc: any) => {
return Promise.all(lawDoc.map(async (val: any, index: any) => {
const lawUrl = val.url.slice(1);
const DocUrl = serverUrl + lawUrl;
let docFileName = ''
let fileType = ".pdf";
const res = await fetch(DocUrl, {
method: "GET",
});
const response = await res.blob();
const timeStamp = Math.floor(Date.now() / 1000);
const fileName = val.name;
docFileName = fileName + "_" + timeStamp;
zip.folder("Compliance/Laws").file(docFileName + fileType, response);
}));
}
Related
Using react-native-vision-camera, I saw that there is a path for the image. It seem readable by react native tag.
I attempted to upload using this path (I used both file:// for android, and same for IOS), however it failed. Each time the file was detected as "jpeg" or "jpg" but I couldn't access it.
After downloading (From S3 amazon where I uploaded) and converting the jpg to txt, I only find the "file://path".
import { Storage } from "aws-amplify";
export default async function s3UploadBackup(file, user) {
let formatted_date = moment().format("DD-MM-YYYY");
let filePath = file.split("/");
let fileImageName = filePath[filePath.length - 1];
try {
console.log("Files contains :" + JSON.stringify(file));
// example of one of the URL I used "file:///storage/emulated/0/Android/data/com.app/files/Pictures/image-c64a66b3-489d-4af6-bf93-7adb507ceda1790666367.jpg"
const fileName = `${formatted_date}---${user.businessName}---${user.phoneNumber}---${user.location}---${fileImageName}`;
return Storage.put(uploadBackup.path + user.sub + "/" + user.phoneNumber + "/" + fileName, file, {
// contentType: "image/jpeg"
contentType: file.mime
})
} catch(error) {
console.log(error);
}
}
AWS-AMPLIFY support uploading file as BLOB and converting to specified file extension (JPEG, PNG,...).
Assume that we have local file URI - file:///storage/emulated/0/Android/data/com.app/files/Pictures/image-c64a66b3-489d-4af6-bf93-7adb507ceda1790666367.jpg
Let we refactor s3UploadBackup function
import { Storage } from "aws-amplify";
export default async function s3UploadBackup(file, user) {
let formatted_date = moment().format("DD-MM-YYYY");
let filePath = file.split("/");
let fileImageName = filePath[filePath.length - 1];
try {
console.log("Files contains :" + JSON.stringify(file));
// example of one of the URL I used "file:///storage/emulated/0/Android/data/com.app/files/Pictures/image-c64a66b3-489d-4af6-bf93-7adb507ceda1790666367.jpg"
const fileName = `${formatted_date}---${user.businessName}---${user.phoneNumber}---${user.location}---${fileImageName}`;
const blob = await new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest();
xhr.onload = function () {
resolve(xhr.response);
};
xhr.onerror = function (e) {
reject(new TypeError("Network request failed"));
};
xhr.responseType = "blob";
xhr.open("GET", localPath, true);
xhr.send(null);
});
await Storage.put("yourKeyHere", blob, {
contentType: "image/jpeg", // contentType is optional
});
// We're done with the blob, close and release it
blob.close();
} catch(error) {
console.log(error);
// We're done with the blob, close and release it
blob.close();
}
}
I get a blob and I treat it like this:
const file = response.data;
var blob = new Blob([file], {
type: 'application/pdf',
});
const fileReaderInstance = new FileReader();
fileReaderInstance.readAsDataURL(blob);
fileReaderInstance.onload = async () => {
const fileUri = `${FileSystem.documentDirectory}file.pdf`;
await FileSystem.writeAsStringAsync(
fileUri,
fileReaderInstance.result.split(',')[1],
{
encoding: FileSystem.EncodingType.Base64,
}
);
console.log(fileUri);
Sharing.shareAsync(fileUri);
};
however when I generate and share the file, I can't access it and if I get its URI and search on the web it returns:
i solved my problem in this way:
This is a func who get other data to request, do the request (generate PDF()) and treat the data and generate by received blob the buffer on (fileReaderInstance.result) who is shared in Sharing.shareAsync()
const generatePDF = async () => {
setAnimating(true);
const companyReponse = await CompanyService.getCompany();
const peopleResponse = await PeopleService.getPerson(sale.customerId);
const company = companyReponse.response.company;
const people = peopleResponse.response;
const quote = false;
const json = await SaleService.generatePDF({
sale,
company,
people,
quote,
});
if (json && json.success) {
try {
const fileReaderInstance = new FileReader();
fileReaderInstance.readAsDataURL(json.data);
fileReaderInstance.onload = async () => {
const base64data = fileReaderInstance.result.split(',');
const pdfBuffer = base64data[1];
const path = `${FileSystem.documentDirectory}/${sale._id}.pdf`;
await FileSystem.writeAsStringAsync(`${path}`, pdfBuffer, {
encoding: FileSystem.EncodingType.Base64,
});
await Sharing.shareAsync(path, { mimeType: 'application/pdf' });
};
} catch (error) {
Alert.alert('Erro ao gerar o PDF', error.message);
}
}
setAnimating(false);
}
This is the func in SaleServicegeneratePDF who do the request to api and pass the parameters that return a blob of pdf using axios:
generatePDF: async ({ sale, company, people, quote }) => {
const token = await AsyncStorage.getItem('token');
const body = { sale, company, people, quote };
try {
const response = await axios(`${BASE_API}/generate-sale-pdf`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: token,
},
responseType: 'blob',
data: body,
});
return {
success: true,
data: response.data,
};
} catch (err) {
return err.error;
}
},
I have solved this problem by passing the blob string to WriteAsStringAsync method of FileSystem library from expo.
const blobDat = data.data[0].data; //blob data coming from an API call
const fileUri = FileSystem.documentDirectory + `testt.pdf`; //Directory Link of the file to be saved
await FileSystem.writeAsStringAsync(fileUri, blobDat, {
encoding: FileSystem.EncodingType.UTF8,
}) //This step writes the blob string to the pdf fileURI
await IntentLauncher.startActivityAsync("android.intent.action.VIEW", {
data: fileUri,
flags: 1,
type: "application/pdf",
});
//prompts user with available application to open the above created pdf.
I am trying to display the image from firebase storage. Below is the file location copied from firebase storage. It is a jpeg file
profile/2186uPKjgo4pMOQNm0Cm/profilepic
My following code returned error.
useEffect(() => {
function geturl(){
const filename = "profile/"+userid+"/profilepic.jpeg";
var ref = firebase.storage().ref(filename);
console.log(filename);
// This returns the exact file name
ref.getDownloadURL().then((url)=> {
console.log(url);
});
}
geturl();
}, []);
I got this error [object Object]. After that, I tried the following code async await
useEffect(() => {
async function geturl(){
const filename = "profile/"+userid+"/profilepic.jpeg";
var ref = firebase.storage().ref(filename);
console.log("inside geturl");
const downloadurl = await ref.getDownloadURL();
console.log(downloadurl);
}
geturl();
}, []);
Now Im getting the following error.
Possible Unhandled Promise Rejection (id: 29):
"code_": "storage/object-not-found",
"message_": "Firebase Storage: Object 'profile/2186uPKjgo4pMOQNm0Cm/profilepic.jpeg' does not exist.",
"name_": "FirebaseError",
"serverResponse_": "{
\"error\": {
\"code\": 404,
\"message\": \"Not Found. Could not get object\",
\"status\": \"GET_OBJECT\"
}
}",
}
Please let me know how I can get the url?
here you go you can use this function it uploads image to firebase storage and get the image uri at the same time
const uploadImage = async () => {
const response = await fetch(image);
const blob = await response.blob();
let filename = image.substring(image.lastIndexOf('/')+1);
const ext = filename.split('.').pop();
const name = filename.split('.').slice(0, -1).join('.');
filename = name + Date.now() + '.' + ext;
try {
var ref = firebase.storage().ref().child('post-images/'+filename);
await ref.put(blob)
.then(snapshot => {
return snapshot.ref.getDownloadURL();
})
.then(downloadURL => {
console.log(`Successfully uploaded file and got download link');
return downloadURL;
});
return null;
} catch (error) {
return null;
}
}
I'm trying to deploy a lambda function allowing me to upload a picture to S3.
The lambda works well in offline but when I'm deploy it to AWS, the function doesn't work.
The first error I encountered was this one :
ERROR (node:7) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
So, I followed the recommendation to use Buffer.from() method instead. But it doesn't work too. The lambda run until the timeout.
Can someone tell me where I was wrong or suggest me another solution ?
Below my lambda function :
const AWS = require("aws-sdk");
const Busboy = require("busboy");
const uuidv4 = require("uuid/v4");
require("dotenv").config();
AWS.config.update({
accessKeyId: process.env.ACCESS_KEY_ID,
secretAccessKey: process.env.SECRET_ACCESS_KEY,
subregion: process.env.SUB_REGION
});
const s3 = new AWS.S3();
const getContentType = event => {
// see the second block of codes
};
const parser = event => {
// see the third block of codes
};
module.exports.main = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;
const uuid = uuidv4();
const uploadFile = async (image, uuid) =>
new Promise(() => {
// const bitmap = new Buffer(image, "base64"); // <====== deprecated
const bitmap = Buffer.from(image, "base64"); // <======== problem here
const params = {
Bucket: "my_bucket",
Key: `${uuid}.jpeg`,
ACL: "public-read",
Body: bitmap,
ContentType: "image/jpeg"
};
s3.putObject(params, function(err, data) {
if (err) {
return callback(null, "ERROR");
}
return callback(null, "SUCCESS");
});
});
parser(event).then(() => {
uploadFile(event.body.file, uuid);
});
};
getContentType() :
const getContentType = event => {
const contentType = event.headers["content-type"];
if (!contentType) {
return event.headers["Content-Type"];
}
return contentType;
};
parser()
const parser = event =>
new Promise((resolve, reject) => {
const busboy = new Busboy({
headers: {
"content-type": getContentType(event)
}
});
const result = {};
busboy.on("file", (fieldname, file, filename, encoding, mimetype) => {
file.on("data", data => {
result.file = data;
});
file.on("end", () => {
result.filename = filename;
result.contentType = mimetype;
});
});
busboy.on("field", (fieldname, value) => {
result[fieldname] = value;
});
busboy.on("error", error => reject(error));
busboy.on("finish", () => {
event.body = result;
resolve(event);
});
busboy.write(event.body, event.isBase64Encoded ? "base64" : "binary");
busboy.end();
});
new Buffer(number) // Old
Buffer.alloc(number) // New
new Buffer(string) // Old
Buffer.from(string) // New
new Buffer(string, encoding) // Old
Buffer.from(string, encoding) // New
new Buffer(...arguments) // Old
Buffer.from(...arguments) // New
You are using callbackWaitsForEmptyEventLoop which basically let lambda function thinks that the work is not over yet. Also, you are wrapping it in promise but not resolving it. You can simplify this logic using following inbuilt promise function on aws-sdk
module.exports.main = async event => {
const uuid = uuidv4();
await parser(event); // not sure if this needs to be async or not. check
const bitmap = Buffer.from(event.body.file, "base64"); // <======== problem here
const params = {
Bucket: "my_bucket",
Key: `${uuid}.jpeg`,
ACL: "public-read",
Body: bitmap,
ContentType: "image/jpeg"
};
const response = await s3.putObject(params).promise();
return response;
};
I need way to upload image to firebase
i tried to use react-native-fetch-blob library
but I think there is something wrong with installing the library
No need to use react-native-fetch-blob. Here is how I do it on my project.
Install both react-native-firebase and react-native-image-picker. Follow the installation steps from their documentation guide.
Then implement 2 small functions to do image pick and upload to firebase. Here is the sample code.
// 1. Import required library
import firebase from 'react-native-firebase';
import ImagePicker from 'react-native-image-picker';
// 2. Create a function to pick the image
const pickImage = () => {
return new Promise((resolve, reject) => {
ImagePicker.showImagePicker(pickerOptions, response => {
if (response.didCancel) return;
if (response.error) {
const message = `An error was occurred: ${response.error}`;
reject(new Error(message));
return;
}
const { path: uri } = response;
resolve(uri);
});
});
};
// 3. Create a function to upload to firebase
const uploadImage = async (fileName, uri) {
return new Promise(
(resolve, reject) => {
firebase
.storage()
.ref(`uploads/${filename}`)
.putFile(uri)
.then(resolve)
.catch(reject);
}
);
}
Then simply firing both function as you need, here is the sample to pick and immediately upload it.
const pickImageAndUpload = async () => {
const uri = await pickImage();
const fileName = 'someImage.jpg';
const { state, downloadURL } = await uploadImage(fileName, uri);
}
async function uploadImageAsync(itemImage, passedParameter, ItemName, ItemDesc, ItemPrice, ItemWeight) {
const response = await fetch(itemImage);
const blob = await response.blob();
console.log("uri of the elements ius", blob)
var storageRef = firebase.storage().ref();
var file = blob
var metadata = {
contentType: 'image/jpeg',
};
const timeStamp = Date.now();
var uploadTask = storageRef.child('CategoryDescription' + "/" + `${passedParameter}` + "/" + `${ItemName}`).put(file, metadata);
//For image pick
pickImage = async () => {
const { CAMERA, CAMERA_ROLL } = Permissions;
const permissions = {
[CAMERA]: await Permissions.askAsync(CAMERA),
[CAMERA_ROLL]: await Permissions.askAsync(CAMERA_ROLL),
};
if (permissions[CAMERA].status === 'granted' && permissions[CAMERA_ROLL].status === 'granted') {
let result = await ImagePicker.launchImageLibraryAsync({
allowsEditing: false,
aspect:[4,3],
quality: 0.5,
});
// console.log(result);
if (!result.cancelled) {
this.setState({ itemImage: result.uri });
}
}