I am using the s3-zip npm package in order to download multiple files from an s3 bucket into a zip file. To do this, I am using the following code taken straight from the documentation https://www.npmjs.com/package/s3-zip:
const fs = require('fs')
const join = require('path').join
const s3Zip = require('s3-zip')
const region = 'bucket-region'
const bucket = 'name-of-s3-bucket'
const folder = 'name-of-bucket-folder/'
const file1 = 'Image A.png'
const file2 = 'Image B.png'
const file3 = 'Image C.png'
const file4 = 'Image D.png'
const output = fs.createWriteStream(join(__dirname, 'use-s3-zip.zip'))
s3Zip
.archive({ region: region, bucket: bucket}, folder, [file1, file2, file3, file4])
.pipe(output)
This works and I am able to download the zip file, but the zip file is automatically downloaded into the location specified by the parameters of createWriteStream. However, when the user downloads the zip file, I want them to be presented with the standard pop-up that allows them to choose where to download the zip file. Does anyone know what code I'll need to allow this?
Related
What I want to do: Download an S3 file (pdf) in a lambda and extract its text, using Rust.
The Error:
ERROR PDF error: Invalid file header
I checked the pdf file in the bucket, downloaded it from the console and everything looks correct, so something is breaking in the way I store the file.
How I am doing it:
let config = aws_config::load_from_env().await;
let client = s3::Client::new(&config);
// Get uploaded object in raw bucket (serde derived the json)
let key = event.records.get(0).unwrap().s3.object.key.clone();
let key = key.replace('+', " ");
let key = percent_encoding::percent_decode_str(&key).decode_utf8().unwrap().to_string();
let content = client
.get_object()
.bucket(raw_bucket_name)
.key(&key)
// .response_content_type("application/pdf") // this did not make any difference
.send()
.await?;
let mut bytes = content.body.into_async_read();
let file = tempfile::NamedTempFile::new()?;
let path = file.into_temp_path();
let mut file = tokio::fs::File::create(&path).await?;
tokio::io::copy(&mut bytes, &mut file).await?;
let content = pdf_extract::extract_text(path)?; // this line breaks
Versions:
tokio = { version = "1", features = ["macros"] }
aws-sdk-s3 = "0.21.0"
aws-config = "0.51.0"
pdf-extract = "0.6.4"
I feel like I misunderstood something in how to store the bytestream, but e.g. https://stackoverflow.com/a/62003659/4986655 do it in the same way afaiks.
Any help or pointers on what the issue might be or how to debug this are very welcome.
I've been looking create-react-app's configuration files.
After ejecting, we can see require.cache is used in config/env.js file.
// config/env.js
"use strict";
const fs = require("fs");
const path = require("path");
const paths = require("./paths");
// Make sure that including paths.js after env.js will read .env variables.
delete require.cache[require.resolve("./paths")];
...
I quite cannot understand why that require.cache is used. I know what require.cache is for and does, but cannot understand why this is used in this case.
I've made config/test.js file and compared the output.
const a = require('./paths')
const b = require('./env')
console.log("A", a)
console.log("B", b)
const c = require('./env')
const d = require('./paths')
console.log("C", c)
console.log("D", d)
node config/test.js
As expected, those are the same. So what is the purpose of using require.cache in the config/env.js file?
I want to use my own S3 storage and display the image that was just uploaded. How can I get the filename that was uploaded to S3?
For example I can upload a jpg image to UploadCare.
This is the output I can get:
fileInfo.cdnUrl: "https://ucarecdn.com/6314bead-0404-4279-9462-fecc927935c9/"
fileInfo.name: "0 (3).jpg"
But if I check my S3 bucket this is the file name that was actually uploaded to S3: https://localdevauctionsite-us.s3.us-west-2.amazonaws.com/6314bead-0404-4279-9462-fecc927935c9/03.jpg
Here is the javascript I have so far:
var widget = uploadcare.Widget('[role=uploadcare-uploader]');
widget.onChange(group => {
group.files().forEach(file => {
file.done(fileInfo => {
// Try to list the file from aws s3
console.log('CDN url:', fileInfo.cdnUrl);
//https://uploadcare.com/docs/file_uploader_api/files_uploads/
console.log('File name: ', fileInfo.name);
});
});
});
Filenames are sanitized before copying to S3, so the output file name can contain the following characters only:
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_
The following function can be used to get a sanitized file name:
function sanitize(filename) {
var extension = '.' + filename.split('.').pop();
var name = filename.substring(0, filename.length - extension.length);
return name.replace(/[^A-Za-z0-9_]+/g, '') + extension;
}
The final S3 URL can be composed of the S3 base URL, a file UUID, which you can obtain from the fileInfo object, and a sanitized name of the uploaded file.
var csvString = ['rest','test','age'];
var fileName_CSV = "Report_1.csv";
var fileName_ZIP = "Report_1.zip";
var blob = new Blob(dd,{type: application/zip"});
var zip = new JSZip();
zip.file(fileName_CSV,csvString),{type:"blob"};
var content = zip.generate({type:"blob"});
saveAs(content,fileName_ZIP);
I have the json data i have converted it to fit in csv format so i created the csv file with the data then saves it in memory and now zipped the csv file and now i want to apply password on it .. so when we open the zip and try to open the csv it should ask for the user defined password.. and either i want to use java script or nodejs for it... please help
The mini-zip-asm package supports creating zip archives with passwords.
https://www.npmjs.com/package/minizip-asm.js
From the docs:
npm install minizip-asm.js
Example Usage:
var Minizip = require('minizip-asm.js');
var fs = require("fs");
var csvString = new Buffer("Abc~~~");
var mz = new Minizip();
mz.append("Report_1.csv", csvString, {password: "insert-password"});
fs.writeFileSync("Report_1.zip", new Buffer(mz.zip()));
This question already has answers here:
List all files and folder in google drive
(2 answers)
Closed 2 years ago.
Hi I'm looking for a way to get a list for all my folders, sub folders and files from my google drive. I would like it to create a spreadsheet in my drive that outputs:
-All folders names, all sub folder names, all files names and there id (if not the id the url or both). Optional if it is possible output the description
Ive tried the code display on this post that works but it only gives me the file name and link in the parent folder only but I would like all mention information.
if someone knows the correct full code?
// replace your-folder below with the folder for which you want a listing
function listFolderContents() {
var foldername = 'your-folder';
var folderlisting = 'listing of folder ' + foldername;
var folders = DriveApp.getFoldersByName(foldername)
var folder = folders.next();
var contents = folder.getFiles();
var ss = SpreadsheetApp.create(folderlisting);
var sheet = ss.getActiveSheet();
sheet.appendRow( ['name', 'link'] );
var file;
var name;
var link;
var row;
while(contents.hasNext()) {
file = contents.next();
name = file.getName();
link = file.getUrl();
sheet.appendRow( [name, link] );
}
};
Execute this URI request which I generated from the Drive API explorer:
https://www.googleapis.com/drive/v3/files?fields=files%2CincompleteSearch%2Ckind%2CnextPageToken
It will return all your files and the its metada - complete info, which is found in File Resource properties.