In my node.js application I'm downloading multiple user files from AWS S3, compress them to single zip (with usage of Archiver npm library) file and send back to client. All the way I'm operating on streams, and yet I can't send files to client (so client would start download after successful http request).
const filesStreams = await this.awsService.downloadFiles(
document?.documentFiles,
);
const zipStream = await this.compressService.compressFiles(filesStreams);
// ts-ignore
response.setHeader('Content-Type', 'application/octet-stream');
response.setHeader(
'Content-Disposition',
'attachment; filename="files.zip"',
);
zipStream.pipe(response);
Where response is express response object. zipStream is created with usage of Archiver:
public async compressFiles(files: DownloadedFileType[]): Promise<Archiver> {
const zip = archiver('zip');
for (const { stream, name } of files) {
zip.append((stream as unknown) as Readable, {
name,
});
}
return zip;
And I know it is correct - because when I pipe it into WriteStream to some file in my file system it works correctly (I'm able to unzip written file and it has correct content). Probably I could temporarily write file in file system, send it back to client with usage of response.download and remove save file afterwards, but it looks like very inefficient solution. Any help will be greatly appreciated.
So I found a source of a problem - I'll post it here just for record, in case if anyone would have same problem. Source of the problem was something totally different - I was trying to initiate a download by AJAX request, which of course won't work. I changed my frontend code and instead AJAX I used HTML anchor element with src attribute set on exactly same endpoint and it worked just fine.
I had a different problem, but rather it came from the lint. I needed to read all files from the directory, and then send them to the client in one zip. Maybe someone finds it useful.
There were 2 issues:
I mismatched cwd with root; see glob doc
Because I used PassThrough as a proxy object between Archiever and an output, lint shows stream.pipe(response) and type issue... what is a mistake - it works fine.
Related
I am working in a mobile app. In this mobile app I need to send a file (.wav file) to a server.
At this moment, I am working locally and testing my app. I am creating an audio file in the mobile app with a defined name and extension. However, I'm wondering, If my app is used by more people,
can it happen that, if the different users use the app and send the file at the same time, as the file name is already defined in the app, in the server this file is going to be overwitten ?
What is usually the behaviour in a situation like this? And if the file is overwitten, how can I overcome this problem, since diffrent users may send a different file?
Details:
API : FAST API :
async def something(file: UploadFile = File(...)):
with open(f"{file.filename}", "wb") as buffer:
shutil.copyfileobj(file.file, buffer)
#saved teh file
developing with React NAtive
Using Expo AV to do the audio
sending the audio using fetch
sending method:
formData.append("file",
uri: path,
name: "audio.wav",
type: "audio/wav",
....
method: "POST",
headers: {
"Content-Type": "multipart/form-data",
},
body: formData,
Thanks !
This scenario should be handled completely by the server. The client should not contain any logic on what files are already located on the server and which are not.
Let the server create a new uuid for each new file that it receives.
Thus, the metadata for a file upload could be the following object.
{
uuid: "11edc52b-2918-4d71-9058-f7285e29d894",
fileUri: "",
}
Thus, the server needs to be adapted to solve this problem. The file is then stored along with its unique identifier. No name clash should occur.
Of course, if you want to display the files in the client later, then you need some name property, which could be the fileUri appended with a counter variable if the same fileUri exists multiple times. This should be handled by the server as well.
I am working on a webapp where the user provides an image file-text sequence. I am compressing the sequence into a single ZIP file uisng JSZip.
On the server I simply use PHP move_uploaded_file to the desired location after having checked the file upload error status.
A test ZIP file created in this way can be found here. I have downloaded the file, expanded it in Windows Explorer and verified that its contents (two images and some HTML markup in this instance) are all present and correct.
So far so good. The trouble begins when I try to fetch that same ZIP file and expand it using JSZip.loadAsync which consistently reports Corrupted zip: missing 210 bytes. My PHP code for squirting back the ZIP file is actually pretty simple. Shorn of the various security checks I have in place the essential bits of that code are listed below
if (file_exists($file))
{
ob_clean();
readfile($file);
http_response_code(200);
die();
} else http_response_code(399);
where the 399 code is interpreted in my webapp as a need to create a new resource locally instead of trying to read existing resource data. The trouble happens when I use the result text (on an HTTP response of 200) and feed it to JSZip.loadAsync.
What am I doing wrong here? I assume there is something too naive about the way I am using readfile at the PHP end but I am unable to figure out what that might be.
What we set out to do
Attempt to grab a server-side ZIP file from JavaScript
If it does not exist send back a reply (I simply set a custom HTTP response code of 399 and interpret it) telling the client to go prepare its own new local copy of that resource
If it does exist send back that ZIP file
Good so far. However, reading the existent ZIP file into PHP and sending it back does not make sense + is fraught with problems. My approach now is to send back an http_response_code of 302 which the client interprets as being an instruction to "go get that ZIP for yourself directly".
At this point to get the ZIP "directly" simply follow the instructions in this tutorial on MDN.
I'm new to MobileFirst development. I need to download a zip file from one of the adapters. I am able to do this when invoking REST API call eg using Postman. But I'm not sure how to handle this when using MFS sdk:
Here's my code to access the adapter:
var resourceRequest = new WLResourceRequest(url,
WLResourceRequest.GET
);
const resp = await resourceRequest.send()
I log the response and it's showing special characters in the responseText.
Is there a way to handle the binary response?
Thank you in advance for your help!
For any binary content, you will have to convert the content to Base 64 in your adapter and revert it back to binary in your app.
This is because the WLResourceRequest APIs are designed for handling text based data.
If your binary content is rather large, then it is best to host the file somewhere and return the link to the file from your adapter.
I'm doing a app which takes input from the user and generates PDF file.i"m trying to upload generated pdf file to api.but, i'm not getting how to do it, i'm totally new to swift.can anyone please give me a exsample on how to upload a file to api
Thanks in advance
First, I'd suggest you consider using Alamofire if you don't want to get lost in the idiosyncrasies of composing network requests. If you do it yourself, it can get pretty hairy (see https://stackoverflow.com/a/26163136/1271826 for example of how to construct multipart/formdata request manually).
Second, in terms of how that request should be formed, it depends entirely upon how you designed the API. But the easiest for file uploads is to support multipart/formdata requests (e.g. in PHP, using the $_FILES mechanism). See http://php.net/manual/en/features.file-upload.php. Or see example here that not only uploads image file (which you can easily modify to accept PDFs), but constructs JSON response, too: https://stackoverflow.com/a/19743872/1271826.
Anyway, if your server is designed to handle multipart/formdata requests, you can create request and parse the response using Alamofire, as shown in the Uploading MultipartFormData section of the README file:
Alamofire.upload(
.POST,
"https://httpbin.org/post",
multipartFormData: { multipartFormData in
multipartFormData.appendBodyPart(fileURL: unicornImageURL, name: "unicorn")
multipartFormData.appendBodyPart(fileURL: rainbowImageURL, name: "rainbow")
},
encodingCompletion: { encodingResult in
switch encodingResult {
case .Success(let upload, _, _):
upload.responseJSON { response in
debugPrint(response)
}
case .Failure(let encodingError):
print(encodingError)
}
}
)
I've run into the following problem when porting an app from REST API to GDAA.
The app needs to download some of (thousands of) JPEG images based on user selection. The way this is solved in the app is by downloading a thumbnail version first, using this construct of the REST API:
private static InputStream getCont(String rsid, boolean bBig){
InputStream is = null;
if (rsid != null) try {
File gFl = bBig ?
mGOOSvc.files().get(rsid).setFields("downloadUrl" ).execute():
mGOOSvc.files().get(rsid).setFields("thumbnailLink").execute();
if (gFl != null){
GenericUrl url = new GenericUrl(bBig ? gFl.getDownloadUrl() : gFl.getThumbnailLink());
is = mGOOSvc.getRequestFactory().buildGetRequest(url).execute().getContent();
}
} catch (UserRecoverableAuthIOException uraEx) {
authorize(uraEx.getIntent());
} catch (GoogleAuthIOException gauEx) {}
catch (Exception e) { }
return is;
}
It allows to get either a 'thumbnail' or 'full-blown' version of an image based on the bBig flag. User can select a thumbnail from a list and the full-blown image download follows (all of this supported by disk-base LRU cache, of course).
The problem is, that GDAA does not have an option to ask for reduced size / thumbnail version of an object (AFAIK), so I have to resort to combining both APIs, which makes the code more convoluted then I like (bottom of the page). Needles to state that the 'Resource ID' needed by the REST may not be immediately available.
So, the question is: Is there a way to ask GDAA for a 'thumbnail' version of a document?
Downloading thumbnails isn't currently available in the Drive Android API, and unfortunately I can't give a timeframe to when it will be available. Until that time, the Drive Java Client Library is the best way to get thumbnails on Android.
We'd appreciate if you go ahead and file a feature request against our issue tracker: https://code.google.com/a/google.com/p/apps-api-issues/
That gives requests more visibility to our teams internally, and issues will be marked resolved when we release updates.
Update: I had an error in the discussion of the request fields.
As Ofir says, you can't get thumbnails with the Drive Android API and you can get thumbnails with the Drive Java Client Library. This page has is a really good primer for getting started:
https://developers.google.com/drive/v3/web/quickstart/android
Oddly, I can't get the fields portion of the request to work as it is on that quick start. As I've experienced, you have to request the fields a little differently.
Since you're doing a custom field request you have to be sure to add the other fields you want as well. Here is how I've gotten it to work:
Drive.Files.List request = mService.files()
.list()
.setFields("files/thumbnailLink, files/name, files/mimeType, files/id")
.setQ("Your file param and/or mime query");
FileList files = request.execute();
files.getFiles(); //Each File in the collection will have a valid thumbnailLink
A sample query might be:
"mimeType = 'image/jpeg' or mimeType = 'video/mp4'"
Hope this helps!