When using multer-s3, a .wav contain non-english filename is broken.
"multer": "^1.4.5-lts.1",
"multer-s3": "^2.10.0",
like this:
á\x84\x91á\x85µá\x84\x8Bá\x85¡á\x84\x82á\x85©.wav
This issue is generated when frontend send non-english file to server. When server is received the file, the filename already is broken like á\x84\x91á\x85µá\x84\x8Bá\x85¡á\x84\x82á\x85©️.wav
What wrong with me ? is it server problem ? or client problem? I am using axios -> nestjs(multer s3 interceptor)
Related
I am running a webserver on an ESP32 chip using the libwebsocket library. The server files are in a ROMFS partition on the ESP32.
Currently I am in the process of trying to improve the loading time by concatenating, minifying and compressing the javascript, html and css files.
The concatenation and minification worked properly, I now only have a concatenated.js and concatenated.css file in my website. But the issue came when I tried to get the compression working.
Initially, I thought my server would compress the files by itself before sending them, however when I looked at the server file transfer using Chrome developper extension, I found out that the javascript file GET request was returned with "content-type: text/javascript".
I tried several solutions I could think of, but none seem to work:
gzip the file before creating the romfs (ie there is now only a concatenated.js.gz in my ROMFS file system)
The server returns 404 when trying to access "concatenated.js"
gzip the file before creating the romfs and make it live alongside the original file (I was thinking maybe libwebsocket would be able to see they were both there and pick the most efficient one)
The server only returns the js file, and never the gz file
Does anybody knows how to enable the gzip compression in libwebsocket ? I am guessing there must be some options I don't have enabled, but it has been hard finding resources on the web. Most of them only discuss about the ability of the libwebsocket to get gzip from a zipped file.
Regards,
The issue ended up coming directly from the libwebsocket code.
When opening a file from the ESP32, there was no logic in place to look for a file with the same name and ".gz" at the end. The logic to look for such a file if the browser accepted gzip file needed to be added to the function.
This change was done on an older version of the libwebsocket, and as such may not apply to the latest version (for anybody looking at this modification). Also, I needed to include <string.h> to have access to the string manipulation functions:
libwebsocket/lib/plate/freertos/esp32/esp32-helpers.c -> function esp32_lws_fops_open
Replace
f->i = romfs_get_info(lws_esp32_romfs, filename, &len, &csum);
By
// check for the gzip file if gzip is allowed by the browser
f->i = NULL;
if((*flags & LWS_FOP_FLAG_COMPR_ACCEPTABLE_GZIP) == LWS_FOP_FLAG_COMPR_ACCEPTABLE_GZIP)
{
char *filename_gz = malloc(strlen(filename) + 3 + 1); // add space for ".gz" and null termination
sprintf(filename_gz, "%s.gz", filename);
f->i = romfs_get_info(lws_esp32_romfs, filename_gz, &len, &csum);
}
// if we haven't found a gz file (not allowed or no gzip), search for the regular file
if(!f->i)
{
f->i = romfs_get_info(lws_esp32_romfs, filename, &len, &csum);
}
// otherwise, add the flags to let the library knows the file transfered is a gzip file
else
{
*flags |= LWS_FOP_FLAG_COMPR_IS_GZIP;
}
I am able to upload files through forms using Flask but what I require is to be able to put the local file path in the URL and then flask can upload the file and process it. How do I do this?
well this is just not how a webserver works. If you want it to process the data locally stored on your computer you need to send it via a POST request.
Cheers, T
We have a critical file storage issue of RavenDb.
The context is we have a Web application server and a separate File service server based on RavenDB. The user tries to upload a file with the name of "xxx_Living #Mapping.pdf" for example. The web application would then send the request to File server by wrapping the file content into the request body, passing the encoded file name as parameter with Put Action. But all the characters after and includes the '#' was blocked upon request received by File Server for unknown network reason, result in "xxx_Living". The file server is able to create file by calling filesStore.AsyncFilesCommands.UploadAsync(fileName, Request.Body) without exception.
The issue is we are able to view the File uploaded from Raven.Studio by sending a request /RavenDbServer/fs/ClientFile/search?query=__directoryName%3A%2Fclientattachments+AND+__level%3A2&start=108&pageSize=27. (see the snapshot attached)
enter image description here
But, We are not able to see the metadata by selecting the target file in the list by sending request /RavenDbServer/studio/index.html#filesystems/edit?&id=clientattachments%2F0aecef9c-6dd0-4a9e-9df3-039228576471_Living%20&filesystem=clientFile. (see the snapshot attached)
enter image description here
We also checked the network flow. the request RavenDbServer/fs/clientFile/files/clientattachments%2F0aecef9c-6dd0-4a9e-9df3-039228576471_Living%20 returns 404 code.
My question is:
1) Are those kind of files stored successfully in RavenDb File server since we can see the id, file size and Last Modify date in List page, but not able to get it by click into the file from RavenDb studio or download it by calling api filesStore.AsyncFilesCommands.DownloadAsync(FileName)?
2) Can we store the file to Raven file server with the file name contains special characters, like '#', or any other ones?
ok, lets start from scratch. I just realized this is apache and not phpmyadmin, my bad.
Anyway, I needed some sort of file storage accessible through the web. I deleted the index.html to list the other files in /var/www. Now if I open the json file (UTF8 w/o BOM) in the browser, the special charakters like ä,ü,ö are not correctly displayed (normal chars are). If I download the file, all is correct on my system.
So the file itself is fine, but the stream from apache to the web is not in UTF8, or something like that. And that I would like to change.
I need this for an android app, where I parse the content of the json file with volley lib. But there it also gets the special charakters wrong.
hope this is more usefull than befor. my apologies for that.
The only thing that is wrong is that your browser doesn't know it should interpret the UTF-8 encoded JSON file as UTF-8. Instead it falls back to its default Latin-1 interpretation, in which certain characters will screw up, because it's using the wrong encoding to interpret the file.
That is all. The file will appear fine if it is interpreted using the correct encoding, UTF-8 in this case.
Use the View → Encoding menu of your browser to force it to UTF-8 and see it work.
Why doesn't the browser use UTF-8? Because there's no HTTP Content-Type header telling it to do so. Why is there no appropriate HTTP header set? Because you didn't tell your web server that it should set this header for .json files. How do you tell Apache to do so? By adding this line in an .htaccess file:
AddCharset UTF-8 .json
I'm writing an apache2 module
by default and when viewed in a web browser, the module would only print the first lines of a large file and convert them to HTML.
if the user choose to 'download as...', the whole raw file would be downloaded.
Is it possible to detect this choice on the server side ? (for example is there a specific http header set ?).
note: I would like to avoid any parameter in the GET url (e.g: "http://example.org/file?mode=raw" )
Pierre
added my own answer to close the question: as said #alexeyten there is no difference. I ended by a javascript code the alter the index.html file generated by apache.