To whom it may concern,
I have a program that records sounds (*.wav(s) converted to *.wma(s) and *.m4a(s)) that I’m trying to upload to SoundCloud (previously the program had burned CD’s of the WMP Playlists and then I added a section that made a *.zip of the Playlist and attached it to an email), now my users want to upload the WMP playlists to their SoundCloud accounts.
I’ve already added a routine to upload the *.zip file to Dropbox and that works fine. Since the oauth2 for Dropbox was very similar to the oauth2 for SoundCloud, I was able to add my App to the SoundCloud list of Apps, and then add a browser control to my program for the user to “allow” my App to “link” to their SoundCloud account (very similar to how Dropbox handles it).
So I’m up to the point of uploading, but this is handled very differently and I’m having a hard time trying to get this to work (find the right HTTP handlers in Visual Basic to do what I need to do).
In my Dropbox section, I simply had to create a POST message, with the Title of the file and the oauth2 access_token in the URL, then I simply out to read the *.zip file (or any file that I was going to upload) and Write its contents to the GetRequestStream. The only stumbling block that I had was the “Content-Type” which had to be “application/zip” or whatever extension the file was and voila, It was uploaded.
I’ve been reading about multi-part forms and I’m getting a little confused (again the mechanics of the HTTP structures in Visual Basic can be a bit different…duh!...then in other languages…but I’ll try and explain).
I came across one example that used another constructor. Rather than the HttpRequest that I had used to build my “request” and “response” communications, this one had a WebClient. The WebClient could add Headers (Accept-Charset, Accept-Encoding, Accept-Language, Content-Type, Content-Disposition), and also a QueryString (this is where I placed the oath_token, track[asset-data], track[title], etc). Finally, the WebClient constructor had a WebClient.UploadFile command (which took the URL and the physical filename…which I believed would then handle the writing of the file contents the Request).
So…after getting through some 400’s, 401’s (have to deal with expiring token stuff), 422’s, a 503, and a bunch of 500’s (which is where I’m currently at), I still can’t upload a simple *.wma file: Track001.wma.
Here’s my code (strFileName is already built with the physical name of
the sound file \|dir|\Track001.wma, and strAccessToken was already
obtained from SoundCloud):
strURL = "https://api.soundcloud.com/tracks.json?oauth_token=" & strAccessToken
Dim myWebClient As New WebClient
myWebClient.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3")
myWebClient.Headers.Add("Accept-Encoding", "gzip,deflate,sdch")
myWebClient.Headers.Add("Accept-Language", "en-US,en;q=0.8,ru;q=0.6")
myWebClient.Headers.Add("Content-Disposition: form-data; name=track[title]")
I also tried this line as well:
myWebClient.Headers.Add("Content-Disposition: form-data; name=" & HttpUtility.UrlEncode(strFileName))
This adds the SoundCloud params
Dim myQueryStringCollection As New NameValueCollection()
myQueryStringCollection.Add("oauth_token", strAccessToken)
myQueryStringCollection.Add("track[asset_data]", strFileName)
myQueryStringCollection.Add("track[title]", "Test Track Upload")
myQueryStringCollection.Add("track[sharing]", "public")
myQueryStringCollection.Add("track[downloadable]", "true")
myQueryStringCollection.Add("track[post_to]", "")
This attaches the params into the Request
myWebClient.QueryString = myQueryStringCollection
Here’s where I try and send it
Dim responseArray As Byte() = myWebClient.UploadFile(strURL, strFileName)
But I’m receiving an error here: {"errors":[{"error_message":"422 -
Unprocessable Entity"}]}
If I set the Content-type to multi-form, I
get: The Content-Type header cannot be set to a multipart type for
this request
I just tried to get a 500 again, but all I’m getting now is the 422 (so maybe it’s fixed on the other end?).
Related
We are participating in a process where we provide a vendor with AWS S3 SignedURLs that allow them to upload files to an S3 bucket.
The process is quite simple:
They request a signedurl from us via an API / Lambda function we've published in AWS.
We return an S3 signedurl.
They use the signedurl to POST a binary file (in this case, a compressed .zip file).
While we don't have all of their code, they shared this snippet regarding the actual POST operation. (interesting to note, they're using the PUT method...not sure if that matters).
The problem occurs in Step 3. Their service uploads the binary file and, if we change the binary file to .txt, we see the following additional elements have been added to the .zip.
--6de81f1b-be80-44de-8a14-7e023c92fcf3
Content-Type: application/octet-stream
Content-Disposition: form-data; name="file"; filename="H22101_0031_087.zip"
This alters the structure of the .zip file considerably, causes checksum operations to fail and prevents us from decompressing the file...essentially it isn't a valid .zip unless we strip these additional elements out.
The code responsible for posting the binary is quite simple:
var dataFileUploadRequest = new RestRequest();
dataFileUploadRequest.Method = Method.Put;
dataFileUploadRequest.AddFile("file", sdpData, documentFileName);
dataFileUploadRequest.RequstFormat = DataFormat.Binary;
They implemented a workaround by using httpclient instead of restsharp...however, we would like to better understand why this is happening.
Does anyone know why restsharp's libraries would append this additional Content-Type, Content-Disposition...and even that strange guid (which appears to be a sort of transaction id) around the .zip's binary contents?
I am working on a webapp where the user provides an image file-text sequence. I am compressing the sequence into a single ZIP file uisng JSZip.
On the server I simply use PHP move_uploaded_file to the desired location after having checked the file upload error status.
A test ZIP file created in this way can be found here. I have downloaded the file, expanded it in Windows Explorer and verified that its contents (two images and some HTML markup in this instance) are all present and correct.
So far so good. The trouble begins when I try to fetch that same ZIP file and expand it using JSZip.loadAsync which consistently reports Corrupted zip: missing 210 bytes. My PHP code for squirting back the ZIP file is actually pretty simple. Shorn of the various security checks I have in place the essential bits of that code are listed below
if (file_exists($file))
{
ob_clean();
readfile($file);
http_response_code(200);
die();
} else http_response_code(399);
where the 399 code is interpreted in my webapp as a need to create a new resource locally instead of trying to read existing resource data. The trouble happens when I use the result text (on an HTTP response of 200) and feed it to JSZip.loadAsync.
What am I doing wrong here? I assume there is something too naive about the way I am using readfile at the PHP end but I am unable to figure out what that might be.
What we set out to do
Attempt to grab a server-side ZIP file from JavaScript
If it does not exist send back a reply (I simply set a custom HTTP response code of 399 and interpret it) telling the client to go prepare its own new local copy of that resource
If it does exist send back that ZIP file
Good so far. However, reading the existent ZIP file into PHP and sending it back does not make sense + is fraught with problems. My approach now is to send back an http_response_code of 302 which the client interprets as being an instruction to "go get that ZIP for yourself directly".
At this point to get the ZIP "directly" simply follow the instructions in this tutorial on MDN.
Thank you for the nice work you put into this amazing library.
I have an issue with my request adapter but only with the MultipartFormData.
I want to be able to update the body of the request but when the request gets in the adapter, I'm getting a nil httpBody. I only get this behaviour on Multipart, not on classic POST requests.
I'm trying to sign the request with an oauth2 token (async), but the particularity of this API is that the token is sent in the body and not in the headers.
There's a body as the metrics in the response say (Request Body Transfer Bytes) 231306
I'm using Alamofire 5.0.0
This is most likely due to multipart forms using UploadRequests (and therefore URLSessionUploadTasks) which do not include the body data as part of the URLRequest for performance reasons. If you update your question with what you're trying to do to the body, perhaps I can provide an alternate solution. If nothing else, you could manually create multipart uploads by using MultipartFormData.encode() directly and adding the Data to a URLRequest, but that's not recommended for large uploads.
I need to invoke a remote REST interface handler and submit it a file in request body. Please note that I don't control the server. I cannot change the request to be multipart, the client has to work in accordance to external specification.
So far I managed to make it work like this (omitting headers etc. for brevity):
byte[] data = readFileCompletely ();
client.target (url).request ().post (Entity.entity (data, "file/mimetype"));
This works, but will fail with huge files that don't fit into memory. And since I have no restriction on filesize, this is a concern.
Question: is it somehow possible to use streams or something similar to avoid reading the whole file into memory?
If possible, I'd prefer to avoid implementation-specific extensions. If not, a solution that works with RESTEasy (on Wildfly) is also acceptable.
ReastEasy as well as Jersey support InputStream out of the box so simply use Entity.entity(inputStream, "application/octet-stream"); or whatever Content-Type header you want to set.
You can go low-level and construct the HTTP request using a library such as the plain java.net.URLConnection.
I have not tried it myself but there is example code which reads a local file and writes it to the request stream without loading it into a byte array.
Upload files from Java client to a HTTP server
Of course this solution requires more manual coding but it should work (unless java.net.URLConnection loads the whole file into memory)
Background - I'm trying to stream an existing webpage to a separate web application, using HttpWebRequest/HttpWebResponse in C#. One issue I'm striking is that I'm trying to set the file upload request content-length using the file download's content-length, HOWEVER the issue seems to be when the source webpage is on a webserver for which the HttpWebResponse doesn't provide a content length.
HttpWebRequest downloadRequest = WebRequest.Create(new Uri("downloaduri")) as HttpWebRequest;
using (HttpWebResponse downloadResponse = downloadRequest.GetResponse() as HttpWebResponse)
{
var uploadRequest = (HttpWebRequest) WebRequest.Create(new Uri("uripath"));
uploadRequest.Method = "POST";
uploadRequest.ContentLength = downloadResponse.ContentLength; // ####
QUESTION: How could I update this approach to cater for this case (when the download response doesn't have a content-length set). Would it be to somehow use a MemoryStream perhaps? Any sample code would be appreciated.
If you're happy to download the response from the other web server completely, that would indeed make life easier. Just repeatedly write into a MemoryStream as you fetch from the first web server, then you know the length to set for the second request and you can write the data in easily (especially as MemoryStream has a WriteTo method to write its contents to another stream).
The downside of this is that you'll take a lot of memory if it's a large file. Is that likely to be a problem in your situation? Alternatives include:
Writing to a file instead of using a MemoryStream. You'll need to clean up the file afterwards, of course - you're basically using the file system as bigger memory :)
Using a chunked transfer encoding to "read a chunk, write a chunk"; this may be fiddly to get right - it's certainly not something I've done before.