I'm uploading a pdf file from an iOS application to a Rails application.
The file is uploaded, but its content type gets corrupted.
Here is the relevant part of the iOS code to upload the file (using AFNetworking library):
NSURL *url = [NSURL URLWithString:kWebserviceHost];
AFHTTPClient *httpClient = [[AFHTTPClient alloc] initWithBaseURL:url];
// httpClient.parameterEncoding = AFFormURLParameterEncoding; // Experimented with different values, no effect
NSData *documentData = [NSData dataWithContentsOfURL:self.documentURL];
if (! documentData) {
NSLog(#"Trying to upload nil document: sorry! - %#", self.documentData);
return;
}
NSMutableURLRequest *request = [httpClient multipartFormRequestWithMethod:#"PUT" // #"POST"
path:#"new_upload"
parameters:#{ #"fooKey" : fooString, #"barKey" : barString }
constructingBodyWithBlock: ^(id <AFMultipartFormData>formData) {
[formData appendPartWithFileData:documentData name:#"document" fileName:#"upload.pfd" mimeType:#"application/pdf"];
}];
AFHTTPRequestOperation *operation = [[AFHTTPRequestOperation alloc] initWithRequest:request];
[operation setUploadProgressBlock:^(NSInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite) {
NSLog(#"Sent %lld of %lld bytes", totalBytesWritten, totalBytesExpectedToWrite);
progressBlock(totalBytesWritten / totalBytesExpectedToWrite);
}];
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
uploadSuccessBlock();
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
NSLog(#"Sorry, could not upload document - %#", error);
failureBlock(error);
}];
[operation start];
Let's go to the other side, a Rails 3.2.3 server.
I'm using the gem PaperClip (version 3.1.4). Here is the definition of the paperclip attachment in my model class:
paperclip_storage = (Rails.env.production?) ? {
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => ':attachment/:id/:style.:extension',
:bucket => 'mybucketname' #,
# :s3_headers => { "Content-Type" => "video/mp4" }
}
: { :headers => { "Content-Type" => "application/pdf" }
}
has_attached_file :document, paperclip_storage.merge( :use_timestamp => false,
:url => "/assets/:id/:basename.:extension",
:path => ":rails_root/public/assets/:id/:basename.:extension",
:processors => nil # Maybe the processing of the file mess things up?
)
I also tried to skip the post process (even though the documentation says none should be applied because no style is defined).
before_post_process :skip_post_process
def skip_post_process
return false
end
An other try: setting the content_type in a post_process filter:
after_post_process :force_content_type_to_pdf
def force_content_type_to_pdf
puts "--------------- Changing content_type"
self.document.instance_write(:content_type, "application/pdf")
end
Here is the controller method to receive and save the file:
def new_upload
#document = Document.create()
document = params[:document]
if (document.nil?)
return
end
#document.document = document
puts "Document type: #{#document.document.content_type}" # logs: 'application/pdf'
puts "Parameters: #{params}" # logs Parameters: {"fooKey"=>"foo", "barKey"=>"bar", "document"=>#<ActionDispatch::Http::UploadedFile:0x007ff5a429be58 #original_filename="upload.pfd", #content_type="application/pdf", #headers="Content-Disposition: form-data; name=\"document\"; filename=\"upload.pfd\"\r\nContent-Type: application/pdf\r\n", #tempfile=#<File:/var/folders/vy/zm_x1bts5hs6pkvzk7clnmvm0000gn/T/RackMultipart20120802-12714-1j0hq6q>>}
#document.foo = params['fooKey']
#document.bar = params['barKey']
if #document.save
render :json => {status: "saved" }
return
else
render json: #document.errors, status: :unprocessable_entity
return
end
end
The file is uploaded, but unreadable. The Finder does not know which application to use to open it.
Here is a screenshot of a QuickLook preview:
The content type is somehow mixed up.
file --mime /file/path/before/or/after/upload.pdf returns the following message on the original file (the one that will be uploaded by the iOS app) or on the file created by the Rails server after a successful upload:
/upload.pfd: application/pdf; charset=binary. Sounds good so far.
mdls -name kMDItemContentType /file/path/before/upload.pdf returns kMDItemContentType = "com.adobe.pdf" on the file to be uploaded. Still ok.
But the same command on the file created by the Rails server returns: kMDItemContentType = "dyn.ah62d4rv4ge81a3xe".
This at least explains why the Finder is confused.
The problem is similar when downloading the file in a browser. Here is the relevant method of my controller:
def download_document
#document = Document.find(params[:id])
if (#document && #document.document)
# Line below propagate the file content-type problem:
send_file Rails.root.join(document.path), :type => "application/pdf", :x_sendfile => false, :stream => false
# This hack works while server is locally hosted
send_data File.read(Rails.root.join(#document.document.path)), :type => "application/pdf"
end
end
Unfortunately, the hack will not be ok when hosting on S3. And it prevents me to conveniently browse my file system to look at the uploaded document when debugging my iOS app.
Downloading the file directly from S3 via an FTP client such as Transmit gets me the same corrupted file.
What is the problem? Where do you think I should further troubleshoot, client or server? How? Any idea, tip, hunch of things to assert or look at?
If you don't have any solution, I would also be glad to be given a temporary fix to set properly again the content_type (or whatever the problem is) on the corrupted file after having downloaded it.
What happens if you name the uploaded file “upload.pdf” instead of “upload.pfd”?
Related
I am trying to send images to my Rails app and then store them via Active Storage.
I tried Base64 and direct upload and researched for hours but nothing really works.
Can somebody point me to a good way?
My last attempt was to use Base64 like so:
def attach_preview
page = Page.first
content = JSON.parse(request.body.read)
decoded_data = Base64.decode64(content["file_content"].force_encoding("UTF-8"))
begin
file = Tempfile.new('test')
file.write decoded_data
#page.thumbnail = file
filename = "foooo"
page.thumbnail.attach(io: File.read(file), filename: filename)
if page.save
render :json => {:message => "Successfully uploaded the profile picture."}
else
render :json => {:message => "Failed to upload image"}
end
ensure
file.close
file.unlink
end
end
But this results in a "\xAB" from ASCII-8BIT to UTF-8 error.
Dont really care if its Base64 or something else, I just need a way :-)
This works, I use IO directly since ActiveStorage needs it anyway.
def attach_thumbnail
content = JSON.parse(request.body.read.force_encoding("UTF-8"))
decoded_data = Base64.decode64(content["file_content"])
io = StringIO.new
io.puts(decoded_data)
io.rewind
#page.thumbnail.attach(io: io, filename: 'base.png')
#page.save
render json: {
success: #page.thumbnail.attached?,
thumbnail_url: url_for(#page.thumbnail),
page: #page
}
end
In this way, I download a image successfully:
GTLServiceDrive *drive = ...;
GTLDriveFile *file = ...;
NSString *url = [NSString stringWithFormat:#"https://www.googleapis.com/drive/v3/files/%#?alt=media",file.identifier];
GTMSessionFetcher *fetcher = [drive.fetcherService fetcherWithURLString:url];
[fetcher beginFetchWithCompletionHandler:^(NSData *data, NSError *error) {
if (error == nil) {
NSLog(#"Retrieved file content");
// Do something with data
} else {
NSLog(#"An error occurred: %#", error);
}
}];
the document of google drive sdk tell me to download a pdf or google docs should use another url:
NSString *url = [NSString stringWithFormat:#"https://www.googleapis.com/drive/v3/files/%#/export?alt=media&mimeType=application/pdf",file.identifier];
but I failed, error is this:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "badRequest",
"message": "Bad Request"
}
],
"code": 400,
"message": "Bad Request"
}
}
In downloading file make sure your app must be authorized with a scope that allows reading of file content. For example, an app using the drive.readonly.metadata scope would not be authorized to download the file contents. Users with edit permission may restrict downloading by read-only users by setting the viewersCanCopyContent field to true.
There is another way to download a file by using the partial download. It involves downloading only a specified portion of a file. You can specify the portion of the file you want to dowload by using a byte range with the Range header.
This SO question is using partial download. I think it can help you.
This code demonstrate how to download a Google Document in PDF format.
String fileId = "1ZdR3L3qP4Bkq8noWLJHSr_iBau0DNT4Kli4SxNc2YEo";
OutputStream outputStream = new ByteArrayOutputStream();
driveService.files().export(fileId, "application/pdf")
.executeMediaAndDownloadTo(outputStream);
I have client code that communicates with the server to create an account. The communication works, but the error response is not received correctly. In the Objective-C code below, I use AFNetworking to send the request. I purposely sent an invalid email address and expected the failure block to get executed, but my code kept going into the success block.
- (void)createAccount:(NSString *)email
andUsername:(NSString *)username
andPassword:(NSString *)password
success:(SuccessBlock)success
failure:(FailureBlock)failure
{
NSDictionary *params = #{
#"email": email,
#"username" : username,
#"password" : password,
#"client_id" : #"12345"
#"client_secret" : #"abcdef"
};
NSString *requestUrl = [self pathForEndpoint:#"users/new"
withVersion:ServiceRemoteRESTApibbPressExtVersion_1_0];
[self.api POST:requestUrl parameters:params
success:^(AFHTTPRequestOperation *operation, id responseObject) {
success(responseObject); // WENT IN THIS PATH
}
failure:^(AFHTTPRequestOperation *operation, NSError *error){
failure(error); // EXPECTS THIS PATH
}
];
}
On the server side, my PHP scripts is coded as follow. I did check the debug.log and saw the printed message: *** users_new_REST_API() email invalid
add_action( 'rest_api_init', function () {
register_rest_route( 'bbpress_ext/v1', '/users/new', array(
'methods' => 'POST',
'callback' => 'users_new_REST_API'
) );
} );
public function users_new_REST_API( $request ) {
$user_name = $request['username'];
$password = $request['password'];
$user_email = $request['email'];
$client_id = $request['client_id'];
$client_secret = $request['client_secret'];
if ( ! is_email( $user_email ) ) {
error_log('*** users_new_REST_API() email invalid');
return new WP_Error('email_invalid', 'Invalid email entered.', array( 'status' => 524) );
}
}
I can't tell if AFNetworking is misbehaving or the REST-API (beta version 2) for Wordpress is broken.
Your understanding of AFNetworking error handling is incomplete. The error handler is for networking errors. It doesn't trigger for HTTP error codes. That's up to your app to decipher and handle. So, whether WP_Error returns a non-2XX HTTP status, or it just returns some kind of JSON object with some error information, AFNetworking is going to report the request succeeded.
It's not the job of AFNetworking to deal with app-level protocol logic.
I'm trying to upload an image to paperclip and save it to s3. However, I get the following error in my console
!! Unexpected error while processing request: invalid byte sequence in UTF-8
There are a few responses on StackOverflow about how to resolve this problem, though most point to the original solution being an update to Rack. However, I'm using Ruby 1.9.3 and Rails 3.1.3, and believe I don't have Rack (I haven't installed it as a Gem, should I??).
The filenames that I've been trying are fairly simple, so I'm assuming the issue is in the actual file, but I'm not sure how to debug which upload variable the error is coming from. Rails isn't putting any of these errors in the log files, so I can't seem to get more details.
My controller is fairly simple, just like the example on the paperclip github documentation
def create
wine_photo = WinePhoto.create(params[:wine_photo])
return render :json => wine_photo
end
though originally I used the more common
wine_photo - WinePhoto.new(params[:wine_photo])
if wine_photo.save
return render :json => wine_photo
else
return render :json => wine_photo.errors
end
My model (which I doubt is very helpful) is
class WinePhoto true
validates_with AttachmentPresenceValidator, :attributes => :photo
belongs_to :wine
belongs_to :user
def photo_url
photo.url
end
end
based on this response on stackoverflow, Ruby Invalid Byte Sequence in UTF-8, I've tried the below in my controller
def create
wine_photo = WinePhoto.new(params[:wine_photo])
wine_photo.photo = IO.read(wine_photo.photo).force_encoding("ISO-8859-1").encode("utf-8", replace: nil)
...
but still got the error.
Any suggestions on how to get past this encoding issue? Is there a way to confirm that the error is coming from the file being uploaded?
My upload code (ajax) is
save_photo: function(){
var file = document.getElementById('file_api').files[0];
console.log(file);
var xhr = new XMLHttpRequest();
if (xhr.upload && file.type == "image/jpeg" ) {
// create progress bar
var o = document.getElementById("progress");
var progress = o.appendChild(document.createElement("p"));
progress.appendChild(document.createTextNode("upload " + file.name));
// progress bar
xhr.upload.addEventListener("progress", function(e) {
var pc = parseInt(100 - (e.loaded / e.total * 100));
progress.style.backgroundPosition = pc + "% 0";
}, false);
// file received/failed
xhr.onreadystatechange = function(e) {
if (xhr.readyState == 4) {
progress.className = (xhr.status == 200 ? "success" : "failure");
}
};
// start upload
xhr.open("POST", document.getElementById("add_photo").action, true);
xhr.setRequestHeader("X_FILENAME", file.name);
xhr.send(file);
}
}
and the params for file are
File {webkitRelativePath: "", lastModifiedDate: Thu Nov 10 2011 09:40:39 GMT+1100 (AUS Eastern Summer Time), name: "WP_000012.jpg", type: "image/jpeg", size: 1344450}
After two days of messing with this, it turns out the problem was in my ajax upload, rails wasn't getting the file field at all.
I followed this blog post to get the ajax upload working,
https://github.com/newbamboo/example-ajax-upload/blob/master/public/index.html
and changed the name in my input file to < input type="file" name="wine_photo[photo]" > where before the name was just photo.
the html form now also has
<input name="utf8" type="hidden" value="✓">
I have a remote directory with several subdirectories and files in them on Dropbox.
remote side:
-Mobile Profiles *(root)*
-- Custom Profiles
--- Profile1
--- Profile2
--- Profile3
Uploading the files and directories / and subdirectories with files is not a problem. I am having a brain fart when it comes to getting the subdirectories and their contents from dropbox to the device.
put
-(void)backupCustomProfiles {
for ( NSString *file in [[NSFileManager defaultManager] contentsOfDirectoryAtPath:MP_CUSTOM error:&error] ) {
[self.restClient uploadFile:file toPath:#"/Mobile Profiles/Custom Profiles/" fromPath:EasyStrings(MP_CUSTOM,file)];
}
}
get
-(void)restoreCustomProfiles {
for ( ) {
/* ? */
}
}
I am not sure how to iterate through the subdirectories on the remote side.
First load the directory metadata, then load the files that it references.
To limit the number of parallel fetches, use a NSOperationQueue for all of the loadMetadata and loadFile calls to limit the number of parallel fetches. And to avoid redundant file downloads, remember the downloaded metadata in a plist.
- (void) restoreCustomProfiles
{
[self.client loadMetadata:#"/Mobile Profiles/Custom Profiles" withHash:hash];
}
- (void) restClient:(DBRestClient*)client loadedMetadata:(DBMetadata*)metadata
{
for (DBMetadata* child in metadata.contents) {
NSString *path = [child.path lowercaseString];
if (child.isDirectory) {
[client loadMetadata:child.path withHash:hash];
} else {
[client loadFile:pathToDownload intoPath:[
self.directory stringByAppendingString:path]];
}
}
}
- (void) restClient:(DBRestClient*)client loadedFile:(NSString*)destPath
{
// successfully downloaded a file to destPath
}