Formerly: Running a model method on a paperclip attachment after create or update (paperclip callbacks don't seem to work)
Edit (later that day)
I figured out my problem. The processor apparently works with the file that is updated, but doesn't save any files until after processing. I changed my Zip::ZipFile to open 'file' rather than 'attachment.path' since the attachment path doesn't actually hold anything yet. This fixed the first problem. Now I'm having other problems that I'll need to track down. But the answer below is mostly correct.
Edit (1/31/2011):
So I have taken the advice to create a processor for my attachment which will perform all the necessary actions. So far, it looks like it should work; the processor starts and does all the initialization stuff, apparently. However, when I get the point where I want to access the zip file that gets uploaded, I get an error saying that the file cannot be found. The code for my processor is below:
class Extractor < Processor
attr_accessor :resolution, :whiny
def initialize(file, options = {}, attachment = nil)
super
#file = file
#whiny = options[:whiny].nil? ? true : options[:whiny]
#basename = File.basename(#file.path, File.extname(#file.path))
#attachment = attachment
#instance = attachment.instance
end
def make
# do your conversions here, you've got #file, #attachment and #basename to work with
export_path = attachment.path.gsub('.zip', '_content')
Zip::ZipFile.open(attachment.path) { |zip_file|
zip_file.each { |image|
image_path = File.join(export_path, image.name)
FileUtils.mkdir_p(File.dirname(image_path))
unless File.exist?(image_path)
zip_file.extract(image, image_path)
# ..stuff that it does..
end
}
}
# clean up source files, but leave the zip
FileUtils.remove_dir(export_path)
# you return a file handle which is the processed result
dst = File.open result_file_path
end
end
And here is the contents of the error that I get:
Zip::ZipError in GalleriesController#create
File /home/joshua/railscamp/moments_on_three/public/assets/archives/delrosario.zip not found
Rails.root: /home/joshua/railscamp/moments_on_three
Application Trace | Framework Trace | Full Trace
config/initializers/extractor.rb:16:in `make'
app/controllers/galleries_controller.rb:32:in `new'
app/controllers/galleries_controller.rb:32:in `create'
Request
Parameters:
{"utf8"=>"✓",
"authenticity_token"=>"0s4L4MrlqjDTMjzjgkUdvUxeHklZNOIShDhT6fgOICY=",
"gallery"=>{"name"=>"DelRosario",
"public"=>"0",
"owner_id"=>"1",
"shoot_date(1i)"=>"2011",
"shoot_date(2i)"=>"1",
"shoot_date(3i)"=>"31",
"category_id"=>"1",
"archive"=>#<ActionDispatch::Http::UploadedFile:0x00000004148d78 #original_filename="delrosario.zip",
#content_type="application/zip",
#headers="Content-Disposition: form-data; name=\"gallery[archive]\"; filename=\"delrosario.zip\"\r\nContent-Type: application/zip\r\n",
#tempfile=#<File:/tmp/RackMultipart20110131-9745-14u347v>>},
"commit"=>"Create Gallery"}
From what I can tell it's looking for the file in the right place, but the file doesn't seem to be uploaded yet to access it. As far as I'm aware, Paperclip is smart enough to know and wait for the attachment to upload before it tries to process it. Can anyone spot what I'm doing wrong here?
Thanks a lot.
Old stuff:
I'm developing a photo gallery app using Rails 3 and Paperclip. The Admin is able to create a gallery and upload a zip file containing a bunch of images.
What I want to happen:
Enter gallery info and zip file to upload into the form.
Hit 'Create Gallery' button.
Form posts, gallery saves, and zip file gets uploaded.
After zip file is uploaded, run the method :extract_photos (btw, this code
works).
4.a. At the end of this method, zip file is destroyed.
Admin is redirected to gallery page with all the photos in it (where
gallery has_many photos).
I've tried to make this work several different ways.
Before, I created a controller method which would allow the Admin to click a link which ran the :extract_photos method. This worked on my computer, but for some reason the server had trouble routing this on the client's computer. So it's a no go. Plus I thought it was an ugly way of doing it.
Recently, I tried using callback methods. after_save didn't work because it apparently interrupts the form POST and the file doesn't get uploaded and the :extract_photos method can't find the file.
I checked out callback methods on the Paperclip github page, and it talks about the callbacks:
Before and after the Post Processing
step, Paperclip calls back to the
model with a few callbacks, allowing
the model to change or cancel the
processing step. The callbacks are
"before_post_process" and
"after_post_process" (which are called
before and after the processing of
each attachment), and the
attachment-specific
"beforepost_process" and
"afterpost_process". The callbacks are
intended to be as close to normal
ActiveRecord callbacks as possible, so
if you return false (specifically -
returning nil is not the same) in a
before filter, the post processing
step will halt. Returning false in an
after filter will not halt anything,
but you can access the model and the
attachment if necessary.
I've tried using before_post_process and after_post_process, but it can't find the file to run the process, so the file obviously isn't getting uploaded by the time those methods are getting called (which I think is strange). Additionally, when I try beforepost_process and afterpost_process, I get a NoMethodError.
So how do I call a method on an attachment when it is created or updated, but after the file is uploaded and in the right place?
UPDATE
I tried the code below, moving my extraction method code into the make method of the processor. I've gotten farther than I have did before with trying to write a processor, but it's still a no-go. The process throws an exception as soon as I try and open the uploaded file for processing, saying the file doesn't exist. The naming scheme is correct and everything, but still nothing is getting uploaded before the process is getting triggered. Does anyone have any idea why this is happening?
You can write your own processor to accomplish this.
in your model when declaring the paperclip stuff add a custom processor
has_attached_file :my_attachment, {
:styles => {:original => {:processors => [:my_processor]}}
}.merge(PAPERCLIP_SETTINGS)
then write your own processor and put it config/initializers:
module Paperclip
class MyProcessor < Processor
attr_accessor :resolution, :whiny
def initialize(file, options = {}, attachment = nil)
super
#file = file
#whiny = options[:whiny].nil? ? true : options[:whiny]
#basename = File.basename(#file.path, File.extname(#file.path))
#attachment = attachment
end
def make
# do your conversions here, you've got #file, #attachment and #basename to work with
# you return a file handle which is the processed result
dst = File.open result_file_path
end
end
end
I am using a custom processor to things similar to what you are doing with lots of processing and converting of the file in the middle and it seems to work well.
Related
I have just switched from carrierwave_backgrounder to carrierwave_direct. I have carrierwave_direct set up and functioning. That is, the main file is being uploaded and can be displayed in the view. However, my uploader versions are not being created.
Following is my job:
class ProcessReceiptJob < ApplicationJob
queue_as :process_receipt
def perform(expense_id, key)
expense = Expense.find expense_id
uploader = expense.receipt
expense.key = key
expense.remote_receipt_url = uploader.direct_fog_url(with_path: true)
expense.save!
# expense.recreate_versions!
end
after_perform do |job|
expense = Expense.find(job.arguments.first)
expense.update_column :receipt_processing, false
end
end
When exactly does carrierwave_direct process the versions---or, when is carrierwave instructed to process the versions? I'm assuming that loading the original image using expense.remote_receipt_url, and then calling save! triggers the uploader to process the versions. Is that correct?
In any case, my original image is being uploaded via a background job---however, the versions are not being created/uploaded.
Do I need to "recreate_versions" even thought they don't previously exist? Do I need to somehow explicitly process versions after pointing to the source file or should that be handled automagically?
I was not saving the model after assigning it :key BEFORE sending it the background worker. I was sending the key to the background worker as an argument and then saving the model in processing the job. This was the problem. It is mentioned in the docs the need to save the model after assigning it :key in the success action.
So, I had to update_attributes(key: params[:key]) and THEN call my background job (where incidentally the model is saved again).
Thoughtbot's Paperclip gem (v 3.2.1) will clean up filenames, replacing spaces and other special chars with underscores. I am reimporting data and need to check whether attachments have already been uploaded, but the original file name may not match the attached file name in Paperclip. What method does Paperclip use for the clean up?
Found the answer digging into the source code. It's a private instance method Paperclip::Attachment#cleanup_filename . (Since this is a short lived (one off) import task I dont mind the (slight) risk of using an unpublished method.)
Thus my code looks something like this (Post has_many :attachments; Attachment has_attached_file :attached )
if #post.attachments.present?
cleaned_filename = #post.attachments.first.attached.send :cleanup_filename, filename
if #post.attachments.map(&:attached_file_name).include? cleaned_filename
puts "already attached: #{filename}"
return
end
end
puts "attaching upload: #{filename}"
I am trying to move files around my S3 bucket using CarrierWave to reorganize the folder structure.
I came to an existing Rails application where all images for a class are being uploaded into a folder called /uploads. This is causing problems where if two users upload different images with the same file-name, the second image overwrites the first. To solve this, I want to reorganize the folders to place each image in its own directory according to the ActiveRecord object instance. We are using CarrierWave to manage file uploads.
The old uploader code had the following method:
def store_dir
"uploads"
end
I modified the method to reflect my new file storage scheme:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
This works great for new images, but breaks the url for old images. Existing images report their URL to be inside the new folder immediately when I change the model, while the image files are still stored in /uploads.
> object.logo.store_dir
=> "uploads/object/logo/133"
This is not correct. This object should report its logo in /uploads.
My solution is to write a script to move the image files, but I haven't found the correct methods in CarrierWave to move the files. My script would look something like this:
MyClass.all.each |image|
filename = file.name #This method exists in my uploader, returns the file name
#Move the file from "/uploads" to "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
What should I do in line three of my script to move the file to a new location?
WARNING: This is untested, so please don't use on production before testing it out.
Here's the thing, once you change the contents of 'store_dir', all your old uploads will become missing. You know this already. Interacting with S3 directly seems like the most obvious way of solving this, since carrierwave doesn't have a move function.
One thing that might work, would be to re-'store' your uploads and change the 'store_dir' path in the 'before :store' callback.
In your uploader:
#Use the old uploads directory so carriewave knows where the original upload is
def store_dir
'uploads'
end
before :store, :swap_out_store_dir
def swap_out_store_dir
self.class_eval do
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
end
end
And then run a script like this:
MyClass.all.each do |image|
image.image.cache! #create a local cache so that store! has something to store
image.image.store!
end
After this, verify that the files have been copied to the correct locations. You'll then have to delete the old upload files. Also, remove the one time use uploader code above and replace it with your new store_dir path:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id} "
end
I haven't tested this out, so I can't guarantee it will work. Please use test data first to see if it works and comment here if you've had any success.
I have an ipad app that uses dropbox to sync images to the cloud so that i can access them with a webapp and process them etc etc.
the part i'm having trouble with is getting the file from dropbox to s3 via carrierwave. i have a photo model and i can create a new photo and upload and an image successfully. I can also put a file on dropbox. However when i try to get a file off of dropbox and put it on s3, the contents of the file is just text.
Are there some sort of mime types i need to set or something?
I am using dropbox_sdk and the get_file_and_metadata method. It returns me the file object successfully, but the contents are all just text. this is me hard coding the image file so i can be sure it exists..
dbsession = DropboxSession.deserialize(session[:dropbox_session])
#client = DropboxClient.new(dbsession, ACCESS_TYPE) #raise an exception if session not authorized
#info = #client.account_info # look up account information
#photo = Photo.new(params[:photo])
#db_image metadata = #client.get_file_and_metadata('/IMG_1575.jpg')
the part i don't know how to do is say take this image #db_image and use that file when creating a new photo and store it on S3.
I'm thinking it might be a 'mime type' issue, but i've read that that is only based on the file ext.
any insight you guys could share would really help me get past this hurdle. thanks!
Figured this out. Instead I used the direct_url.url method that is part of the dropbox-api gem used with the carrierwave gem. the direct_url.url method returns a secure full url path to that file that you can use as the remote_url value for carrierwave.
#client = Dropbox::API::Client.new(:token => 'derp', :secret => 'derp')
#dropbox_files = #client.ls "images/#{#event.keyword}/#{photo_size}/"
#dropbox_files.each do |f|
photo_exists = Photo.where(:dropbox_path => f.direct_url.url).count
if photo_exists == 0
#photo = Photo.create(:remote_filename_url => f.direct_url.url,
:dropbox_path => f.direct_url.url,
:event_id => #event.id)
end
end
Now, i'm pretty new at ruby, so i'll be posting a better way to step through the results, as that seems pretty slow and clunky.
I have a strange bug in my Rails 3 app. I am using this code to send images that are not public:
image = open(f, "rb") { |io| io.read }
send_data(image, :disposition => 'inline')
I am using this code to display images in admin pages and user pages. If I use development environment this code works OK and the images are displayed on both pages. But if I use production environment, this images are displayed only in admin pages but not user pages. I can click on the images that are not displayed, and select "properties". Under image type I see:
application/xhtml+xml
But other public images has under the type JPG image/PNG image or something like this.
Which difference between the environemnts could be causing images not to work and how can I fix this, so the images will be properlly displayed on the all pages?
I had a distinctly similar symptom. I know this is an old issue and already resolved but I thought I would contribute the findings of my situation which turned out to be a different cause.
I was building a CSV file and using send_file to send the file to the browser. In development it worked great, in production the browser reported a page not found.
Here is the action from the controller.
def export
#campaign = LiveEmailCampaign.find params[:id]
#campaign.recipients_csv do |csv_file|
send_file csv_file,
filename: #campaign.name,
type: Mime::CSV
end
end
And the CSV is build from this code in a model.
def recipients_csv
tempfile = Tempfile.new(self.name.downcase.dasherize)
CSV.open tempfile, 'w' do |csv|
recipients.each do |recipient|
csv << [recipient]
end
end
yield tempfile
end
After a few minutes of research I determined that the culprit was a conflict between the XSendFile directive in Apache on the production server and the temporary path being used to write the CSV data to. In my case XSendFile was set for only the app root and the temp file was in /tmp on the server.
Instead of tampering with the XSendFile config at the server level I just instructed Tempfile to use the tmp folder in the Rails app.
So, I changed the call to Tempfile in the model method to this
tempfile = Tempfile.new(self.name.downcase.dasherize)
Now, Rails and Apache are friends again. I just need to refactor this code because it doesn't explicitly unlink the created temporary file. Best practice is to explicitly unlink temporary files.