Upload with paperclip very slow (unicorn) - ruby-on-rails-3

Question

Sitting here with a simple rails 3 app in which I have a simple Gallery model and each gallery has many images. The image model is extended with paperclip and with the following options
has_attached_file :local,
:styles => {
:large => "800x800>",
:medium => "300x300>",
:thumb => "100x100#",
:small => "60x60#"
}
In my galleries_controller I have the following action that is implemented in order to work with the jQuery-File-Upload plugin. thereby the json response.
def add_image
gallery = Gallery.find params[:id]
image = gallery.images.new({:local => params[:local]})
if image.save
render :json => {:thumb => image.url(:thumb), :original => image.url}
else
render :json => { :result => 'error'}
end
end
To me this is fairly straight forward. But here comes the issue. In Development under mongrel any kind of upload works just fine with about 500-1000ms/upload.
However when I push it in to production I constantly get timeouts of my unicorn workers and when it does send an image through it takes anywhere from 30-55 seconds for one file.
the files I upload are around 100k in size
I have done some testing of the bandwidth between my VPS and my dev computer witH ipref and got an average speed of about 77kbps so the upload should not be a problem.
Note I also did a test with a non ajax file upload using the same app with user model that has an avatar.
Development => Completed 302 Found in 693ms
Production => Completed 302 Found in 21618ms
Anyone experienced a similar issue with (rails3, unicorn) file uploads?

Answer 1

So after digging around I managed to determine that on my VPS it was the OpenMP Option in ImageMagick that was causing the very slow operation. So my first attempt was to rebuild the native Ubuntu 10.04 package with the --disable-openmp flag added. This failed for some reason and while I am not sure why the package kept comming out with openMP still active. My current solution is now instead to backport ImageMagick from Ubuntu 10.10. Below follows the steps I took:
Step 1 download the following files:
imagemagick_6.6.2.6-1ubuntu1.1.dsc
imagemagick_6.6.2.6.orig.tar.bz2
imagemagick_6.6.2.6-1ubuntu1.1.debian.tar.bz2
from here
Step 2 unpack the package
$ dpkg-source -x imagemagick_6.6.2.6-1ubuntu1.1.dsc
Step 3 edit the rules
$ cd imagemagick-6.6.2.6
$ vim debian/rules
Add the the follwing line to the ./configure statment on line 25-39. I added mine on line 34.
34: --disable-openmp \
Step 4 add dependencies and build ( I needed these dependencies)
$ sudo apt-get install liblqr-1-0-dev librsvg2-dev
$ dpkg-buildpackage -b
Step 5 Out with the old, in with the new
$ sudo apt-get remove --purge imagemagick
$ sudo dpkg -i libmagickcore3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i libmagickwand3_6.6.2.6-1ubuntu1.1_amd64.deb
$ sudo dpkg -i imagemagick_6.6.2.6-1ubuntu1.1_amd64.deb
Step 6 Once again have fast image conversions
_before_ (with openmp)
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real 0m11.602s
user 0m11.414s
sys 0m0.069s
_after_
$ time utilities/convert 'image.jpg' -resize "x60" -crop "60x60+10+0" +repage 'thumb'
real 0m0.077s
user 0m0.058s
sys 0m0.019s

Answer 2

If processing takes a long time, consider processing the thumbnails in a separate workers.
Request: accept file; save it to disk; post job to queue
Worker: pop job from queue; create thumbnails; repeat
Delayed::Job and Resque are great solutions for this.