I followed this guide to get drag and drop file uploads through AJAX: http://dannemanne.com/posts/drag-n-drop_upload_that_works_with_ror_and_paperclip
Everything was working fine on my development environment with WebBrick but if I deploy to PhusionPassenger then I get:
Paperclip::AdapterRegistry::NoHandlerError (No handler found for #<PhusionPassenger::Utils::RewindableInput:0x000000041aef38 #io=#<PhusionPassen...
I'm using this in my controller:
before_filter :parse_raw_upload, :only => :bulk_submissions
def bulk_submissions
...
#submission = Submission.create!(url: "", file: #raw_file, description: "Please edit this description", work_type: "other", date_completed: DateTime.now.to_date)
...
end
private
def parse_raw_upload
if env['HTTP_X_FILE_UPLOAD'] == 'true'
#raw_file = env['rack.input']
#raw_file.class.class_eval { attr_accessor :original_filename, :content_type }
#raw_file.original_filename = env['HTTP_X_FILE_NAME']
#raw_file.content_type = env['HTTP_X_MIME_TYPE']
end
end
Looking at the request itself all the headers are set (X_MIME_TYPE, X_FILE_NAME) etc.
Any ideas?
Thanks in advance!
The example you're cribbing from expects the file stream to be a StringIO object, but Passenger is giving you a PhusionPassenger::Utils::RewindableInput object instead.
Fortunately, a RewindableInput is duckalike to StringIO for this case, so Paperclip's StringioAdapter can be used to wrap your upload stream.
Inside the if block in your parse_raw_upload, at the end, do:
if #raw_file.class.name == 'PhusionPassenger::Utils::RewindableInput'
#raw_file = Paperclip::StringioAdapter.new(#raw_file)
end
Related
I am trying to send images to my Rails app and then store them via Active Storage.
I tried Base64 and direct upload and researched for hours but nothing really works.
Can somebody point me to a good way?
My last attempt was to use Base64 like so:
def attach_preview
page = Page.first
content = JSON.parse(request.body.read)
decoded_data = Base64.decode64(content["file_content"].force_encoding("UTF-8"))
begin
file = Tempfile.new('test')
file.write decoded_data
#page.thumbnail = file
filename = "foooo"
page.thumbnail.attach(io: File.read(file), filename: filename)
if page.save
render :json => {:message => "Successfully uploaded the profile picture."}
else
render :json => {:message => "Failed to upload image"}
end
ensure
file.close
file.unlink
end
end
But this results in a "\xAB" from ASCII-8BIT to UTF-8 error.
Dont really care if its Base64 or something else, I just need a way :-)
This works, I use IO directly since ActiveStorage needs it anyway.
def attach_thumbnail
content = JSON.parse(request.body.read.force_encoding("UTF-8"))
decoded_data = Base64.decode64(content["file_content"])
io = StringIO.new
io.puts(decoded_data)
io.rewind
#page.thumbnail.attach(io: io, filename: 'base.png')
#page.save
render json: {
success: #page.thumbnail.attached?,
thumbnail_url: url_for(#page.thumbnail),
page: #page
}
end
I'm having issues with an API in Rails4/mongoid application. I need to manipulate the data with a python 3 script through an API but I'm getting
NoMethodError (undefined method `permit' for "note_id":String):
error when I try to submit the request.
My python codes looks like this
import requests
import json
url = 'http://0.0.0.0:3000/api/v1/note_proc_logs.json'
payload = {'note_proc_log' : { 'note_id' : '120904'}}
head = {"Authorization":"Token token=xxxxxxxxxxxxxxxxxxx"}
r = requests.post(url, payload, headers=head)
The API controler
module Api
module V1
# This class does not inherit from ApplicationController like the rest to skip Devise authentication
class NoteProcLogsController < ActionController::Base
before_filter :restrict_access if Rails.env.development? == false
respond_to :json
def create
Rails.logger.warn "note_proc_log_params: #{params}" if Rails.env.development?
#note_proc_log = NoteProcLog.new(note_proc_log_params)
respond_to do |format|
if #note_proc_log.save
format.json { render :show, status: :created, location: #note_proc_log }
else
format.json { render json: #note_proc_log.errors, status: :unprocessable_entity }
end
end
end
private
def restrict_access
authenticate_or_request_with_http_token do |token, options|
ApiKey.where(access_token: token).exists?
end
end
# Never trust parameters from the scary internet, only allow the white list through.
def note_proc_log_params
params.require(:note_proc_log).permit(:note_id)
end
end
end
end
I saw few question with the same error but couldn't find a solution to my problem.
Any help would be greatly appreciated.
UPDATE:
Rails.logger.warn "note_proc_log_params: #{params}" if Rails.env.development?
gives me
W, [2016-07-25T15:10:38.362848 #48352] WARN -- : params: {"note_proc_log"=>"note_id", "format"=>"json", "controller"=>"api/v1/note_proc_logs", "action"=>"create"}
The problem was in the python script. A simple python dictionary is ok as payload but nested ones appear not to be.
My final python script look like this
import requests
import json
url = 'http://0.0.0.0:3000/api/v1/note_proc_logs.json'
payload='note_proc_log[chip_id]=120904¬e_proc_log[test_timestamp]=2016-07-19T13:24:49'
head = {"Authorization":"Token token=xxxxxxxxxxxxxxxxxxxx"}
r = requests.post(url=url, data=payload, headers=head)
On Rails side everything will be treated as string so no need for adding additional quotation marks, even for strings with spaces, parent attribute has to be specified for each child attribute and elements separated with &.
This is what works for me, it would be interesting to know if there are other/better ways to do it, in particular how to include an Array of values.
I'd like a caching version of Mechanize. The idea is that #get(uri...) checks to see if that uri has been previously fetched, and if so, fetch the response from the cache rather than hitting the web. If not in the cache, it hits the web and saves the response in the cache.
My naive approach doesn't work. (I probably don't need to mention that CachedWebPage is a subclass of ActiveRecord::Base):
class CachingMechanize < Mechanize
def get(uri, parameters = [], referer = nil, headers = {})
page = if (record = CachedWebPage.find_by_uri(uri.to_s))
record.contents
else
super.tap {|contents| CachedWebPage.create!(:uri => uri, :contents => contents)}
end
yield page if block_given?
page
end
end
This fails because the object returned by Mechanize#get() is a complex, circular structure that neither YAML nor JSON want to serialize for storage into the database.
I realize that what I want is to capture the low-level contents before Mechanize parses it.
Is there clean way to do this? I think I can use Mechanize's post_connect hook to access the raw page coming in, but I don't see how to subsequently pass the cached raw page to Mechanize for parsing.
Is there some package I should be using that does web page caching already?
It turns out the solution was simple, albeit not entirely clean. It's a simple matter to cache the results of Mechanize#get() like this:
class CachingMechanize < Mechanize
def get(uri, parameters = [], referer = nil, headers = {})
WebCache.with_web_cache(uri.to_s) { super }
end
end
... where with_web_cache() uses YAML to serialize and cache the object returned by super.
My problem was that by default, Mechanize#get() returns a Mechanize::Page object containing some lambda object, which cannot be dumped and loaded by YAML. The fix was to eliminate those lambdas, which turned out to be rather simple. Full code follows.
class CachingMechanize < Mechanize
def initialize(*args)
super
sanitize_scheme_handlers
end
def get(uri, parameters = [], referer = nil, headers = {})
WebCache.with_web_cache(uri.to_s) { super }
end
# private
def sanitize_scheme_handlers
scheme_handlers['http'] = SchemeHandler.new
scheme_handlers['https'] = scheme_handlers['http']
scheme_handlers['relative'] = scheme_handlers['http']
scheme_handlers['file'] = scheme_handlers['http']
end
class SchemeHandler
def call(link, page) ; link ; end
end
end
the moral: don't try to YAML.dump and YAML.load objects containing lambda or proc
This goes beyond just this example: if you see a YAML error that reads:
TypeError: allocator undefined for Proc
Check to see if there's a lambda or proc in the object you're trying to serialize and deserialize. If you are able (as I was in this case) to replace the lambda with a method call to an object, you should be able to work around the problem.
Hope this helps someone else.
Update
In response to #Martin's request for the definition of WebCache, here 'tis:
# Simple model for caching pages fetched from the web. Assumes
# a schema like this:
#
# create_table "web_caches", :force => true do |t|
# t.text "key"
# t.text "value"
# t.datetime "expires_at"
# t.datetime "created_at", :null => false
# t.datetime "updated_at", :null => false
# end
# add_index "web_caches", ["key"], :name => "index_web_caches_on_key", :unique => true
#
class WebCache < ActiveRecord::Base
serialize :value
# WebCache.with_web_cache(key) {
# ...body...
# }
#
# Searches the web_caches table for an entry with a matching key. If
# found, and if the entry has not expired, the value for that entry is
# returned. If not found, or if the entry has expired, yield to the
# body and cache the yielded value before returning it.
#
# Options:
# :expires_at sets the expiration date for this entry upon creation.
# Defaults to one year from now.
# :expired_prior_to overrides the value of 'now' when checking for
# expired entries. Mostly useful for unit testing.
#
def self.with_web_cache(key, opts = {})
serialized_key = YAML.dump(key)
expires_at = opts[:expires_at] || 1.year.from_now
expired_prior_to = opts[:expired_prior_to] || Time.zone.now
if (r = self.where(:key => serialized_key).where("expires_at > ?", expired_prior_to)).exists?
# cache hit
r.first.value
else
# cache miss
yield.tap {|value| self.create!(:key => serialized_key, :value => value, :expires_at => expires_at)}
end
end
# Prune expired entries. Typically called by a cron job.
def self.delete_expired_entries(expired_prior_to = Time.zone.now)
self.where("expires_at < ?", expired_prior_to).destroy_all
end
end
I'm trying to get an ajax upload working with rails 3.1.3 and paperclip.
I found this solution to my problem Rails 3 get raw post data and write it to tmp file, but using this, I get an 'encoding undefined conversion error "\xFF" from ASCII-8BIT to UTF-8.
The error occurs at the line #user.photo = #user.photo = QqFile.parse(params[:qqfile], request)
I have not edited the code supplied in the previous answer, but I'll include it here so you don't have to switch back and forth.
the gem list paperclip, returns 2.5.2, 2.4.5, 2.3.8
my controller
def create
#user = User.new(params[:user])
#user.photo = QqFile.parse(params[:qqfile], request)
if #user.save
return render :json => #user
else
return render :json => #user.errors
end
end
qq_file.rb
# encoding: utf-8
require 'digest/sha1'
require 'mime/types'
# Usage (paperclip example)
# #asset.data = QqFile.new(params[:qqfile], request)
class QqFile < ::Tempfile
def initialize(filename, request, tmpdir = Dir::tmpdir)
#original_filename = filename
#request = request
super Digest::SHA1.hexdigest(filename), tmpdir
fetch
end
def self.parse(*args)
return args.first unless args.first.is_a?(String)
new(*args)
end
def fetch
self.write #request.raw_post
self.rewind
self
end
def original_filename
#original_filename
end
def content_type
types = MIME::Types.type_for(#request.content_type)
types.empty? ? #request.content_type : types.first.to_s
end
end
This was an encoding error related to Ruby 1.9.2 (or I believe Ruby 1.9+).
this github post lead to the answer
https://github.com/lassebunk/webcam_app/issues/1
You must specify raw_post.force_encoding("UTF-8") when reading an upload as far as I could tell (I'm not a great programmer).
I was using paperclip for file upload. with validations as below:
validates_attachment_content_type :upload, :content_type=>['application/pdf'],
:if => Proc.new { |module_file| !module_file.upload_file_name.blank? },
:message => "must be in '.pdf' format"
But, my client complained today that he is not able to upload pdf. After investigating I come to know from request headers is that the file being submitted had content_type=application/octet-stream.
Allowing application/octet-stream will allow many type of files for upload.
Please suggest a solution to deal with this.
Seems like paperclip doesn't detect content type correctly. Here is how I was able to fix it using custom content-type detection and validation (code in model):
VALID_CONTENT_TYPES = ["application/zip", "application/x-zip", "application/x-zip-compressed", "application/pdf", "application/x-pdf"]
before_validation(:on => :create) do |file|
if file.media_content_type == 'application/octet-stream'
mime_type = MIME::Types.type_for(file.media_file_name)
file.media_content_type = mime_type.first.content_type if mime_type.first
end
end
validate :attachment_content_type
def attachment_content_type
errors.add(:media, "type is not allowed") unless VALID_CONTENT_TYPES.include?(self.media_content_type)
end
Based on the above, here's what I ended up with which is compatible with PaperClip 4.2 and Rails 4:
before_post_process on: :create do
if media_content_type == 'application/octet-stream'
mime_type = MIME::Types.type_for(media_file_name)
self.media_content_type = mime_type.first.to_s if mime_type.first
end
end
For paperclip 3.3 and Rails 3, I did this a bit differently
before_validation on: :create do
if media_content_type == 'application/octet-stream'
mime_type = MIME::Types.type_for(media_file_name)
self.media_content_type = mime_type.first if mime_type.first
end
end
validates_attachment :media, content_type: { content_type: VALID_CONTENT_TYPES }
By the way, i needed to do this because testing with Capybara and phantom js using attach_file did not generate the correct mime type for some files.