I'm trying to add cache_control to images within s3 buckets via a Ruby script, yet I keep running into an Access Denied (Aws::S3::Errors::AccessDenied) error. All my environment variables are correct, and I'm having no issue creating new buckets with the script, it's just every time I try to add cache_control it throws an error.
I've tried digging into the aws-sdk documentation, https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Object.html, but I haven't been able to wrap my mind around what is wrong with my script.
Here's what the code looks like when it's functional:
def initialize(region: 'us-west-2', bucket_name: 'project-images')
#bucket = Aws::S3::Resource.new(region: region).bucket(bucket_name)
end
def copy(to:, from:)
bucket.objects(prefix: from).each do |object|
_, filename = object.key.split("/")
object.copy_to(bucket: bucket.name, key: "#{to}/#{filename}")
end
end
Here's what the code looks like when I'm getting the Access Denied errors from trying to add in the cache_control and additional options:
def initialize(region: 'us-west-2', bucket_name: 'project-images')
#bucket = Aws::S3::Resource.new(region: region).bucket(bucket_name)
end
def copy(to:, from:)
bucket.objects(prefix: from).each do |object|
_, filename = object.key.split("/")
object.copy_to(bucket: bucket.name, key: "#{to}/#{filename}", acl: "public-read", cache_control: "max-age=154400", metadata_directive: "REPLACE")
end
end
Any help would be greatly appreciated!
Related
I have a user module and I have generated paperclip attachment: profile_pic
user.rb:
has_attached_file :profile_pic,
style: { :medium => "300x300>", thumb: "100x100>" },
default_url: "/images/:style/missing.png"
controller:
image_base = params[:manager][:profile_pic]
if image_base != nil
image = Paperclip.io_adapters.for(image_base)
image.original_filename = params[:manager][:file_name]
current_user.profile_pic = image
current_user.errors.delete(:profile_pic)
current_user.save
end
config/initializers/paperclip.rb:
Paperclip::DataUriAdapter.register
It's not showing any errors but If I am trying to display the image, it gives me following error:
ActionController::RoutingError (No route matches [GET] "/system/managers/profile_pics/000/000/008/original/icon_new.png"):
When I am trying in console like:
user.profile_pic.display
/system/managers/profile_pics/000/000/008/original/icon_new.png?
1556082410 => nil
Picture was saved in public folder
The default folder is the following, so Paperclip uploaded to the right place:
:rails_root/public/system/:class/:attachment/:id_partition/:style/:filename
But it looks like you are having troubles accessing the right location. Have you tried user.profile_pic.url? I tried on my working example, both .display and .url are the same but I use an AWS S3 bucket.
user.profile_pic.url should be where to find the file on the web.
user.profile_pic.path should be where to find the file on the file system.
The problem is with production mode and Solved this by adding the following line in production.rb
config.public_file_server.enabled = true
I have a Gallery and Attachment models. A gallery has_many attachments and essentially all attachments are images referenced in the ':content' attribute of Attachment.
The images are uploaded using Carrierwave gem and are stored in Aws S3 via fog-aws gem. This works OK. However, I'd like to conduct image recognition to the uploaded images with Amazon Rekognition.
I've installed aws-sdk gem and I'm able to instantiate Rekognition without a problem until I call the detect_labels method at which point I have been unable to use my attached images as arguments of this method.
So fat I've tried:
#attachement = Attachment.first
client = Aws::Rekognition::Client.new
resp = client.detect_labels(
image: #attachment
)
# I GET expected params[:image] to be a hash... and got class 'Attachment' instead
I've tried using:
client.detect_labels( image: { #attachment })
client.detect_labels( image: { #attachment.content.url })
client.detect_labels( image: { #attachment.content })
All with the same error. I wonder how can I fetch the s3 object form #attachment and, even if I could do that, how could I use it as an argument in detect_labels.
I've tried also fetching directly the s3 object to try this last bit:
s3 = AWS:S3:Client.new
s3_object = s3.list_objects(bucket: 'my-bucket-name').contents[0]
# and then
client.detect_labels( image: { s3_object })
Still no success...
Any tips?
I finally figured out what was the problem, helped by the following AWS forum
The 'Image' hash key takes as a value an object that must be named 's3_object' and which subsequently needs only the S3 bucket name and the path of the file to be processed. As a reference see the correct example below:
client = Aws::Rekognition::Client.new
resp = client.detect_labels(
image:
{ s3_object: {
bucket: "my-bucket-name",
name: #attachment.content.path,
},
}
)
# #attachment.content.path => "uploads/my_picture.jpg"
While in the Rails development environment, I am attempting to add a Sinatra app as a middleware. The Sinatra app uses the geoip gem that processes a user's ip address and returns json with their city.
I can view the returned json by going directly to the example url in the browser or using curl in the command line, http://local.fqdn.org/geoip/locate.json?ip=24.18.211.123. However when I attempt to call the url with wget from within a Rails controller, the Rails app stops responding often crashing my browser and my rails server wont exit using the control+C command.
Any clue to what is happening here? Why would going directly to the url in the browser return the proper response but my call in the controller results in a Time Out?
sinatra-geoip.rb
require 'sinatra'
require 'geoip'
require 'json'
# http://localhost/geoip/locate.json?ip=24.18.211.123
#
# {
# latitude: 47.684700012207
# country_name: "United States"
# area_code: 206
# city: "Seattle"
# region: "WA"
# longitude: -122.384803771973
# postal_code: "98117"
# country_code3: "USA"
# country_code: "US"
# dma_code: 819
# }
class GeoIPServer < Sinatra::Base
get '/geoip/locate.json' do
c = GeoIP.new('/var/www/mywebsite.org/current/GeoLiteCity.dat').city(params[:ip])
body c.to_h.to_json
end
end
routes.rb
mount GeoIPServer => "/geoip"
config/environments/development.rb
Website::Application.configure do
require "sinatra-geoip"
config.middleware.use "GeoIPServer"
...
end
controller
raw_geo_ip = Net::HTTP.get(URI.parse("http://#{geoip_server}/geoip/locate.json?ip=#{request.ip}"))
#geo_ip = JSON.parse(raw_geo_ip)
Our solution was difficult to find. We ended up finding a method in the sinatra source code call forward.
new sinatra-geoip.rb
class GeoIPServer < Sinatra::Base
if defined?(::Rails)
get '/properties.json' do
env["geo_ip.lookup"] = geo_ip_lookup(request.ip)
forward
end
end
def geo_ip_lookup(ip = nil)
ip = ip.nil? ? params[:ip] : ip
result = GeoIP.new('/var/www/mywebsite.org/current/GeoLiteCity.dat').city(ip)
result.to_h.to_json
end
end
Essentially, we removed the /geoip/locate.json route from the file and converted it to a simple method. We needed the geoip lookup to occur when the properties.json was being called, so a new param was added with the geoip information. Then we set the new param equal to #geo_ip variable in the controller.
New properties controller
if Rails.env.development? or Rails.env.test?
# Retrieves param set by sinatra-geoip middleware.
#geo_ip = JSON.parse(env["geo_ip.lookup"] || "{}")
else
# Production and staging code
end
Rather obscure problem and solution. Hopefully it will help someone out there. Cheers.
I followed this guide to get drag and drop file uploads through AJAX: http://dannemanne.com/posts/drag-n-drop_upload_that_works_with_ror_and_paperclip
Everything was working fine on my development environment with WebBrick but if I deploy to PhusionPassenger then I get:
Paperclip::AdapterRegistry::NoHandlerError (No handler found for #<PhusionPassenger::Utils::RewindableInput:0x000000041aef38 #io=#<PhusionPassen...
I'm using this in my controller:
before_filter :parse_raw_upload, :only => :bulk_submissions
def bulk_submissions
...
#submission = Submission.create!(url: "", file: #raw_file, description: "Please edit this description", work_type: "other", date_completed: DateTime.now.to_date)
...
end
private
def parse_raw_upload
if env['HTTP_X_FILE_UPLOAD'] == 'true'
#raw_file = env['rack.input']
#raw_file.class.class_eval { attr_accessor :original_filename, :content_type }
#raw_file.original_filename = env['HTTP_X_FILE_NAME']
#raw_file.content_type = env['HTTP_X_MIME_TYPE']
end
end
Looking at the request itself all the headers are set (X_MIME_TYPE, X_FILE_NAME) etc.
Any ideas?
Thanks in advance!
The example you're cribbing from expects the file stream to be a StringIO object, but Passenger is giving you a PhusionPassenger::Utils::RewindableInput object instead.
Fortunately, a RewindableInput is duckalike to StringIO for this case, so Paperclip's StringioAdapter can be used to wrap your upload stream.
Inside the if block in your parse_raw_upload, at the end, do:
if #raw_file.class.name == 'PhusionPassenger::Utils::RewindableInput'
#raw_file = Paperclip::StringioAdapter.new(#raw_file)
end
I'm trying to get an ajax upload working with rails 3.1.3 and paperclip.
I found this solution to my problem Rails 3 get raw post data and write it to tmp file, but using this, I get an 'encoding undefined conversion error "\xFF" from ASCII-8BIT to UTF-8.
The error occurs at the line #user.photo = #user.photo = QqFile.parse(params[:qqfile], request)
I have not edited the code supplied in the previous answer, but I'll include it here so you don't have to switch back and forth.
the gem list paperclip, returns 2.5.2, 2.4.5, 2.3.8
my controller
def create
#user = User.new(params[:user])
#user.photo = QqFile.parse(params[:qqfile], request)
if #user.save
return render :json => #user
else
return render :json => #user.errors
end
end
qq_file.rb
# encoding: utf-8
require 'digest/sha1'
require 'mime/types'
# Usage (paperclip example)
# #asset.data = QqFile.new(params[:qqfile], request)
class QqFile < ::Tempfile
def initialize(filename, request, tmpdir = Dir::tmpdir)
#original_filename = filename
#request = request
super Digest::SHA1.hexdigest(filename), tmpdir
fetch
end
def self.parse(*args)
return args.first unless args.first.is_a?(String)
new(*args)
end
def fetch
self.write #request.raw_post
self.rewind
self
end
def original_filename
#original_filename
end
def content_type
types = MIME::Types.type_for(#request.content_type)
types.empty? ? #request.content_type : types.first.to_s
end
end
This was an encoding error related to Ruby 1.9.2 (or I believe Ruby 1.9+).
this github post lead to the answer
https://github.com/lassebunk/webcam_app/issues/1
You must specify raw_post.force_encoding("UTF-8") when reading an upload as far as I could tell (I'm not a great programmer).