Carrierwave: how to crop and resize so that the final picture has exactly the same width and height? - ruby-on-rails-3

I'm using Rails 3 with carrierwave and Rmagick.
I want the picture to be resized exactly to 192 x 135
This seamed simple at first but nothing I tried worked until now.
Anyone found a solution for this? Here is my code for the uploader.
class AvatarUploader < CarrierWave::Uploader::Base
#Include RMagick or MiniMagick support:
include CarrierWave::RMagick
include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
def default_url
# # For Rails 3.1+ asset pipeline compatibility:
ActionController::Base.helpers.asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
end
process :resize_to_fit => [250, 250]
# Process files as they are uploaded:
# process :scale => [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
version :thumb do
process :resize_to_fit => [200, 200]
end
version :medium do
process :resize_to_fit => [250, 250]
end
version :mini do
process :resize_to_fit => [100, nil]
process crop: '100x100+0+0'
end
version :grid do
process :resize_to_fit => [192, 135]
process crop: '192x135+0+0'
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
# def extension_white_list
# %w(jpg jpeg gif png)
# end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
private
# Simplest way
def crop(geometry)
manipulate! do |img|
img.crop(geometry)
img
end
end
# Resize and crop square from Center
def resize_and_crop(size)
manipulate! do |image|
if image[:width] < image[:height]
remove = ((image[:height] - image[:width])/2).round
image.shave("0x#{remove}")
elsif image[:width] > image[:height]
remove = ((image[:width] - image[:height])/2).round
image.shave("#{remove}x0")
end
image.resize("#{size}x#{size}")
image
end
end
end

Change your line
process :resize_to_fit => [250, 250]
To
process :resize_to_fill => [192, 135]
I asked a similar question about a month ago. Hope this helps! :)

Related

Convert jpeg image data stored in AWS s3 to TFRecords where the sub-directory is the unique label associated with these images using AWS SageMaker

I have a directory of jpeg images in AWS s3 where the sub-directory is the unique label associated with these images. I am attempting to follow this example using AWS SageMaker and I am making a mess of input and output paths while being inexperienced with flags. Any guidance on applying the linked solution using s3 and SageMaker or another approach to achieve the output of TFRecords then saved back to s3 would be greatly appreciated.
Below is a slimmed down example that might help you. I adapted this from some code I use which includes object detection labels (multiple bounding boxes per image), so there are some TODOs added where you can tweak this for your labels. Also, not sure based on your question if an alternative is to work on the files locally. This example uses those files locally, and creates a TFRecord file that you would then upload to S3.
def create_tfrecords(train_img_dir, tfrecord_file):
""" Create TFRecord file from images/labels.
train_img_dir: A directory that contains .jpg images
tfrecord_file: Name of a file where the tfrecords are written to
"""
from object_detection.utils import dataset_util
# these same functions are available here:
# https://www.tensorflow.org/tutorials/load_data/tfrecord
with tf.io.TFRecordWriter(tfrecord_file) as writer:
#TODO: Modify this to recursivley list files in subdirs of labeled images or pull from S3
train_img_files = [f for f in listdir(train_img_dir) if isfile(join(train_img_dir, f))]
for i, f in enumerate(train_img_files):
try:
file_path = os.path.join(train_img_dir, f)
name, ext = path.splitext(f)
#TODO extract labe name based on path
label = ...
#TODO: convert the label name to an index if you want to store the index value
label_index = ...
# Skip non-jpegs
if ext not in ['.jpg', '.jpeg']:
continue
with tf.io.gfile.GFile(file_path, 'rb') as fid:
encoded_jpg = fid.read()
# Pil image to extract h/w
im = Image.open(file_path)
image_w, image_h = im.size
# Create TFRecord
tf_example = tf.train.Example(features=tf.train.Features(feature={
'image/height': dataset_util.int64_feature(image_h),
'image/width': dataset_util.int64_feature(image_w),
'image/filename': dataset_util.bytes_feature(f.encode('utf8')),
'image/encoded': dataset_util.bytes_feature(encoded_jpg),
'image/format': dataset_util.bytes_feature('jpeg'.encode('utf8')),
'image/label': dataset_util.int64_feature(label_index),
}))
if tf_example:
writer.write(tf_example.SerializeToString())
except ValueError:
print('Invalid example, ignoring.')
pass
except IOError:
print("Can't read example, ignoring.")
pass
print('TFRecord file created: ', tfrecord_file)
Then call it like this:
create_tfrecords(train_image_dir, tfrecord_file)
EDIT:
Use these methods if you don't have the object detection framework installed source:
# The following functions can be used to convert a value to a type compatible
# with tf.train.Example.
def _bytes_feature(value):
"""Returns a bytes_list from a string / byte."""
if isinstance(value, type(tf.constant(0))):
value = value.numpy() # BytesList won't unpack a string from an EagerTensor.
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
def _float_feature(value):
"""Returns a float_list from a float / double."""
return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))
def _int64_feature(value):
"""Returns an int64_list from a bool / enum / int / uint."""
return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

Update ListView dynamically when Log file updated

I have a log file in .txt format, that is being updated continuously. Then I want to show the file content in ListView (PyQt5) dynamically
Use a QFileSystemWatcher to detect when you file has been modified.
A quick example:
class LogWatcher(QObject):
def __init__(self, parent=None):
super().__init__(parent)
self.watcher = QFileSystemWatcher()
self.watcher.addPath("./foobar.txt") # The file you want to check
self.watcher.fileChanged.connect(self.displayLine)
def displayLine(self, path): # Will be called when the file has changed
print(path, "has changed")
if __name__ == '__main__':
app = QtWidgets.QApplication([])
logger = LogWatcher()
sys.exit(app.exec_())
Each time the file foobar.txt changes, the method displayLine is called

TypeError (can't cast File): Rails Carrierwave File upload

I am using Carrierwave and Rails getting the following error while uploading a file.
Started POST "/upload_client_input_file?project_id=7" for 192.168.1.101 at 2018-08-06 10:46:43 +0530
Processing by InputFilesController#upload_client_input_file as /
Parameters: {"file"=>#, #original_filename="input.csv.part_1.1", #content_type="application/octet-stream", #headers="Content-Disposition: form-data; name=\"file\"; filename=\"input.csv.part_1.1\"\r\nContent-Type: application/octet-stream\r\n">, "project_id"=>"7"}
Completed 500 Internal Server Error in 27ms (ActiveRecord: 7.9ms)
TypeError (can't cast File):
app/controllers/input_files_controller.rb:61:in `upload_client_input_file'
My model code:
class InputFile < ApplicationRecord
belongs_to :project
mount_uploader :file, AvatarUploader
end
My uploader code:
class AvatarUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
# include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url(*args)
# # For Rails 3.1+ asset pipeline compatibility:
# # ActionController::Base.helpers.asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process scale: [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
# version :thumb do
# process resize_to_fit: [50, 50]
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
# def extension_whitelist
# %w(jpg jpeg gif png)
# end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
My Controller code:
def upload_client_input_file
params.permit!
project_id = params["project_id"]
tempfile = params["file"].tempfile
file = CSV.read(tempfile)
input_file_id = InputFile.find_or_create_by(batch_name: "Test", flag: true, project_id: project_id, file: File.open(tempfile,"r")).id
end

CarrierWave isn't uploading image

I am trying to upload user profile picture from my application to a centralized application using carrierwave gem. In params avatar is getting passed but not getting updated.
Here my applicaton log:
Started PATCH "/api/v1/users/47" for ::1 at 2018-07-01 13:45:14 +0530
Processing by Api::V1::UsersController#update as JSON
Parameters: {"user"=>{"avatar"=>#<ActionDispatch::Http::UploadedFile:0x00007f7c7cda6e20 #tempfile=#<Tempfile:/tmp/RackMultipart20180701-10347-otl55f.jpg>, #original_filename="leotolstoy1.jpg", #content_type="image/jpeg", #headers="Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"leotolstoy1.jpg\"\r\nContent-Type: image/jpeg\r\n">, "first_name"=>"Mayuresh"}, "id"=>"47"}
And centralized application log:
Started POST "/api/v1/users/update" for 127.0.0.1 at 2018-07-01 13:45:15 +0530
Processing by Api::V1::UsersController#update as */*
Parameters: {"first_name"=>"Mayuresh", "last_name"=>nil, "avatar"=>{"tempfile"=>"#<File:0x00007f7c7c6ba468>", "original_filename"=>"leotolstoy1.jpg", "content_type"=>"image/jpeg", "headers"=>"Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"leotolstoy1.jpg\"\r\nContent-Type: image/jpeg\r\n"}, "user"=>{"first_name"=>"Mayuresh", "last_name"=>nil, "avatar"=>{"tempfile"=>"#<File:0x00007f7c7c6ba468>", "original_filename"=>"leotolstoy1.jpg", "content_type"=>"image/jpeg", "headers"=>"Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"leotolstoy1.jpg\"\r\nContent-Type: image/jpeg\r\n"}}}
Application Load (0.4ms) SELECT "oauth_applications".* FROM "oauth_applications" WHERE "oauth_applications"."secret" = $1 ORDER BY "oauth_applications"."id" ASC LIMIT $2 [["secret", "<secret_token>"], ["LIMIT", 1]]
AccessToken Load (0.4ms) SELECT "oauth_access_tokens".* FROM "oauth_access_tokens" WHERE "oauth_access_tokens"."active" = $1 AND "oauth_access_tokens"."token" = $2 ORDER BY "oauth_access_tokens"."id" ASC LIMIT $3 [["active", true], ["token", "<access_token>"], ["LIMIT", 1]]
User Load (0.6ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 1], ["LIMIT", 1]]
(1.8ms) BEGIN
User Exists (1.5ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = $1 AND ("users"."id" != $2) AND "users"."type" = $3 LIMIT $4 [["email", "<email>"], ["id", 1], ["type", "User"], ["LIMIT", 1]]
SQL (0.4ms) UPDATE "users" SET "first_name" = $1, "updated_at" = $2 WHERE "users"."id" = $3 [["first_name", "Mayuresh"], ["updated_at", 2018-07-01 08:15:15 UTC], ["id", 1]]
(12.7ms) COMMIT
As you can see only first name is getting updated not avatar.
Avatar uploader in centralized application:
class AvatarUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
# include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
# "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
'tmp/upload/'
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url(*args)
# # For Rails 3.1+ asset pipeline compatibility:
# # ActionController::Base.helpers.asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process scale: [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
# version :thumb do
# process resize_to_fit: [50, 50]
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_whitelist
%w(jpg jpeg png)
end
def content_type_whitelist
/image\//
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
Even if I am trying to update from rails console at centralized application, it is not getting updated.
2.3.4 :049 > user.update avatar: 'https://static1.squarespace.com/static/586af0132994caa37cca0067/5876e27f9f74561d8c5ce4df/5876e296c534a514869fe56e/1484187414154/Tree+On+The+Bank.jpg'
(0.7ms) BEGIN
User Exists (0.8ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = $1 AND ("users"."id" != $2) AND "users"."type" = $3 LIMIT $4 [["email", "<email>"], ["id", 1], ["type", "User"], ["LIMIT", 1]]
SQL (0.4ms) UPDATE "users" SET "updated_at" = $1, "avatar" = $2 WHERE "users"."id" = $3 [["updated_at", 2018-07-01 07:39:16 UTC], ["avatar", "abc.jpg"], ["id", 1]]
(13.4ms) COMMIT
=> true
Not able in figuring out what is wrong here.
The problem was that file object which was getting passed to centralized application was not the original file object because as per code I was converting parameters to json and then was making connection through faraday to connect to centralized application.
What I did is made a separate faraday connection for uploading files and passed parameters as it is in payload as below:
conn = Faraday.new url: url do |f|
f.request :multipart
f.request :url_encoded
f.adapter Faraday.default_adapter
end
payload = { avatar: Faraday::UploadIO.new(update_params[:avatar], update_params[:avatar].content_type, update_params[:avatar].original_filename) }
conn.put '/api/v1/users/update', payload
Now my application parameters:
Started PATCH "/api/v1/users/47" for ::1 at 2018-07-01 13:45:14 +0530
Processing by Api::V1::UsersController#update as JSON
Parameters: {"user"=>{"avatar"=>#<ActionDispatch::Http::UploadedFile:0x00007f7c7cda6e20 #tempfile=#<Tempfile:/tmp/RackMultipart20180701-10347-otl55f.jpg>, #original_filename="leotolstoy1.jpg", #content_type="image/jpeg", #headers="Content-Disposition: form-data; name=\"user[avatar]\"; filename=\"leotolstoy1.jpg\"\r\nContent-Type: image/jpeg\r\n">, "first_name"=>"Mayuresh"}, "id"=>"47"}
And centralized application parameters:
Started PUT "/api/v1/users/update" for 127.0.0.1 at 2018-07-05 02:12:04 +0530
Processing by Api::V1::UsersController#update as */*
Parameters: {"avatar"=>#<ActionDispatch::Http::UploadedFile:0x0000000775e1d8 #tempfile=#<Tempfile:/tmp/RackMultipart20180705-23054-1cbcry3.png>, #original_filename="nio.png", #content_type="image/png", #headers="Content-Disposition: form-data; name=\"avatar\"; filename=\"leotolstoy1.jpg\"\r\nContent-Length: 272286\r\nContent-Type: image/png\r\nContent-Transfer-Encoding: binary\r\n">}
As you can see avatar parameter is actual ActionDispatch object and that's why images are getting uploaded properly.
And for remote url's I got solution here. It will be done as below:
2.3.4 :049 > user.update remote_avatar_url: 'https://static1.squarespace.com/static/586af0132994caa37cca0067/5876e27f9f74561d8c5ce4df/5876e296c534a514869fe56e/1484187414154/Tree+On+The+Bank.jpg'

Carrierwave: Scale image if the size is larger than (conditionally create versions)

Is possible with carrierwave create a version (for example thumb) only if the image is larger than the size of the version??
Example:
version :thumb, :if => :is_thumbnable? do
process :resize_to_fit => [32,nil]
end
protected
def is_thumbnable?(file)
image ||= MiniMagick::Image.open(file.path)
if image.nil?
if image['width'] >= 32 || image['height'] >= 32
true
else
false
end
else
false
end
end
I tried them and it didn't work for me.
I get the server blocked when resizing to large images in development.
carrierwave (0.9.0)
rmagick (2.13.2)
So I get a look at the documentation: http://carrierwave.rubyforge.org/rdoc/classes/CarrierWave/RMagick.html
There is a wonderful function: resize_to_limit(width, height)
Resize the image to fit within the specified dimensions while retaining the original aspect ratio. Will only resize the image if it is larger than the specified dimensions. The resulting image may be shorter or narrower than specified in the smaller dimension but will not be larger than the specified values.
My code look like this:
version :version_name, from_version: :parent_version_name do
process resize_to_limit: [width, nil]
end
It resize to fit the width only if it's larger, respecting the ratio w/h.
I defined method in which if image exceed given width then manipulate it to your size the 32 pixels in this case. Put this code in your ImageUploader:
version :thumb do
process :resize_to_width => [32, nil]
end
def resize_to_width(width, height)
manipulate! do |img|
if img[:width] >= width
img.resize "#{width}x#{img[:height]}"
end
img = yield(img) if block_given?
img
end
end
Actually #Roza solution didn't work for me.
I had to modify method like this:
process :resize_to_width => [650, nil]
def resize_to_width(width, height)
manipulate! do |img|
if img.columns >= width
img.resize(width)
end
img = yield(img) if block_given?
img
end
end
I use rmagick (2.13.2) and rails 3.2.13, carrierwave (0.8.0)