I have the following code in my user_ransaker.rb file:
ransacker :new_donors do
sql = %{(
users.id IN (
#{User.new_donor_sql}
)
)}
Arel.sql(sql)
end
On user.rb model:
def self.new_donor_sql
part_1 = %{(
SELECT distinct(user_id)
FROM donations
}
part_1
end
I get the following Brakeman warning for above statement:
Confidence: High
Category: SQL Injection
Check: SQL
Message: Possible SQL injection
Code: Arel.sql("(\n users.id IN (\n #{User.new_donor_sql}\n)\n)")
File: app/models/concerns/user_ransackers.rb
Is this a valid error? If I used ActiveRecord to write the SQL statement, I could have used ? placeholder if I needed to interpolate values. I am not really sure how to fix this warning. If this is a valid warning, how do I remediate it?
If you gonna Arel then do some relational algebra:
class User < ApplicationRecord
def self.new_donor_sql
arel_table.project(arel_table[:user_id]).distinct
end
end
ransacker :new_donors do
User.arel_table.then do |users|
users.where(users[:id].in(User.new_donor_sql)).where_sql
end
end
You could also just drop the class method:
ransacker :new_donors do
User.arel_table.then do |users|
subquery = users.project(users[:user_id]).distinct
users.where(users[:id].in(subquery)).where_sql
end
end
Related
When using Rails with ActiveRecord (and PostgreSQL), executing "simple" queries adds a name to them, e.g. calling
Article.all
# => Article Load (2.6ms) SELECT "articles".* FROM "articles"
names the query Article Load. However, when executing slightly more complex queries, no name is being generated, as for example with
Article.group(:article_group_id).count
# => (1.2ms) SELECT COUNT(*) AS count_all, "articles"."article_group_id" AS articles_article_group_id FROM "articles" GROUP BY "articles"."article_group_id"
I can add a name if executing a custom query using the execute method:
ActiveRecord::Base.connection.execute("SELECT * FROM articles", "My custom query name")
# => My custom query name (2.5ms) SELECT * FROM articles
But is there a way to add a custom name to a query built with the ActiveRecord-methods?
If you wonder why: The name is useful for all kinds of monitoring, e.g. when looking at slow queries in AppSignal.
Since you just want to custom query name for monitoring purpose, so i think you only need to change the query name in the ActiveRecord::ConnectionAdapters#log method, this method is the one log the sql query that be executed, include the query name.
Here is my solution:
# lib/active_record/base.rb
# note that MUST be base.rb
# otherwise you need to add initializer to extend Rails core
#
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
attr_accessor :log_tag
private
alias old_log log
def log(sql, name = "SQL", binds = [], type_casted_binds = [], statement_name = nil)
if name != 'SCHEMA'
name = #log_tag
#log_tag = nil # reset
end
old_log(sql, name, binds, type_casted_binds, statement_name) do
yield
end
end
end
end
module QueryMethods
def log_tag(tag_name) # class method
spawn.log_tag(tag_name)
self
end
end
module Querying
delegate :log_tag, to: :all
end
class Relation
def log_tag(tag_name) # instance method
conn = klass.connection
conn.log_tag = tag_name
self
end
end
end
Demo
Task.log_tag("DEMO").group(:status).count
# DEMO (0.7ms) SELECT COUNT(*) AS count_all, "tasks"."status" AS tasks_status FROM "tasks" GROUP BY "tasks"."status"
Task.where(status: 6).log_tag("SIX").first(20)
# SIX (0.8ms) SELECT "tasks".* FROM "tasks" WHERE "tasks"."status" = ? ORDER BY "tasks"."id" ASC LIMIT ?
Task.where(status: 6).first(20)
# (0.8ms) SELECT "tasks".* FROM "tasks" WHERE "tasks"."status" = ? ORDER BY "tasks"."id" ASC LIMIT ?
Note
In case you want to fix query name for specific query, you can use a hash with key is the whole the specific sql string (or hash of whole sql, such as the way Rails core cache query: query_signature = ActiveSupport::Digest.hexdigest(to_sql)) and the value is the query name you want.
# set up before hand
ActiveRecord::ConnectionAdapters::LogTags[Product.where...to_sql] = "DEMO"
# AbstractAdapter
LogTags = Hash.new
def log(sql, name...)
name = LogTags[sql]
# ...
end
I have the following setup:
question.rb
class Question < ActiveRecord::Base
has_many :answers
#validations, methods, etc
...
#Returns the questions with the most answers
def Question.top_questions(max = 10)
sql = "SELECT question_id, COUNT('question_id') as aCount FROM answers GROUP BY question_id ORDER BY aCount DESC LIMIT #{max.to_i}" # Probably shouldn't use string interpolation here :D
Question.connection.execute(sql)
end
end
answer.rb
class Answer < ActiveRecord::Base
belongs_to :question
...
end
And if I call Question.top_questions(), then it returns this:
[{"question_id"=>1, "aCount"=>25, 0=>1, 1=>25}, {"question_id"=>38, "aCount"=>3, 0=>38, 1=>3}, {"question_id"=>45, "aCount"=>3, 0=>45, 1=>3}, {"question_id"=>26, "aCount"=>2, 0=>26, 1=>2}, {"question_id"=>46, "aCount"=>2, 0=>46, 1=>2}, {"question_id"=>48, "aCount"=>2, 0=>48, 1=>2}, {"question_id"=>51, "aCount"=>2, 0=>51, 1=>2}, {"question_id"=>5, "aCount"=>1, 0=>5, 1=>1}, {"question_id"=>15, "aCount"=>1, 0=>15, 1=>1}, {"question_id"=>20, "aCount"=>1, 0=>20, 1=>1}]
I'm not sure how I would use the data returned in a view while still keeping the code clean.
So I'm wondering if I could write the Question.top_questions() method using rails' read methods(find(), where(), etc). Or how I could get it to return an array of Question objects.
It returns an array of hashes, you could use it in a view as you like.
But if you don't want to write native sql, you could rewrite it as below.
def self.top_questions(max = 10)
Question.joins('LEFT JOIN answers ON questions.id = answers.question_id')
.select('questions.*, count(answers.id) as answers_count')
.group('questions.id')
.order('answers_count desc')
.limit(max)
end
Following on the question how can I use like query in ruby with sinatra? I have the following problem securing my sql from injection.Here is my method to make a query from the type string, it receives a v(alue) to search for and a k(ey) (=field) to look in.
After that the various selctions are joined by selection.join(' and ')
def string_selector(k, v)
case
when v[/\|/]
v.scan(/([^\|]+)(\|)([^\|]+)/).map {|p| "lower(#{k}) LIKE '%#{p.first.downcase}%' or lower(#{k}) LIKE '%#{p.last.downcase}%'"}
when v[/[<>=]/]
v.scan(/(<=?|>=?|=)([^<>=]+)/).map { |part| p part; "#{k} #{part.first} '#{part.last.strip}'"}
else
# "lower(#{k}) LIKE '%#{v.downcase}%'" #(works)
("lower(#{k}) LIKE ?", '%#{v.downcase}%') #doesn't work
end
end
But i get the error
selectors.rb:38: syntax error, unexpected keyword_end, expecting $end
from C:/../1.9.1/rubygems/core_ext/kernel_require.rb:55:in `require'
What could i be doing wrong ?
There's got to be a better way to do what you are trying to do if you are using ActiveRecord... However, if you need to support your string_selector functionality for some reason, I would at least use Arel:
def string_selector(k, v)
tbl = Arel::Table.new(:test) # your table, or you could pass this in...
condition = case v
when /\|/
vals = v.split(/\|/)
first = vals.shift
vals.inject(tbl[k].matches("%#{first.strip}%")) do |acc, val|
acc.or(tbl[k].matches("%#{val.strip}%"))
end
when /<>/
tbl[k].not_eq(v.gsub(/<>/, '').strip)
when /\=/
tbl[k].eq(v.gsub(/\=/, '').strip)
else
tbl[k].matches(v.strip)
end
tbl.where(condition).to_sql
end
Please note that matches will perform a case insensitive query for you (e.g., by using ILIKE in PostgreSQL).
I'm using postgres full text search with the pg_search gem. The search itself is working well, but I need to further filter the results and here are the details:
class Notebook < ActiveRecord::Base
has_many :invites
def self.text_search(query)
if query.present?
search(query)
else
scoped
end
end
Notebooks Controller:
def index
if params[:query].present?
#notebooks = Notebook.text_search(params[:query]).includes(:invites).where("invites.email = :email OR notebooks.access = :access OR notebooks.access = :caccess OR notebooks.user_id = :uid", email: current_user.email, access: "open", caccess: "closed", uid: current_user.id)
else
#notebooks = Notebook.includes(:invites).where("invites.email = :email OR notebooks.access = :access OR notebooks.access = :caccess OR notebooks.user_id = :uid", email: current_user.email, access: "open", caccess: "closed", uid: current_user.id)
end
The error I get is 'missing FROM-clause entry for table 'invites'. I have tried many different things including:
replacing 'includes' with 'joins'
replacing 'includes(:invites) with joins('LEFT JOIN "invites" ON "invites"."email" = "email" ')
changing the order of the .text_search and the .includes calls.
adding the includes call in the controller, in the model, in a scope, and in the text_search function definition.
I keep getting the same error, and when using the joins call with SQL it does not filter by invite emails, and shows multiple repeats of each search result.
I would just remove the include(:invites) because the text_search itself is working just fine. But I really need this condition to be included.
Any help would be greatly appreciated. Maybe I'm just getting my SQL call wrong, but I also would like to understand why the .includes(:invites) works without the pg text_search but won't work with it.
Edit #1 - more specific question
I think there are 2 slightly different questions here. The first seems to be some issue with combining pg_search gem and an 'includes(:invites)' call. The second question is what is the equivalent SQL statement that I can use in order to avoid making the 'includes(:invites)' call. I think it should be a LEFT JOIN of some sort, but I don't think I'm making it correctly. In my db, a Notebook has_many invites, and invites have an attribute 'email'. I need the the notebooks with invites that have an email equal to the current_user's email.
Help with either of these would be great.
Here is the link that showed me the solution to my problem:
https://github.com/Casecommons/pg_search/issues/109
Here is my specific code:
class Notebook < ActiveRecord::Base
has_many :invites
include PgSearch
pg_search_scope :search, against: [:title],
using: {tsearch: {dictionary: "english"}},
associated_against: {user: :name, notes:[:title, :content]}
scope :with_invites_and_access, lambda{ |c_user_email|
joins('LEFT OUTER JOIN invites ON invites.notebook_id = notebooks.id').where('invites.email = ? OR notebooks.access = ? OR notebooks.access = ?', c_user_email, 'open', 'closed')
}
def self.text_search(query)
if query.present?
search(query).with_invites_and_access(current_user_email)
else
scoped
end
end
end
The key was the joins statement. joins(:invites) doesn't work, includes(:invites) doesn't work. The full SQL statement is required:
joins('LEFT OUTER JOIN invites ON invites.notebook_id = notebooks.id')
I can see a join but I cannot see what makes joined invites fields to appear in the SELECT statement.
I think You may need to add the fields from the invites table into select() like this
select('invites.*').joins('LEFT OUTER JOIN invites ON invites.notebook_id = notebooks.id').where('invites.email = ? OR notebooks.access = ? OR notebooks.access = ?', c_user_email, 'open', 'closed')
}
I am using Ruby on Rails 3.2.2 and I would like to generate the following SQL query:
SELECT `articles`.* FROM `articles` WHERE (`articles`.`user_id` = 1 OR `articles`.`status` = 'published' OR (`articles`.`status` = 'temp' AND `articles`.`user_id` IN (10, 11, 12, <...>)))
By using Arel this way
Article
.where(
arel_table[:user_id].eq(1)
.or(arel_table[:status].eq("published"))
.or(
arel_table[:status].eq("temp")
.and(
arel_table[:user_id].in(10, 11, 12, <...>)
)
)
)
it generates the following (note: brackets are not the same as the first SQL query):
SELECT `articles`.* FROM `articles` WHERE (((`articles`.`user_id` = 1 OR `articles`.`status` = 'published') OR `articles`.`status` = 'temp' AND `articles`.`user_id` IN (10, 11, 12, <...>)))
Since I think the latter SQL query doesn't "work" as the first one, how could I use Arel (or, maybe, something else) so to generate the SQL query as the first one?
Update (after comments)
Given SQL queries above "work" the same but I still would like to generate the exact SQL query as the first one in the question (the main reason to make this is that the first SQL query is more readable than the second since in the first one are used less and "explicit" brackets), how could I make that by using Arel?
I had the same problem. I was searching the web for some hours and finally found a method named grouping in Arel::FactoryMethods which simply adds brackets around an expression.
You should wrap your groups with a arel_table.grouping(...) call.
Example of how to use arel_table.grouping(...) as part of scope
# app/model/candy.rb
class Candy < ActiveRecord::Base
has_many :candy_ownerships
has_many :clients, through: :candy_ownerships, source: :owner, source_type: 'Client'
has_many :users, through: :candy_ownerships, source: :owner, source_type: 'User'
# ....
scope :for_user_or_global, ->(user) do
# ->() is new lambda syntax, lamdba{|user| ....}
worldwide_candies = where(type: 'WorldwideCandies').where_values.reduce(:and)
client_candies = where(type: 'ClientCandies', candy_ownerships: { owner_id: user.client.id, owner_type: 'Client'}).where_values.reduce(:and)
user_candies = where(type: 'UserCandies', candy_ownerships: { owner_id: user.id, owner_type: 'User' }).where_values.reduce(:and)
joins(:candy_ownerships).where( worldwide_candies.or( arel_table.grouping(client_candies) ).or( arel_table.grouping(user_candies) ) )
end
# ....
end
call
Candy.for_user_or_global(User.last)
#=> SELECT `candies`.* FROM `candies` INNER JOIN `candy_ownerships` ON `candy_ownerships`.`candy_id` = `candies`.`id` WHERE (`candies`.`deleted_at` IS NULL) AND (((`candies`.`type` = 'WorldwideCandies' OR (`candies`.`type` = 'ClientCandies' AND `candy_ownerships`.`owner_id` = 19 AND `candy_ownerships`.`owner_type` = 'Client')) OR (`candies`.`type` = 'UserCandies' AND `candy_ownerships`.`owner_id` = 121 AND `candy_ownerships`.`owner_type` = 'User')))
thx micha for the tip
I've successfully used this gem: squeel which comes on top of Arel so you don't have to mess with it. So in order to generate your query you would do something like this in Squeel:
#articles = Article.
where{
( user_id.eq(1) | status.eq('published') ) |
( user_id.in([10, 11, 12, '<...>']) & status.eq('temp') )
}
# since this is an ActiveRecord::Relation we can play around with it
#articles = #articles.select{ [ user_id, status ] }
# and you can also inspect your SQL to see what is going to come out
puts #articles.to_sql
The more complicated your queries get the more you're going to like this gem.