I want to do a database-side string concatenation in a Rails query, and do it in database-independent way.
SQL-92 specifies double-bar (||) as the concatenation operator. Unfortunately it looks like MS SQL Server doesn't support it; it uses + instead.
I'm guessing that Rails' SQL grammar abstraction has solved the db-specific operator problem already. If it does exist, how do I use it?
I had the same problem and never came up with anything that was built into Rails. So I wrote this little method.
# Symbols should be used for field names, everything else will be quoted as a string
def db_concat(*args)
adapter = configurations[RAILS_ENV]['adapter'].to_sym
args.map!{ |arg| arg.class==Symbol ? arg.to_s : "'#{arg}'" }
case adapter
when :mysql
"CONCAT(#{args.join(',')})"
when :sqlserver
args.join('+')
else
args.join('||')
end
end
I'm thinking somebody should really write some sort of SQL helper plugin that could automatically format simple SQL expressions based using the correct functions or operators for the current adapter. Maybe I'll write one myself.
It hasn't received much usage yet but I wrote the following code which seems to solve the problem. This monkey-patches the adapters to have a method to support it:
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
# Will return the given strings as a SQL concationation. By default
# uses the SQL-92 syntax:
#
# concat('foo', 'bar') -> "foo || bar"
def concat(*args)
args * " || "
end
end
class AbstractMysqlAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MySQL format:
#
# concat('foo', 'bar') -> "CONCAT(foo, bar)"
def concat(*args)
"CONCAT(#{args * ', '})"
end
end
class SQLServerAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MS-SQL format:
#
# concat('foo', 'bar') -> foo + bar
def concat(*args)
args * ' + '
end
end
end
end
With this you should be able to do the following in your code:
class User < ActiveRecord::Base
def self.find_by_name(name)
where("#{connection.concat('first_name', 'last_name')} = ?", name)
end
end
This outputs the following SQL query on a SQL-92 database (Oracle, SQLite, PostgreSQL):
SELECT * FROM users WHERE first_name || last_name = ?
For MySQL it outputs:
SELECT * FROM users WHERE CONCAT(first_name, last_name) = ?
For SQL Server it outputs
SELECT * FROM users WHERE first_name + last_name = ?
Obviously you could extend this concept to other database adapters.
If you want something Rails neutral, you're going to need to return the values you want concatenated and do that once the data has been delivered to rails (or do it in rails before you give it to the database).
It looks like Mysql uses CONCAT(), Postgres ||, Oracle CONCAT() or ||, T-SQL +.
Any rails abstraction of the same would have to take place at a point where you could just be doing concatenation using regular Ruby, or I've completely misunderstood the question.
Related
I am currently making an API using Ruby on Rails. I was just wondering in general if there are built in Rails methods or libraries/gems to sanitize Json and SQL or if Rails 4 does this by default? I am most worried about such cases where I have an SQL statement such as
User.where("users.first_name IS NOT NULL")
or something like
Event.where(:location => params[:location]).
Essentially, what should I watch out for in my SQL syntax and in incoming JSON requests?
By default, using the following will sanitize the str and make it safe from SQL injections:
User.where(name: str)
User.where('name ILIKE ?', str)
However, the following code (direct string interpolation then given to the where method) make it unsafe from SQL injections:
User.where("name = '#{str}'")
In your case, you can use ActiveRecord::Base.sanitize(your_string_from_user_input). It will use your DB adapter to escape/quote the relevant parts, preventing from SQL injections.
In a Model, you can directly access to the sanitize method (since you are in a context already inheriting from ActiveRecord::Base):
class User < ActiveRecord::Base
def self.search(string)
terms = string.split
searchable_columns = [:name, :username, :whatever]
query = terms.map do |term|
fields = searchable_columns.map |column|
" #{self.table_name}.#{column} LIKE '#{sanitize("%#{term}%")}'"
end
"(#{fields.join(' OR ')})"
end.join(' AND ')
where(query)
end
end
The above code will produce a SQL WHERE clause like the following:
# str is 'bob doe'
WHERE
(users.name LIKE 'bob' OR users.username LIKE 'bob' OR users.whatever LIKE 'bob')
AND
(users.name LIKE 'doe' OR users.username LIKE 'doe' OR users.whatever LIKE 'doe')
Rails will do it automatically if you format your queries properly
From the guides - don't do things like:
Project.where("name = '#{params[:name]}'")
Instead do
Project.where("name = ?", params[:name])
I know it's not safe to use interpolated strings when calling .where.
e.g. this:
Client.where("orders_count = #{params[:orders]}")
should be rewritten as:
Client.where("orders_count = ?", params[:orders])
Is it safe to use interpolated strings when calling .order? If not, how should the following be rewritten?
Client.order("#{some_value_1}, #{some_value_2}")
Yes, ActiveRecord's “order” method is vulnerable to SQL injection.
No, it is not safe to use interpolated strings when calling .order.
The above answers to my question have been confirmed by Aaron Patterson, who pointed me to http://rails-sqli.org/#order . From that page:
Taking advantage of SQL injection in ORDER BY clauses is tricky, but a
CASE statement can be used to test other fields, switching the sort
column for true or false. While it can take many queries, an attacker
can determine the value of the field.
Therefore it's important to manually check anything going to order is safe; perhaps by using methods similar to #dmcnally's suggestions.
Thanks all.
Short answer is you need to sanitize your inputs.
If the strings you are planning to interpolate come from an untrusted source (e.g. web browser) then you need to first map them to trusted values. You could do this via a hash:
# Mappings from known values to SQL
order_mappings = {
'first_name_asc' => 'first_name ASC',
'first_name_desc' => 'first_name DESC',
'last_name_asc' => 'last_name ASC',
'last_name_desc' => 'last_name DESC',
}
# Ordering options passed in as an array from some source:
order_options = ['last_name_asc', 'first_name_asc']
# Map them to the correct SQL:
order = order_options.map{|o| order_mappings[o] }.compact.join(', ')
Client.order(order)
#Mike explanation is correct. #dmcnally workaround would work. I'm following in a slightly different path mentioned in [Railscast][1] http://railscasts.com/episodes/228-sortable-table-columns
In a nutshell, if you can construct a private method in the controller, in order to sanitize the user input:
Order by name of one your table columns:
private
def sort_column
Client.column_names.include?(params[:sort]) ? params[:sort] : "first_name"
end
Order by other criteria, then use the whitelist construct such as below:
def sort_direction
%w[asc desc].include?(params[:direction]) ? params[:direction] : "asc"
end
And your controller method should then look like this:
Client.all.order(sort_column + " " + sort_direction)
Just anther way to Rome. Hope this help.
Let's try this!
# app/models/concern/ext_active_record.rb
module ExtActiveRecord
extend ActiveSupport::Concern
included do
scope :sortable, -> (params) do
return unless params[:sort_by] && params[:sort_dir]
reorder("#{params[:sort_by]}" => "#{params[:sort_dir]}")
end
end
end
# app/models/user.rb
class User < ActiveRecord::Base
include ExtActiveRecord
# ....
end
# app/controllers/user_controller.rb
class UserController < ApplicationController
def index
#users = User.sortable(params).page(params[:page]).per(params[:per])
end
end
Client.order("#{some_value_1}, #{some_value_2}")
should be written as
order = sanitize_sql_array(['%s, %s', some_value_1, some_value_2])
Client.order(order)
How do you invoke database functions from grails?
Such as the CONVERT(decimal, timestamp)
Without using groovy SQL like this:
def sql = new Sql(dataSource)
I would like to avoid any SQL statements and take full advantage of the grails magic.
My understanding is that grails does not have native support for database functions. You will need to write SQL. Gorm is an abstraction of Hibernate, you may want to refer to:
How can you call custom database functions with Hibernate?
In your Domain class First you must -
import groovy.sql.Sql
Then you can create Sql instance and call a function like below -
def date = "'21-Feb-19 14:10:10.123000'"
def timestamp_format = "'DD-Mon-RR HH24:MI:SS.FF'"
String query = 'select to_timestamp('+date+','+timestamp_format+') from dual'
def sql = Sql.newInstance('url', 'username', 'password', 'driverClassName')
def result= sql.rows(query);
def rows = result.collect{it.values()}
println rows
I have this site with the following parameters:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc
I use the values of each of the parameters as a value in a SQL query.
I am trying to test my application and ultimately hack my own application for learning purposes.
I'm trying to inject this statement:
http://www.example.com.com/pagination.php?page=4&order=comment_time&sc=desc' or 1=1 --
But It fails, and MySQL says this:
Warning: mysql_fetch_assoc() expects parameter 1 to be resource,
boolean given in /home/dir/public_html/pagination.php on line 132
Is my application completely free from SQL injection, or is it still possible?
EDIT: Is it possible for me to find a valid sql injection statement to input into one of the parameters of the URL?
The application secured from sql injection never produces invalid queries.
So obviously you still have some issues.
Well-written application for any input produces valid and expected output.
That's completely vulnerable, and the fact that you can cause a syntax error proves it.
There is no function to escape column names or order by directions. Those functions do not exist because it is bad style to expose the DB logic directly in the URL, because it makes the URLs dependent on changes to your database logic.
I'd suggest something like an array mapping the "order" parameter values to column names:
$order_cols = array(
'time' => 'comment_time',
'popular' => 'comment_score',
... and so on ...
);
if (!isset($order_cols[$_GET['order'])) {
$_GET['order'] = 'time';
}
$order = $order_cols[$_GET['order']];
Restrict "sc" manually:
if ($_GET['sc'] == 'asc' || $_GET['sc'] == 'desc') {
$order .= ' ' . $_GET['sc'];
} else {
$order .= ' desc';
}
Then you're guaranteed safe to append that to the query, and the URL is not tied to the DB implementation.
I'm not 100% certain, but I'd say it still seems vulnerable to me -- the fact that it's accepting the single-quote (') as a delimiter and then generating an error off the subsequent injected code says to me that it's passing things it shouldn't on to MySQL.
Any data that could possibly be taken from somewhere other than your application itself should go through mysql_real_escape_string() first. This way the whole ' or 1=1 part gets passed as a value to MySQL... unless you're passing "sc" straight through for the sort order, such as
$sql = "SELECT * FROM foo WHERE page='{$_REQUEST['page']}' ORDER BY data {$_REQUEST['sc']}";
... which you also shouldn't be doing. Try something along these lines:
$page = mysql_real_escape_string($_REQUEST['page']);
if ($_REQUEST['sc'] == "desc")
$sortorder = "DESC";
else
$sortorder = "ASC";
$sql = "SELECT * FROM foo WHERE page='{$page}' ORDER BY data {$sortorder}";
I still couldn't say it's TOTALLY injection-proof, but it's definitely more robust.
I am assuming that your generated query does something like
select <some number of fields>
from <some table>
where sc=desc
order by comment_time
Now, if I were to attack the order by statement instead of the WHERE, I might be able to get some results... Imagine I added the following
comment_time; select top 5 * from sysobjects
the query being returned to your front end would be the top 5 rows from sysobjects, rather than the query you try to generated (depending a lot on the front end)...
It really depends on how PHP validates those arguments. If MySQL is giving you a warning, it means that a hacker already passes through your first line of defence, which is your PHP script.
Use if(!preg_match('/^regex_pattern$/', $your_input)) to filter all your inputs before passing them to MySQL.
I want to write a script in Ruby to clean up some messed up keys in several copies of the same MySQL schema. I'd like to do something like SHOW CREATE TABLE, then look at what comes back and delete keys if they exist.
I know in the Rails environment you can do this...
ActiveRecord::Base.connection.execute( some sql )
But what you get back is a "Result" object. For this task I need a String so I can analyze it and act accordingly.
This should help you:
>> result = ActiveRecord::Base.connection.execute("SHOW TABLES")
=> #<Mysql::Result:0x37ecb30>
>> result.class.instance_methods - Object.instance_methods
=> ["all_hashes", "field_seek", "row_tell", "fetch_field_direct", "free", "field_tell", "fetch_lengths", "num_fields", "data_seek", "fetch_row", "num_rows", "fetch_field", "each", "each_hash", "fetch_hash", "row_seek", "fetch_fields"]
Look at #all_hashes on the MySql::Result instance
I would use the mysql-ruby gem and you would do something like this:
require 'mysql'
m = MySQL.new("localhost", "username", "password", "database")
r = m.query("SELECT * FROM people ORDER BY name")
r.each_hash do |f|
print "#{f['name']} - #{f['email']}"
end
You could check the mysql-ruby gem.
Here is a write-up on how to use it: Using the Ruby MySQL Module
More can be found via google
If you don't want to use ActiveRecord an ORM may be a bit complicated for your usage right now), you can still use the ruby-mysql library or even better IMHO is to use the Ruby DBI/DBD library (here) which has DBD drivers for mysql & postgresql out-of-the-box.
That way, you can issue straight SQL statements like this
require "dbi"
require "dbi/dbrc"
# == Configuration
DB = "sympa"
HOST = "saphir"
cnt = 0
dup = 0
# == Crude option processing
#
list_name = ARGV.shift.to_s
file = ARGV.shift.to_s
db = DBI::DBRC.new(DB)
DBI.connect(db.dsn + ":#{HOST}", db.user, db.password) do |dbh|
date = Time.now.asctime
if not list_name or list_name == "" then
puts "List name is mandatory"
exit 1
end
req1 = <<-"EOR"
insert into user_table (email_user,lang_user)
values (?, ?)
EOR
...
req2 = <<-"EOR"
insert into subscriber_table
(user_subscriber, list_subscriber, visibility_subscriber,
date_subscriber, reception_subscriber) values (?, ?, ?, NOW(), ?)
EOR
sth1 = dbh.prepare(req1)
sth2 = dbh.prepare(req2)
...
#
# Insertion in user_table
#
begin
sth1.execute(line, "en")
cnt += 1
rescue DBI::DatabaseError => err
$stderr.puts("DBI: #{err}")
end
dbi/dbrc is a useful module that enables you to avoid putting login&password directly in the script. See there.
There is probably a better way to do that programmatically, however if you really want to drive the interactive commands and parse the results, then expect may be more suitable. You could still kick off expect from your ruby script.
Use Mysql2
To update this thread: I'd suggest Mysql2 now: http://rubygems.org/gems/mysql2