I'm a fan of Yesql-style templates that consist of raw SQL with placeholders for parameters.
Here is an example from the docs:
SELECT *
FROM users
WHERE country_code = ?
This snipped is stored in a file and pulled into the application like so:
(defquery users-by-country "some/where/users_by_country.sql")
(users-by-country db-spec "GB")
Is there any gem with the same functionality in Ruby? Or is there a way to at least load raw SQL from a file and execute it, storing the result in an array or json?
Absolutely, and this (a separate SQL file) is a good technique for longer queries. Any of the database adapter gems can do this. In pg, which I'm most familiar with,
Read your query from a SQL file, e.g. File.read
Open a connection, e.g. PG::Connection.open
Call exec_params
Example from pg documentation:
require 'pg'
conn = PG::Connection.open(:dbname => 'test')
res = conn.exec_params('SELECT $1 AS a, $2 AS b, $3 AS c', [1, 2, nil])
# Equivalent to:
# res = conn.exec('SELECT 1 AS a, 2 AS b, NULL AS c')
http://deveiate.org/code/pg/PG/Connection.html
As you can see, the placeholders here are sequentially numbered, an improvement on simple question marks, I think.
Related
I'm using tiny_tds to pull data from a few different databases. As of right now, I have a Ruby file with multiple methods, each one devoted to a specific query (since the databases are very large, and not all of the scripts I'm using require the same kind/amount of data). To make things cleaner and easier, I wanted to separate out the SQL queries themselves into a single file, rather than have them embedded into the Ruby file containing the functions. But the SQL queries depend on certain fields having specific values. In essence, what I'm trying to do is send a variable to the SQL query, get data based on that particular value in a field, and feed that data back into the Ruby file.
So a simplified version of what I'm currently doing is this:
def initialize
#client = TinyTds::Client.new(:username => '', :password => '', :host => '', timeout: 0)
end
def query_example(value)
results = #client.execute("SELECT DISTINCT field1, field2, field3
FROM db
WHERE field1 = '#{value}'
")
results.each {|x| return x}
end
The script calls the query_example(value) function, and based on the value variable, gets the relevant data for that case.
But what I would like is to essentially have one file that has nothing but the raw SQL queries, like:
SELECT DISTINCT field1, field2, field3
FROM db
WHERE field1 = '#{value}'
where the #{value} is populated by an external value fed to it (although I'm not sure how that kind of wildcard would be declared here). Let's say I save this file as "query.sql", then I would just want to read that file into the Ruby function like:
def query_example(value)
query = File.read("query.sql")
results = #client.execute(query)
end
The problem is I don't actually know how to pass that value argument into the SQL file itself, so that the data that is pulled with the execute command is the specific data for that value. Is this possible with tiny_tds, or is tiny_tds simply not made for this kind of two-way interaction between external SQL queries, and the Ruby functions that call them? I'm open to considering other SQL libraries, I'm just very unfamiliar with the options as I deal mostly with the Ruby side of things.
You can use format method to replace placeholders with actual values. Here is the simplest example:
template = "Hello, %{name}!"
format(template, name: "World")
=> "Hello, World!"
And your code may look like this:
# query.sql
SELECT DISTINCT field1, field2, field3
FROM db
WHERE field1 = '%{value}'
# ruby file
def query_example(value)
query = File.read("query.sql")
results = #client.execute(format(query, value: value))
end
I have a quite simple Sinatra application for which I don't want to bother to create a model. I need to insert data into a Postgresql database without a model and, perhaps if possible, without ActiveRecord and also via only pure Sql. I've not found any examples of such a matter. How can I do that then?
You can use pg gem directly.
require 'pg'
conn = PG::Connection.open(:dbname => 'test')
res = conn.exec_params('SELECT $1 AS a, $2 AS b, $3 AS c', [1, 2, nil])
# Equivalent to:
# res = conn.exec('SELECT 1 AS a, 2 AS b, NULL AS c')
For specify more connection options check it out the PG::Connection constructor documentation.
I wrote a Perl script to check the data in an Oracle database. Because the query process is very complex I chose to create a VIEW in the middle. Using this view the code could be largely simplified.
The Perl code run well when I used it to query the database starting from a file, like Perl mycode.pl file_a. The Perl code reads lines from file_a and creates/updates the view until the end of the input. The results I achieved are completely right.
The problem came when I simultaneously run
perl mycode.pl file_a
and
perl mycode.pl file_b
to access the same database. According to my observation, the VIEW used by the first process will be modified by the second process. These two processes were intertwined on the same view.
Is there any suggestion to make these two processes not conflict with one another?
The Perl code for querying database is normally like this, but the details in each real query is more complex.
my ($gcsta,$gcsto,$cms) = #t; #(details of #t is read from a line in file a or b)
my $VIEWSS = 'CREATE OR REPLACE VIEW VIEWSS AS SELECT ID,GSTA,GSTO,GWTA FROM TABLEA WHERE GSTA='.$gcsta.' AND GSTO='.$gcsto.' AND CMS='.$cms;
my $querying = q{ SELECT COUNT(*) FROM VIEWSS WHERE VIEWSS.ID=1};
my $inner_sth = $dbh->prepare($VIEWSS);
my $inner_rv = $inner_sth->execute();
$inner_sth = $dbh->prepare($querying);
$inner_rv = $inner_sth->execute();
You must
Create the view only once, and use it everywhere
Use placeholders in your SQL statements, and pass the actual parameters with the call to execute
Is this the full extent of your SQL? Probably not, but if so it really is fairly simple.
Take a look at this refactoring for some ideas. Note that is uses a here document to express the SQL. The END_SQL marker for the end of the text must have no whitespace before or after it.
If your requirement is more complex than this then please describe it to us so that we can better help you
my $stmt = $dbh->prepare(<<'END_SQL');
SELECT count(*)
FROM tablea
WHERE gsta = ? AND gsto = ? AND cms= ? AND id = 1
END_SQL
my $rv = $stmt->execute($gcsta, $gcsto, $cms);
If you must use a view then you should use placeholders in the CREATE VIEW as before, and make every set of changes into a transaction so that other processes can't interfere. This involves disabling AutoCommit when you create the database handle $dbh and adding a call to $dbh->commit when all the steps are complete
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect('dbi:Oracle:mydbase', 'user', 'pass',
{ AutoCommit => 0, RaiseError => 1 } );
my $make_view = $dbh->prepare(<<'END_SQL');
CREATE OR REPLACE VIEW viewss AS
SELECT id, gsta, gsto, gwta
FROM tablea
WHERE gsta = ? AND gsto = ? AND cms= ? AND id = 1
END_SQL
my $get_count = $dbh->prepare(<<'END_SQL');
SELECT count(*)
FROM viewss
WHERE id = 1
END_SQL
while (<>) {
my ($gcsta, $gcsto, $cms) = split;
my $rv = $make_view->execute($gcsta, $gcsto, $cms);
$rv = $get_count->execute;
my ($count) = $get_count->fetchrow_array;
$dbh->commit;
}
Is the view going to be the same or different?
If the views are all the same then create it only once, or check if it exists with the all_views table : http://docs.oracle.com/cd/B12037_01/server.101/b10755/statviews_1202.htm#i1593583
You can easily create a view including your pid with the $$ variable to be the pid, but it wont be unique across computers, oracle has also some unique ids, see http://docs.oracle.com/cd/B14117_01/server.101/b10759/functions150.htm, for example, the SESSIONID.
But do you really need to do this? why dont you prepare a statement and then execute it? http://search.cpan.org/dist/DBI/DBI.pm#prepare
thanks,
mike
I want to do a database-side string concatenation in a Rails query, and do it in database-independent way.
SQL-92 specifies double-bar (||) as the concatenation operator. Unfortunately it looks like MS SQL Server doesn't support it; it uses + instead.
I'm guessing that Rails' SQL grammar abstraction has solved the db-specific operator problem already. If it does exist, how do I use it?
I had the same problem and never came up with anything that was built into Rails. So I wrote this little method.
# Symbols should be used for field names, everything else will be quoted as a string
def db_concat(*args)
adapter = configurations[RAILS_ENV]['adapter'].to_sym
args.map!{ |arg| arg.class==Symbol ? arg.to_s : "'#{arg}'" }
case adapter
when :mysql
"CONCAT(#{args.join(',')})"
when :sqlserver
args.join('+')
else
args.join('||')
end
end
I'm thinking somebody should really write some sort of SQL helper plugin that could automatically format simple SQL expressions based using the correct functions or operators for the current adapter. Maybe I'll write one myself.
It hasn't received much usage yet but I wrote the following code which seems to solve the problem. This monkey-patches the adapters to have a method to support it:
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
# Will return the given strings as a SQL concationation. By default
# uses the SQL-92 syntax:
#
# concat('foo', 'bar') -> "foo || bar"
def concat(*args)
args * " || "
end
end
class AbstractMysqlAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MySQL format:
#
# concat('foo', 'bar') -> "CONCAT(foo, bar)"
def concat(*args)
"CONCAT(#{args * ', '})"
end
end
class SQLServerAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MS-SQL format:
#
# concat('foo', 'bar') -> foo + bar
def concat(*args)
args * ' + '
end
end
end
end
With this you should be able to do the following in your code:
class User < ActiveRecord::Base
def self.find_by_name(name)
where("#{connection.concat('first_name', 'last_name')} = ?", name)
end
end
This outputs the following SQL query on a SQL-92 database (Oracle, SQLite, PostgreSQL):
SELECT * FROM users WHERE first_name || last_name = ?
For MySQL it outputs:
SELECT * FROM users WHERE CONCAT(first_name, last_name) = ?
For SQL Server it outputs
SELECT * FROM users WHERE first_name + last_name = ?
Obviously you could extend this concept to other database adapters.
If you want something Rails neutral, you're going to need to return the values you want concatenated and do that once the data has been delivered to rails (or do it in rails before you give it to the database).
It looks like Mysql uses CONCAT(), Postgres ||, Oracle CONCAT() or ||, T-SQL +.
Any rails abstraction of the same would have to take place at a point where you could just be doing concatenation using regular Ruby, or I've completely misunderstood the question.
I want to write a script in Ruby to clean up some messed up keys in several copies of the same MySQL schema. I'd like to do something like SHOW CREATE TABLE, then look at what comes back and delete keys if they exist.
I know in the Rails environment you can do this...
ActiveRecord::Base.connection.execute( some sql )
But what you get back is a "Result" object. For this task I need a String so I can analyze it and act accordingly.
This should help you:
>> result = ActiveRecord::Base.connection.execute("SHOW TABLES")
=> #<Mysql::Result:0x37ecb30>
>> result.class.instance_methods - Object.instance_methods
=> ["all_hashes", "field_seek", "row_tell", "fetch_field_direct", "free", "field_tell", "fetch_lengths", "num_fields", "data_seek", "fetch_row", "num_rows", "fetch_field", "each", "each_hash", "fetch_hash", "row_seek", "fetch_fields"]
Look at #all_hashes on the MySql::Result instance
I would use the mysql-ruby gem and you would do something like this:
require 'mysql'
m = MySQL.new("localhost", "username", "password", "database")
r = m.query("SELECT * FROM people ORDER BY name")
r.each_hash do |f|
print "#{f['name']} - #{f['email']}"
end
You could check the mysql-ruby gem.
Here is a write-up on how to use it: Using the Ruby MySQL Module
More can be found via google
If you don't want to use ActiveRecord an ORM may be a bit complicated for your usage right now), you can still use the ruby-mysql library or even better IMHO is to use the Ruby DBI/DBD library (here) which has DBD drivers for mysql & postgresql out-of-the-box.
That way, you can issue straight SQL statements like this
require "dbi"
require "dbi/dbrc"
# == Configuration
DB = "sympa"
HOST = "saphir"
cnt = 0
dup = 0
# == Crude option processing
#
list_name = ARGV.shift.to_s
file = ARGV.shift.to_s
db = DBI::DBRC.new(DB)
DBI.connect(db.dsn + ":#{HOST}", db.user, db.password) do |dbh|
date = Time.now.asctime
if not list_name or list_name == "" then
puts "List name is mandatory"
exit 1
end
req1 = <<-"EOR"
insert into user_table (email_user,lang_user)
values (?, ?)
EOR
...
req2 = <<-"EOR"
insert into subscriber_table
(user_subscriber, list_subscriber, visibility_subscriber,
date_subscriber, reception_subscriber) values (?, ?, ?, NOW(), ?)
EOR
sth1 = dbh.prepare(req1)
sth2 = dbh.prepare(req2)
...
#
# Insertion in user_table
#
begin
sth1.execute(line, "en")
cnt += 1
rescue DBI::DatabaseError => err
$stderr.puts("DBI: #{err}")
end
dbi/dbrc is a useful module that enables you to avoid putting login&password directly in the script. See there.
There is probably a better way to do that programmatically, however if you really want to drive the interactive commands and parse the results, then expect may be more suitable. You could still kick off expect from your ruby script.
Use Mysql2
To update this thread: I'd suggest Mysql2 now: http://rubygems.org/gems/mysql2