I want to write a script in Ruby to clean up some messed up keys in several copies of the same MySQL schema. I'd like to do something like SHOW CREATE TABLE, then look at what comes back and delete keys if they exist.
I know in the Rails environment you can do this...
ActiveRecord::Base.connection.execute( some sql )
But what you get back is a "Result" object. For this task I need a String so I can analyze it and act accordingly.
This should help you:
>> result = ActiveRecord::Base.connection.execute("SHOW TABLES")
=> #<Mysql::Result:0x37ecb30>
>> result.class.instance_methods - Object.instance_methods
=> ["all_hashes", "field_seek", "row_tell", "fetch_field_direct", "free", "field_tell", "fetch_lengths", "num_fields", "data_seek", "fetch_row", "num_rows", "fetch_field", "each", "each_hash", "fetch_hash", "row_seek", "fetch_fields"]
Look at #all_hashes on the MySql::Result instance
I would use the mysql-ruby gem and you would do something like this:
require 'mysql'
m = MySQL.new("localhost", "username", "password", "database")
r = m.query("SELECT * FROM people ORDER BY name")
r.each_hash do |f|
print "#{f['name']} - #{f['email']}"
end
You could check the mysql-ruby gem.
Here is a write-up on how to use it: Using the Ruby MySQL Module
More can be found via google
If you don't want to use ActiveRecord an ORM may be a bit complicated for your usage right now), you can still use the ruby-mysql library or even better IMHO is to use the Ruby DBI/DBD library (here) which has DBD drivers for mysql & postgresql out-of-the-box.
That way, you can issue straight SQL statements like this
require "dbi"
require "dbi/dbrc"
# == Configuration
DB = "sympa"
HOST = "saphir"
cnt = 0
dup = 0
# == Crude option processing
#
list_name = ARGV.shift.to_s
file = ARGV.shift.to_s
db = DBI::DBRC.new(DB)
DBI.connect(db.dsn + ":#{HOST}", db.user, db.password) do |dbh|
date = Time.now.asctime
if not list_name or list_name == "" then
puts "List name is mandatory"
exit 1
end
req1 = <<-"EOR"
insert into user_table (email_user,lang_user)
values (?, ?)
EOR
...
req2 = <<-"EOR"
insert into subscriber_table
(user_subscriber, list_subscriber, visibility_subscriber,
date_subscriber, reception_subscriber) values (?, ?, ?, NOW(), ?)
EOR
sth1 = dbh.prepare(req1)
sth2 = dbh.prepare(req2)
...
#
# Insertion in user_table
#
begin
sth1.execute(line, "en")
cnt += 1
rescue DBI::DatabaseError => err
$stderr.puts("DBI: #{err}")
end
dbi/dbrc is a useful module that enables you to avoid putting login&password directly in the script. See there.
There is probably a better way to do that programmatically, however if you really want to drive the interactive commands and parse the results, then expect may be more suitable. You could still kick off expect from your ruby script.
Use Mysql2
To update this thread: I'd suggest Mysql2 now: http://rubygems.org/gems/mysql2
Related
Is there a way to prepare an SQL statement like that built by ActiveRecord where() method (using named placeholders):
Client.where("created_at >= :start_date AND created_at <= :end_date",
{start_date: params[:start_date], end_date: params[:end_date]})
I have to use this in CASE .. END statement after ORDER BY clause (or in SELECT to create a computed column) to protect it from SQL injection.
EDIT:
I have to retrieve all the ActiveRecord models by this query too. So can I use find_by_sql()? (trying...).
EDIT2:
find_by_sql() can't use named placeholders (:start_date in the above code example).
It's Rails 3.2.11
EDIT3:
Sorry, it can use with an Array as one parameter (find_by_sql with array format in Rails 3).
Yes, you can write / prepare anything that ActiveRecord does. Do you mean to use a direct DBI query?
I recommend you trace the SQL that ActiveRecord is generating.
Retrieving sql queries from Active-record queries in Rails 3
Tracing Rails 3 SQL queries
http://guides.rubyonrails.org/v2.3.11/debugging_rails_applications.html
See query_trace for one approach.
A tutorial on how to use Ruby DBI and prepared statements:
http://www.tutorialspoint.com/ruby/ruby_database_access.htm
Quoted directly from the tutorial:
#!/usr/bin/ruby -w
require "dbi"
begin
# connect to the MySQL server
dbh = DBI.connect("DBI:Mysql:TESTDB:localhost",
"testuser", "test123")
sth = dbh.prepare( "INSERT INTO EMPLOYEE(FIRST_NAME,
LAST_NAME,
AGE,
SEX,
INCOME)
VALUES (?, ?, ?, ?, ?)" )
sth.execute('John', 'Poul', 25, 'M', 2300)
sth.execute('Zara', 'Ali', 17, 'F', 1000)
sth.finish
dbh.commit
puts "Record has been created"
rescue DBI::DatabaseError => e
puts "An error occurred"
puts "Error code: #{e.err}"
puts "Error message: #{e.errstr}"
dbh.rollback
ensure
# disconnect from server
dbh.disconnect if dbh
end
Using find_by_sql() solved this.
But this can't be use as scoped query. (And thus cannot be chained).
I wrote a Perl script to check the data in an Oracle database. Because the query process is very complex I chose to create a VIEW in the middle. Using this view the code could be largely simplified.
The Perl code run well when I used it to query the database starting from a file, like Perl mycode.pl file_a. The Perl code reads lines from file_a and creates/updates the view until the end of the input. The results I achieved are completely right.
The problem came when I simultaneously run
perl mycode.pl file_a
and
perl mycode.pl file_b
to access the same database. According to my observation, the VIEW used by the first process will be modified by the second process. These two processes were intertwined on the same view.
Is there any suggestion to make these two processes not conflict with one another?
The Perl code for querying database is normally like this, but the details in each real query is more complex.
my ($gcsta,$gcsto,$cms) = #t; #(details of #t is read from a line in file a or b)
my $VIEWSS = 'CREATE OR REPLACE VIEW VIEWSS AS SELECT ID,GSTA,GSTO,GWTA FROM TABLEA WHERE GSTA='.$gcsta.' AND GSTO='.$gcsto.' AND CMS='.$cms;
my $querying = q{ SELECT COUNT(*) FROM VIEWSS WHERE VIEWSS.ID=1};
my $inner_sth = $dbh->prepare($VIEWSS);
my $inner_rv = $inner_sth->execute();
$inner_sth = $dbh->prepare($querying);
$inner_rv = $inner_sth->execute();
You must
Create the view only once, and use it everywhere
Use placeholders in your SQL statements, and pass the actual parameters with the call to execute
Is this the full extent of your SQL? Probably not, but if so it really is fairly simple.
Take a look at this refactoring for some ideas. Note that is uses a here document to express the SQL. The END_SQL marker for the end of the text must have no whitespace before or after it.
If your requirement is more complex than this then please describe it to us so that we can better help you
my $stmt = $dbh->prepare(<<'END_SQL');
SELECT count(*)
FROM tablea
WHERE gsta = ? AND gsto = ? AND cms= ? AND id = 1
END_SQL
my $rv = $stmt->execute($gcsta, $gcsto, $cms);
If you must use a view then you should use placeholders in the CREATE VIEW as before, and make every set of changes into a transaction so that other processes can't interfere. This involves disabling AutoCommit when you create the database handle $dbh and adding a call to $dbh->commit when all the steps are complete
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect('dbi:Oracle:mydbase', 'user', 'pass',
{ AutoCommit => 0, RaiseError => 1 } );
my $make_view = $dbh->prepare(<<'END_SQL');
CREATE OR REPLACE VIEW viewss AS
SELECT id, gsta, gsto, gwta
FROM tablea
WHERE gsta = ? AND gsto = ? AND cms= ? AND id = 1
END_SQL
my $get_count = $dbh->prepare(<<'END_SQL');
SELECT count(*)
FROM viewss
WHERE id = 1
END_SQL
while (<>) {
my ($gcsta, $gcsto, $cms) = split;
my $rv = $make_view->execute($gcsta, $gcsto, $cms);
$rv = $get_count->execute;
my ($count) = $get_count->fetchrow_array;
$dbh->commit;
}
Is the view going to be the same or different?
If the views are all the same then create it only once, or check if it exists with the all_views table : http://docs.oracle.com/cd/B12037_01/server.101/b10755/statviews_1202.htm#i1593583
You can easily create a view including your pid with the $$ variable to be the pid, but it wont be unique across computers, oracle has also some unique ids, see http://docs.oracle.com/cd/B14117_01/server.101/b10759/functions150.htm, for example, the SESSIONID.
But do you really need to do this? why dont you prepare a statement and then execute it? http://search.cpan.org/dist/DBI/DBI.pm#prepare
thanks,
mike
Is there a more efficient method for doing a Rails SQL statement of the following code?
It will be called across the site to hide certain content or users based on if a user is blocked or not so it needs to be fairly efficient or it will slow everything else down as well.
users.rb file:
def is_blocked_by_or_has_blocked?(user)
status = relationships.where('followed_id = ? AND relationship_status = ?',
user.id, relationship_blocked).first ||
user.relationships.where('followed_id = ? AND relationship_status = ?',
self.id, relationship_blocked).first
return status
end
In that code, relationship_blocked is just an abstraction of an integer to make it easier to read later.
In a view, I am calling this method like this:
- unless current_user.is_blocked_by_or_has_blocked?(user)
- # show the content for unblocked users here
Edit
This is a sample query.. it stops after it finds the first instance (no need to check for a reverse relationship)
Relationship Load (0.2ms) SELECT "relationships".* FROM "relationships" WHERE ("relationships".follower_id = 101) AND (followed_id = 1 AND relationship_status = 2) LIMIT 1
You can change it to only run one query by making it use an IN (x,y,z) statement in the query (this is done by passing an array of ids to :followed_id). Also, by using .count, you bypass Rails instantiating an instance of the model for the resulting relationships, which will keep things faster (less data to pass around in memory):
def is_blocked_by_or_has_blocked?(user)
relationships.where(:followed_id => [user.id, self.id], :relationship_status => relationship_blocked).count > 0
end
Edit - To get it to look both ways;
Relationship.where(:user_id => [user.id, self.id], :followed_id => [user.id, self.id], :relationship_status => relationship_blocked).count > 0
I'm using RoR and I want to do concurrency safe update queries. For example when I have
var user = User.find(user_id)
user.visits += 1
I get the following SQL code:
SELECT * FROM Users WHERE ID=1 -- find user's visits (1)
UPDATE Users SET Visits=2 WHERE ID=1 -- 1+1=2
But if there are several queries taking place at the same time, there will be locking problems.
According to RoR API I can use :lock => true attribute, but that's not what I want.
I found an awesome function update_counters:
User.update_counters(my_user.id, :visits => 1)
This gives the following SQL code
UPDATE Users SET Visits=Visits+1 WHERE ID=#something
Works great!
Now my question is how can I override the += function to do the update_counters thing instead?
Because
user.visits += 1 # better
User.update_counters(my_user.id, :visits => 1) # than this
UPDATE
I just created the following function
class ActiveRecord::Base
def inc(column, value)
User.update_counters(self.id, column => value)
end
end
Are there any other better ideas?
Don't know about better ideas, but I would define that method to the User class, not the ActiveRecord. And maybe increment_counter (that uses update_counters) would make it a bit more readable?
def inc(column, value)
self.increment_counter(column, value)
end
Haven't tested that and not saying this is definitely better idea, but that's probably how I'd do it.
Update:
And AFAIK you can't override the "+=", because "a += b" just a shortcut for "a = a + b" and you probably don't want to override "=" to use the update_counters :)
I want to do a database-side string concatenation in a Rails query, and do it in database-independent way.
SQL-92 specifies double-bar (||) as the concatenation operator. Unfortunately it looks like MS SQL Server doesn't support it; it uses + instead.
I'm guessing that Rails' SQL grammar abstraction has solved the db-specific operator problem already. If it does exist, how do I use it?
I had the same problem and never came up with anything that was built into Rails. So I wrote this little method.
# Symbols should be used for field names, everything else will be quoted as a string
def db_concat(*args)
adapter = configurations[RAILS_ENV]['adapter'].to_sym
args.map!{ |arg| arg.class==Symbol ? arg.to_s : "'#{arg}'" }
case adapter
when :mysql
"CONCAT(#{args.join(',')})"
when :sqlserver
args.join('+')
else
args.join('||')
end
end
I'm thinking somebody should really write some sort of SQL helper plugin that could automatically format simple SQL expressions based using the correct functions or operators for the current adapter. Maybe I'll write one myself.
It hasn't received much usage yet but I wrote the following code which seems to solve the problem. This monkey-patches the adapters to have a method to support it:
module ActiveRecord
module ConnectionAdapters
class AbstractAdapter
# Will return the given strings as a SQL concationation. By default
# uses the SQL-92 syntax:
#
# concat('foo', 'bar') -> "foo || bar"
def concat(*args)
args * " || "
end
end
class AbstractMysqlAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MySQL format:
#
# concat('foo', 'bar') -> "CONCAT(foo, bar)"
def concat(*args)
"CONCAT(#{args * ', '})"
end
end
class SQLServerAdapter < AbstractAdapter
# Will return the given strings as a SQL concationation.
# Uses MS-SQL format:
#
# concat('foo', 'bar') -> foo + bar
def concat(*args)
args * ' + '
end
end
end
end
With this you should be able to do the following in your code:
class User < ActiveRecord::Base
def self.find_by_name(name)
where("#{connection.concat('first_name', 'last_name')} = ?", name)
end
end
This outputs the following SQL query on a SQL-92 database (Oracle, SQLite, PostgreSQL):
SELECT * FROM users WHERE first_name || last_name = ?
For MySQL it outputs:
SELECT * FROM users WHERE CONCAT(first_name, last_name) = ?
For SQL Server it outputs
SELECT * FROM users WHERE first_name + last_name = ?
Obviously you could extend this concept to other database adapters.
If you want something Rails neutral, you're going to need to return the values you want concatenated and do that once the data has been delivered to rails (or do it in rails before you give it to the database).
It looks like Mysql uses CONCAT(), Postgres ||, Oracle CONCAT() or ||, T-SQL +.
Any rails abstraction of the same would have to take place at a point where you could just be doing concatenation using regular Ruby, or I've completely misunderstood the question.