Rails 3.1 / mysql2 error : "MySQL server has gone away" - ruby-on-rails-3

Question

I'm experiencing trouble upgrading my rails 2.3.14 / ruby 1.8.7 app to 3.1.1/1.9.2 : I have some
(ActiveRecord::StatementInvalid) "Mysql2::Error: MySQL server has gone away"
errors happening sporadically. It's important to precise that I never had such issues with the 'mysql' gem on 2.3.14 and the exactly same db (so the bug shouldn't come from mysql (v5.5.10)).
Example :
$ rails c production
Loading production environment (Rails 3.1.1)
ruby-1.9.2-p290 :001 > ActiveRecord::Base.connection.active?
=> false
ruby-1.9.2-p290 :002 > exit
$ rails c production
Loading production environment (Rails 3.1.1)
ruby-1.9.2-p290 :001 > ActiveRecord::Base.connection.active?
=> true
This happens only with my (remote) production database, no problem with my local development db. I've tried to set "reconnect: true" in my database.yml but it led to a
Mysql2::Error: Host '****' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts':...
I've tried to isolate the problem with a little rb script only loading mysql2 and activerecord but I didn't manage to reproduce the bug that way (so it may be linked to the rails stack).
I can't go back from the 'mysql2' to the 'mysql' gem because of encoding issues ( http://www.rorra.com.ar/2010/07/30/rails-3-mysql-and-utf-8/ ). As a consequence, I had to rollback my production to my rails 2.3.14 app, which saddens me very much...
Do you see what I can do to debug this ? I can't even find a sure way to reproduce the error... Have anyone met the same bug ?
I just found few people mentionning this bug (ex : https://github.com/brianmario/mysql2/issues/213) but not a solution.
Thanks for your help.

Answer 1

Ok, I think I solved my problem. I didn't notice it when I posted my question, but it seemed that the error was timeout related : after about 20s, activerecord losts its connection.
$ rails runner "sleep 23; puts ActiveRecord::Base.connection.active?"
=> true
$ rails runner "sleep 25; puts ActiveRecord::Base.connection.active?"
=> false
So I dug further and I realized that mysql and mysql2 gems didn't deal with the MySQL 'wait_timeout' param the same way : mysql gem doesn't set it thus uses the MySQL default value 28800, whereas mysql2 gem sets it at 2592000 if not defined in the database.yml.
But I have the impression that the value 2592000 is over the max value for this param : 2147483 ! Which could lead to the unexpected behavior I described...
I build a script test showing the bug : https://gist.github.com/1514154
And if I had some apparently random disconnect while loading rails console (cf my question), I think it's because of my app taking a long time to load and me sometimes waiting a few seconds before typing my command.
I can't explain why we are so few to encounter this problem. Perhaps it's specific to my conf (remote database, MySQL version ?). I've tried with another remote staging database : the bug didn't reproduce...
So as a conclusion, I will set wait_timeout: 2147483 in my database.yml. And maybe pull request rails...

Answer 2

Had a lot of lost connections - but I couldn't say if they went away due to the following tweak or elsewise :/
Had to throw the following script into initializers and add a line of configuration to each of my databases in my database.yml like this:
...
flags: <%= 65536 | 131072 %>
...
The script looks like this:
/config/initializers/mysql2.rb
module ActiveRecord
class Base
# Overriding ActiveRecord::Base.mysql2_connection
# method to allow passing options from database.yml
#
# Example of database.yml
#
# login: &login
# socket: /tmp/mysql.sock
# adapter: mysql2
# host: localhost
# encoding: utf8
# flags: 131072
#
# #param [Hash] config hash that you define in your
# database.yml
# #return [Mysql2Adapter] new MySQL adapter object
#
def self.mysql2_connection(config)
config[:username] = 'root' if config[:username].nil?
if Mysql2::Client.const_defined? :FOUND_ROWS
config[:flags] = config[:flags] ? config[:flags] | Mysql2::Client::FOUND_ROWS : Mysql2::Client::FOUND_ROWS
end
client = Mysql2::Client.new(config.symbolize_keys)
options = [config[:host], config[:username], config[:password], config[:database], config[:port], config[:socket], 0]
ConnectionAdapters::Mysql2Adapter.new(client, logger, options, config)
end
end
end