This question is a follow up to this question, where should I place this code?
connection = ActiveRecord::Base.connection
class << connection
alias :original_exec :execute
def execute(sql, *name)
# try to log sql command but ignore any errors that occur in this block
# we log before executing, in case the execution raises an error
begin
file = File.open(RAILS_ROOT + "/log/sql.txt",'a'){|f| f.puts Time.now.to_s+": "+sql}
rescue Exception => e
;
end
# execute original statement
original_exec(sql, *name)
end
end
I have tried placing it inside of the model, but what happens is when I execute some sql query more then once it returns "stack level is to deep" error.
Put it in config/initializers. Most likely it's because of reloading classes each time in dev env. This code need to be executed only once though.
Related
My application is crashing really, really hard, and it appears to be related to the database. The application deals with lots and lots of data, and hundreds of simultaneous users. In an effort to speed up data loads, I am loading some records like this:
def load(filename)
rc = Publication.connection.raw_connection
rc.exec("COPY invoice_line_items FROM STDIN WITH CSV HEADER")
# open up your CSV file looping through line by line and getting the line into a format suitable for pg's COPY...
error = false
begin
CSV.foreach(filename) do |line|
until rc.put_copy_data( line.to_csv )
ErrorPrinter.print " waiting for connection to be writable..."
sleep 0.1
end
end
rescue Errno => err
User.inform_admin(false, User.me, "Line Item import failed with #{err.class.name} the following error: #{err.message}", err.backtrace)
error = true
else
rc.put_copy_end
while res = rc.get_result
if (res.result_status != 1)
User.inform_admin(false, User.me, "Line Item import result of COPY was: %s" % [ res.res_status(res.result_status) ], "")
error = true
end
end
end
end
I also have Sidekiq running with about 90 threads. Does this method of loading put an exclusive lock on that table? Is it possible that these jobs are running into each other? If they are, am I better off just doing inserts?
COPY takes the same level of lock as INSERT. (It's missing from the explicit locking chapter, but visible in the source code). So whatever's giving you trouble, it's probably not that.
You should be looking at pg_locks and pg_stat_activity to see if anything's stuck on a lock. More info on other questions on SO or DBA.SE, the manual, and the PostgreSQL wiki.
I have two service programs: mySrvPgm and myErr
mySrvPgm has a procedure which contains:
/free
...
exec sql INSERT INTO TABLE VALUES(:RECORD_FMT);
if sqlError() = *ON;
//handle error
endif;
...
/end-free
myErr has a procedure sqlError:
/free
exec sql GET DIAGNOSTICS CONDITION 1
:state = RETURNED_SQLSTATE;
...
/end-free
Background info: I am using XMLSERVICE to call the given procedure in mySrvPgm from PHP. I am not using a persistent connection. myErr is bound-by-reference via a binding directory used by mySrvPgm. Its activation is set to *IMMED, its activation group is set to *CALLER.
The problem: Assume there is an error on the INSERT statement in mySvrPgm. The first time sqlError() is called it will return SQLSTATE 00000 despite the error. All subsequent calls to sqlError() return the expected SQLSTATE.
A workaround: I added a procedure to myErr called initSQL:
/free
exec sql SET SCHEMA MYLIB;
/end-free
If I call initSQL() before the INSERT statement in mySrvPgm, sqlError() functions correctly. It doesn't have to be SET SCHEMA, it can be another GET DIAGNOSTICS statement. However, if it does not contain an executable SQL statement it does not help.
The question: I believe the myErr service program is activating properly and has the correct scope, but I am wondering if there is something more I need to do to activate the SQL part of it. Is there some way to set it up so SQL auto-initializes when the service program is activated, or do I have to execute an unneeded SQL statement in order to get it started?
There is some more background information available here.
Thank you for reading.
What version an release of the OS? Are you upto date on PTFs?
Honestly, seems to me that it's possibly a bug. Or the manual(s) need clarification.. I'd open a PMR.
In our PostgreSQL-backed Rails project, when running rspec using spork, sometimes we receive the following error:
ActiveRecord::StatementInvalid:
PG::Error: ERROR: prepared statement "a1" already exists
Originally, it occurred only a few times a day, but recently, it has begun happening every 3-4 test runs which slows our development efforts to a crawl.
Is there a way to reset/delete the prepared statements in PostgreSQL somewhere inside of our spec_helper.rb file?
After searching the PostgreSQL docs and a fair amount of trial and error testing it in different spots inside the spec_helper.rb, I finally figured out that I could add the following to delete all existing prepared statements and I have not since seen the error:
RSpec.configure do |config|
# ... other code
config.after(:suite) do
ActiveRecord::Base.connection.execute("DEALLOCATE ALL")
end
# ... other code
end
You can get this if you're doing something like:
class ActiveRecord::Base
mattr_accessor :shared_connection
##shared_connection = nil
def self.connection
##shared_connection || retrieve_connection
end
end
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection
(which was recommended here).
If you have that or similar, try removing it.
I have an iteration in which I am calling set_table_name for some model. The idea is that in each iteration, the model changes its table. Sometimes, the table won't exist, in that case, an error of this kind will happen:
Mysql2::Error: Table 'db_name.table_name_xyz' doesn't exist
I want the iteration to keep running and not abort because of the error. I've wrapped the set_table_name line of code with a begin and rescue, but no exception seems to be raised as the script instantly aborts on error (it doesn't execute the rescue code). Here's the code:
((start_year)..(start_actual_year)).each do |year|
begin
Data.set_table_name("Secciones#{year}#{year + 1}")
rescue Exception => e
next
end
end
Can I rescue this kind of error? What should I do? Thanks!
Are you sure you are catching the exception correctly? Here's more info if needed.
begin
# Your code here.
rescue Exception => e
# Output the exception that it is catching.
puts e.message
ensure
# The ensure block is used to close a db connection if needed.
end
I want to save to a log file some SQL query rails performs, (namely the CREATE, UPDATE and DELETE ones)
therefore I need to intercept all queries and then filter them maybe with some regexp and log them as needed.
Where would I put such a thing in the rails code?
Here a simplified version of what c0r0ner linked to, to better show it:
connection = ActiveRecord::Base.connection
class << connection
alias :original_exec :execute
def execute(sql, *name)
# try to log sql command but ignore any errors that occur in this block
# we log before executing, in case the execution raises an error
begin
File.open(Rails.root.join("/log/sql.txt"),'a'){|f| f.puts Time.now.to_s+": "+sql}
rescue Exception => e
;
end
# execute original statement
original_exec(sql, *name)
end
end
SQL logging in rails -
In brief - you need to override ActiveRecord execute method. There you can add any logic for logging.
As a note for followers, you can "log all queries" like Rails - See generated SQL queries in Log files and then grep the files for the ones you want, if desired.
If you are using mysql I would look into mysqlbinlog . It is going to track everything that potentially updates data. you can grep out whatever you need from that log easily.
http://dev.mysql.com/doc/refman/5.0/en/mysqlbinlog.html
http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
SQL Server? If so...
Actually, I'd do this at the SQL end. You could set up a trace, and collect every query that comes through a connection with a particular Application Name. If you save it to a table, you can easily query that table later.
Slightly updated version of #luca's answer for at least Rails 4 (and probably Rails 5)
Place this in config/initializers/sql_logger.rb:
connection = ActiveRecord::Base.connection
class << connection
alias :original_exec :execute
def execute(sql, *name)
# try to log sql command but ignore any errors that occur in this block
# we log before executing, in case the execution raises an error
begin
File.open(Rails.root.join("log/sql.log"), 'a') do |file|
file.puts Time.now.to_s + ": " + sql
end
rescue Exception => e
"Error logging SQL: #{e}"
end
# execute original statement
original_exec(sql, *name)
end
end