This question already has answers here:
How to show SQL queries run in the Rails console?
(8 answers)
Closed 8 years ago.
My terminal used to display SQL table insert statements (as it should by default), and now it doesn't. Everything else - for example, GET requests, or local server startup info, or job completion notifications - appear as normal.
For the life of me, I can't figure out what the problem could be. I've been searching for a long time and can't figure out any reason this would happen. Do you have any ideas to point me in the right direction?
Put this in either in your .irbrc or .pryrc in your home directory
if defined?(Rails) && !Rails.env.nil?
puts '... ActiveRecord and ActiveResource Logger set to STDOUT'
logger = Logger.new(STDOUT)
ActiveRecord::Base.logger = logger
ActiveResource::Base.logger = logger
end
You should start to see the sql statements executing in the rails console.
Hope this helps
Related
I keep getting a sql syntax error when I run the script to import the data into sql.
here is the data in question,
filtered_interfaces = (['interface Gi1/0/7']
filtered_tech = ['description TECH_5750'])
cab = u'10.210.44.5'
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '['description TECH_5750'],['interface Gi1/0/7'],10.20.94.5)' at line 1")
Ive attempted dictation and tried to match values as it gets placed in to the database, I have also tried %s inplace of {}.
Im getting the same error as above on both of these methods.
sql = "INSERT INTO device (hostname, cab, localint) VALUES ({0},{1},{2})".format(filtered_tech,filtered_interface,cab)
mycursor.execute(sql)
Long story short this application logs into switches at our DC's. It looks for CDP LLDP devices then spits out all that info into an array. I then filter the info looking for hostname and local interface. The goal is to import that data into sql (Its different on each switch). Then retrieve data on a html page and edit the hostname if applicable. I had the editable piece working but it wasn't scalable hints the sql additions. Apologies if this is an easy fix im not a programmer by trade, just trying to make life easier and learn python.
If you are using MySQL behind your Python script, then you should be following the usage pattern in the documentation for prepared statements:
params = (filtered_tech, filtered_interface, cab,)
sql = """INSERT INTO device (hostname, cab, localint)
VALUES (%s, %s, %s)"""
cursor.execute(sql, params)
# retrieve data here
I am running a very simple query and trying to extract the results to a text file. The entire query is essentially what is below, I am selecting everything from one single table with one piece of where criteria which is limiting the data to one month's worth. After it has extracted around 1.2 gig this error shows up. Is there any way that I can work around this other than extracting smaller date ranges? I am trying to pull a couple of years worth of data so if I can only get it a few days at a time it will take a lot of manual work.
I am currently using the free trial of a DB2 query tool - Razor SQL if that makes a difference, I can probably purchase different software if it would help. I am trying to get IBM's tool but for some reason it freezes during the download so I am still working on that. I have searched about this error but everything I see seems much more complex than what I am doing and I can't tell if it applies or not. Thanks in advance.
select *
from MyTable
where date_col between date '2014-01-01' and date '2014-01-31'
I stumbled at this error too, found out it is related to db2jcc.jar (type 4) driver.
Excerpt: If there are no items in the result set left (or to begin with), the Result set is closed automatically and therefore the Exception. Suggestion is to handle it in the application, perhaps in my case, I started checking if(rs.next()) but otherwise, there is a work around. Check out the source link below for how you can set some properties to Data source and avoid exception.
Source :
"Invalid operation: result set is closed" error with Data Server Driver for JDBC
In my case, i missed some properties in WAS, after add allowNextOnExhaustedResultSet the issue is fixed.
1.Log in to the WebSphere Application Server administration console.
2.Select Resources > JDBC > Data sources > Application Center DataSource name > Custom properties and click New.
3.In the Name field, enter allowNextOnExhaustedResultSet.
4.In the Value field, type 1.
5.Change the type to java.lang.Integer.
6.Click OK.
Sometimes you need also check whether resultSetHoldability properties exists. Details refer to here.
I encountered this failure also when ugrading from JDBC Type 2 driver (db2java.zip) JDBC type 4 driver (db2jcc4.jar)
Statement statement = results.getStatement();
if (statement != null)
{
connection = statement.getConnection(); // ** failed here
statement.close();
}
Solution was to check if the statement is closed or not as follows.
Changed to:
Statement statement = results.getStatement();
if (statement != null && !statement.isClosed()) {
{
connection = statement.getConnection();
statement.close();
}
Creating property bellow with type Integer it's worked for me:
allowNextOnExhaustedResultSet:
I had the same issue on WAS 7 so i had to add and change few this on Admin Console.
This TeamWorksRuntimeException exception should be fixed by applying APAR JR50863 which is available on top of BPM V8.5.5 or included on BPM V8.5 refresh pack 6.
For the case that the APAR does not solve the problem, try following workaround:
Log in to the WebSphere Application Server admin console
Select Resources > JDBC > Data sources > DataSource name (TeamWorksDB) > Custom properties and click New
In the Name field, enter downgradeHoldCursorsUnderXa
In the Value field, type true
Change the type to java.lang.Boolean
Click OK to save your changes
Select custom property resultSetHoldability
In the Value field, type 1
Click OK to save your changes
Source of the Answer : https://developer.ibm.com/answers/questions/194821/invalid-operation-result-set-is-closed-errorcode-4/
Restarting the app may fix the problem if connection pool lost session to Db2. If using Tomcat then connection pool property of 'testonBorrow' may reestablish the connection to Db2.
I recollect getting log files that were nicely ordered, so that you could follow one request, then the next, and so on.
Now, the log files are, as my 4 year old says "all scroggled up", meaning that they are no longer separate, distinct chunks of text. Loggings from two requests get intertwined/mixed up.
For instance:
Started GET /foobar
...
Completed 200 OK in 2ms (Views: 0.4ms | ActiveRecord: 0.8ms)
Patient Load (wait, that's from another request that has nothing to do with foobar!)
[ blank space ]
Something else
This is maddening, because I can't tell what's happening within one single request.
This is running on Passenger.
I tried to search for the same answer but couldn't find any good info. I'm not sure if you should fix server or rails code.
If you want more info about the issue here is the commit that removed old way of logging https://github.com/rails/rails/commit/04ef93dae6d9cec616973c1110a33894ad4ba6ed
If you value production log readability over everything else you can use the
PassengerMaxInstancesPerApp 1
configuration. It might cause some scaling issues. Alternatively you could stuff something like this in application.rb:
process_log_filename = Rails.root + "log/#{Rails.env}-#{Process.pid}.log"
log_file = File.open(process_log_filename, 'a')
Rails.logger = ActiveSupport::BufferedLogger.new(log_file)
Yep!, they have made some changes in the ActiveSupport::BufferedLogger so it is not any more waiting until the request has ended to flush the logs:
http://news.ycombinator.com/item?id=4483390
https://github.com/rails/rails/commit/04ef93dae6d9cec616973c1110a33894ad4ba6ed
But they have added the ActiveSupport::TaggedLogging which is very funny and you can stamp every log with any kind of mark you want.
In your case could be good to stamp the logs with the request UUID like this:
# config/application.rb
config.log_tags = [:uuid]
Then even if the logs are messed up you still can follow which of them correspond to the request you are following up.
You can make more funny things with this feature to help you in your logs study:
How to log user_name in Rails?
http://zogovic.com/post/21138929607/running-time-in-rails-logs
Well, for me the TaggedLogging solution is a no go, I can live with some logs getting lost if the server crashes badly, but I want my logs to be perfectly ordered. So, following advice from the issue comments I'm applying this to my app:
# lib/sequential_logs.rb
module ActiveSupport
class BufferedLogger
def flush
#log_dest.flush
end
def respond_to?(method, include_private = false)
super
end
end
end
# config/initializers/sequential_logs.rb
require 'sequential_logs.rb'
Rails.logger.instance_variable_get(:#logger).instance_variable_get(:#log_dest).sync = false
As far as I can say this hasn't affected my app, it is still running and now my logs make sense again.
They should add some quasi-random reqid and write it in every line regarding one single request. This way you won't get confused.
I haven't used it, but I believe Lumberjack's unit_of_work method may be what you're looking for. You call:
Lumberjack.unit_of_work do
yield
end
And all logging done either in that block or in the yielded block are tagged with a unique ID.
I'm encountering a hang when the program tries to access the fruit database. I've already enabled network access MSDTC on both my development computer and the SQL Server server.
Code:
(pardon the code coloring...SO's misinterpreting my VB .NET)
Using ts As New TransactionScope
Dim fruit As New FruitDataContext
Dim thingies As New ThingiesDataContext
If (From f In fruit.tblApples Where f.Rotten = "Yes" AndAlso f.batch = 1).Count >= 1 Then
'Record today's date as the day that the rotten apples were dumped.
End If
'Other complicated code that uses ThingiesDataContext and FruitDataContext
du.SubmitChanges()
ts.Complete()
End Using
Edit:
I've dug around a bit more and it turns out that the problem lies in the line of LINQ. When I tried to view it with the LINQ to SQL Visualizer, I get the following error:
System.InvalidCastException: Specified cast is not valid.
at LinqToSqlQueryVisualizer.SqlQueryInfo.deserialize(Stream stream)
at LinqToSqlQueryVisualizer.Visualizer.Display(IDialogVisualizerService windowService, Stream rawStream)
at LinqToSqlQueryVisualizer.DialogChooser.Show(IDialogVisualizerService windowService, IVisualizerObjectProvider objectProvider)
at Microsoft.VisualStudio.DebuggerVisualizers.DebugViewerShim.ManagedShim.DelegatedHost.CreateViewer(IntPtr hwnd, HostServicesHelper hsh, SafeProxyWrapper proxy)
I've also edited the LINQ statement to be closer to my real code.
Final edit:
I tried using a normal SqlConnection instead of a "thingies as New ThingiesDataContext" and the problem still occurs.
It appears that TransactionScope cannot handle multiple SQL connections inside the same transaction.
Official Microsoft Note
parallel transactions are not supported by SQL Server.
From MSDN: http://msdn.microsoft.com/en-us/library/bb896149.aspx
This is not an MSDTC issue. If it were, you would get an error saying DTC is not enabled and needs to be. It's also not a deadlock issue, because you would get a specific error about that as well.
If I had to guess, I would say that the 'Other complicated code...' is attempting to perform a database operation and is being blocked by one or the other database context objects.
One way you can determine this is to run SQL Profiler to see what SQL statements are actually being executed on the server, and check for blocks.
I want to save to a log file some SQL query rails performs, (namely the CREATE, UPDATE and DELETE ones)
therefore I need to intercept all queries and then filter them maybe with some regexp and log them as needed.
Where would I put such a thing in the rails code?
Here a simplified version of what c0r0ner linked to, to better show it:
connection = ActiveRecord::Base.connection
class << connection
alias :original_exec :execute
def execute(sql, *name)
# try to log sql command but ignore any errors that occur in this block
# we log before executing, in case the execution raises an error
begin
File.open(Rails.root.join("/log/sql.txt"),'a'){|f| f.puts Time.now.to_s+": "+sql}
rescue Exception => e
;
end
# execute original statement
original_exec(sql, *name)
end
end
SQL logging in rails -
In brief - you need to override ActiveRecord execute method. There you can add any logic for logging.
As a note for followers, you can "log all queries" like Rails - See generated SQL queries in Log files and then grep the files for the ones you want, if desired.
If you are using mysql I would look into mysqlbinlog . It is going to track everything that potentially updates data. you can grep out whatever you need from that log easily.
http://dev.mysql.com/doc/refman/5.0/en/mysqlbinlog.html
http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
SQL Server? If so...
Actually, I'd do this at the SQL end. You could set up a trace, and collect every query that comes through a connection with a particular Application Name. If you save it to a table, you can easily query that table later.
Slightly updated version of #luca's answer for at least Rails 4 (and probably Rails 5)
Place this in config/initializers/sql_logger.rb:
connection = ActiveRecord::Base.connection
class << connection
alias :original_exec :execute
def execute(sql, *name)
# try to log sql command but ignore any errors that occur in this block
# we log before executing, in case the execution raises an error
begin
File.open(Rails.root.join("log/sql.log"), 'a') do |file|
file.puts Time.now.to_s + ": " + sql
end
rescue Exception => e
"Error logging SQL: #{e}"
end
# execute original statement
original_exec(sql, *name)
end
end