I have a Rails app that uses MongoDb on the back. I have these messages that say MONGODB [WARNING] Please note that logging negatively impacts client-side performance. You should set your logging level no lower than :info in production in my logs. OK, I never worried about it but decided to look it up just now.
This page on the mongo site doesn't really discuss logging levels, but it does discuss -v vs -vvvv for verbosity. Is that the same thing as log level? As in -vvvvv is the same as a debug log level and -v is the same as an error log level? The docs are very unclear on this topic.
I had problems with this in my tests, so I ended up doing the following in my spec_helper.rb:
Mongoid.logger.level = Logger::INFO
However if you are inside of rails you should probably (untested) use this to access the logger instead:
config.mongoid.logger
Logging levels refer to rails logging levels whereas the -v flag refers to verbosity.
Rails automatically sets the logging level higher in production than when in development so you shouldn't have anything to worry about.
If you're using mongoid 2.2 or higher, you can set it in mongoid.yml:
production:
hosts:
...
database: ...
logger: false
Also, this does have a performance impact. When I turned off mongo logging in production, I saw fewer garbage collections and app instance memory footprints were about 15 megabytes smaller during 30-minute load tests using apachebench.
Related
I'm wondering if there is some way in ModSecurity Apache2 module (version 2.9.1) how to log error messages into log file specified by the SecDebugLog option but don't duplicate them into the standard Apache error log file?
According to ModSecurity documentation the error messages are always doubled in both log files: Messages with levels 1–3 are designed to be meaningful, and are copied to the Apache’s error log. But I'd like to keep the ModSecurity stuff separate and don't mess the standard error log.
You can remove log from any of the Rules and just leave auditlog.
If using the OWASP CRS then change the default action from this:
SecDefaultAction "phase:1,deny,log"
SecDefaultAction "phase:2,deny,log"
to this:
SecDefaultAction "phase:1,deny,nolog,auditlog"
SecDefaultAction "phase:2,deny,nolog,auditlog"
Which will turn off all logging, but then turn on auditlogging again.
You may also want to add similar for phase 3 and 4 depending on whether you are also checkout outbound traffic.
However I would really, really, really caution against this for a number of reasons:
You will eventually block something with a ModSecurity rule and wonder why it's happening and skip over the Audit log and blame Apache. Trust me. "Why is this request returning 403 when I can see the page exists?!?!" At least if in the error log then you've another chance to see why this is so.
The entry in the error log is in one line. This makes it much easier to collect, parse and deal with errors in tools like Splunk. The audit log is spread over several lines so is less machine readable. And you should be reviewing your WAF logs regularly and not just assuming it's working correctly and only look at logs when something goes wrong. Maybe not in detail at each log level but in summary. Ivan Ristic, the original creator of ModSecurity, recently tweeted:
"If you’re not using your WAF as an IDS, you’re doing it wrong."
These are errors. And the error log is therefore the right place for them. The audit log is then a useful place to get extra detail if you cannot explain the errors.
I am curious how should I force jboss 7.1.1 to use Apaches Log4j 2 instead of org.jboss.as.logging, because I would like to do some performance comparison of log4j2 and jboss.as.logging (I have given up on log4j because it seems to have similar performance as jboss.as.logging).
Log4j2 Official website: http://logging.apache.org/log4j/2.x/
I suppose I need to create a new module for the log4j2 library in jboss modules.
Then what? Do I need any changes in standalone.xml? Any changes for jboss-deployment-structure.xml?
How can I tell jboss where to search for the log4j2 library?
Thanks for any suggestions. I am a bit stuck here.
JBoss Logging is just a logging facade similar to slf4j. JBoss AS 7 uses JBoss Log Manager for it's log manager.
Without changing some code you and removing the logging subsystem you cannot use another log manager like log4j2 for the server wide log manager. You'd have to make some changes here and remove the STDIO stuff. It's probably not worth the effort TBH.
JBoss Log Manager is fairly fast. You could try using an async-handler to see if that helps performance at all. It probably won't make a significant difference though if you're just using a standard console-handler and file-handler of some sort.
Some ruff measuring results between JBoss Default Logging and Log4J 2 (by configuring it native and therefore skipping the LogManager), by logging with 10 Threads at the same time:
JBoss Default Log Async Rolling File Appender -> 200.000 Logs/minute
Log4j 2 ASync File Appender -> 5.000.000 Logs/minute
These are really only ruff results, the second case uses a different logger and does not use the Log Manager, these things must be measured independently ... maybe I will do that too. Nevertheless, the bottom line is that default logging is damn slow.
In the configuration file example for Puma, it says the following for the on_restart function:
Code to run before doing a restart. This code should close log files,
database connections, etc.
Do I need to implement this for a Rails app, to close connections to the db and the logfile, or is that taken care of automatically? If not, how do I actually do all that?
No you don't, Rails takes care of reloading your code automatically. But this code reloading support is limited. For example changes to application.rb are not applied until you restart the app server.
But I would recommend Phusion Passenger over Puma. Phusion Passenger is a lot easier to setup, especially when you hit production. Phusion Passenger integrates into Apache and Nginx directly and provides advanced features like dynamic worker management. Phusion Passenger is very mature, stable and performant and used by the likes of New York Times, Symantec, AirBnB, etc.
I've found that using Redis as my Rails.cache provider causes an error page upon the first request every time my Rails/Puma server is restarted. The error I got was:
Redis::InheritedError (Tried to use a connection from a child process
without reconnecting. You need to reconnect to Redis after forking.)
To get around this error, I didn't add anything to on_restart, but did have to add code to on_worker_boot ( I am running Puma with workers=4 ):
puma-config.rb
on_worker_boot do
puts "Reconnecting Rails.cache"
Rails.cache.reconnect
end
I am testing the tomcat7 clickstack for our application which has some config parameters set using the built in Config features of Cloudbees. The tomcat7 clickstack does not find them, but the standard tomcat6 container does. I have double checked them and reset them through the cloudbees sdk and they are there and correct, but are coming back as null for tomcat7.
The switch to clickstacks requires us to refactor how the servlet container gets configured so that the injection points such as cloudbees-web.xml and jvm system properties behave consistently across all the servlet container clickstacks.
Some of that refactoring has been committed but some of the work is still in my backlog... Assuming none of the other bees steal that task from my backlog before I get to it ;-)
IF I recall correctly, the parameters should be available as environment variables (sub optimal I know, but all containers should be giving this as a consistent UX for all clickstacks, eg both non-java based and java-based) and may be already available as system properties (again sub optimal, but the java container refactoring should be giving this as a consistent UX for all java based clickstacks). The consistent java servlet UX has not been committed yet but should be available soon.
Rails 3 on Apache with Passenger: I need to take down Rails so that the DB connection(s) are closed -- this is necessary to perform some regular database maintenance -- but I want Apache up so that it can respond to requests with a static maintenance page.
I am using Capistrano and have seen the threads on how to invoke maintenance mode, but I need to know where to hook my DB tasks, and cannot figure out where.
Any links, or even pointers to where to look in the Capistrano code would be greatly appreciated.
TIA
You can use capistrano's deploy:web:disable task to block access to your site, allowing you to do database maintenance, etc:
cap deploy:web:disable REASON="a Database Upgrade" UNTIL="in a few minutes"
Then, once you're done:
cap deploy:web:enable