I am running a workflow on a n1-ultramem-40 instance that will run for several days. If an error occurs, I would like to catch and log the error, be notified, and automatically terminate the Virtual Machine. Could I use StackDriver and gcloud logging to achieve this? How could I automatically terminate the VM using these tools? Thanks!
Let's break the puzzle into two parts. The first is logging an error to Stackdriver and the second is performing an external action automatically when such an error is detected.
Stackdriver provides a wide variety of language bindings and package integrations that result in log messages being written. You could include such API calls in your application which detects the error. If you don't have access to the source code of your application but it instead logs to an external file, you could use the Stackdriver agents to monitor log files and relay the log messages to Stackdriver.
Once you have the error messages being sent to Stackdriver, the next task would be defining a Stackdriver log export definition. This is the act of defining a "filter" that looks for the specific log entry message(s) that you are interested in acting upon. Associated with this export definition and filter would be a PubSub topic. A pubsub message would then be written to this topic when an Stackdriver log entry is made.
Finally, we now have our trigger to perform your action. We could use a Cloud Function triggered from a PubSub message to execute arbitrary API logic. This could be code that performs an API request to GCP to terminate the VM.
I am trying to enable mqtt in rabbitmq. Plugin has been enabled successfully but when I make the changes in the config for rabbitmq_mqtt, it fails to start the service. Even after googling a lot, I am not able to see the same issue being raised.
RabbitMQ_MQTT is failing to load even when the port is available.
Starting broker...
BOOT FAILED
===========
Error description:
{could_not_start,rabbitmq_mqtt,
{{function_clause,
[{rabbit_networking,tcp_listener_addresses,
[{1993}],
[{file,"src/rabbit_networking.erl"},{line,176}]},
{rabbit_mqtt_sup,'-listener_specs/3-lc$^0/1-0-',3,
[{file,"src/rabbit_mqtt_sup.erl"},{line,55}]},
{rabbit_mqtt_sup,init,1,
[{file,"src/rabbit_mqtt_sup.erl"},{line,47}]},
{supervisor2,init,1,[{file,"src/supervisor2.erl"},{line,305}]},
{gen_server,init_it,2,[{file,"gen_server.erl"},{line,365}]},
{gen_server,init_it,6,[{file,"gen_server.erl"},{line,333}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]},
{rabbit_mqtt,start,[normal,[]]}}}
Log files (may contain more information):
/var/log/rabbitmq/rabbit.log
/var/log/rabbitmq/rabbit-sasl.log
{"init terminating in do_boot",{could_not_start,rabbitmq_mqtt,{{function_clause,[{rabbit_networking,tcp_listener_addresses,[{1993}],[{file,"src/rabbit_networking.erl"},{line,176}]},{rabbit_mqtt_sup,'-listener_specs/3-lc$^0/1-0-',3,[{file,"src/rabbit_mqtt_sup.erl"},{line,55}]},{rabbit_mqtt_sup,init,1,[{file,"src/rabbit_mqtt_sup.erl"},{line,47}]},{supervisor2,init,1,[{file,"src/supervisor2.erl"},{line,305}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,365}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,333}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]},{rabbit_mqtt,start,[normal,[]]}}}}
You need to check the log in /var/log/rabbitmq/startup_log or /var/log/rabbitmq/startup_err. It is very possible that your changes for the config file is causing the problem. Usually, it's the syntax of the config file causing the problem. If you are using the classic format, it's array like syntax, having extra comma or missing comma could also prevent you from starting the service.
I'm wondering if there is some way in ModSecurity Apache2 module (version 2.9.1) how to log error messages into log file specified by the SecDebugLog option but don't duplicate them into the standard Apache error log file?
According to ModSecurity documentation the error messages are always doubled in both log files: Messages with levels 1–3 are designed to be meaningful, and are copied to the Apache’s error log. But I'd like to keep the ModSecurity stuff separate and don't mess the standard error log.
You can remove log from any of the Rules and just leave auditlog.
If using the OWASP CRS then change the default action from this:
SecDefaultAction "phase:1,deny,log"
SecDefaultAction "phase:2,deny,log"
to this:
SecDefaultAction "phase:1,deny,nolog,auditlog"
SecDefaultAction "phase:2,deny,nolog,auditlog"
Which will turn off all logging, but then turn on auditlogging again.
You may also want to add similar for phase 3 and 4 depending on whether you are also checkout outbound traffic.
However I would really, really, really caution against this for a number of reasons:
You will eventually block something with a ModSecurity rule and wonder why it's happening and skip over the Audit log and blame Apache. Trust me. "Why is this request returning 403 when I can see the page exists?!?!" At least if in the error log then you've another chance to see why this is so.
The entry in the error log is in one line. This makes it much easier to collect, parse and deal with errors in tools like Splunk. The audit log is spread over several lines so is less machine readable. And you should be reviewing your WAF logs regularly and not just assuming it's working correctly and only look at logs when something goes wrong. Maybe not in detail at each log level but in summary. Ivan Ristic, the original creator of ModSecurity, recently tweeted:
"If you’re not using your WAF as an IDS, you’re doing it wrong."
These are errors. And the error log is therefore the right place for them. The audit log is then a useful place to get extra detail if you cannot explain the errors.
I set up a zookeeper (version 3.4.6) for my solr5.4. my zoo.conf looks like this.I am not able to find error logs in /var/log or var/log/zookeeper. am I missing anything?
Thanks in advance
tickTime=2000
dataDir=/var/zookeeper/
dataLogDir=/var/log/
clientPort=2181
initLimit=5
syncLimit=2
server.1=0.0.0.0:2888:3888
server.2=h1:2888:3888
server.3=h2:2888:3888
ZooKeeper uses log4j for actual logging, so you have to see your log4j.properties. The way ZK is set up by default, you can also override some stuff via JVMs system properties, so you might need to check your startup script as well.
The properties in zoo.cfg (dataDir and dataLogDir) are used for ZK application data, like the transaction log, so don't confuse that with standard log4j logging.
We are using MobileFirst 6.3 + Liberty 8.5.5 and we need remove this entry for the message.log file:
[11/12/15 12:00:21:808 ART] 00000108 SystemOut O com.worklight.analytics.api.AnalyticsHttpService.sendDataToURL target server response code: 201
It is increasing our log files a lot. I was reviewing other posts here and the Liberty logging options but these settings applied only for the trace.log file (if it is enabled).
Here is my current bootstrap.properties file:
websphere.log.provide=binaryLogging-1.0
com.ibm.ws.logging.max.file.size=1
com.ibm.ws.logging.max.files=10
com.ibm.ws.logging.console.log.level=WARNING
com.ibm.ws.logging.message.file.name=messages.log
Thanks
If you want to block anything written to System.out from being written to your Liberty messages.log and console.log, set your traceSpecification in your server.xml's logging element as follows:
<logging traceSpecification="SystemOut=off"/>
Since you are using binary logging, you can also control what loggers are filtered out when running the binaryLog command as follows:
binarylog view daytrader3sample --excludeLogger="SystemOut"
By changing the traceSpecification you would avoid having the System.out content being added to your logs. By using the filter on the binaryLog command you would just omit SystemOut from the binaryLog output without affecting what actually is stored in the logs.
Login to WAS as the image below and set it to "No Logging":