Spinnaker logs an unholy amount of INFO messages. I could not find any documentation on how to raise the minimum log level of spinnaker to, let's say, WARN.
How can I set a different default log level for spinnaker, preferably using halyard?
You can amend spinnaker-local.yml:
logging:
level:
com:
netflix:
spinnaker: DEBUG
retrofit: TRACE
You can update it to log only WARN or ERROR events and higher.
Related
I'm trying to migrate data from Aurora MySQL to S3. Since Aurora MySQL does not support replicating ongoing changes from cluster reader endpoint, my source endpoint is attached to cluster writer endpoint.
When I choose full-load migration only, DMS works. However, i get error Last Error Task 'courral-membership-s3-writer' was suspended after 9 successive recovery failures Stop Reason FATAL_ERROR Error Level FATAL when i choose full-load + ongoing replication or ongoing replication.
Thanks in advance.
This could be an error caused by - Replication instance class, you may need to upgrade it.
I am running a workflow on a n1-ultramem-40 instance that will run for several days. If an error occurs, I would like to catch and log the error, be notified, and automatically terminate the Virtual Machine. Could I use StackDriver and gcloud logging to achieve this? How could I automatically terminate the VM using these tools? Thanks!
Let's break the puzzle into two parts. The first is logging an error to Stackdriver and the second is performing an external action automatically when such an error is detected.
Stackdriver provides a wide variety of language bindings and package integrations that result in log messages being written. You could include such API calls in your application which detects the error. If you don't have access to the source code of your application but it instead logs to an external file, you could use the Stackdriver agents to monitor log files and relay the log messages to Stackdriver.
Once you have the error messages being sent to Stackdriver, the next task would be defining a Stackdriver log export definition. This is the act of defining a "filter" that looks for the specific log entry message(s) that you are interested in acting upon. Associated with this export definition and filter would be a PubSub topic. A pubsub message would then be written to this topic when an Stackdriver log entry is made.
Finally, we now have our trigger to perform your action. We could use a Cloud Function triggered from a PubSub message to execute arbitrary API logic. This could be code that performs an API request to GCP to terminate the VM.
I have deployed my ASP.NET Core application to GKE and I am now seeing output logged in Stackdriver. However for some reason all of the log entries have severity of INFO. It doesn't matter if it's exception log (with severity ERROR) or something else... everything is logged as INFO.
How can I instruct Stackdriver Logger to tag log entries from .NET Core application with appropriate severity types?
Well, first things first, that configuration should be done inside the log generator of your .NET application, like this:
Sometimes the application logs have some (some times none) string, such as stderr or stdout, that Stackdriver reads as the severity, so you could add a "Severity" field to your logs with the proper value and, this way, Stackdriver would read it as you specify, check the values here.
or in the GKE cluster with Fluentd, you can refer to this documentation for that. Also this could work.
In general your logs are missing the severity field or have other strings.
I'm not able to find the logs of application I have submitted on the the yarn on ambari cluster via log search service.
You can access logs from CLI using yarn logs -applicationId <application ID>
Try to find the log file associated with the log message that you are searching for. Then see if this particular component is configured in log search. Maybe it's not configured. That is one reason why log search is not picking up the log message
If it is configured there could be additional reasons why log search is not picking the log message
1) Check the filter set for the component. Maybe the filter is set to pick up only ERROR and WARN and the log message you are trying to see could be INFO
2)While configuring log search components you can set a filter. Make sure the log message matches the filter
I'm running an NServiceBus endpoint on an Azure Workerrole. I send all diagnostics to table storage at the moment. I was getting messages in my DLQ, and I couldn't figure out why I wasn't getting any exceptions logged in my table storage.
It turns out that NSB logs the exceptions as INFO, which is why I couldn't easily spot them in between all the actual verbose logging.
In my case, a command handler's dependencies couldn't be resolved so Autofac throws an exception. I totally get why the exception is thrown, I just don't understand why they're logged as INFO. The message ends up in my DLQ, and I only have a INFO-trace to understand why.
Is there a reason why exceptions are handled this way in NSB?
NServiceBus is not logging container issue as an error because it's happening during an attempt to process a message. First Level Retry and Second Level Retry will be attempted. When SLR is executed, it will log a WARN about the retry. Ultimately, a message will fail processing and an ERROR message will be logged. NSB and Autofac sample can be used to reproduce this.
When endpoint is running with a scaled out role and MadDeliveryCount is not big enough to accommodate all the role instances and retry count that each instance would hold, this will result in DeliveryCount reaching it's max while NServiceBus endpoint instance still thinks it has attempts before sending message to an error queue and logging an error. Similar to the question here I'd recommend to increase MaxDeliveryCount.
There is an open NServiceBus issue to have a native support for SLR counter. You can add your voice to the issue. The next version of NServiceBus (V6) will be logging message id along with the exception so that you at least could correlate between message in DLQ and log file.