Can PatternLayout be used with HTTP appender in log4j2.xml? - mule

We have a requirement to push our application logs to ELK from the Mule applications in CloudHub. For this, I am using ELK configs in the log4j HTTP appender.
Below is the log4j Config in my mule application app1:
<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<!--These are some of the loggers you can enable.
There are several more you can find in the documentation.
Besides this log4j configuration, you can also use Java VM environment variables
to enable other logs like network (-Djavax.net.debug=ssl or all) and
Garbage Collector (-XX:+PrintGC). These will be append to the console, so you will
see them in the mule_ee.log file. -->
<Appenders>
<RollingFile name="file" fileName="/logs/mule/application/app1.log"
filePattern="/logs/mule/application/app1-%d{dd-MMM-yyyy}-%i.log">
<PatternLayout pattern="%-5p %d [%t] [event: %X{correlationId}] %c: %m%n" />
<SizeBasedTriggeringPolicy size="10 MB" />
<DefaultRolloverStrategy max="20"/>
</RollingFile>
<Http name="ELK" url="https://<url to elk>/mule-cloudhub-logs/_doc">
<JsonLayout compact="true" eventEol="true" properties="true" />
<Property name="kbn-xsrf" value="true" />
<Property name="Content-Type" value="application/json" />
<Property name="Authorization" value="ApiKey apikey" />
<PatternLayout pattern="%-5p %d [%t] [event: %X{correlationId}] %c: %m%n" />
</Http>
</Appenders>
<Loggers>
<!-- Http Logger shows wire traffic on DEBUG. -->
<!--AsyncLogger name="org.mule.service.http.impl.service.HttpMessageLogger" level="DEBUG" /-->
<AsyncLogger name="org.mule.service.http" level="WARN"/>
<AsyncLogger name="org.mule.extension.http" level="WARN"/>
<!-- Mule logger -->
<AsyncLogger name="org.mule.runtime.core.internal.processor.LoggerMessageProcessor" level="INFO"/>
<AsyncRoot level="INFO">
<AppenderRef ref="file" />
<AppenderRef ref="ELK" />
</AsyncRoot>
</Loggers>
</Configuration>
Now, the output in ELK for this, comes out to be :
[MuleRuntime].uber.08: [app1].get:\v1\dummy\uri:app1-config.CPU_INTENSIVE #232321
No matter what kind of pattern I put in the PatternLayout in the HTTP appender, the output is still the same in elk.
Additionally, when am deploying the application, the deployment logs show a one line error as below :
2022-09-15 12:54:09,270 WrapperListener_start_runner ERROR appender Http has no parameter that matches element PatternLayout
Is there is an issue in my config or some other bug?

You can define only 1 layout per Appender. Your Http appender has two, the JsonLayout and PatternLayout

Related

How do you debug the Azure Java SDK CosmosClient SSL handshake?

How do you debug the Azure Java SDK CosmosClient connection SSL handshake?
I tried with Commons-Logging and with SLF4j, and cannot get the SSL handshake to put out debug output. Does anyone know how to do this.
Here is my log4j.properties :
log4j.rootLogger=INFO, stdout
log4j.logger.io.netty=DEBUG
log4j.logger.io.projectreactor=DEBUG
log4j.logger.reactor.netty.http=DEBUG
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n
Here is my logging.properties :
handlers = java.util.logging.ConsoleHandler
.level = ALL
java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=[%1$tF %1$tT] [%4$s] %5$s %n
io.netty.level=ALL
io.projectreactor.level=ALL
reactor.netty.http=ALL
I solved it. I'm using Lombok SLF4j and I needed this log4j2.xml config:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF">
<Properties>
<Property name="LOG_LOC">logs</Property>
<Property name="MAX">5</Property>
<Property name="LOG_PATTERN">%d{yyyy.MM.dd HH:mm:ss.SSS} [%p] %c: %m%n
</Property>
</Properties>
<Appenders>
<RollingFile name="FILE" fileName="${LOG_LOC}/main.log"
filePattern="${LOG_LOC}/main.%i.log">
<PatternLayout>
<Pattern>${LOG_PATTERN}</Pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy />
</Policies>
<DefaultRolloverStrategy max="${MAX}" />
</RollingFile>
<Console name="STDOUT" target="SYSTEM_OUT" follow="true">
<PatternLayout pattern="${LOG_PATTERN}" />
</Console>
</Appenders>
<Loggers>
<Logger name="io.netty" level="debug" />
<Logger name="io.projectreactor" level="debug" />
<Logger name="reactor.netty.http" level="debug" />
<Logger name="com.azure.cosmos" level="debug" />
<Logger name="file" level="debug" additivity="false">
<appender-ref ref="FILE" />
</Logger>
<Root level="warn">
<AppenderRef ref="FILE" />
<AppenderRef ref="STDOUT" />
</Root>
</Loggers>
</Configuration>
Also works with a log4j.properties if you add these dependencies in addition to Lombok:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.36</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.36</version>
</dependency>

Is there any similar kind of configure to rolingStyle="once" of log4net is available for NLog?

I am using log4net currently and on every hour log file archive is being performed.
Now I am changing log4net to NLog
Is there similar setting available in NLog Like rolingStyle="once"? which was configure setting available in log4net.
For example earlier while using log4net the files created were used to be like:
LogFile.log
LogFile.log.1 <-last archive file
Following is configuration I used in log4net and I want to use the exact configure settings so that archived file naming should remains as it was in log4net:
<appender name="Work" type="RMM.Common.Logger.LogFileRollingAppender, Common">
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<dateTimeStrategy type="log4net.Appender.RollingFileAppender+UniversalDateTime" />
<file type="log4net.Util.PatternString" value="%property{EdgeAPILogPath}\WebAPI_Work.log" />
<param name="AppendToFile" value="true"/>
<rollingStyle value="Once" />
<rollingStyle value="Composite" />
<datePattern value=".yyyyMMdd-HH" />
<maxSizeRollBackups value="300" />
<maximumFileSize value="20MB" />
<Encoding value="UTF-8" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%utcdate{yyyy/MM/dd HH:mm:ss,fff} [%-5p] [%3t] %m%n" />
</layout>
<filter type="log4net.Filter.LevelRangeFilter">
<levelMin value="DEBUG" />
<levelMax value="FATAL" />
</filter>
</appender>
You can:
Use fileName="${basedir}/logs/${cached:${date:format=yyyy-MM-dd HH_mm_ss}}.log" as a way to ensure one log file per application instance.
Use archiveFileName="${archiveLogDirectory}/LogFile.log.{####}" to append numbers at the end (feel free to add or remove # as required, depending on your maxArchiveFiles).
Use archiveNumbering="Sequence" to achieve the order you want (higher numbers = newer logs).
Source: this piece of documentation and some personal experience.
Hopefully this basic example will help you getting closer to your final target:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
internalLogLevel="Error"
internalLogFile="./internal-nlog.txt"
throwExceptions="true">
<variable name="logDirectory" value="./logs"/>
<variable name="archiveLogDirectory" value="./logs/archive"/>
<targets>
<target name="errors"
xsi:type="File"
fileName="${logDirectory}/${cached:${date:format=yyyy-MM-dd HH_mm_ss}}.log"
archiveFileName="${archiveLogDirectory}/LogFile.log.{#}"
maxArchiveFiles="9"
archiveEvery="Hour"
archiveNumbering="Sequence"
/>
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="errors"/>
</rules>
</nlog>
Not an expert on log4net, but it sounds like that rolingStyle="once" is the same as NLog FileTarget ArchiveOldFileOnStartup. So maybe something like this:
<nlog>
<variable name="EdgeAPILogPath" layout="${basedir}" />
<targets>
<target xsi:type="file" name="work"
fileName="${EdgeAPILogPath}/WebAPI_Work.log"
encoding="utf-8"
archiveNumbering="DateAndSequence"
archiveFileName="${EdgeAPILogPath}/WebAPI_Work.{#}.log"
archiveDateFormat="yyyyMMdd-HH"
archiveEvery="Hour"
archiveAboveSize="20000000"
archiveOldFileOnStartup="true"
maxArchiveFiles="300" />
</targets>
<rules>
<logger name="*" minLevel="Debug" writeTo="work" />
</rules>
</nlog>
See also: https://github.com/NLog/NLog/wiki/FileTarget-Archive-Examples

How to log to a file from an Eclipse RCP application?

I can't log to a file from my eclipse rcp application.
So, I followed the guide on https://www.vogella.com/tutorials/EclipseLogging/article.html.
I installed Logback SLF4J Binding, logback classic module and logback core module from orbit repository.
I created a plugin that read the config file logback.xml. That plugin require ch.qos.logback.classic & ch.qos.logback.core. I also imported org.sl4j.
I imported org.slf4j on the other plugins.
Here is the logback.xml file :
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true">
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<property name="HOME_LOG" value="d://logFiles/app.log"/>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>${HOME_LOG}</file>
<encoder>
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} -FILE- %msg%n
</Pattern>
</encoder>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} -OUT- %msg%n
</Pattern>
</layout>
</appender>
<logger name="fr.test.sirius.logger" level="debug"
additivity="false">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE"/>
</logger>
<root level="debug">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE"/>
</root>
</configuration>
When I launch my Eclipse RCP application with "run as" everything is working as attended.
When I create an update site and install the feature on Eclipse, the log file is not created. I tried to launch eclipse with -console argument, the logs are printed correctly in the OSGi console.
edit: The logs are not printed correctly in the OSGI console, seems like an issue with the activator.
Thanks

How to route log messages based on category using log4j2.xml

I am after routing log messages based on category using log4j2.xml.
I found below link but not sure how to achieve it
https://docs.mulesoft.com/mule-user-guide/v/3.8/logger-component-reference#configuring-custom-logging-settings
Can someone please provide an example config for log4j2.xml.
Below is config from my log4j2.xml
<Appenders>
<RollingFile name="file"
fileName="${sys:mule.home}${sys:file.separator}logs
${sys:file.separator}splunktest.log"
filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}splunktest-%i.log">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n" />
<SizeBasedTriggeringPolicy size="10 MB" />
<DefaultRolloverStrategy max="10"/>
</RollingFile>
<RollingFile name="splunk"
fileName="${sys:mule.home}${sys:file.separator}logs
${sys:file.separator}ForSplunk.log"
filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}ForSplunk-%i.log">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n" />
<SizeBasedTriggeringPolicy size="10 MB" />
<DefaultRolloverStrategy max="10"/>
</RollingFile>
</Appenders>
<AsyncRoot level="INFO">
<AppenderRef ref="file" />
</AsyncRoot>
<AsyncLogger name="test" level="INFO" category="splunk">
<AppenderRef ref="splunk" />
</AsyncLogger>
Logger output for category APP and splunk. In this case i want category splunk logger message to go to file ForSplunk.log.
[[splunktest].HTTP_Listener_Configuration.worker.01] APP: Start
Logger
[[splunktest].HTTP_Listener_Configuration.worker.01] splunk:
End Logger
#user3366906 We had used logging category in our project.
You need to specify some value to category field and that same value should be specified in a cloudHub logging.

Mule log is not working if i use synch and asyn in same log4 file

I am trying to use Asyn log and Syn log in the same log4j2.xml file and want to set the root level is "INFO" and the specific mule package(org.mule) is "FATAL".but its automatically pick the info level for org.mule package and its not restricting the log level to specific package if i use sync and asynch level at same file. please help us if you have any idea. Thanks in advance.
Mule ESB - 3.6
<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<Appenders>
<RollingFile name="file"
fileName="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}eztutorial.log"
filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}eztutorial-%i.log">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n" />
<SizeBasedTriggeringPolicy size="10 MB" />
<DefaultRolloverStrategy max="10" />
</RollingFile>
</Appenders>
<Loggers>
<!-- CXF is used heavily by Mule for web services -->
<AsyncLogger name="org.apache.cxf" level="WARN" />
<!-- Apache Commons tend to make a lot of noise which can clutter the log -->
<AsyncLogger name="org.apache" level="WARN" />
<!-- Reduce startup noise -->
<AsyncLogger name="org.springframework.beans.factory"
level="WARN" />
<logger name="org.mule">
<level value="FATAL" />
</logger>
<logger name="com.mulesoft">
<level value="FATAL" />
</logger>
<category name="org.mule">
<priority value="FATAL" />
</category>
<category name="com.mulesoft">
<priority value="FATAL" />
</category>
<!-- Reduce DM verbosity -->
<AsyncLogger name="org.jetel" level="WARN" />
<AsyncLogger name="Tracking" level="WARN" />
<AsyncRoot level="INFO">
<AppenderRef ref="file" />
</AsyncRoot>
</Loggers>
</Configuration>
You can remove the <Category ...> entries from your config. (Log4j2 ignores them.)
For all named <Logger> and <AsyncLogger> entries, add additivity="false". Without this, the INFO level statements will still appear in your log because the root logger picks them up.
See http://logging.apache.org/log4j/2.x/manual/configuration.html#Additivity