Collectd Theshold plugin not warnings to rsyslog - collectd

I am attempting to properly configure the collectd (5.4.2 ) plugins installed in FreeBSD 10.1. I expect based on my configuration included below to
see warning events in my /var/log/messages file which is managed by rsyslog
which writes this file for any facility reporting at warning level and above.
I do not receive any threshold warnings. I have used tools like "stress -c "
to force down the idle "jiffies".
I appear to be successfully collecting information based on using rrdtool lastupdate to display
/var/db/collectd/rrd/localhost/cpu-average/cpu-idle.rrd
/var/db/collectd/rrd/localhost/tail-messages/counter-os.rrd
I suspect this is something I am not quite getting right in the configuration declarations of tail, aggregate, Chain or threshold with respect to plugin,type and instance keywords.
In tail where I am looking for problems from various facilities I
think GaugeInc would be the more appropriate DSType but that is not
supported with my current collectd revision.
Appreciate any insights on what is likely a setup matter.
cat /usr/local/etc/collectd.conf
Hostname "localhost"
FQDNLookup true
BaseDir "/var/db/collectd"
PIDFile "/var/run/collectd.pid"
TypesDB "/usr/local/share/collectd/types.db"
#ReadThreads 5
#WriteThreads 5
#https://collectd.org/wiki/index.php/Main_Page
#https://collectd.org/wiki/index.php/Naming_schema
#A value is identified by a unique name, which we usually call The "identifier" consists of five parts, two of which are optional:
#host
#plugin
#plugin instance (optional)
#type
#type instance (optional)
# e.g. host "/" plugin ["-" plugin instance] "/" type ["-" type instance]
# localhost/cpu-0/cpu-idle
LoadPlugin syslog
<plugin syslog>
LogLevel warning
NotifyLevel "OKAY"
</plugin>
LoadPlugin cpu
LoadPlugin aggregation
<LoadPlugin df >
Interval 300
</LoadPlugin>
LoadPlugin interface
LoadPlugin load
LoadPlugin memory
LoadPlugin match_regex
LoadPlugin rrdtool
LoadPlugin threshold
<LoadPlugin tail >
Interval 60
</LoadPlugin>
<plugin "df">
FSType zfs
MountPoint "/"
#ReportInodes false
ValuesPercentage true
</plugin>
<plugin rrdtool>
DataDir "/var/db/collectd/rrd"
CacheTimeout 120
CacheFlush 900
</plugin>
<plugin aggregation>
<Aggregation>
Plugin "cpu"
Type "cpu"
SetPlugin "cpu"
SetPluginInstance "%{aggregation}"
GroupBy "Host"
GroupBy "TypeInstance"
CalculateAverage true
</Aggregation>
</plugin>
<Chain "PostCache">
<Rule> # send cpu values for aggregation
<Match regex>
Plugin "^cpu$"
PluginInstance "[0-9]+$"
</Match>
<Target write>
Plugin "aggregation"
</Target>
Target stop
</Rule>
<Target write> # Write everything else via rrdtool.
Plugin "rrdtool"
</Target>
</Chain>
<plugin "tail">
<File "/var/log/messages">
Instance "messages"
<Match>
# localhost/tail-messages/counter-ace
Regex "local1.(err|warn|alert|crit)"
DSType "CounterInc"
Type "counter"
Instance "ace"
</Match>
<Match>
Regex "local0.(err|warn|alert|crit)"
ExcludeRegex "smdr:"
DSType "CounterInc"
Type "counter"
Instance "postgres"
</Match>
<Match>
Regex "local4.(err|warn|alert|crit)"
DSType "CounterInc"
Type "counter"
Instance "mec"
</Match>
<Match>
Regex "local5.(err|warn|alert|crit)"
DSType "CounterInc"
Type "counter"
Instance "web"
</Match>
<Match>
Regex "(local6|local7).(err|warn|alert|crit)"
DSType "CounterInc"
Type "counter"
Instance "apache"
</Match>
<Match>
Regex "^.*$"
ExcludeRegex " local[0-7] "
DSType "CounterInc"
Type "counter"
Instance "os"
</Match>
</File>
</plugin>
#https://collectd.org/documentation/manpages/collectd-threshold.5.shtml
<Plugin "threshold">
<Plugin "interface">
Instance "eth0"
<Type "if_octets">
FailureMax 10000000
DataSource "rx"
</Type>
</Plugin>
<plugin "df">
<type "df">
Instance "/zroot/ROOT/default"
WarningMax 75
</type>
</plugin>
<Host "Hostname">
<plugin "aggregation">
<type "cpu-average">
Instance "idle"
WarningMin 17000
FailureMin 15000
Hits 1
</type>
</plugin>
<Plugin "memory">
<Type "memory">
Instance "free"
WarningMin 10000000
</Type>
</Plugin>
<plugin "load">
<type "load">
DataSource "midterm"
FailureMax 4
Hits 3
Hysteresis 3
</type>
</plugin>
<Plugin "tail">
Instance "messages"
<type "counter">
Instance "os"
WarningMax .001
</type>
<type"counter">
Instance "ace"
WarningMax .001
</type>
</Plugin>
</Host>
</Plugin>

The most common error with thresholds is to have a too restrictive filter. Try removing sections like Host, Instance etc. until you can see the notifications. Also you can use the unixsock plugin to PUTVAL fake values instead of trying to burn your system.

Related

How to make Log4j2 configurable by environment using Spring Boot 1.3.6.RELEASE

I would like to change some properties from the log4j2.xml file depending on the my application.properties, for example define some properties and then substitute in the log4j2 those properties that are parameters.
I ran different approaches but I still did not get the right thing. I would like to have different configs depending on the environment (DEV, QA or PROD). How to accomplish this?
I'm trying to have this in my properties
#Place holders for log4j2.xml file
log.file.path=/opt/tomcat/logs
log.file.name=dummydummy
log.file.size=100 MB
log.level=DEBUG
My log4j2 below.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30">
<Properties>
<Property name="PID">????</Property>
<property name="name">my-log</property>
</Properties>
<Appenders>
<RollingFile name="file" fileName="${log.file.path}${log.file}.log"
filePattern="${log.file.path}${log.file}-%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout
pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %5p ${sys:PID} --- [%t] %c{1}(%M:%L) : %m%n%wEx" />
<Policies>
<TimeBasedTriggeringPolicy /><!-- Rotated everyday -->
<SizeBasedTriggeringPolicy size="${log.file.size}" /> <!-- Or every 100 MB -->
</Policies>
</RollingFile>
<Console name="Console" target="SYSTEM_OUT" follow="true">
<PatternLayout
pattern="%clr{%d{yyyy-MM-dd HH:mm:ss.SSS}}{faint} %clr{%5p} %clr{${sys:PID}}{magenta} %clr{---}{faint} %clr{[%t]}{faint} %clr{%c{1}(%M:%L)}{cyan} %clr{:}{faint} %m%n%wEx" />
</Console>
</Appenders>
<Loggers>
<Logger name="org.hibernate.validator.internal.util.Version"
level="warn" />
<Logger name="org.apache.coyote.http11.Http11NioProtocol" level="warn" />
<Logger name="org.apache.tomcat.util.net.NioSelectorPool" level="warn" />
<Logger name="org.apache.catalina.startup.DigesterFactory" level="error" />
<Logger name="org.springframework.web" level="error" />
<Root level="${log.level}">
<AppenderRef ref="Console" />
<AppenderRef ref="file" />
</Root>
</Loggers>
</Configuration>
The properties lookup element allows to refer properties from an external properties file in the log4j configuration.
For your example it should be something like this:
A file env.properties contains the following properties:
log.file.path=/opt/tomcat/logs
log.file.name=dummydummy
log.file.size=100 MB
log.level=DEBUG
The properties lookup should be defined as properties of the log4j2.xml:
<Configuration>
<Properties>
<property name="log.file.path">${bundle:env:log.file.path}</property>
<property name="log.file.name">${bundle:env:log.file.name}</property>
<property name="log.file.size">${bundle:env:log.file.size}</property>
<property name="log.level">${bundle:env:log.level}</property>
</Properties>
Now the properties may be referred in appenders with ${property_name} notation. Each property reference will be interpolated with the real value from the env.properties.
You can find another example of properties lookup here.
As of Log4j 2.13.0 Log4j 2 now provides a Spring Lookup as part of its Spring Cloud Config support. It will allow you to reference properties defined in your application.properties or application.yml file of your Spring Boot application in the log4j2.xml.
Since log4j 2.14.0, you can now use Spring Boot environment variables without Spring Cloud and without doing direct reference to the properties file. You'll need at least Spring Boot 2.0.3
<property name="applicationName">${spring:spring.application.name}</property>
Documentation: https://logging.apache.org/log4j/2.x/log4j-spring-boot/index.html
Maven repository: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-spring-boot
Ensure that log4j2 starter is added in the classpath and then remove logging related properties in application.properties then spring will load your log4j2.xml from resources folder.
This way you can have full control over logging. If you want to substitute values then refer this link
Note:: If you have actuator in your project then remove spring boot logger starter
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
Workaround in case you need just different property values in log4j2.xml config for different spring profiles (dev, test, prod, etc.).
You could access spring profile files like: ${bundle:application-${sys:spring.profiles.active}:log.file.name}
application-dev.properties file:
log.file.name=dev_log
application-prod.properties file:
log.file.name=app_name
log4j2.xml config:
<Configuration>
<Properties>
<Property name="file_name">${bundle:application-${sys:spring.profiles.active}:log.file.name}</Property>
</Properties>
<Appenders>
<RollingFile name="RollingFile"
fileName="/logs/${file_name}.log"
...
NOTE: You have to add log.file.name property to each spring profile, log4j2 sees it just as separate text files and will not resolve default values from application.properties file and etc.

How i can add some jvm options to arquillian test

Its possible to add some jvm options to embedded glassfish using arquillian ?
I need to add that jvm options:
-Djavax.net.ssl.keyStorePassword=changeit
-Djavax.net.ssl.trustStorePassword=changeit
Java properties on Glassfish are configured in domain.xml. Since you are running an embedded Glassfish, you don't really have a domain.xml file you could modify. You can try to do this in arquillian.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<arquillian xmlns="http://www.jboss.org/arquillian-1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.jboss.org/arquillian-1.0 http://jboss.org/schema/arquillian/arquillian-1.0.xsd">
<engine>
<property name="deploymentExportPath">target/</property>
</engine>
<container qualifier="glassfish" default="true">
<configuration>
<property name="configurationXml">file:src/test/resources/domain.xml</property>
...
</configuration>
</container>
</arquillian>
The configurationXml property is used to pass the configuration file to use for the embedded insance. See also https://docs.jboss.org/author/display/ARQ/GlassFish+3.1+-+Embedded. domain.xml itself has a section for JVM arguments.

IntelliJ Enterprise Plugin Repository - Plugins' Metadata won't display

I have several proprietary IntelliJ plugins that I am hosting on an Enterprise Repository. When I view them in IntelliJ's Repository Browser, the metadata does not display, besides the version.
(Note: None of the other plugins have metadata either)
I am hosting these plugins at my.enterprise.repository.com/updatePlugins.xml, with the file
<plugins>
<plugin
id="Plugin 1"
url="http://my.enterprise.repository.com/Plugin1.jar"
version="12.001" />
<plugin
id="Plugin 2"
url="http://my.enterprise.repository.com/Plugin2.jar"
version="12.001"/>
<plugin
id="Plugin 3"
url="http://my.enterprise.repository.com/Plugin3.jar"
version="12.001"/>
<plugin
id="Plugin 4"
url="http://my.enterprise.repository.com/Plugin4.jar"
version="12.001"/>
<plugin
id="Plugin 5"
url="http://my.enterprise.repository.com/Plugin5.jar"
version="12.001"/>
</plugins>
And the plugin.xml for Plugin 5 is
<idea-plugin version="2">
<name>Plugin 5</name>
<description>Is a cool plugin.</description>
<version>12.001</version>
<vendor>My Company</vendor>
<idea-version since-build="8000"/>
<application-components>
<!-- Add your application components here -->
</application-components>
<project-components>
<!-- Add your project components here -->
</project-components>
<extensions defaultExtensionNs="com.intellij">
...
</extensions>
</idea-plugin>
Is the metadata specified in a different file? Or is my plugin.xml file wrong?
Note: These plugins successfully download, install, and function.
It's a known issue, the work is in progress. You can track the state here:
IDEA-88075 Plugins in enterprise / custom plugin repository do not show plugin details

Maven - POM: How to make the jetty port changeable so that it can be retrieved later?

I'm working on an integration test suite and I've got a question for you.
My parent pom defines the use of the jetty plugin with the goal: run-war. I need to make the port that jetty listens on changeable via the command-line. This can be achieved by passing -Djetty.port=8099 for example.
In the child project, I need to use this port number to configure the endpoint for some SOAP tests that I'll need to run on the service hosted by jetty.
If I use ${jetty.port} in my child pom in the end-point configuration this works fine IF and only IF I explicitly pass -Djetty.port when invoking maven.
In my child pom:
<endpoint>http://127.0.0.1:${jetty.port}/{artifactId}<endpoint>
I need jetty.port to be filled in with 8080 which is what jetty defaults to if -Djetty.port is not explicitly passed, and still catch any other port values if the command line argument is specified.
Use the properties section, and add a jetty.port property with a default value:
<properties>
<jetty.port>8080</jetty.port>
</properties>
config maven jetty plugin:
<plugins>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1H.14.1</version>
<configuration>
<connectors>
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<port>8085</port>
<maxIdleTime>60000</maxIdleTime>
</connector>
</connectors>
</configuration>
</plugin>
</plugins>
If you want to use a newer version of jetty plugin, use the following configuration:
From http://www.eclipse.org/jetty/documentation/current/jetty-maven-plugin.html:
You could instead configure the connectors in a standard jetty xml config file and put its location into the jettyXml parameter. Note that since jetty-9.0 it is no longer possible to configure a https connector directly in the pom.xml: you need to use jetty xml config files to do it.
Something like:
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.0.5.v20130815</version>
<configuration>
<jettyXml>src/main/resources/jetty.xml</jettyXml>
<webApp>
<contextPath>/yourCtxPath</contextPath>
</webApp>
</configuration>
</plugin>
would do the trick, with
jetty.xml file content:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<Call id="httpsConnector" name="addConnector">
<Arg>
<New class="org.eclipse.jetty.server.ServerConnector">
<Arg name="server"><Ref refid="Server" /></Arg>
<Set name="host"><Property name="jetty.host" /></Set>
<Set name="port"><Property name="jetty.port" default="8085" /></Set>
<Set name="idleTimeout">30000</Set>
</New>
</Arg>
</Call>
</Configure>
See the log after 'mvn jetty:run', at the end should show something like:
2013-09-05 09:49:05.047:INFO:oejs.ServerConnector:main: Started ServerConnector#a6e9cb4{HTTP/1.1}{0.0.0.0:8085}
You will need to use maven 3 and java 7 for this version of plugin.

How to execute tasks conditionally using the maven-antrun-plugin?

I need to execute some ant commands depending on an environment variable passed in as a parameter to the maven build command.
At the moment I have 3 tasks blocks and only the tasks block with no condition is being executed.
<tasks name="isProdCheck">
<condition property="isProd">
<equals arg1="${environment}" arg2="PROD" />
</condition>
</tasks>
<tasks if="isProd" depends="isProdCheck">
...
</tasks>
<tasks>
... I am the only block executed
</tasks>
What am I doing wrong, is there a better way to do this?
First, according to Maven 1.x website, the current stable release for maven 1.x is version 1.1, not 1.4. Second, there is no AntRun Plugin version 1.7 and, to my knowledge, this is a Maven 2 plugin. Third, the syntax you are using seems very similar to Using Attributes which, again, is about Maven 2.
So, I may be missing something but, this is very confusing and you should maybe clarify these points in your question.
Anyway, as you explicitly mentioned Maven 1, I'll try to answer. If I remember well, I would write a custom goal and use Jelly's core:if or core:when. To do so, provide something like this in maven.xml:
<project xmlns:j="jelly:core" xmlns:ant="jelly:ant">
<goal name="my-goal">
<j:if test="${environment == 'PROD'}">
<ant:xxx .../>
</j:if>
</goal>
</project>
I'm really not sure of the syntax, all this Maven 1 stuff is just too far away, and I didn't test it (I'm too lazy to install Maven 1). But I guess you will. The scripting reference may help you.
To be honest, I really hope you have a good reason to prefer Maven 1.x over Maven 2.x :)
UPDATE: It appears that the OP is actually using Maven 2 so I'll update my question accordingly. To implement the desired behavior, you could use Ant-contrib's if task as shown below:
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<taskdef resource="net/sf/antcontrib/antcontrib.properties"
classpathref="maven.plugin.classpath" />
<if>
<equals arg1="${foo}" arg2="bar" />
<then>
<echo message="The value of property foo is bar" />
</then>
<else>
<echo message="The value of property foo is not bar" />
</else>
</if>
</tasks>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>ant-contrib</groupId>
<artifactId>ant-contrib</artifactId>
<version>20020829</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
And then call mvn compile -Dfoo=bar (this is just an example).
But, all this is not the "maven way" to do things. Now that I understand a bit better what you are trying to do (but not entirely as you didn't explain your ultimate goal), I think that using build profiles would be more appropriate and, having read your own answer, I think that you are over complicating things (and that you are on the wrong path).
I understand that you are a Maven beginner but I'd suggest to try to use it though instead of falling back on Ant or you won't get the benefits of it. Also, when opening a question, instead of asking for a specific solution, you should rather explain your general problem, you'll get better answers. Here, I can't provide more guidance as I don't know what you are really trying to achieve.
You don't need to use AntContrib after maven-antrun-plugin 1.5 that uses <target> instead of <tasks> according to plugin usage. This tag works in the same way as , but in this one you can add conditions like the example below.
<properties>
<execute.my.target>true</execute.my.target>
</properties>
<build>
...
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>config</id>
<phase>package</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<!-- this target will be executed always -->
<target>
<echo message="Hello there, I'm a simple target" />
</target>
<!--
This target will be executed if and only if
the property is set to true
-->
<target name="my-target" if="execute.my.target">
<echo message="Conditional target..." />
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
...
</build>
The code above always executes the first target, but the second target depends on a property value. You can configure it for a parent and a sub-module project too, defining the plugin on <pluginsManagement> tag and calling some properties at the sub-modules, then calling the plugin.
Update:
Be sure to use the if parameter without ${}, since it can cause troubles mentioned in the comments section.
Resolved this issue by creating multiple named targets with "if" attributes and a condition property in a build.xml file in the project root as follows.
<target name="prod" if="isProd" depends="isProdCheck">
// do something
</target>
Passed properties of the command line switches I required, and called the ant targets in the build.xml file from the tasks section in the Maven POM as follows:
<tasks>
<ant antfile="${basedir}/build.xml">
<property name="environment" value="${environment}"/>
<target name="prod"/>
</ant>
</tasks>
runable example here https://www.surasint.com/run-ant-with-if-from-maven/
Another solution would be: keep the ant-contrib-1.0b3.jar to a path and then define it like this
<property name="runningLocation" location="" />
<taskdef resource="net/sf/antcontrib/antcontrib.properties">
<classpath>
<pathelement location="${runningLocation}/ant-contrib-1.0b3.jar" />
</classpath>
</taskdef>
then
<target name="doSomething">
<if>
<equals arg1="${someProp}" arg2="YES" />
<then>
<echo message="It is YES" />
</then>
<else>
<echo message="It is not YES" />
</else>
</if>
</target>
I put full code example here which you can download https://www.surasint.com/2017/04/09/run-ant-with-if-from-maven/