I want to send syslog messages to console and file. For sending it to file I am using following configuration.
destination d_mycode { file("/var/log/app.log"); };
filter f_mycode { program(mycode); };
log { source(s_src); filter(f_mycode); destination(d_mycode); };
How can I send same message to console also? What changes do I need to make in configuration file?
just create a console destination and include it in your log path:
destination d_usertty { usertty("root"); };
log { source(s_src); filter(f_mycode); destination(d_mycode); destination (d_usertty); };
Related
With syslog-ng (v3.3) configuration how do I modify the message format with templates for non file destination. I want to be able to do something like this:
destination d_syslog_to_agent {
syslog( host.name.company.com transport("udp") port(12345) template("${ISODATE} ${TAGS} ${HOST} ${MESSAGE}\n"));
};
The reason I want to enable templating here is because I want to tag my messages with the help of the ${TAGS} macro. If there are other ways to tag then please do let me know.
The config you wrote should do the job.
If you would like to add tags to the log message, you can use the set-tag() rewrite rule.
For example:
source s_net8888
{
network(
ip("localhost")
port(8888));
};
destination d_syslog_to_agent
{
syslog(
"localhost"
transport("tcp")
port(12345)
template("${ISODATE} ${TAGS} ${HOST} ${MESSAGE}\n"));
};
log
{
source(s_net8888);
rewrite
{
set-tag(".mytags.test-TAG");
};
destination(d_syslog_to_agent);
};
Sending a log message to localhost:8888 will generate the following output on localhost:12345:
338 <38>1 2019-01-09T13:16:13+01:00 localhost prg00000 1234 - - 2019-01-09T13:16:13+01:00 .mytags.test-TAG,.source.s_net8888 localhost seq: 0000000000, thread: 0000, runid: 1547036173, stamp: 2019-01-09T13:16:13 PADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADD
This Meteor server code tries to attach a pdf file but giving errors on server startup. I want avoid saving the file locally first.
it is using pascoual:pdfkit.
The error I am getting is:
Error: Message failed: 421 Timeout waiting for data from client.
Meteor doc point to mailcomposer documentation, but the issue is how to integrate the pdf doc in the attachments. Any ideas? Thanks
Meteor.startup(() => {
Invoice.email();
};
// invoice.js
use strict";
let PDFDocument = require ('pdfkit');
let metaData = {
'Title': 'Invoice',
'Author': 'myName',
};
export const Invoice = {
'make': function () {
let doc = new PDFDocument(metaData);
doc.text('my company').text('company number');
return doc;
},
'email': function () {
let inv = Invoice.make();
Email.send({
to: 'myemail#comp.com',
from: 'personal#company.com',
subject: 'Invoice',
text: ' Please see the attached invoice',
attachments: {
filename: 'invoice.pdf',
content: inv // <=== this is the issue
});
}
};
As per Documentation, the content definition is
content - String, Buffer or a Stream contents for the attachment
Link to mailcomposer documentation.
If I am not wrong the Invoice.make() is not returning expecting format of blob. Kindly check the Stream and type so that you know you get what you are creating.
Link to more Supporting details
I installed syslog-ng by using "yum install syslog-ng" in both local machine and server end.
I am using an open source version of syslog-ng.
My need is to pass the log file name from client to server end . I explicitly set the .SDATA.file at 18372.4.name field on my
client side, as the name of the file is available in the $FILE_NAME macro. But ".SDATA.file at 18372.4.name" is empty
in server side. When I am using some static file name the log beings to work.
Below is my code I dont know where i am going wrong If you need more information I can provide you can anyone help me.
MY CLIENT END SYSLOG-NG CODE:
source s_application_logs {
file(
"/var/log/test.log"
flags(no-parse)
);
};
destination d_access_system {
syslog(
"52.38.34.160"
transport("tcp")
port(6514)
);
};
rewrite r_set_filename {
set(
"$FILE_NAME",
value(".SDATA.file at 18372.4.name")
);
};
rewrite r_rename_filename {
subst(
"/var/log/",
"",
value(".SDATA.file at 18372.4.name")
type("string")
flags("prefix")
);
};
log {
source(s_application_logs);
rewrite(r_set_filename);
rewrite(r_rename_filename);
destination(d_access_system);
};
MY SERVER END SYSLOG-NG CODE:
source s_server_end {
syslog(
port(6514)
max_connections(1000)
keep_hostname(yes)
);
};
destination d_log_files {
file(
"/var/log/test/${.SDATA.file at 18372.4.name}"
create_dirs(yes)
);
};
log {source(s_server_end);destination(d_log_files);};
The problem is that the $FILE_NAME macro is currently available only in the commercial version of syslog-ng. For a possible workaround, see this blogpost: Forwarding filenames with syslog-ng
I am trying to get syslog-ng to support postfix's multiple lines.
As I understand it, syslog-ng can use the " flags(no-multi-line)" config option, but it is either not working, or I am applying it incorrectly.
My configurations:
Client:
source s_src {
system();
internal();
};
filter f_mail { facility(mail) and not filter(f_debug); };
log { source(s_src); filter(f_mail); destination(d_mail); };
destination d_tls {
tcp("logs.myserver.com" port(999)
tls( ca_dir("/etc/syslog-ng/ssl/")) );
};
log {
source(s_src);
destination(d_tls);
};
Server:
source s_tls {
tcp(port(999)
tls( key_file("/etc/syslog-ng/ssl/logs.key")
cert_file("/etc/syslog-ng/ssl/logs.crt")
peer_verify(optional-untrusted))
flags(no-multi-line) # no worky
);
};
From syslog documentations:
no-multi-line: The no-multi-line flag disables line-breaking in the messages; the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the syslog, udp, unix-dgram drivers support multi-line messages; other drivers, for example, the tcp driver does not.
Thus it is not clear how multi-line messages get to your server in the first place. As a workaround, you might put the flag into file parameter:
destination d_farm_cmtslog_debug {
file("/var/log/cm/cmts.debug" flags(no-multi-line));
};
We're receiving logs using Logstash with the following configuration:
input {
udp {
type => "logs"
port => 12203
}
}
filter {
grok {
type => "tracker"
pattern => '%{GREEDYDATA:message}'
}
date {
type => "tracker"
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
}
}
output{
tcp{
type => "logs"
host => "host"
port => 12203
}
}
We're then picking the logs up on the machine "host" with the following settings:
input {
tcp {
type => "logs"
port => 12203
}
}
output {
pipe {
command => "python /usr/lib/piperedis.py"
}
}
From here, we're doing parsing of the lines and putting them into a Redis database. However, we've discovered an interesting problem.
Logstash 'wraps' the log message in a JSON style package i.e.:
{\"#source\":\"source/\",\"#tags\":[],\"#fields\":{\"timestamp\":[\"2013-09-16 15:50:47,440\"],\"thread\":[\"ajp-8009-7\"],\"level\":[\"INFO\"],\"classname\":[\"classname\"],\"message\":[\"message"\]}}
We then, on receiving it and passing it on on the next machine, take that as the message and put it in another wrapper! We're only interested in the actual log message and none of the other stuff (source path, source, tags, fields, timestamp e.t.c.)
Is there a way we can use filters or something to do this? We've looked through the documentation but can't find any way to just pass the raw log lines between instances of Logstash.
Thanks,
Matt
The logstash documentation is wrong - it indicates that the default "codec" is plain but in fact it doesn't use a codec - it uses an output format.
To get a simpler output, change your output to something like
output {
pipe {
command => "python /usr/lib/piperedis.py"
message_format => "%{message}"
}
}
Why not just extract those messages from stdout?
line = sys.stdin.readline()
line_json = json.loads(line)
line_json['message'] # will be your #message