I want to send syslog messages to console and file. For sending it to file I am using following configuration.
destination d_mycode { file("/var/log/app.log"); };
filter f_mycode { program(mycode); };
log { source(s_src); filter(f_mycode); destination(d_mycode); };
How can I send same message to console also? What changes do I need to make in configuration file?
just create a console destination and include it in your log path:
destination d_usertty { usertty("root"); };
log { source(s_src); filter(f_mycode); destination(d_mycode); destination (d_usertty); };
I keep any hard coding information inside models/config.js, but i'm not sure that the models/config.js file is the correct place for a port.
Keep a ./config/my_database_config.js and put all there.
Similar for ./config/main_server_config.js
usually all other config files can also go there.
You can hardcode values in this my_database_config.js file . Or this file could suppose make a request to the server for the config file that returns a following json.
The config could be a json of type :
configJson = {
"env_production": {
"db_host_production": "www.host.production.url",
"db_password_production": "www.host.production.password"
},
"env_staging": {
"db_host_staging": "www.host.staging.url",
"db_password_staging": "www.host.staging.password"
},
"env_local": {
"db_host_local": "www.host.local.url",
"db_password_local": "www.host.local.password"
}
}
If its is just for local testing puposes you could even pass in config values as env variables to the json in config.js
i try to run my first deepstream.io server from this link but i get this error :
error:
CONNECTION_ERROR | Error: listen EADDRINUSE 127.0.0.1:3003
PLUGIN_ERROR | connectionEndpoint wasn't initialised in time
f:\try\deep\node_modules\deepstream.io\src\utils\dependency-
initialiser.js:96
throw error
^
Error: connectionEndpoint wasn't initialised in time
at DependencyInitialiser._onTimeout
(f:\try\deep\node_modules\deepstream.io\src\utils\dependency-
initialiser.js:94:17)
at ontimeout (timers.js:386:14)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)
and this is my code:
const DeepStreamServer = require("deepstream.io")
const C = DeepStreamServer.constants;
const server = new DeepStreamServer({
host:'localhost',
port:3003
})
server.start();
In deepstream 3.0 we released our HTTP endpoint, by default this runs alongside our websocket endpoint.
Because of this, passing the port option at the root level of the config no longer works (it overrides both the HTTP and websocket port options, as you can see in the screen capture provided, both endpoints are trying to start on the same port).
You can override each of these ports as follows:
const deepstream = require('deepstream.io')
const server = new deepstream({
connectionEndpoints: {
http: {
options: {
port: ...
}
},
websocket: {
options: {
port: ...
}
}
}
})
server.start()
Or you can define your config in a file and point to that while initialising deepstream[1].
[1] deepstream server configuration
One solution that i find is passing empty config object so inseted of :
const server = new DeepStreamServer({
host:'localhost',
port:3003
})
i'm just using this :
const server = new DeepStreamServer({})
and now everything work's well.
All the bellow is for Version 4.2.2 (last version by now)
I was having the same Port in use or config file not found errors. And i was using typescript and i didn't pay attention too to the output dir and build (which can be a problem when one use typescript and build). I was able to run the server in the end. And i had a lot of analysis.
I checked up the code source and i have seen how the config is loaded
const SUPPORTED_EXTENSIONS = ['.yml', '.yaml', '.json', '.js']
const DEFAULT_CONFIG_DIRS = [
path.join('.', 'conf', 'config'), path.join('..', 'conf', 'config'),
'/etc/deepstream/config', '/usr/local/etc/deepstream/config',
'/usr/local/etc/deepstream/conf/config',
]
DEFAULT_CONFIG_DIRS.push(path.join(process.argv[1], '..', 'conf', 'config'))
DEFAULT_CONFIG_DIRS.push(path.join(process.argv[1], '..', '..', 'conf', 'config'))
Also i tested different things and all. Here what i came up with:
First of all if we don't precise any parameter in the constructor. A config from the default directories will get to load. If there isn't then the server fail to run.
And one of the places where we can put a config is ./conf in the same folder as the server node script.
Secondly we can precise a config as a string path (parameter in the constructor). config.yml or one of the supported extensions. That will allow the server to load the server config + the permission.yml and users.yml configs. Which too are supposed to be in the same folder. If not in the same folder there load will fail, and therefor the permission plugin will not load. And so does the users config. And no fall back to default will happen.
Thirdly the supported extensions for the config files are: yml, yaml, json, js.
In nodejs context. If nothing precised. There is no fallback to some default config. The config need to be provided in one of the default folders, or by precising a path to it. Or by passing a config object. And all the optional options will default to some values if not provided ( a bit bellow there is an example that can show that ). Know however that precising an end point is very important and required.
To precise the path, we need to precise the path to the config.yml file (the server config) [example: path.join(__dirname, './conf/config.yml')]. Then from the same dir permission.yml and users.yml will be retrieved (the extension can be any of the supported one). We can not precise a path to a directory, it will fail.
We can precise the path to permission config or user config separatly within config.yaml as shown bellow:
# Permissioning example with default values for config-based permissioning
permission:
type: config
options:
path: ./permissions.yml
maxRuleIterations: 3
cacheEvacuationInterval: 60000
Finally we can pass an object to configure the server, or by passing null as a parameter and use .set methods (i didn't test the second method). For configuring the server we need to follow the same structure as the yml file. With sometimes a bit different naming. The typescript declaration files or types show us the way. With an editor like vscode. Even if we are not using typescript we can keep get the auto completion and type definitions.
And the simplest for equivalent to the previous version is :
const webSocketServer = new Deepstream({
connectionEndpoints: [
{
type: 'ws-websocket',
options: {
port: 6020,
host: '127.0.0.1',
urlPath: '/deepstream'
}
}
]
});
webSocketServer.start();
the above is the new syntax and way.
const server = new DeepStreamServer({
host:'localhost',
port:3003
})
^^^^^^^ is completely deprecated and not supported in version 4 (the doc is not updated).
I installed syslog-ng by using "yum install syslog-ng" in both local machine and server end.
I am using an open source version of syslog-ng.
My need is to pass the log file name from client to server end . I explicitly set the .SDATA.file at 18372.4.name field on my
client side, as the name of the file is available in the $FILE_NAME macro. But ".SDATA.file at 18372.4.name" is empty
in server side. When I am using some static file name the log beings to work.
Below is my code I dont know where i am going wrong If you need more information I can provide you can anyone help me.
MY CLIENT END SYSLOG-NG CODE:
source s_application_logs {
file(
"/var/log/test.log"
flags(no-parse)
);
};
destination d_access_system {
syslog(
"52.38.34.160"
transport("tcp")
port(6514)
);
};
rewrite r_set_filename {
set(
"$FILE_NAME",
value(".SDATA.file at 18372.4.name")
);
};
rewrite r_rename_filename {
subst(
"/var/log/",
"",
value(".SDATA.file at 18372.4.name")
type("string")
flags("prefix")
);
};
log {
source(s_application_logs);
rewrite(r_set_filename);
rewrite(r_rename_filename);
destination(d_access_system);
};
MY SERVER END SYSLOG-NG CODE:
source s_server_end {
syslog(
port(6514)
max_connections(1000)
keep_hostname(yes)
);
};
destination d_log_files {
file(
"/var/log/test/${.SDATA.file at 18372.4.name}"
create_dirs(yes)
);
};
log {source(s_server_end);destination(d_log_files);};
The problem is that the $FILE_NAME macro is currently available only in the commercial version of syslog-ng. For a possible workaround, see this blogpost: Forwarding filenames with syslog-ng
I am trying to get syslog-ng to support postfix's multiple lines.
As I understand it, syslog-ng can use the " flags(no-multi-line)" config option, but it is either not working, or I am applying it incorrectly.
My configurations:
Client:
source s_src {
system();
internal();
};
filter f_mail { facility(mail) and not filter(f_debug); };
log { source(s_src); filter(f_mail); destination(d_mail); };
destination d_tls {
tcp("logs.myserver.com" port(999)
tls( ca_dir("/etc/syslog-ng/ssl/")) );
};
log {
source(s_src);
destination(d_tls);
};
Server:
source s_tls {
tcp(port(999)
tls( key_file("/etc/syslog-ng/ssl/logs.key")
cert_file("/etc/syslog-ng/ssl/logs.crt")
peer_verify(optional-untrusted))
flags(no-multi-line) # no worky
);
};
From syslog documentations:
no-multi-line: The no-multi-line flag disables line-breaking in the messages; the entire message is converted to a single line. Note that this happens only if the underlying transport method actually supports multi-line messages. Currently the syslog, udp, unix-dgram drivers support multi-line messages; other drivers, for example, the tcp driver does not.
Thus it is not clear how multi-line messages get to your server in the first place. As a workaround, you might put the flag into file parameter:
destination d_farm_cmtslog_debug {
file("/var/log/cm/cmts.debug" flags(no-multi-line));
};