perl6 'do(file)' equivalent - raku

In perl5 I used to 'do (file)' for configuration files like this:
---script.pl start ---
our #conf = ();
do '/path/some_conf_file';
...
foreach $item (#conf) {
$item->{rules} ...
...
---script.pl end ---
---/path/some_conf_file start ---
# arbitrary code to 'fill' #conf
#conf = (
{name => 'gateway',
rules => [
{verdict => 'allow', srcnet => 'gw', dstnet => 'lan2'}
]
},
{name => 'lan <-> lan2',
rules => [
{srcnet => 'lan', dstnet => 'lan2',
verdict => 'allow', dstip => '192.168.5.0/24'}
]
},
);
---/path/some_conf_file end ---
Also Larry Wall's "Programming Perl" also mentions this method:
But do FILE is still useful for such things as reading program
configuration files. Manual error checking can be done this way:
# read in config files: system first, then user
for $file ("/usr/share/proggie/defaults.rc",
"$ENV{HOME}/.someprogrc") {
unless ($return = do $file) {
warn "couldn't parse $file: $#" if $#;
warn "couldn't do $file: $!" unless defined $return;
warn "couldn't run $file" unless $return;
} }
Benefits:
does not require write your own parser each time - perl parse and
create data structures for you;
faster/simpler: native perl data
structures/types without overheads for converting from external format (like YAML);
does not require manipulate #INC to load the
module from somewhere compared to module as conf file;
less extra
code compared to modules as conf file;
"syntax" of "configuration file" is powerful as perl itself;
"ad hoc" format;
Disadvantages:
no isolation: we can execute/destroy anything from "configuration
file";
How do I get the same with perl6?
Is there way to do it better in perl6 (without Disadvantages) and without parsing own syntax, grammars, module including?
Something like "Load hashes or arrays from text representation from file"?

You can use EVALFILE($file) (ref. http://doc.perl6.org/language/5to6-perlfunc#do).
As you pointed out, using EVALFILE has disadvantages, so I'm not going to add anything in that direction :-)
Here's a sample configuration file:
# Sample configuration (my.conf)
{
colour => "yellow",
pid => $*PID,
homedir => %*ENV<HOME> ~ "/.myscript",
data_source => {
driver => "postgres",
dbname => "test",
user => "test_user",
}
}
and here's a sample script using it:
use v6;
# Our configuration is in this file
my $config_file = "my.conf";
my %config := EVALFILE($config_file);
say "Hello, world!\n";
say "My homedir is %config<homedir>";
say "My favourite colour is %config<colour>";
say "My process ID is %config<pid>";
say "My database configuration is:";
say %config<data_source>;
if $*PID != %config<pid> {
say "Strange. I'm not the same process that evaluated my configuration.";
}
else {
say "BTW, I am still the same process after reading my own configuration.";
}

Related

How to update Puppet ini_setting or ini_subsetting resource without section header in conf file?

I wonder if someone can help me with my conf file problem. I need to get the output like below but I get problems in using the inifile. I have put below my code and testing output. My service won't start because of the '[]'. Your comments and ideas are highly appreciated. Thanks!
Expected output
cat /etc/service.conf
info something something...
without section header
setting1=value1
Testings
testscript1.pp
ini_setting {'setx':
ensure => present,
path => '/etc/service.conf',
key_val_separator => '=',
setting => 'setting1',
value => 'value1',
}
output of testscript1.pp
cat /etc/service.conf
info something something...
[setx]
setting1=value1
testscript2.pp
$defaults = {
ensure => present,
path => '/etc/service.conf',
key_val_separator => '=',
}
$settings = {
' ' => {
'setting1' => 'value1',
}
}
create_ini_settings($settings,$defaults)
output of testscript2.pp
cat /etc/service.conf
info something something...
[ ]
setting1=value1
Since I really wanted to delete the [] character because it's causing error during service restart, I used section_prefix => '#',. The first puppet agent run is smooth and working. Problem now is if puppet agent runs on its frequency time (like let's say every hour), it will auto-append details in conf file due to lack of section header. I decided to use ini_subsetting but I'm getting errors with it.
testscript3.pp
ini_subsetting {'subset':
ensure => present,
section => '',
key_val_separator => '=',
path => '/etc/service.conf',
setting => 'setting1',
subsetting => '',
value => 'value1',
}
output of testscript3.pp
Error: Failed to apply catalog: Parameter path failed on Ini_subsetting[subset]: File paths must be fully qualified, not '/etc/service.conf'.
Any suggestions or advises are highly appreciated.
Thank you.
If the file you are managing does not have section markers of some kind, then it is not an INI file, not even in the generalized sense that the puppetlabs/inifile module supports. To the best of my knowledge, you'll need to choose a different approach to managing the file.
You could consider templating the whole file, or writing a custom type and provider for it, but before going to so much trouble, you should consider whether a good old file_line resource from puppetlabs/stdlib would be adequate for your needs.
Have you tried your testscript1.pp with section => ''?
It would look like this:
ini_setting {'setx':
ensure => present,
path => '/etc/service.conf',
key_val_separator => '=',
section => '',
setting => 'setting1',
value => 'value1',
}
And the output would be:
cat /etc/service.conf
info something something...
setting1=value1
Or you could try to use force_new_section_creation => false, as it is true by default and forces the creation of a section, as stated in the module’s reference.
As for your 3rd example, it probably fails because of the blank subsetting parameter. The ini_subsetting resource type requires both setting and subsetting parameters to work.

Puppet conditional only if file exists in a particular directory

I have the following file resource in my puppet manifest:
file{
'example.dat'
path => "/path/to/example.dat",
owner => devops,
mode => "0644",
source => "/path/to/example.txt"
}
I want to run the above snippet only when the .txt file is present in a particular directory. Otherwise, I do not want the snippet to run.
How do I go about doing that in puppet?
In general, you would create a custom fact that returns true when the directory in question satisfies your condition. For example:
Facter.add('contains_txt') do
setcode do
! Dir.glob("/path/to/dir/*.txt").empty?
end
end
Then you would write:
if $facts['contains_txt'] {
file { 'example.dat':
path => "/path/to/example.dat",
owner => devops,
mode => "0644",
source => "/path/to/example.txt",
}
}

Template Toolkit cannot find template files

I have the following .htaccess file:
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_URI} !/views/.*/.*\.html
RewriteRule ^([a-zA-Z0-9_-]+)/([a-zA-Z0-9_-]+)/(.*)$ index.pl?controller=$1&action=$2&id=$3
This allows me to forward requests like /Users/show/1 to my main controller. I can also access the template files; at least in my browser i can open the file:
/views/Users/add.html
However when I try to access the file in my controller when i want to render the template, it cannot be found (No such file or directory)
This is the code where i try to render the template:
package UsersController;
use Moose;
use Controllers::ControllerRole;
use Template;
with 'ControllerRole';
has 'example' => (
is => 'rw',
isa => 'Str',
default =>'test',
);
sub show {
my $self = shift;
my $id = $self->{params}->{id};
my $config = {
INTERPOLATE => 1, # expand "$var" in plain text
POST_CHOMP => 1, # cleanup whitespace
PRE_PROCESS => 'header', # prefix each template
EVAL_PERL => 1, # evaluate Perl code blocks
};
my $vars = {
id => $id,
};
my $template = Template->new();
$template->process('views/Users/add.html', $vars) || die "$!";
}
1;
I also tried using absolute or relative paths, but apache complained that this is not allowed
It maybe to helpful to see the file structure of my project:
Update:
If i use chdir i can open the file with Perl's open function, but trying to process the file i still get the error
file error - header: not found
Here is a second iteration of my code:
sub show {
my $self = shift;
chdir '/srv/http/perlweb';
my $id = $self->{params}->{id};
my $config = {
INTERPOLATE => 1, # expand "$var" in plain text
POST_CHOMP => 1, # cleanup whitespace
PRE_PROCESS => 'header', # prefix each template
EVAL_PERL => 1, # evaluate Perl code blocks
ABSOLUTE => 1,
RELATIVE => 1,
};
my $vars = {
id => $id,
planet => 'earth',
captain => 'kirk',
};
open my $fh, "views/Users/add.html" or die "$!"; #this works
close $fh;
my $tt = Template->new($config);
$tt->process('views/Users/add.html', $vars) or die $tt->error(); # this does not work
}
Template Toolkit has the configuration option INCLUDE_PATH that allows you to set up one or more directories in which Template Toolkit looks to find template files. If you set the include_path variable, rather than putting the path in the $tt->process() directive, you can then set up paths in a global config and not hardcode paths in every $tt->process() directive or have to use the ABSOLUTE or RELATIVE options. You can specify numerous directories in the INCLUDE_PATH, and use a subroutine to generate include paths, so you are not limited to keeping all your templates in a single directory.
I would therefore change the config and process directives to the following:
my $config = {
INTERPOLATE => 1, # expand "$var" in plain text
POST_CHOMP => 1, # cleanup whitespace
PRE_PROCESS => 'header', # prefix each template
EVAL_PERL => 1, # evaluate Perl code blocks
INCLUDE_PATH => [ "/srv/http/perlweb/views/Users/",
"/wherever/header/is",
\&my_cool_sub_to_generate_a_path_on_the_fly ]
};
my $tt = Template->new($config);
$tt->process('add.html', $vars) or die $tt->error();
The documentation on INCLUDE_PATH has full information on options for this setting.
Note that the error you are getting is about the file header that is automatically included in each template -- make sure that you have its directory in your INCLUDE_PATH.

Multiple Logstash instances causing duplication of lines

We're receiving logs using Logstash with the following configuration:
input {
udp {
type => "logs"
port => 12203
}
}
filter {
grok {
type => "tracker"
pattern => '%{GREEDYDATA:message}'
}
date {
type => "tracker"
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
}
}
output{
tcp{
type => "logs"
host => "host"
port => 12203
}
}
We're then picking the logs up on the machine "host" with the following settings:
input {
tcp {
type => "logs"
port => 12203
}
}
output {
pipe {
command => "python /usr/lib/piperedis.py"
}
}
From here, we're doing parsing of the lines and putting them into a Redis database. However, we've discovered an interesting problem.
Logstash 'wraps' the log message in a JSON style package i.e.:
{\"#source\":\"source/\",\"#tags\":[],\"#fields\":{\"timestamp\":[\"2013-09-16 15:50:47,440\"],\"thread\":[\"ajp-8009-7\"],\"level\":[\"INFO\"],\"classname\":[\"classname\"],\"message\":[\"message"\]}}
We then, on receiving it and passing it on on the next machine, take that as the message and put it in another wrapper! We're only interested in the actual log message and none of the other stuff (source path, source, tags, fields, timestamp e.t.c.)
Is there a way we can use filters or something to do this? We've looked through the documentation but can't find any way to just pass the raw log lines between instances of Logstash.
Thanks,
Matt
The logstash documentation is wrong - it indicates that the default "codec" is plain but in fact it doesn't use a codec - it uses an output format.
To get a simpler output, change your output to something like
output {
pipe {
command => "python /usr/lib/piperedis.py"
message_format => "%{message}"
}
}
Why not just extract those messages from stdout?
line = sys.stdin.readline()
line_json = json.loads(line)
line_json['message'] # will be your #message

How to order resources from a specific module?

I got some trouble ordering resources from a module.
class { 'postgres' :
charset => 'UTF8',
locale => 'fr_FR',
require => Service['postgresqld'],
}->
class { 'postgresql::server':
}
postgresql::role { 'role1' :
namevar => 'redmine',
password_hash => 'random_md5',
createdb => true,
require => Class['postgres'],
}
postgresql::database_user {'charly':
password => 'random',
role => 'redmine',
require => postgresql::role['role1'],
}
I want to order this, but it appears to have a syntax error on the last line at role.
I'm pretty sure it comes from the capitalized first letter. But Puppet doesn't want to run the manifest if I put a capital letter Postgresql::role['role1] or postgresql::Role['role1]. Without capital letter, I "just" get a warning :
warning: Deprecation notice: Resource references should now be capitalized on line 61 in file /home/charly/testManifests/part1.pp
I'm doing something wrong, but I don't know what. I searched for an answer on the Internet but can't find what I want neither in tutorials nor on the forums.
Try using chaining arrows to your resource group references e.g.
Class['postgres'] -> Class['postgresql::server']
class { 'postgres' :
charset => 'UTF8',
locale => 'fr_FR',
require => Service['postgresqld']
}
class { 'postgresql::server': }
More detail can be found here in the puppet reference Chaining Arrows