How come data is not coming from my hiera yaml file? - hiera

I am using Puppet Enterprise 3.7.2 and on one of my nodes I create the file:
[root#vii-osc4-mgmt-001 ~]# cat /etc/profile.d/POD_prefix.sh
export FACTER_pod_prefix=vii-osc4
Then I rebooted that node and logged back in and verified that
the FACTER_pod_prefix gets set and facter pod_prefix outputs the
expected value.
[root#vii-osc4-mgmt-001 ~]# env | grep FACTER_pod_prefix
FACTER_pod_prefix=vii-osc4
[root#vii-osc4-mgmt-001 ~]# facter pod_prefix
vii-osc4
On my PE 3.7 Puppet master I created the file /var/lib/hiera/vii-osc4.yaml.
I created the /var/lib/hiera/vii-osc4.yaml from the /var/lib/hiera/defaults.yaml
file that I had been using like so:
# cp /var/lib/hiera/defaults.yaml /var/lib/hiera/vii-osc4.yaml
This file has a bunch of class parameter values. For example there is this
line in the file:
controller_vip_name: vii-osc4.example.com
Then I changed my hiera.yaml file to look like this:
[root#osc4-ppt-001 ~]# cat /etc/puppetlabs/puppet/hiera.yaml
---
:backends:
- yaml
:hierarchy:
- "%{pod_prefix}"
- defaults
- "%{clientcert}"
- "%{environment}"
- global
:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /var/lib/hiera on *nix
# - %CommonAppData%\PuppetLabs\hiera\var on Windows
# When specifying a datadir, make sure the directory exists.
:datadir:
Then I restarted my pe-httpd service like so (RHEL7):
# systemctl restart pe-httpd
Then I make a small change to the /var/lib/hiera/vii-osc4.yaml for example
I change the line ...
controller_vip_name: vii-osc4.example.com
... to ...
controller_vip_name: VII-osc4.example.com
But when I run puppet agent -t --noop on my node, vii-osc4-mgmt-001, I do not see the change
that I expected to see. If I make the change in the /var/lib/hiera/defaults.yaml and then
run puppet agent -t --noop on my node I do see the expected changes. What am I doing wrong here?
UPDATE: using /etc/facter/facts.d method of setting custom facts.
I looked into using /etc/facter/facts.d for what I am trying to do. What I am trying to do is set a custom fact "pod_prefix". I want to use this fact in my hiera.yaml like so ...
---
:backends:
- yaml
:hierarchy:
- "%{::pod_prefix}"
- defaults
- "%{clientcert}"
- "%{environment}"
- global
:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /var/lib/hiera on *nix
# - %CommonAppData%\PuppetLabs\hiera\var on Windows
# When specifying a datadir, make sure the directory exists.
:datadir:
... so that nodes that have pod_prefix set to vii-osc4 will obtain their class parameters from the file /var/lib/hiera/vii-osc4/yaml and host that pod_prefix set to ix-xyz will get their class params from /var/lib/hiera/ix-xyz.yaml. I do not see how creating the file /etc/facter/facts.d/pod_prefix.txt on my puppet master that contains something like this ...
# cat pod_prefix.txt
pod_prefix=vii-osc4
... could possibly be a solution to my problem. I guess I must be misunderstanding something here. Can someone help?
UPDATE 2.
The /etc/facter/facts.d/pod_prefix.txt file goes on my nodes.
I think my biggest problem is that just execute systemctl restart pe-httpd was not sufficient and things didn't start working until I did a full reboot of my puppet master. I need to go look at the docs and figure out what is the correct way to restart the "puppet master".

The very approach of managing custom facts through environment variables is quite brittle. In this case, I suspect it does not work because you changed the environment of login shells via /etc/profile.d. System services don't run in such shells, though.
A clean approach would be to define your fact value in /etc/facter/facts.d instead.

Related

How configure multiple Redis instances on Debian

I've got a Debian server running Redis and I'd like to run a second copy using a different port. There are plenty of guides explaining how to do it on Ubuntu and other flavours of Linux but I'm having a hard time translating those to Debian.
So far I've created a copy of the /etc/redis/redis.conf and have renamed it /etc/redis/redis_6380.conf. In the new file I've changed the name of the PID file, location of the log file, the listening port (to 3680) so that they do not conflict with the existing instance of Redis.
The problem I have is knowing which changes to make so that systemd can start the new instance.
I've made a duplicate of /lib/systemd/system/redis-server.service and called it redis-server-6380.service and have changed the EXECStart and PIDFile lines to point to the new files:
ExecStart=/usr/bin/redis-server /etc/redis/redis_6380.conf
PIDFile=/var/run/redis/redis-server_6380.pid
Doing:
systemctl enable redis-server-6380.service
results in:
Failed to enable unit: File /etc/systemd/system/redis.service already exists and is a symlink to /lib/systemd/system/redis-server.service
How do I fix this ? I'm guessing that I've missed out a vital step but I'm not that familiar with configuring systemd supervised processes on Debian.
The end of the redis-server unit file would say:
[Install]
WantedBy=multi-user.target
Alias=redis.service
Either remove that Alias, or make sure it would be unique.
Also: make sure your Redis instances would have their own database and log files.

docker-compose override application properties

Having a Spring Boot application we are using application.yml file to store properties. I got a task to give a user a possibility to override some properties while starting an application. Taking into consideration we have dockerised our app docker-compose file is the very right place I believe for that. I found one option which works actually, env_file:
backend:
build:
context: backend
dockerfile: Dockerfile.backend
restart: always
ports:
- 3000:3000
env_file:
- backend/custom.env
volumes:
- ../m2_repo:/root/.m2/
- ../{APP_NAME}/data_sources:/backend/data_sources/
links:
- database
networks:
main:
aliases:
- backend
This solves perfectly my task and all the KEY=VALUE pairs override existing in application.yml properties. However, I have 2 questions:
It appeared that having multiple services in my docker-compose file I need specify a separate env_file for each service, which is probably not very convenient. Is there a possibility to have one common env_file for the whole docker-compose file?
I know that for docker-compose run command there is an option -e where i can put key=value pairs of env variables. Is there any similar option for docker-compose up? I mean in order not to use env_file at all.
Ad 1: It is not possible. I also believe it is intentional - to make the developer define what container has access to what .env data.
Ad 2: No, you cannot supply the variables using a runtime parameter of up command of docker-compose (run docker-compose help up to see the available runtime params). But you can define these using environment clause from within a compose file, like:
restart: always
ports:
- 3000:3000
env_file:
- backend/custom.env
environment:
- DB_PASSWORD # <= #1
- APP_ENV=production # <= #2
ie.
either just a name of the env var - its value is then taken from the host machine
or the whole definition to create a new one to be available within a container
See docs on environment clause for more clarification.
Another thing you can do in order to override some settings is to extend the compose file using a "parent" one. Docs on extends clause
Unfortunately as of now, extends won't work when using compose file of version 3, but it is being discussed in this github issue, so hopefully it will be available soon:)

Changing permissions of added file to a Docker volume

In the Docker best practices guide it states:
You are strongly encouraged to use VOLUME for any mutable and/or user-serviceable parts of your image.
And by looking at the source code for e.g. the cpuguy83/nagios image this can clearly be seen done, as everything from nagios to apache config directories are made available as volumes.
However, looking at the same image the apache service (and cgi-scripts for nagios) are run as the nagios user by default. So now I'm in a pickle, as I can't seem to figure how to add my own config files in order to e.g. define more hosts for nagios monitoring. I've tried:
FROM cpuguy83/nagios
ADD my_custom_config.cfg /opt/nagios/etc/conf.d/
RUN chown nagios: /opt/nagios/etc/conf.d/my_custom_config.cfg
CMD ["/opt/local/bin/start_nagios"]
I build as normal, and try to run it with docker run -d -p 8000:80 <image_hash>, however I get the following error:
Error: Cannot open config file '/opt/nagios/etc/conf.d/my_custom_config.cfg' for reading: Permission denied
And sure enough, the permissions in the folder looks like (whist the apache process runs as nagios):
# ls -l /opt/nagios/etc/conf.d/
-rw-rw---- 1 root root 861 Jan 5 13:43 my_custom_config.cfg
Now, this has been answered before (why doesn't chown work in Dockerfile), but no proper solution other than "change the original Dockerfile" has been proposed.
To be honest, I think there's some core concept here I haven't grasped (as I can't see the point of declaring config directories as VOLUME nor running services as anything other than root) - so provided a Dockerfile as above (which follows Docker best practices by adding multiple volumes) is the solution/problem:
To change NAGIOS_USER/APACHE_RUN_USER to 'root' and run everything as root?
To remove the VOLUME declarations in the Dockerfile for nagios?
Other approaches?
How would you extend the nagios dockerfile above with your own config file?
Since you are adding your own my_custom_config.cfg file directly into the container at build time just change the permissions of the my_custom_config.cfg file on your host machine and then build your image using docker build. The host machine permissions are copied into the container image.

How to set PATH variable in crontab via whenever gem

Is it possible to set the PATH or SHELL variable in a crontab via the whenever schedule.rb file?
# here I want to set the PATH and SHELL variable somehow
every 3.hours do
# some cronjob
end
I want this output in my crontab after my capistrano deploy:
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/bin/X11
# some cronjobs
Ok, it seems as I found the solution. I found it here: https://gist.github.com/jjb/950975
I will update this answer when I have tested it
I have to put this into my schedule.rb
# If your ruby binary isn't in a standard place (for example if it's in /usr/local/bin,
# because you installed it yourself from source, or from a thid-party package like REE),
# this tells whenever (or really, the rails runner) where to find it.
env :PATH, '/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin'
You are already doing it when running zenity when setting DISPLAY, LANG etc.
If you want to set the shell, set it in the first line of /home/username/script/script1.sh using #!/bin/bash.
If you want to set the path, one way to do it is to set it before running the command:
5 9-20 * * * PATH=/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/bin/X11 /home/username/script/script1.sh > /dev/null
A alternate/better way is to create a simple wrapper script like so:
#!/bin/bash
export PATH=/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/bin/X11
# Absolute path to this script
SCRIPT=`readlink -f $0`
# Absolute directory this script is in
SCRIPTPATH=`dirname $SCRIPT`
#make sure we are in the same directory as the script1.sh - this is useful in case the script assumes it is running from the same directory it's in and makes relative directory/file references
cd $SCRIPTPATH
##run final script, and pass through all parameters that were passed to wrapper script.
/home/username/script/script1.sh "$#"

How to trace where php5-fpm umask settings are coming from on ubuntu

I'd really appreciate any help in tracking down and diagnosing an umask issue on Ubuntu:
I'm running php5-fpm with Apache via proxy_fcgi. The process is running with a umask of 0022 (confirmed by having PHP send the results of umask() into a file [the result is '18' == 0022]). I'd like to change this to 0002, but can't track down where the umask is coming from.
Apache is set with umask 0002, and as a test, if I disable proxy_fcgi and run my test above, I get a file with u+g having rw access (and the file contents confirm the umask as '2' == 0002).
If I sudo -iu fpmuser and run umask the results are 0002.
System info:
PHP: 5.5.3-1ubuntu2.1
Apache: 2.4.6
Ubuntu: 13.10
PHP-PFM is listing using TCP ports (as Unix ports aren't yet working/support)
So far I've tried the following (each followed by a system restart and a retest):
adding umask 0002 to the start of /etc/init.d/php5-fpm
adding --umask 0002 into the start-stop-daemon calls in /etc/init.d/php5-fpm
adding umask 0002 to .profile in the home of the fpm user
Something is clearly adjusting the umask of the php-fpm process - so, how can I begin tracing what is forcing the umask 0022 onto the php-fpm process?
EDIT (1):
adjusting the system wide umask via /etc/login.defs (see How to set system wide umask?) affects the umask elsewhere (e.g. comannds via sudo now have a umask of 0002), but still php-fpm creates files with a umask of 0022. Note that I verified that session optional pam_umask.so was also present in /etc/pam.d/common-session-noninteractive and I tested umasks of 002 and 0002.
EDIT (2):
I have been able to replicate the issue using nginx and php5-fpm (using unix sockets set to listen mode '0666').
I would love to trace where the umask is coming from but I'd settle for some way to force it to what I want.
I should add that the first test was done on an Amazon Ubuntu 13.10 image. My tests in 'edit 2' where completed using a copy of the Ubuntu13.10 server ISO setup from scratch in a virtual machine. All installations were completed via apt-get rather than by downloading the source and building.
EDIT (3):
I have confirmed I can manipulate the umask manually by either of the following (verified by checking the permissions on the test file created):
a. In a shell, set a umask then run /usr/sbin/php-fpm from the shell
b. In a shell, run the following with whatever umask value I like:
start-stop-daemon --start --quiet --umask 0002 --pidfile /var/run/php5-fpm.pid --exec /usr/sbin/php5-fpm -- --daemonize --fpm-config /etc/php5/fpm/php-fpm.conf
However this exact same command in the /etc/init.d/php5-fpm file fails to adjust the umask when running sudo service php5-fpm stop; sudo service php5-fpm start or at reboot.
Not a solution for generically tracing where umask settings are coming from on ubuntu (the only way I've found so far is the good old hard work approach of replicating the issue, attempting to isolate it to a script or a function, then stepping back through each script/function that is called recursively) but a solution to the php5-fpm umask issue. I've found a lot of hits on google, stackoverflow, and elsewhere for the problem, but so far no solution. Hopefully this is useful for people.
Edit /etc/init/php-fpm.conf to include the line umask 0002 (or whatever umask you wish). My version of the file now looks like this:
# php5-fpm - The PHP FastCGI Process Manager
description "The PHP FastCGI Process Manager"
author "Ondřej Surý <ondrej#debian.org>"
start on runlevel [2345]
stop on runlevel [016]
### my edit - change umask setting
umask 0002
pre-start exec /usr/lib/php5/php5-fpm-checkconf
respawn
exec /usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf
Explanation
Having traced through the service command which launches php5-fpm at startup, it runs some checks (line 118 on my copy) for /etc/init/${SERVICE}.conf, along with verifying initctl is present and can report it's version. If these tests are passed then upstart is used which in the case of php5-fpm uses the /etc/init/php-fpm.conf file.
The ubuntu upstart site gives pretty clear instructions. In particular you can check out the upstart cookbook for the specifics you need.
As best I can work out that means that therefore the 'service' command was never actually running the start-stop-daemon … commands found in /etc/init.d/php5-fpm which is why my previous edits were having no effect. Instead it passes off to upstart (actually initctl) when you use something like service php5-fpm start, etc.
If you use systemd, in the /etc/systemd/system directory, create a new directory called php7.2-fpm.service.d. The name of this directory will vary depending on your distro and PHP version. Run systemctl list-units --type=service | grep --ignore-case php to find out what to call it. Inside of this directory, place a file called umask.conf with the contents:
# /etc/systemd/system/php7.2-fpm.service.d/umask.conf
[Service]
UMask=0002
For the changes to take effect, run:
systemctl daemon-reload && systemctl restart php7.2-fpm
The benefit of this solution is that your customizations are not lost when packages get updated.
Explanation of how this works from the systemd manual:
Along with a unit file foo.service, a "drop-in" directory foo.service.d/ may exist. All files with the suffix ".conf" from this directory will be parsed after the file itself is parsed. This is useful to alter or add configuration settings for a unit, without having to modify unit files. Each drop-in file must have appropriate section headers. Note that for instantiated units, this logic will first look for the instance ".d/" subdirectory and read its ".conf" files, followed by the template ".d/" subdirectory and the ".conf" files there.
In addition to /etc/systemd/system, the drop-in ".d" directories for system services can be placed in /usr/lib/systemd/system or /run/systemd/system directories. Drop-in files in /etc take precedence over those in /run which in turn take precedence over those in /usr/lib. Drop-in files under any of these directories take precedence over unit files wherever located. Multiple drop-in files with different names are applied in lexicographic order, regardless of which of the directories they reside in.
better copy systemd script before editing php5-fpm.service or it will be overwritten on next update:
cp /lib/systemd/system/php5-fpm.service /etc/systemd/system/
vi /etc/systemd/system/php5-fpm.service
Add: UMask=0002 in [Service] section.
systemctl daemon-reload
systemctl restart php5-fpm
Source: https://ispire.me/running-php-fpm-with-different-user-group-using-umask/
okey, but this applies to all the pools.
Would be handy to be able to set it with something like
env[umask] = 0002
(no chance for this to work)
been googling, but doesn't seem to be a way to do this on a per host basis.