How is the correct syntax/call to get the sls-file working - salt - module

I'm trying to build a reactor sls file, which starts running when an event occurs.
The content of the sls file should be as the following cli commands:
sudo salt minion git.add /srv/salt .
sudo salt minion git.commit /srv/salt test
sudo salt minion git.push /srv/salt origin master identity=/home/autogit/.ssh/id_rsa
If i run the code bellow triggered by the reactor. I get the following error message.
[DEBUG ] Reactor is populating module client cache
[ERROR ] An un-handled exception from the multiprocessing process 'Reactor-9:1' was caught:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/utils/process.py", line 765, in _run
return self._original_run()
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 271, in run
self.call_reactions(chunks)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 228, in call_reactions
self.wrap.run(chunk)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 330, in run
self.populate_client_cache(low)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 324, in populate_client_cache
self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'module'
[CRITICAL] Engine 'reactor' could not be started!
I've tried different syntax (old style and new style) but couldn't figure out what the problem is. Always getting an KeyError: u'module' or u'git'.
Also tried it with runner function to run it locally on the master.
git pull:
module.run:
- git.pull:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- identity: /home/autogit/.ssh/id_rsa
- git.add:
- cwd: /srv/salt
- filename: .
- git.commit:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- git.push:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- identity: /home/autogit/.ssh/id_rsa
salt --versions-report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: unknown
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-46-generic
system: Linux
version: Ubuntu 18.04 bionic
Since i'm quite new to Salt, hopefully you can give me a hint what i'm doing wrong:

You didn't provide the master config.
About module.run confusion: add in your settings (minion and maybe to master since I don't know your use-case)
use_superseded:
- module.run
That will enable your syntax, more doc about this here: https://docs.saltstack.com/en/latest/ref/states/all/salt.states.module.html#salt.states.module.run
In general: you are executing execution modules from the place that state modules are allowed only (the term module is heavily overused in salt...)
You didn't provide the full Master config. Reactor requires dedicated config to match events to sls files:
https://docs.saltstack.com/en/latest/ref/configuration/master.html#master-reactor-settings
You can also check the doc I've written some time ago about events and reactors:
https://github.com/kiemlicz/util/wiki/Salt-Events-and-Reactor
Assuming you've configured your events-to-sls-files-matching in master config, your provided sls:
git pull:
module.run:
- git.pull:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/
...
will not work.
Mind that reaction happens on Salt Master thus the reaction sls file need to provide type of reaction (local, runner etc.) since it's no longer 'view of one minion' but possibly of tons of minions!
First create runner reaction type (which delegates to some orchestration sls file which will contain your logic wrapped with (I think) salt.function )
Help yourself with aforementioned github link to my attempt of explaining Reactor.
Refer to official doc as well: https://docs.saltstack.com/en/latest/topics/reactor/index.html

Related

CodeDeploy not properly copying code from GitHub

I have CodeDeploy pull code from my GitHub repo. In the deployment Commit ID (for GitHub) I have specified the Commit ID that I want to deploy. My repo has the following structure:
my-service/
README.md
.gitignore
scripts/
deploy.sh
src/
<lots of code here>
pm2.dev.json
appspec.yml
My appspec.yml file looks like:
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu
hooks:
BeforeInstall:
- location: scripts/deploy.sh
timeout: 300
runas: root
My scripts/deploy.sh looks like:
sudo npm install pm2 -g
pwd
pm2 start /home/ubuntu/my-service/pm2.dev.json
When I run the CodeDeployment deployment for this, it fails with the following error:
Script at specified location: scripts/deploy.sh run as user root failed with exit code 1
When I look at the logs I see:
LifecycleEvent - BeforeInstall
Script - scripts/deploy.sh
[stderr]npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
[stdout]changed 182 packages, and audited 183 packages in 8s
[stdout]
[stdout]12 packages are looking for funding
[stdout] run `npm fund` for details
[stdout]
[stdout]found 0 vulnerabilities
[stdout]/opt/codedeploy-agent
[stderr][PM2][ERROR] File /home/ubuntu/my-service/pm2.dev.json not found
Sure enough when I look in /home/ubuntu/my-service, I do not see a pm2.dev.json file, because this server had been manually configured several weeks ago before a pm2.dev.json file was added to the project. I would have expected CodeDeploy to have written whats in the repo to the server under /home/ubuntu.
Can anyone spot anything wrong with my appspec.yml or other configuration? Could it be a bad GitHub setup?
Had to change BeforeInstall to Install.
BeforeInstall runs before it copies over the source code (specified under files/source). But Install run just after that copy occurs, hence files will be available on the file system.

Error using brownie on Vscode while running scripts

I get the following ERROR when I try to run scripts with brownie, using the following PowerShell command;
brownie run scripts/simple_collectible/deploy_simple
I have look all over stacked and other pages for info on this and I can't seem to find much, I would really like to carry on with my project but I am stuck at this point. any hekp would be wonderful.
Cheers!
INFO MESSAGE:
PS C:\Users\charl\OneDrive\Desktop\NFT Development\NFT-mix-main> brownie run scripts/simple_collectible/deploy_simple
INFO: Could not find files for the given pattern(s).
Brownie v1.17.2 - Python development framework for Ethereum
NftMixMainProject is the active project.
Launching 'ganache-cli.cmd --port 8545 --gasLimit 12000000 --accounts 10 --hardfork istanbul --mnemonic brownie'...
File "C:\Users\charl.local\pipx\venvs\eth-brownie\lib\site-packages\brownie_cli_main_.py", line 64, in main
importlib.import_module(f"brownie._cli.{cmd}").main()
File "C:\Users\charl.local\pipx\venvs\eth-brownie\lib\site-packages\brownie_cli\run.py", line 46, in main
path, _ = _get_path(args[""])
File "C:\Users\charl.local\pipx\venvs\eth-brownie\lib\site-packages\brownie\project\scripts.py", line 130, in _get_path
raise FileNotFoundError(f"Cannot find {path_str}")
FileNotFoundError: Cannot find scripts/simple_collectible/deploy_simple
Terminating local RPC client...
I have the following packages installed:
ganache-cli
pip
pipx
Brownie (installed through pipx, and initialized)
I have run the brownie command to make sure the install is good.
I have install Python Venv
I have tried uninstalling all packaged and reinstalling
I have done the same with my VScode and Vsbuildtools
I have done the same with Python itself (reinstalled from the website)
The code snippet I have for my Script that I am trying to run is here:
#!/usr/bin/python3
import os
from brownie import SimpleCollectible, accounts, config, network
def main():
dev = accounts.add(config["wallets"]["from_key"])
print(network.show_active())
publish_source = True if os.getenv("ETHERSCAN_TOKEN") else False
SimpleCollectible.deploy({"from": dev}, publish_source=publish_source)
And finally for your reference I have my brownie-config.yaml contents here:
# exclude SafeMath when calculating test coverage
# https://eth-brownie.readthedocs.io/en/v1.10.3/config.html#exclude_paths
reports:
exclude_contracts:
- SafeMath
dependencies:
- smartcontractkit/chainlink-brownie-contracts#1.1.1
- OpenZeppelin/openzeppelin-contracts#3.4.0
compiler:
solc:
remappings:
- '#chainlink=smartcontractkit/chainlink-brownie-contracts#1.1.1'
- '#openzeppelin=OpenZeppelin/openzeppelin-contracts#3.4.0'
# automatically fetch contract sources from Etherscan
autofetch_sources: True
dotenv: .env
# set a custom mnemonic for the development network
networks:
default: development
kovan:
vrf_coordinator: '0xdD3782915140c8f3b190B5D67eAc6dc5760C46E9'
link_token: '0xa36085F69e2889c224210F603D836748e7dC0088'
keyhash: '0x6c3699283bda56ad74f6b855546325b68d482e983852a7a82979cc4807b641f4'
fee: 100000000000000000
oracle: '0x2f90A6D021db21e1B2A077c5a37B3C7E75D15b7e'
jobId: '29fa9aa13bf1468788b7cc4a500a45b8'
eth_usd_price_feed: '0x9326BFA02ADD2366b30bacB125260Af641031331'
rinkeby:
vrf_coordinator: '0xb3dCcb4Cf7a26f6cf6B120Cf5A73875B7BBc655B'
link_token: '0x01be23585060835e02b77ef475b0cc51aa1e0709'
keyhash: '0x2ed0feb3e7fd2022120aa84fab1945545a9f2ffc9076fd6156fa96eaff4c1311'
fee: 100000000000000000
oracle: '0x7AFe1118Ea78C1eae84ca8feE5C65Bc76CcF879e'
jobId: '6d1bfe27e7034b1d87b5270556b17277'
eth_usd_price_feed: '0x8A753747A1Fa494EC906cE90E9f37563A8AF630e'
mumbai:
eth_usd_price_feed: '0x0715A7794a1dc8e42615F059dD6e406A6594651A'
binance:
# link_token: ??
eth_usd_price_feed: '0x9ef1B8c0E4F7dc8bF5719Ea496883DC6401d5b2e'
binance-fork:
eth_usd_price_feed: '0x9ef1B8c0E4F7dc8bF5719Ea496883DC6401d5b2e'
mainnet-fork:
eth_usd_price_feed: '0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419'
matic-fork:
eth_usd_price_feed: '0xF9680D99D6C9589e2a93a78A04A279e509205945'
wallets:
from_key: ${PRIVATE_KEY}
from_mnemonic: ${MNEMONIC}
# You'd have to change the accounts.add to accounts.from_mnemonic to use from_mnemonic
Change scripts/simple_collectible/deploy_simple -> scripts/simple_collectible/deploy_simple.py

screen fails on NetBSD, reporting "poll: Invalid argument"

I have installed and used screen many times on several different operating systems. Recently I installed it on a NetBSD-8.0 virtual machine.
$ sudo pkgin install screen
calculating dependencies...done.
1 package to install:
screen-4.8.0nb1
0 to refresh, 0 to upgrade, 1 to install
0B to download, 1098K to install
proceed ? [Y/n] Y
installing screen-4.8.0nb1...
screen-4.8.0nb1: setting permissions on /usr/pkg/bin/screen-4.8.0 (o=root, g=wheel, m=4511)
screen-4.8.0nb1: adding /usr/pkg/bin/screen to /etc/shells
screen-4.8.0nb1: registering info file /usr/pkg/info/screen.info
===========================================================================
$NetBSD: MESSAGE,v 1.5 2005/12/28 17:53:24 reed Exp $
[snip]
===========================================================================
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
marking screen-4.8.0nb1 as non auto-removable
However, when I went to use it, I got an immediate failure.
$ uname -mrs
NetBSD 8.0 amd64
$ ls -l /usr/pkg/bin/screen
lrwxr-xr-x 1 root wheel 12 Apr 6 02:50 /usr/pkg/bin/screen -> screen-4.8.0
$ groups
users wheel
$ screen
poll: Invalid argument
This problem persists even when I first remove, then reinstall the screen package. Any suggestions as to what's wrong?
My guess is that the system used to build binary packages for 8.0 (as of the 8.0_2020Q1 pkgsrc release) is no longer quite compatible with the NetBSD-8.0 release. It is likely running on a newer release, inside a chroot(8) sandbox.
I would recommend using NetBSD-9.0 instead, as that is the latest NetBSD release, or NetBSD-8.2, as that is the latest release in the netbsd-8 branch. Using the latest NetBSD and pkgsrc releases provides better coverage against unpatched vulnerabilities.
However, if you want to keep using NetBSD-8.0, you can get a working screen(1) from the 8.0_2019Q4 pkgsrc release. To have pkgin(1) pull from that release, edit the /usr/pkg/etc/pkgin/repositories.conf file to use this repository URL:
http://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/8.0_2019Q4/All
There is currently likely just one line in the file that is not commented out, and it points to a URL with just 8.0 in it (which on the server is a symbolic link to the latest pkgsrc release). Just replace that line, or comment it out and add the above line.
Then remove and re-install screen:
sudo pkgin remove screen && sudo pkgin install screen

Installing GNU Radio using PyBOMBS

I am following the installation instructions on https://github.com/gnuradio/pybombs.
I get the following error message:
→ sudo -H pybombs prefix init ~/prefix -a myprefix -R gnuradio-default
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.6
PyBOMBS - INFO - PyBOMBS Version 2.3.3a0
PyBOMBS.prefix - WARNING - There already is a prefix in `~/prefix'.
Continue using this path Y/[N]? y
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.6
Alias `myprefix' already exists, overwrite Y/[N]? y
PyBOMBS.ConfigManager - INFO - Prefix Python version is: 2.7.6
PyBOMBS.prefix - INFO - Installing default packages for prefix...
PyBOMBS.prefix - INFO -
- <ruamel.yaml.comments.CommentedSeq object at 0x7f3891d44d20>
- <ruamel.yaml.comments.CommentedSeq object at 0x7f3891d44d70>
PyBOMBS.install_manager - INFO - Phase 1: Creating install tree and installing binary packages:
PyBOMBS.get_recipe - ERROR - Error fetching recipe `<ruamel.yaml.comments.CommentedSeq object at 0x7f3891d44d20>':
Package <ruamel.yaml.comments.CommentedSeq object at 0x7f3891d44d20> has no recipe file!
It seems like a mismatch in the use of ruamel.yamel. Reinstalling ruamel.yamel does not solve this:
→ sudo -H pip install --upgrade --force-reinstall ruamel.yaml
Collecting ruamel.yaml
Downloading https://files.pythonhosted.org/packages/3c/bf/6be7b1510270d785ca53193d06ba64c2f2bd969db732d108e2a82cbc9a0e/ruamel.yaml-0.15.57-cp27-cp27mu-manylinux1_x86_64.whl (590kB)
Collecting ruamel.ordereddict; platform_python_implementation == "CPython" and python_version <= "2.7" (from ruamel.yaml)
Using cached https://files.pythonhosted.org/packages/f3/2c/fa6d75dc459b371ed3b88fdbf8042785ce1655073c884fd97bdbb9f48e01/ruamel.ordereddict-0.4.13-cp27-cp27mu-manylinux1_x86_64.whl
Installing collected packages: ruamel.ordereddict, ruamel.yaml
Found existing installation: ruamel.ordereddict 0.4.13
Uninstalling ruamel.ordereddict-0.4.13:
Successfully uninstalled ruamel.ordereddict-0.4.13
Found existing installation: ruamel.yaml 0.15.55
Uninstalling ruamel.yaml-0.15.55:
Successfully uninstalled ruamel.yaml-0.15.55
Successfully installed ruamel.ordereddict-0.4.13 ruamel.yaml-0.15.57
I using Ubuntu 14.04.4 LTS (for various reasons I cannot upgrade).
Any suggestion will be greatly appreciated.
PyBOMBS could not cope with recent changes in ruamel.yaml where CommentedMap and CommentedSeq were no longer subclasses of dict resp list but of the abstract baseclasses MutableMapping resp. MutableSequence (from collections.abc). PyBombs actually checked using isinstance() against dict resp. list.
Installing an older version of ruamel.yaml (anything <= 0.15.52) should work, you can do so with pip install 'ruamel.yaml<=0.15.52'
In the mean time a PR was submitted and merge to fix this issue, so make sure you run with the latest version of PyBOMBS checked out from github.
It is likely that your config.yml was truncated, so make sure you restore/regenerate that one.

Error installing libapache2-mod-rpaf for detecting IPs using nginx as a reverse proxy

I've seen here, that I should install mod_rpaf so that Apache will be able to detect the IPs coming from the Nginx reverse proxy.
Even seen that there is a bug on Ununtu SO, and it should be solved changing the first line on /etc/apache2/mods-available/rpaf.conf.
The first line contains <IfModule rpaf_module>. I've tried replacing with <IfModule mod_rpaf-2.0.c>, but nothing changes when I restart the Apache.
This is the output when I try to install this module:
apt-get install libapache2-mod-rpaf
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
libapache2-mod-rpaf
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/9,126 B of archives.
After this operation, 67.6 kB of additional disk space will be used.
Selecting previously unselected package libapache2-mod-rpaf.
(Reading database ... 28714 files and directories currently installed.)
Unpacking libapache2-mod-rpaf (from .../libapache2-mod-rpaf_0.6-2ubuntu1_amd64.deb) ...
Setting up libapache2-mod-rpaf (0.6-2ubuntu1) ...
Enabling module rpaf.
* Restarting web server apache2 [fail]
* The apache2 configtest failed.
Output of config test was:
apache2: Syntax error on line 140 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/rpaf.load: API module structure 'rpaf_module' in file /usr/lib/apache2/modules/mod_rpaf.so is garbled - expected signature 41503234 but saw 41503232 - perhaps this is not an Apache module DSO, or was compiled for a different Apache version?
Action 'configtest' failed.
The Apache error log may have more information.
invoke-rc.d: initscript apache2, action "restart" failed.
dpkg: error processing libapache2-mod-rpaf (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
libapache2-mod-rpaf
E: Sub-process /usr/bin/dpkg returned an error code (1)
Per the error, you already have a version of /usr/lib/apache2/modules/mod_rpaf.so on your system with a filehash that differs from what the package config is expecting (expected signature 41503234 but saw 41503232), so you either need to:
1) Work out what installed the conflicting version and uninstall it:
dpkg -S /usr/lib/apache2/modules/mod_rpaf.so
apt-get --purge remove <package>
Or if manually added:
mv /usr/lib/apache2/modules/mod_rpaf.so /usr/lib/apache2/modules/mod_rpaf.so.bak
apt-get install libapache2-mod-rpaf
This may break other stuff.
2) Manually download, re-compile and install from source, should solve the immediate issue, but you'll hit the same conflicting signature issue if you try to update via apt.
3) Unpack the deb file, hack the hash in the config, repackage and install.