When I execute my cp-sshkey.yml playbook (logged in as myself, not the vagrant user) from my top-level Vagrantfile directory...
ansible-playbook cp-sshkey.yml
I'm getting this error:
TASK: [authorized_key user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}"] ***
fatal: [web1] => Failed to template user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}": could not locate file in lookup: ./files/id_rsa_vagrant.pub
I don't understand why this error is occurring. It's a very simple playbook and the public key file is where I say it is:
.
├── .vagrant
│ └── machines
├── Vagrantfile
├── ansible.cfg
├── bootstrap-mgmt.sh
├── files
│ └── id_rsa_vagrant.pub
├── inventory.ini
├── secrets.yml
├── site.yml
├── website
└── cp-sshkey.yml
Here's my config and host files and the playbook:
# ansible.cfg
[defaults]
hostfile = inventory.ini
remote_user = vagrant
private_key_file = .vagrant/machines/default/virtualbox/private_key
host_key_checking = False
# inventory.ini
[local]
localhost ansible_connection=local
[web]
web1 ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222
# cp-sshkey.yml
- name: Install vagrant's public key on VM
hosts: web1
sudo: True
tasks:
- authorized_key: user=vagrant key="{{ lookup('file', './files/id_rsa_vagrant.pub') }}"
What am I doing wrong here? Thanks.
Quick answer will refine - am playing with this myself but just learning:
I assume you are trying to add your public key (or other key on your ansible console that is no related to the vagrant keys ) to the vagrant machine to allow you to ssh into it without vagrant ssh
I assume that you have checked all the file permissions etc and that you aren't juggling multiple instances. Tried with 127.0.0.1 and localhost and that you've tried with the full file path instead of relative to working directory - my examples use files in subfolders with templates although not in the working snippet below.
Are you able to vagrant ssh
and perhaps check the .ssh/authorized_keys file ?
Are you able to confirm that ansible can connect doing something like ansible web -a df
are you able to ssh into the Vagrant machine using ssh -i .vagrant/machines/default/virtualbox/private_key vagrant#127.0.0.1 -p 2222
In my role task file I have this task.
- name: Copy origin public key to auth keys
authorized_key: user=vagrant key="{{ lookup('file', lookup('env','HOME') + '/.ssh/id_rsa.pub') }}"
Also my host definition has the user:
web1 ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222 ansible_ssh_user=vagrant
but assume that the config you use should work.
This play worked is working for me although my directory structure is different - I expect you're comfortable that your is fine.
Some things to watch out for that caught me:
when you rebuild the Vagrant machine you will need to flush out your .ssh/known_hosts if you have already ssh'd - can remove from known_hosts with sh-keygen -R [localhost]:2222
makes me uneasy seeing localhost as a machine tag
My Setup:
ansible 2.2.0.0
vagrant Version: 1.9.1
Mac OSX
VBox 5.1.6
Vagrant Instance - ubuntu/trusty64
Related
My corporate firewall policy allows only 20 connections per minute 60 seconds between the same source and destinations.
Owing to this the ansible play hangs after a while.
I would like multiple tasks to use the same ssh session rather than creating new sessions. For this purpose i set the below pipelining = True in the local folder ansible.cfg as well as in the command line.
cat /opt/automation/startservices/ansible.cfg
[defaults]
host_key_checking = False
gathering = smart
[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=600s
control_path = %(directory)s/%%h-%%r
pipelining = True
ANSIBLE_SSH_PIPELINING=0 ansible-playbook -i /opt/automation/startservices/finalallmw.hosts /opt/automation/startservices/va_action.yml -e '{ dest_host: myremotehost7 }' -e dest_user=oracle
The playbook is too big to be shared here but it is this task which loops and this is where it hangs due to more than 20 ssh connections in 60 seconds.
- name: Copying from "{{ inventory_hostname }}" to this ansible server.
synchronize:
src: "{{ item.path }}"
dest: "{{ playbook_dir }}/homedirbackup/{{ inventory_hostname }}/{{ dtime }}/"
mode: pull
copy_links: yes
with_items:
- "{{ to_copy.files }}"
With the pipelining settings set, my play still hangs after 20 connections.
Below are the playbook settings:
hosts: "{{ groups['dest_nodes'] | default(groups['all']) }}"
user: "{{ USER | default(dest_user) }}"
any_errors_fatal: True
gather_facts: false
tags: always
vars:
ansible_host_key_checking: false
ansible_ssh_extra_args: -o StrictHostKeyChecking=no -o ConnectionAttempts=5
Post suggestions this far on this thread the issue persists. Below is my local directory ansible.cfg
$ cat /opt/automation/startservices/ansible.cfg
# config file for ansible -- http://ansible.com/
# ==============================================
# nearly all parameters can be overridden in ansible-playbook
# or with command line flags. ansible will read ANSIBLE_CONFIG,
# ansible.cfg in the current working directory, .ansible.cfg in
# the home directory or /etc/ansible/ansible.cfg, whichever it
# finds first
[defaults]
host_key_checking = False
roles_path = roles/
gathering = smart
[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=1200s -o ControlPath=~/.ansible/cp/%r#%h:%p
[persistent_connection]
control_path_dir = ~/.ansible/cp
$
Can you please suggest any solution to the issue on the ansible side where all tasks use the same ssh session and is pipelining not working here?
First: pipelining = True does not do what you are looking for. It reduces the number of network operations, but not the number of ssh connections. Check the docs for more information.
Imho, it is still a good thing to use, as it will speed up your playbooks.
What you want to use is the "persistent control mode" which is a feature of OpenSSH to keep a connection open.
You could for example do this in your ansible.cfg:
[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=1200
This will keep the connection open for 1200 seconds.
The problem is not with ansible controller running the module(s) (i.e. copying the necessary temporary AnsibleZ files to your target and execute them - you already have the correct options for that in ansible.cfg to use master sessions) but with the synchronize module itself which needs to spawn its own ssh connections to transfer files between relevant servers while it is running on the target.
The latest synchronize module version is now part of the ansible.posix collection and has recently gained 2 options that will help you work around your problem to apply the use of master sessions to the module itself while using rsync.
ssh_multiplexing: yes
use_ssh_args: yes
Although it is possible to install this collection in ansible 2.9 to override the older stock module version (which does not have those options), I strongly suggest you use ansible version 2.10 or 2.11. My personal preferred installation method for ansible is through pip as it will let you install any ansible version on any OS for any user in any number of (virtual) environment.
Regarding pip, the versioning has changed (and is quite a mess IMO...)
ansible is now a meta package with its own independent versioning.
the usual ansible binaries (ansible, ansible-playbook, ....) are packaged in ansible-core which has the version the corresponding to what you get when running ansible -v
the meta ansible package installs a set of collections by default (including the ansible.posix one if I'm not wrong)
=> To get ansible -v => 2.10 you want to install ansible pip package 3.x
=> To get ansible -v => 2.11 you want to install ansible pip package 4.x
You will have to remove any previous version installed via pip in your current environment before proceeding.
Question
Is it possible to use a socks tunnel (or any other form of tunnel on port 80 or 443) to control the local machine that is creating the tunnel from the remote machine? Basically, a ssh -R [...] when ssh is not an option and only TCP connection on port 80 and 443 are possible?
Concrete scenario
Due to a very restrictive security policy of one of our customers, we currently have to connect to a Windows jump host without the ability to copy-and-paste stuff there. From there, we download needed files via web browser and copy via ssh to the target machine, or use ssh directly to do maintenance work on the target machine. However, this workflow is time-consuming, and honestly quite annoying.
Unfortunately, the firewall seems to be able to distinguish between real HTTP traffic and ssh as opening instructing sshd on our server to accept connections on 443 did not work.
Firewall
(HTTP only)
┌──────────────┐
│ │
│ ┌─────────┐ │ ??? ┌──────────┐
│ │Jumphost ├─┼───────►│Our Server│
│ │(Windows)│ │ └───▲──────┘
│ └──┬──────┘ │ │
│ │ssh │ │ssh
│ │ │ │
│ ┌─▼─────┐ │ ┌───┴─────┐
│ │Target │ │ │Developer│
│ │(Linux)│ │ │Machine │
│ └───────┘ │ └─────────┘
│ │
└──────────────┘
Any hints are highly appreciated 👍🏻
The problem seems to be a firewall with deep packet inspection.
You can overcome it with using ssh over ssl, using stunnel or openssl.
From the windows box you can tunnel with a stunnel client to our-server stunnel server.
That encapsulate all (ssh) data into ssl therefore there is no difference to a HTTPS connection.
Another option could be ptunnel-ng, it supports a tcp connection over ICMP (ping).
Most firewalls ignores ICMP, if you can ping your our-server this should work, too.
But ptunnel-ng seems sometimes a bit unstable.
If you can't install/execute programs on the windows jumbBox, you can open ports, redirect them by ssh and use them directly by the target-linux.
On your windows jumpbox:
ssh target -R target:7070:our-server:443
On the target (linux) you can use localhost:7070 to connect to our-server:443
I would recommend to use docker for the client and server parts.
I only can't use the ptunnel server inside a container, probably because of the required privileges.
Using ptunnel
On the server
The ptunnel binary is build inside docker, but used by the host directly
This sample expects an ubuntu server
Dockerfile.server
FROM ubuntu:latest
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y build-essential autoconf automake git
RUN mkdir -p /workdir
WORKDIR /workdir
RUN git clone https://github.com/lnslbrty/ptunnel-ng.git && cd ptunnel-ng && ./autogen.sh
start-server.sh
#!/bin/sh
# Starts the icmp tunnel server, this doesn't work inside a docker container
# Or perhaps it works, but I dont't know how
script_dir=$(cd "$(dirname "$0")"; pwd)
if [ ! -f $script_dir/ptunnel-ng ]; then
# Build the ptunnel binary and copy it to the host
docker build -t ptunnel-ng-build-server -f $script_dir/Dockerfile.server $script_dir
docker run --rm -v $script_dir:/shared ptunnel-ng-build-server cp /workdir/ptunnel-ng/src/ptunnel-ng /shared
fi
magic=${1-123456}
sudo $script_dir/ptunnel-ng --magic $magic
On the client
FROM alpine:latest as builder
ARG DEBIAN_FRONTEND=noninteractive
RUN apk add --update alpine-sdk bash autoconf automake git
RUN mkdir -p /workdir
WORKDIR /workdir
RUN git clone https://github.com/lnslbrty/ptunnel-ng.git && cd ptunnel-ng && ./autogen.sh
FROM alpine:latest
WORKDIR /workdir
COPY --from=builder /workdir/ptunnel-ng/src/ptunnel-ng .
start-client.sh
#!/bin/sh
image=ptunnel-ng
if ! docker inspect $image > /dev/null 2> /dev/null; then
docker build -t $image .
fi
magic=${1-123456}
ptunnel_host=${2-myserver.de}
port=${3-2001}
docker run --rm --detach -ti --name 'ptunnel1' -v $PWD:/shared -p 2222:2222 $image //workdir/ptunnel-ng --magic ${magic} -p${ptunnel_host} -l${port}
If you try to run the ptunnel client on termux, this can be done, but requires some small code changes
I'm developing a website on a totally offline environment. also, I use gitlab runner for CI and the host is CentOS 7.
the problem is that gitlab runner uses gitlab-runner user on centos for deploying laravel application and apache uses apache user for running laravel.
I got Permission denied error on apache til I changed ownership of files. after that I get this error on apache log:
Uncaught UnexpectedValueException: The stream or file "storage/logs/laravel.log" could not be opened: failed to open stream: Permission denied
it seems that some vendor libraries like monolog want to write error or debug logs onto storage/logs/laravel.log but it gets permission denied. :(
.gitlab-ci.yml
stages:
- build
- test
- deploy
buildBash:
stage: build
script:
- bash build.sh
testBash:
stage: test
script:
- bash test.sh
deployBash:
stage: deploy
script:
- sudo bash deploy.sh
build.sh
#!/bin/bash
set -xe
# creating env file from production file
cp .env.production .env
# initializing laravel
php artisan key:generate
php artisan config:cache
# database migration
php artisan migrate --force
deploy.sh
#!/bin/bash
PWD=$(pwd)'/public'
STG=$(pwd)'/storage'
ln -s $PWD /var/www/html/public
chown apache.apache -R /var/www/html/public
chmod -R 755 /var/www/html/public
chmod -R 775 $STG
Am I using gitlab runner correct? how can I fix the permission denied error?
SELinux
I found the problem and it was selinux, like always it was selinux and I ignored it at the begining
What's the problem:
you can see selinux context on files with ls -lZ command, by default all files on www are httpd_sys_content_t, the problem is that selinux just allow apache to read these files. you should change storage and bootstrap/cache context so it can be writable.
there are 4 apache context type:
httpd_sys_content_t: read-only directories and files
httpd_sys_rw_content_t: readable and writable directories and files used by Apache
httpd_log_t: used by Apache for log files and directories
httpd_cache_t: used by Apache for cache files and directories
What to do:
first of all install policycoreutils-python for better commands
yum install -y policycoreutils-python
after installing policycoreutils-python the semanage command is available, so you can change file context like this:
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/storage(/.*)?"
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/bootstrap/cache(/.*)?"
don't forget to commit the changes by this command:
restorecon -Rv /var/www/html/laravel/storage
restorecon -Rv /var/www/html/laravel/bootstrap/cache
the problem is solved :)
ref: http://www.serverlab.ca/tutorials/linux/web-servers-linux/configuring-selinux-policies-for-apache-web-servers/
I would like to provision with my three nodes from the last one by using Ansible.
My host machine is Windows 10.
My Vagrantfile looks like:
Vagrant.configure("2") do |config|
(1..3).each do |index|
config.vm.define "node#{index}" do |node|
node.vm.box = "ubuntu"
node.vm.box = "../boxes/ubuntu_base.box"
node.vm.network :private_network, ip: "192.168.10.#{10 + index}"
if index == 3
node.vm.provision :setup, type: :ansible_local do |ansible|
ansible.playbook = "playbook.yml"
ansible.provisioning_path = "/vagrant/ansible"
ansible.inventory_path = "/vagrant/ansible/hosts"
ansible.limit = :all
ansible.install_mode = :pip
ansible.version = "2.0"
end
end
end
end
end
My playbook looks like:
---
# my little playbook
- name: My little playbook
hosts: webservers
gather_facts: false
roles:
- create_user
My hosts file looks like:
[webservers]
192.168.10.11
192.168.10.12
[dbservers]
192.168.10.11
192.168.10.13
[all:vars]
ansible_connection=ssh
ansible_ssh_user=vagrant
ansible_ssh_pass=vagrant
After executing vagrant up --provision I got the following error:
Bringing machine 'node1' up with 'virtualbox' provider...
Bringing machine 'node2' up with 'virtualbox' provider...
Bringing machine 'node3' up with 'virtualbox' provider...
==> node3: Running provisioner: setup (ansible_local)...
node3: Running ansible-playbook...
PLAY [My little playbook] ******************************************************
TASK [create_user : Create group] **********************************************
fatal: [192.168.10.11]: FAILED! => {"failed": true, "msg": "ERROR! Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."}
fatal: [192.168.10.12]: FAILED! => {"failed": true, "msg": "ERROR! Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."}
PLAY RECAP *********************************************************************
192.168.10.11 : ok=0 changed=0 unreachable=0 failed=1
192.168.10.12 : ok=0 changed=0 unreachable=0 failed=1
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
I extended my Vagrantfile with ansible.limit = :all and added [all:vars] to the hostfile, but still cannot get through the error.
Has anyone encountered the same issue?
Create a file ansible/ansible.cfg in your project directory (i.e. ansible.cfg in the provisioning_path on the target) with the following contents:
[defaults]
host_key_checking = false
provided that your Vagrant box has sshpass already installed - it's unclear, because the error message in your question suggests it was installed (otherwise it would be "ERROR! to use the 'ssh' connection type with passwords, you must install the sshpass program"), but in your answer you add it explicitly (sudo apt-get install sshpass), like it was not
I'm using Ansible version 2.6.2 and solution with host_key_checking = false doesn't work.
Adding environment variable export ANSIBLE_HOST_KEY_CHECKING=False skipping fingerprint check.
This error can also be solved by simply export ANSIBLE_HOST_KEY_CHECKING variable.
export ANSIBLE_HOST_KEY_CHECKING=False
source: https://github.com/ansible/ansible/issues/9442
This SO post gave the answer.
I just extended the known_hosts file on the machine that is responsible for the provisioning like this:
Snippet from my modified Vagrantfile:
...
if index == 3
node.vm.provision :pre, type: :shell, path: "install.sh"
node.vm.provision :setup, type: :ansible_local do |ansible|
...
My install.sh looks like:
# add web/database hosts to known_hosts (IP is defined in Vagrantfile)
ssh-keyscan -H 192.168.10.11 >> /home/vagrant/.ssh/known_hosts
ssh-keyscan -H 192.168.10.12 >> /home/vagrant/.ssh/known_hosts
ssh-keyscan -H 192.168.10.13 >> /home/vagrant/.ssh/known_hosts
chown vagrant:vagrant /home/vagrant/.ssh/known_hosts
# reload ssh in order to load the known hosts
/etc/init.d/ssh reload
I had a similar challenge when working with Ansible 2.9.6 on Ubuntu 20.04.
When I run the command:
ansible all -m ping -i inventory.txt
I get the error:
target | FAILED! => {
"msg": "Using a SSH password instead of a key is not possible because Host Key checking is enabled and sshpass does not support this. Please add this host's fingerprint to your known_hosts file to manage this host."
}
Here's how I fixed it:
When you install ansible, it creates a file called ansible.cfg, this can be found in the /etc/ansible directory. Simply open the file:
sudo nano /etc/ansible/ansible.cfg
Uncomment this line to disable SSH key host checking
host_key_checking = False
Now save the file and you should be fine now.
Note: You could also try to add the host's fingerprint to your known_hosts file by SSHing into the server from your machine, this prompts you to save the host's fingerprint to your known_hosts file:
promisepreston#ubuntu:~$ ssh myusername#192.168.43.240
The authenticity of host '192.168.43.240 (192.168.43.240)' can't be established.
ECDSA key fingerprint is SHA256:9Zib8lwSOHjA9khFkeEPk9MjOE67YN7qPC4mm/nuZNU.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.43.240' (ECDSA) to the list of known hosts.
myusername#192.168.43.240's password:
Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-53-generic x86_64)
That's all.
I hope this helps
run the below command, it resolved my issue
export ANSIBLE_HOST_KEY_CHECKING=False && ansible-playbook -i
all provided solutions require changes in global config file or adding environment variable what create problems to onboard new people.
Instead you can add following variable to your inventory or host vars
ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
Adding ansible_ssh_common_args='-o StrictHostKeyChecking=no'
to either your inventory
like:
[all:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
[all:children]
servers
[servers]
host1
OR:
[servers]
host1 ansible_ssh_common_args='-o StrictHostKeyChecking=no'
I'm following a tutorial showing how to setup a vagrant vm to practice using ansible.
I have the following directory structure, files, and configuration:
➜ trusty64 tree
.
├── Vagrantfile
├── ansible
│ ├── hosts
│ ├── playbooks
│ └── roles
└── ansible.cfg
3 directories, 3 files
➜ trusty64 cat ansible/hosts
[vagrantboxes]
vagrant ansible_ssh_host=127.0.0.1 ansible_ssh_port=2222
[vagrantboxes:vars]
ansible_ssh_user=vagrant
ansible_ssh_private_key_file=~/.vagrant.d/insecure_private_key
➜ trusty64 cat ansible.cfg
[defaults]
host_key_checking = False
hostfile = ./ansible/hosts
roles_path = ./ansible/roles
➜ trusty64 vagrant ssh-config
Host default
HostName 127.0.0.1
User vagrant
Port 2222
UserKnownHostsFile /dev/null
StrictHostKeyChecking no
PasswordAuthentication no
IdentityFile /Users/max/Desktop/vagrantboxes/trusty64/.vagrant/machines/default/virtualbox/private_key
IdentitiesOnly yes
LogLevel FATAL
But when I try pinging my vm it doesn't work:
➜ trusty64 ansible all -m ping
vagrant | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh.",
"unreachable": true
}
➜ trusty64 ansible all -m ping -u vagrant
vagrant | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh.",
"unreachable": true
}
What am I doing wrong?
Thanks :)
You might also want to put your ansible management node into a vagrant vm itself. This tutorial shows it quite well.
Vagrant changes insecure key on the first VM run.
Change ansible_ssh_private_key_file to the actual key from vagrant ssh-config:
ansible_ssh_private_key_file=/Users/max/Desktop/vagrantboxes/trusty64/.vagrant/machines/default/virtualbox/private_key