Is there some cli tool I can use to validate the contents of known_hosts? Maybe try to ping all the hosts in there and see if I can connect to each?
Probably using either ssh-keygen or ssh-keyscan?
If you have list of all hosts available you can do it like this:
ssh-keyscan -t rsa,dsa -f hosts_list > ~/.ssh/known_hosts_revised
This will generate a new known_hosts_revised which you can make a diff with your current know_hosts to see the differences.
If you don't need to compare it you can simply do ... > ~/.ssh/known_hosts to overwrite it (WARNING: the original known_hosts will be lost!)
The source of information are the OpenBSD man pages for ssh-keyscan(1).
Edit
The hosts_list expected in for:
1.2.3.4,1.2.4.4 name.my.domain,name,n.my.domain,n,1.2.3.4,1.2.4.4
At least for my setup, using ssh-keyscan is impossible due to my extensive ~/.ssh/config file. I use lots of proxy commands, jump hosts, and alternate Hostname declarations.
For example:
# Connect to Tor nodes
Host *.onion
ProxyCommand socat - SOCKS4A:localhost:%h:%p,socksport=9050
# Work jump box
Host bastion
Hostname bastion.work.com
# Office system, e.g. bob.office -> bastion -> bob.work.com
Host *.office
ProxyCommand ssh bastion nc -w600s $(echo "%h" |sed 's/\.office$/work.com/') %p
# Home system, e.g. adam -> home.com -> adam-laptop.local
Host adam
Hostname adam-laptop.local
ProxyJump home.com
None of the above will work.
Here's a script that should work for the rest though:
#!/usr/bin/awk -f
!/^#/ && NF > 2 {
split($1, hosts, ",")
key_type = $2
gsub(/^ssh-/, "", key_type)
gsub(/-.*/, "", key_type)
for (h in hosts) {
p = index(hosts[h], "]:") # [host]:port (supports raw IPv6 hosts)
if (!p && hosts[h] ~ /^[^:]+:[0-9]+$/) p = index(hosts[h], ":") # host:port
if (p > 0) {
port = substr(hosts[h], p + 2)
gsub("\[|\]?:" port, "", hosts[h])
} else {
port = 22
}
if (seen[key_type,port,hosts[h]]++) next # prevent duplicate lookups
if (port_list[key_type,port]) { comma = "," } else { comma = "" }
port_list[key_type,port] = port_list[key_type,port] comma hosts[h]
}
}
END {
for (tp in port_list) {
split(tp, a, SUBSEP)
system("echo ssh-keyscan -t " a[1] " -p " a[2] " " port_list[tp])
}
}
Remove the echo parts to run once you're convinced this will do what you desire.
This parses non-commented lines and with 3+ fields (since the format is host_list key_type key_hash). It splits the host list since it can be comma-delimited, and further parsing is needed because it can contain ports but ssh-keyscan cannot accept hosts in the format used by known_hosts.
There are two ways a port can be specified:
The old style, which does not work with bare IPv6 addresses, is host:port
The new style, which is required for bare IPv6 addresses, is [host]:port
p is set to the position of ]: if present (the new style). If that string isn't present, we check for the old style and reset p.
If p is positive, we have a port specification. Extract the port and remove (it and the square brackets) from the host name. Otherwise, the port is 22.
Just in case there are duplicate entries, we check for them and continue if we've already seen the type,port,host combination (x++ is false (0) only when first run). Finally, we push the host to a comma-delimited list string in the port_list array as keyed by a tuple of type and port.
After reading in the entire known_hosts file, we iterate on the type,port tuple pairs that key the port_list array, split them into an array named a, and run ssh-keyscan on them.
Run this like awk -f 'this_script.awk' ~/.ssh/known_hosts and if you like the ssh-keyscan commands that it spits out, remove the echo from the system command and re-run.
Do not pipe this output into ~/.ssh/known_hosts! You will want to manually review the results (and probably filter out the comments). Also, you can't redirect output onto one of the files used in the input.
Related
I am looking for a way using ssh-keyscan to possibly define a port within the keyscan file specified with the -f flag instead of having to specify it on the command line.
The following is how I currently do it:
/usr/bin/ssh-keyscan -f /home/ansible/.ssh/test-scanner-keyscan_hosts -p 16005 >> /home/ansible/.ssh/known_hosts;
Contents of the keyscan file:
mainserver1.org,shortname1,shortname2
mainserver2.org,shortname1,shortname2
The issue is, each "mainserver" has a unique ssh port that is different from the others. While this will cause mainserver1 to work, since it's port is 16005, mainserver2 will fail because it's port is 17005. The only way around it currently is to try to do each ssh-keyscan command separately and specifying each different port such that it works.
Instead, I want to be able to specify them within the file, and/or utilize a method that allows for a scanning of a list allowing for unique ports. The issue is, there doesn't seem to be any way to do that.
I tried the following within the keyscan file, and it does not work:
mainserver1.org:16005,shortname1,shortname2
mainserver2.org:17005,shortname1,shortname2
Is there any way to make this work, or any way other than ssh-keyscan, or some other way within ansible to make this function like I hope it does? Otherwise, I have to do an ssh-keyscan task for EVERY server because the ssh ports are all different.
Thanks in advance!
You're actually welcome to use that format, and then use it to drive the actual implementation since ssh-keyscan -f accepts "-" to read from stdin; thus:
scan_em() {
local fn
local port
fn="$1"
for port in $(grep -Eo ':[0-9]+' $fn | sed s/:// | sort | uniq); do
sed -ne "s/:${port}//p" $fn | ssh-keyscan -f - -p $port
done
}
scan_em /home/ansible/.ssh/test-scanner-keyscan_hosts >> /home/ansible/.ssh/known_hosts
I am trying to set up a function on my .zshrc file to create an easy way to do local port forwarding. My goal is starting a Jupyter server on my university's cluster and then using local port forwarding to access it. I have the following on my .zshrc file:
function jptt(){
# Forwards port $1 into port $2 and listens to it
ssh -N -f -L $1:localhost:$2 rgr6291#klc0201.ci.northwestern.edu
}
However, whenever I try to use it, for example with jptt 8888 8888, I get the following error:
Bad local forwarding specification '8888ocalhost:8888'
Am I handling strings or arguments in a wrong way? I can't understand why it isn't able to execute the command. Thanks in advance!
In zsh, you can add history-style expansion operators to the end of a variable expansion. $1:localhost is treated as the expansion $1:l, followed by the text localhost. To prevent this interpretation, use explicit braces to delimit the parameter expansion.
ssh -N -f -L ${1}:localhost:$2 rgr6291#klc0201.ci.northwestern.edu
You can also quote the parameter expansion to separate the : from the expansion syntactically:
ssh -N -f -L "$1":localhost:$2 rgr6291#klc0201.ci.northwestern.edu
:l, by the way, causes the expansion to be converted to lowercase. For example,
% x=FOO
% echo $x
FOO
% echo $x:l
foo
With braces, the operator would go inside the braces, immediately after the parameter name.
% echo ${x}:l
FOO:l
% echo ${x:l}
foo
Let's say.
I have one file with the name of the computer and some other information.
E.g.
Computer1
There's another file with the ip address and some other information.
192.168.100.2
I have 2 greps for example:
grep -i computer /etc/hosts
grep -i ips /etc/hosts
They give me answers like
Computer1
19.168.100.2
Well, I would like to get a file with headers and the information organized as this:
Name
Ip
oser1313
19.168.100.1
I'm quite lost I have no idea how could I format this I usually copy-paste it on Excel but I don't want to do it anymore and since I have to do this on several computers from a server It would be great if I can format it.
Just do something like this:
awk '
{ lc = tolower($0) }
lc ~ /computer/ { name = $0 }
lc ~ /ips/ { ip = $0 }
END {
print "Name", "Ip"
print name, ip
}
' /etc/hosts
The above is untested since you didn't provide a sample input file to test with and it's just mimicing what your grep commands do but there may be a better way to do it if we knew what your input looked like.
I suppose that your two files have the same number of lines and that line numbers match between one file and the other: if oser1313 is line n in the output of grep from /etc/hosts then same for 19.168.100.1 in /etc/hosts.
So it turns pretty simple as bash script:
grep -i computer /etc/hosts > part1.dat
grep -i ips /etc/hosts > part2.dat
echo "Name,IP" > out.dat
paste -d"," part1.dat part2.dat >> out.dat
rm part1.dat part2.dat
Or a oneliner, as suggested in comments:
printf "Name,IP\n$(grep -i computer /etc/hosts),$(grep -i ips /etc/hosts)\n" > out.dat
Imagine we have dynamic number of hosts of pattern
test-1.mydomain.com
test-2.mydomain.com
test-3.mydomain.com
...
test-n.mydomain.com
I'd like to ssh on each of those machines by not using full name
ex. ssh test-7.mydomain.com
but simply by doing
ssh test-7
Is there a way to use ssh config to do pattern like aliases like this??
Yes!
Yes, there is a way to do this directly in your ssh configuration.
host allows PATTERNS
hostname allows TOKENS
How:
Add this snippet to your ssh configuration ~/.ssh/config:
# all test-* hosts should match to test-*.mydomain.example.com
host test-*
hostname %h.mydomain.example.com
Verify:
You can verify this with the -G flag to ssh. If your ssh is older and does not support -G you can try parsing verbose ssh output.
# if your ssh supports -G
% ssh test-17 -G | grep hostname
hostname test-17.mydomain.example.com
# if your ssh does not support -G
% ssh -v -v test-n blarg >/dev/null 2>&1 | grep resolv
debug2: resolving "test-n.mydomain.example.com" port 22
ssh: Could not resolve hostname test-n.mydomain.example.com: Name or service not known
Notes:
Ssh uses the first host line that matches. It is good practice to add your PATTERN host lines at the bottom of your configuration file.
If your test-n name patterns contain only a single character suffix, then you can use a ? pattern to make a less greedy match. test-? will match test-1 test-a test-z but not test-10
You can create a ssh config file and pre setting your servers.
See this tutorial, i hope it helps you!
ssh config
You can also create a function in your bash file for ssh access.
Like this:
function ssh_test () {
[[ $1 =~ ^('test-1'|'test-2'|'test-3')$ ]] || { echo 'Not a valid value !!' && return ;}
domain=$1.mydomain.com
ssh my_user#"$domain"
}
If it is an option for you, you could add a search domain in the resolv.conf file (I'm assuming you are on Linux).
You would need to add a line like this:
search mydomain.com
which will have SSH (and most other apps) look for test-n, then test-n.mydomain.com.
If you are not managing the resolv.conf file yourself (if you use systemd-networkd or NetworkManager for example), you will have to ajust the search domains in their configuration files).
I use a cluster of about 30 machines that have all recently been reconfigured with new OpenSSH host keys. When I try to log into one, I get this error message (many lines removed for brevity):
# WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! #
The fingerprint for the RSA key sent by the remote host is
52:bb:71:83:7e:d0:e2:66:92:0e:10:78:cf:a6:41:49.
Add correct host key in /home/nr/.ssh/known_hosts to get rid of this message.
Offending key in /home/nr/.ssh/known_hosts:50
I can go remove the offending line manually, in which case I get a different complaint about the IP addresss, which requires removing another line manually, and I have no desire to repeat this exercise 29 times. I would like to write a program to do it. Unfortunately, the line in the .ssh file no longer contains the host name and IP address in clear text, as it did in earlier versions.
So here's my question:
Given a host name and an IP address, how can I write a program to find out which lines of my ~/.ssh/known_hosts store an SSH host key for that host or IP address?
If I can recover this info, I think I can do the rest myself.
Footnote: I would prefer to code in bash/ksh/sh or C or Lua; my Perl and Python are very rusty.
Clarifications:
I don't want to remove the whole file and repopulate it; it contains over a hundred validated keys that I prefer not to re-validate.
Whether I maintain a single master copy or multiple replicas, the problem of scrubbing away a large group of obsolete host keys remains.
Answer
Here's the Lua script I wrote using ssh-keygen -F:
#!/usr/bin/env lua
require 'osutil'
require 'ioutil'
local known = os.getenv 'HOME' .. '/.ssh/known_hosts'
local function lines(name)
local lines = { }
for l in io.lines(name) do
table.insert(lines, l)
end
return lines
end
local function remove_line(host)
local f = io.popen('ssh-keygen -F ' .. os.quote(host))
for l in f:lines() do
local line = l:match '^# Host %S+ found: line (%d+) type %u+$'
if line then
local thelines = lines(known)
table.remove(thelines, assert(tonumber(line)))
table.insert(thelines, '')
io.set_contents(known, table.concat(thelines, '\n'))
return
end
end
io.stderr:write('Host ', host, ' not found in ', known, '\n')
end
for _, host in ipairs(arg) do
local ip = os.capture('ipaddress ' .. host)
remove_line(host)
remove_line(ip)
end
ssh-keygen -R hostname
ssh-keygen -R ipaddress
personally I scrub the IP addresses with a loop and perl, and remove the conflicts by hand.
$!/usr/bin/perl
for (1..30){
`ssh keygen -R 192.168.0.$_`; #note: backticks arent apostrophies
}
If I want to find out on what line the entry for a host lives,
ssh-keygen -F hostname
The same trick works with IP addresses.
touch and edit "clearkey.sh" or what ever name makes you happy.
#! /bin/bash
# $1 is the first argument supplied after calling the script
sed -i "$1d" ~/.ssh/known_hosts
echo "Deleted line $1 from known_hosts file"
Should be able to do "clearkey.sh 3" and it will delete the offending line!
I usually do the following in bash script checkssh to automatically remove the line:
#!/bin/bash
# Path to "known_hosts" file
KH=~/.ssh/known_hosts
# Find the host in the file, showing line number
L=`grep -i -n $1 $KH`
# If line is not found, exit
[[ $? -ne 0 ]] && exit
# Isolate line number
L=`echo $L | cut -f 1 -d :`
sed -i "${L}d" $KH
You can add ssh $1 exit at the end to automatically re-create an entry in the file, if your ssh is configured to do so.
Call it like checkssh <hostname>.
You might like to try the following when script writing:
declare CHANGED_HOST_NAME="host.yourpublic.work";
declare CHANGED_HOST_IP=$(dig +short ${CHANGED_HOST_NAME});
# Remove old IP address if found
[ -z ${CHANGED_HOST_IP} ] || ssh-keygen -R ${CHANGED_HOST_IP};
# Remove old host key
ssh-keygen -R ${CHANGED_HOST_NAME};
# Add new host key
ssh-keyscan ${CHANGED_HOST_NAME} >> $HOME/.ssh/known_hosts;
Big thanks to #Storm Knight (#289844)