The standard format of the file is:
Host example
HostName example.com
Port 2222
Host example2
Hostname two.example.com
Host three.example.com
Port 4444
Knowing the host's short name, I want to remove an entire entry in order to re-add it with new details.
The closest I have got is this, except I need to keep the following Host declaration and the second search term will capture two many terms (like HostName):
sed '/Host example/,/\nHost/d'
With GNU sed:
host="example"
sed 's/^Host/\n&/' file | sed '/^Host '"$host"'$/,/^$/d;/^$/d'
Output:
Host example2
Hostname two.example.com
Host three.example.com
Port 4444
s/^Host/\n&/: Insert a newline before every line that begins with "Host"
/^Host '"$host"'$/,/^$/d: delete all lines matching "Host $host" to next empty line
/^$/d: clean up: delete every empty line
My final solution (which is somewhat OSX orientated) was this. It's largely based on Cyrus' earlier answer.
sed < $SOURCE "/^$/d;s/Host /$NL&/" | sed '/^Host '"$HOST"'$/,/^$/d;' > $SOURCE
This is more resilient to HostName directives which aren't indented.
Related
I often use this to check website access logs by IP address. The problem is that it only includes IPV4 and not IPV6.
Any idea what regex I can use so that it includes (or runs a separate) command for IPV6?
cat access.log | sed -e 's/^\([[:digit:]\.]*\).*"\(.*\)"$/\1 \2/' | sort -n | uniq -c | sort -nr | head -50
Matching IP addresses via regular expressions can be tricky - yours matches lots of things that aren't valid IPv4 addresses, like 100000.55, for example.
There's a perl module, Regexp::Common that provides well tested regular expressions for matching all sorts of things, including both IPv4 and IPv6 addresses. If you install it (The Ubuntu package is libregexp-common-perl), you can replace the sed part of that pipeline with
perl -MRegexp::Common=net -lne '/^($RE{net}{IPv4}|$RE{net}{IPv6}).*"(.*)"$/ && print "$1 $2"'
to match both address families.
Imagine we have dynamic number of hosts of pattern
test-1.mydomain.com
test-2.mydomain.com
test-3.mydomain.com
...
test-n.mydomain.com
I'd like to ssh on each of those machines by not using full name
ex. ssh test-7.mydomain.com
but simply by doing
ssh test-7
Is there a way to use ssh config to do pattern like aliases like this??
Yes!
Yes, there is a way to do this directly in your ssh configuration.
host allows PATTERNS
hostname allows TOKENS
How:
Add this snippet to your ssh configuration ~/.ssh/config:
# all test-* hosts should match to test-*.mydomain.example.com
host test-*
hostname %h.mydomain.example.com
Verify:
You can verify this with the -G flag to ssh. If your ssh is older and does not support -G you can try parsing verbose ssh output.
# if your ssh supports -G
% ssh test-17 -G | grep hostname
hostname test-17.mydomain.example.com
# if your ssh does not support -G
% ssh -v -v test-n blarg >/dev/null 2>&1 | grep resolv
debug2: resolving "test-n.mydomain.example.com" port 22
ssh: Could not resolve hostname test-n.mydomain.example.com: Name or service not known
Notes:
Ssh uses the first host line that matches. It is good practice to add your PATTERN host lines at the bottom of your configuration file.
If your test-n name patterns contain only a single character suffix, then you can use a ? pattern to make a less greedy match. test-? will match test-1 test-a test-z but not test-10
You can create a ssh config file and pre setting your servers.
See this tutorial, i hope it helps you!
ssh config
You can also create a function in your bash file for ssh access.
Like this:
function ssh_test () {
[[ $1 =~ ^('test-1'|'test-2'|'test-3')$ ]] || { echo 'Not a valid value !!' && return ;}
domain=$1.mydomain.com
ssh my_user#"$domain"
}
If it is an option for you, you could add a search domain in the resolv.conf file (I'm assuming you are on Linux).
You would need to add a line like this:
search mydomain.com
which will have SSH (and most other apps) look for test-n, then test-n.mydomain.com.
If you are not managing the resolv.conf file yourself (if you use systemd-networkd or NetworkManager for example), you will have to ajust the search domains in their configuration files).
I have two files as follows
config:
which [box-address]
Command ssh user[box2-address]
address:
192.168.0.10
I want to replace in place the [box-address] in "config" with the contents from "address"
I am able to insert the content but overwrite the entire config file each time so instead of
which 192.168.0.10
Command ssh user 192.168.0.20
I end up with
192.168.0.10
as the only thing in the file
What am I doing wrong
awk '/\[box-address\]/{system("cat config");next}1' address > config
You're providing address, not config, as the input to awk. It will never match /\[box-address\]/. Your command is effectively just overwriting config with the content of address.
The following approach should meet your requirements, it reads the content of address into a variable in the BEGIN block and then replaces any instances of [box-address] with it:
awk 'BEGIN{getline l < "address"}/\[box-address\]/{gsub(/\[box-address\]/,l)}1' config
Result:
which 192.168.0.10
Command ssh user[box2-address]
2I wanted to get a host list of linux servers on my network using nmap, but I'm not sure how to filter the results based on multiple strings. So I'm using something like:
nmap -sT -R --dns-servers 192.168.1.1,192.168.1.2 -p 22 192.168.1.0/24
which gives me the results but of course it's also showing every IP with the closed ports. How can I use logic with awk so that if it finds an IP with a hostname AND finds that the lines below have 'tcp open' it will spit those two lines to an output file?
The typical input to use from nmap would look like:
Interesting ports on server.domain.com (192.168.1.1):
PORT STATE SERVICE
22/tcp open ssh
Interesting ports on server2.domain.com (192.168.1.2):
PORT STATE SERVICE
22/tcp closed ssh
but this would be for every single IP that is scanned, so I'm trying to find a way to have a file with only the hostname and IP "if" it shows as 'tcp open' from the nmap command above. Basically giving you a file with:
server.domain.com (192.168.1.1)
So that I only get the server name and IP if SSH is actually listening on port 22
Based on the sample input and the sample output posted, something like this could work:
awk '/^Interesting/ {
domain = $(NF-1)
ip = substr($NF, 0, length($NF) - 1)
}
$2 == "open" {print domain, ip}' file
This works like this:
If a line starts with "Interesting", then the second to last field is stored as the domain, and the last field (minus the trailing colon) is stored as the ip.
If it finds a line where the second token is "open", then it prints the domain and ip found previously.
I use a cluster of about 30 machines that have all recently been reconfigured with new OpenSSH host keys. When I try to log into one, I get this error message (many lines removed for brevity):
# WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! #
The fingerprint for the RSA key sent by the remote host is
52:bb:71:83:7e:d0:e2:66:92:0e:10:78:cf:a6:41:49.
Add correct host key in /home/nr/.ssh/known_hosts to get rid of this message.
Offending key in /home/nr/.ssh/known_hosts:50
I can go remove the offending line manually, in which case I get a different complaint about the IP addresss, which requires removing another line manually, and I have no desire to repeat this exercise 29 times. I would like to write a program to do it. Unfortunately, the line in the .ssh file no longer contains the host name and IP address in clear text, as it did in earlier versions.
So here's my question:
Given a host name and an IP address, how can I write a program to find out which lines of my ~/.ssh/known_hosts store an SSH host key for that host or IP address?
If I can recover this info, I think I can do the rest myself.
Footnote: I would prefer to code in bash/ksh/sh or C or Lua; my Perl and Python are very rusty.
Clarifications:
I don't want to remove the whole file and repopulate it; it contains over a hundred validated keys that I prefer not to re-validate.
Whether I maintain a single master copy or multiple replicas, the problem of scrubbing away a large group of obsolete host keys remains.
Answer
Here's the Lua script I wrote using ssh-keygen -F:
#!/usr/bin/env lua
require 'osutil'
require 'ioutil'
local known = os.getenv 'HOME' .. '/.ssh/known_hosts'
local function lines(name)
local lines = { }
for l in io.lines(name) do
table.insert(lines, l)
end
return lines
end
local function remove_line(host)
local f = io.popen('ssh-keygen -F ' .. os.quote(host))
for l in f:lines() do
local line = l:match '^# Host %S+ found: line (%d+) type %u+$'
if line then
local thelines = lines(known)
table.remove(thelines, assert(tonumber(line)))
table.insert(thelines, '')
io.set_contents(known, table.concat(thelines, '\n'))
return
end
end
io.stderr:write('Host ', host, ' not found in ', known, '\n')
end
for _, host in ipairs(arg) do
local ip = os.capture('ipaddress ' .. host)
remove_line(host)
remove_line(ip)
end
ssh-keygen -R hostname
ssh-keygen -R ipaddress
personally I scrub the IP addresses with a loop and perl, and remove the conflicts by hand.
$!/usr/bin/perl
for (1..30){
`ssh keygen -R 192.168.0.$_`; #note: backticks arent apostrophies
}
If I want to find out on what line the entry for a host lives,
ssh-keygen -F hostname
The same trick works with IP addresses.
touch and edit "clearkey.sh" or what ever name makes you happy.
#! /bin/bash
# $1 is the first argument supplied after calling the script
sed -i "$1d" ~/.ssh/known_hosts
echo "Deleted line $1 from known_hosts file"
Should be able to do "clearkey.sh 3" and it will delete the offending line!
I usually do the following in bash script checkssh to automatically remove the line:
#!/bin/bash
# Path to "known_hosts" file
KH=~/.ssh/known_hosts
# Find the host in the file, showing line number
L=`grep -i -n $1 $KH`
# If line is not found, exit
[[ $? -ne 0 ]] && exit
# Isolate line number
L=`echo $L | cut -f 1 -d :`
sed -i "${L}d" $KH
You can add ssh $1 exit at the end to automatically re-create an entry in the file, if your ssh is configured to do so.
Call it like checkssh <hostname>.
You might like to try the following when script writing:
declare CHANGED_HOST_NAME="host.yourpublic.work";
declare CHANGED_HOST_IP=$(dig +short ${CHANGED_HOST_NAME});
# Remove old IP address if found
[ -z ${CHANGED_HOST_IP} ] || ssh-keygen -R ${CHANGED_HOST_IP};
# Remove old host key
ssh-keygen -R ${CHANGED_HOST_NAME};
# Add new host key
ssh-keyscan ${CHANGED_HOST_NAME} >> $HOME/.ssh/known_hosts;
Big thanks to #Storm Knight (#289844)