I have a Linux server from which i need to download files(server contains many zipped files for a day) to my local machine that is generated for a particular day. is it possible to do so? please help. Thank you
suggesting 2 scripts
show-files.sh on server
#!/bin/sh
stat -c "%y %n" * | grep $1 | awk '{print $4}' > files.list
copy-files.sh on local machine
#!/bin/sh
ssh $1 ./show-files.sh $2
LIST=`ssh $1 cat files.list`
for f in $LIST
do
scp $1:$f ./
done
on local machine call
./copy-files.sh SERVER_IP 2023-01-13
Related
I am trying to read some files located on a server and write only a certain number of columns from those files onto my local machine. I am tried to do this in a for loop to avoid inputting my password for each file. Below is what I was able to cobble till now.
The following code works but writes all the output to a single file which is not manageable due to its large size.
ssh user#xx.xxx.xxx.xx 'for loc in /hel/insur/*/201701*; do zcat $loc | grep -v NUMBER | awk -F',' -v OFS="," '\''{print $1,$2,$3,$4,$5}'\'' | gzip; done' > /cygdrive/c/Users/user1/Desktop/test/singlefile.csv.gz
So, I tried to write each file individually as shown below but it gives me an error saying that it cannot find the location(possibly because I am sshed ito the remote server).
ssh user#xx.xxx.xxx.xx 'for loc in /hel/insur/*/201701*; do zcat $loc | grep -v NUMBER | awk -F',' -v OFS="," '\''{print $1,$2,$3,$4,$5}'\'' | gzip > /cygdrive/c/Users/user1/Desktop/test/`echo $loc | cut -c84-112` ; done'
Any ideas on how to solve this?
The greppable Nmap output looks like the following:
Host: 9.2.1.100 (hello.world.com) Status: Up
Host: 9.2.1.100 (hello.world.com) Ports: 21/closed/tcp//ftp///, 22/closed/tcp//ssh///,
23/closed/tcp//telnet///, 25/closed/tcp//smtp///, 80/open/tcp//http//Citrix Metaframe ICA Browser/,
110/filtered/tcp//pop3///, 139/open/tcp//netbios-ssn//Microsoft Windows netbios-ssn/,
443/closed/tcp//https///, 445/open/tcp//microsoft-ds//Windows Server 2003 3790 Service Pack 2 microsoft-ds/,
3389/open/tcp//ms-wbt-server//Microsoft Terminal Service/ Seq Index: 256 IP ID Seq: Incremental
My question is how do I use awk or grep to parse the output so that I get the following:
IP Address, Host, Port Status (limited to only open ports), Protocol, Service, and Service Version (if there is one)?
9.2.1.100\thello.world.com\t80\topen\ttcp\thttp\tCitrix Metaframe ICA Browser\n
9.2.1.100\thello.world.com\t139\topen\ttcp\tnetbios-ssn\tMicrosoft Windows netbios-ssn\n
...
There are a million different ways to parse this output, and this is probably not the prettiest, but I do believe this should work:
grep 'Ports:' $1 | while read -r line
do
ip=$(echo $line | grep -oP '(?<=^Host: )(\d{1,3}\.){3}\d{1,3}')
name=$(echo $line | grep -Po "(?<=^Host: $ip \()[^\)]*")
echo "$line" | grep -Po '(?<=\tPorts: )[^\t]*\t' | sed -e 's/, /\n/g' | sed -e 's%/%\t%g' | cut -d/ -f 1-3,5,7 | sed -e "s/^/$ip\t$name\t/g"
done
in this specific example you have to save this code to a file and execute it with a the name of the file that contains the nmap output as an argument.
Anyway, here you can find the full explanation about the grepable output of the nmap command. They even explain how it should be parsed, but i did not have the time to read it so i just wrote here this little script.
I hope you will find this helpful.
Inspired by this post, I'm attempting to use docker-machine on my Mac to stop a running container. However, awk appears to behave differently in docker-machine ssh than when run directly "in" the container.
When I'm ssh'd to the container directly, everything works as expected:
me#myMac:~$ docker-machine ssh default
docker#default:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
028fd7148881 myrepo/mycontainer "apachectl -DFOREGROU" 14 minutes ago Up 14 minutes 0.0.0.0:80->80/tcp gloomy_mayer
docker#default:~$ docker ps | awk 'NR > 1 {print $1}'
028fd7148881
docker#default:~$ docker ps | awk 'NR > 1 {print $1}' | xargs --no-run-if-empty docker kill
028fd7148881
docker#default:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
But when run remotely, awk behaves differently:
me#myMac:~$ docker-machine ssh default "docker ps"
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7927b10b8a28 myrepo/mycontainer "apachectl -DFOREGROU" 11 seconds ago Up 10 seconds 80/tcp happy_wozniak
me#myMac:~$ docker-machine ssh default "docker ps | awk 'NR > 1 {print $1}'"
7927b10b8a28 myrepo/mycontainer "apachectl -DFOREGROU" About a minute ago Up About a minute 80/tcp happy_wozniak
me#myMac:~$ docker-machine ssh default "docker ps | awk 'NR > 1 {print $1}' | xargs --no-run-if-empty docker kill"
7927b10b8a28
Failed to kill container (myrepo/mycontainer): Error response from daemon: Cannot kill container myrepo/mycontainer: No such container: myRepo/myContainer
Failed to kill container (apachectl -DFOREGROU): Error response from daemon: Cannot kill container apachectl -DFOREGROU: No such container: apachectl -DFOREGROU
Failed to kill container (3): Error response from daemon: Cannot kill container 3: No such container: 3
...
Why does awk behave differently under docker-machine?
"docker ps | awk 'NR > 1 {print \$1}' | xargs --no-run-if-empty docker kill"
$ ==> \$
docker ps -q
I am new to awk and shell based programming. I have a bunch of files name file_0001.dat, file_0002.dat......file_1000.dat. I want to change the file names such as the number after file_ will be a multiple of 4 in comparison to previous file name. SO i want to change
file_0001.dat to file_0004.dat
file_0002.dat to file_0008.dat
and so on.
Can anyone suggest a simple script to do it. I have tried the following but without any success.
#!/bin/bash
a=$(echo $1 sed -e 's:file_::g' -e 's:.dat::g')
b=$(echo "${a}*4" | bc)
shuf file_${a}.dat > file_${b}.dat
This script will do that trick for you:
#!/bin/bash
for i in `ls -r *.dat`; do
a=`echo $i | sed 's/file_//g' | sed 's/\.dat//g'`
almost_b=`bc -l <<< "$a*4"`
b=`printf "%04d" $almost_b`
rename "s/$a/$b/g" $i
done
Files before:
file_0001.dat file_0002.dat
Files after first execution:
file_0004.dat file_0008.dat
Files after second execution:
file_0016.dat file_0032.dat
Here's a pure bash way of doing it (without bc, rename or sed).
#!/bin/bash
for i in $(ls -r *.dat); do
prefix="${i%%_*}_"
oldnum="${i//[^0-9]/}"
newnum="$(printf "%04d" $(( 10#$oldnum * 4 )))"
mv "$i" "${prefix}${newnum}.dat"
done
To test it you can do
mkdir tmp && cd $_
touch file_{0001..1000}.dat
(paste code into convert.sh)
chmod +x convert.sh
./convert.sh
Using bash/sed/find:
files=$(find -name 'file_*.dat' | sort -r)
for file in $files; do
n=$(sed 's/[^_]*_0*\([^.]*\).*/\1/' <<< "$file")
let n*=4
nfile=$(printf "file_%04d.dat" "$n")
mv "$file" "$nfile"
done
ls -r1 | awk -F '[_.]' '{printf "%s %s_%04d.%s\n", $0, $1, 4*$2, $3}' | xargs -n2 mv
ls -r1 list file in reverse order to avoid conflict
the second part will generate new filename. For example: file_0002.dat will become file_0002.dat file_0008.dat
xargs -n2 will pass two arguments every time to mv
This might work for you:
paste <(seq -f'mv file_%04g.dat' 1000) <(seq -f'file_%04g.dat' 4 4 4000) |
sort -r |
sh
This can help:
#!/bin/bash
for i in `cat /path/to/requestedfiles |grep -o '[0-9]*'`; do
count=`bc -l <<< "$i*4"`
echo $count
done
Hi i have the following process which i cant kill:
I am running cygwin in windows xp 32 bit.
I have tried issuing the following commands:
/bin/kill -f 4760
/bin/kill -9 5000
kill -9 5000
kill 5000
When i write /bin/kill -f 4760 i get the message, 'kill: couldn't open pid 4760'.
When i write /bin/kill -9 5000 i get the message, 'kill: 5000: No such process'.
I simply don't understand why this process cant be killed.
Since it has a WINID shouldnt it be killed by /bin/kill -f 4760?
hope someone can help thx :)
The process is locked from Windows most likely. The error you are getting "couldnt open PID XXX" points to this.
To confirm try killing it with windows taskkill
taskkill /PID 4760
Strangely, the following works in Cygwin:
echo PID1 PID2 PID3 | xargs kill -f
For example:
ps -W | grep WindowsPooPoo | awk '{print $1}' | while read line; do echo $line | xargs kill -f; done;
Different Windows programs will handle the signals that kill sends differently; they've never been designed to deal with them in the same way that Linux/Cygwin programs are.
The only reliable method for killing a Windows program is to use a Windows specific tool, such as Task Manager or Process Explorer.
That said, if you've not already, you may have luck with running your Cygwin terminal in administrator mode (right click on your shortcut and select "Run as administrator").
The method presented by #Donal Tobin is correct:
kill -f <pid>
However, I don't need to log in as administrator.
Create a file called killall.sh with this line
ps -W | grep $1 | awk '{print $1}' | while read line; do echo $line | xargs kill -f; done;
Then give it execute permissions.
chmod 777 killall.sh
In your .bash_profile add this line
alias killall="~/killall.sh" (point it to the correct location)
Then you just have to type "killall [name]"
killall.sh - Kill by process name.
#/bin/bash
ps -W | grep "$1" | awk '{print $1}' | xargs kill -f;
Usage:
$ killall <process name>
For me this command does not work on Windows 10 in Cygwin:
$ kill -f 15916
bash: kill: (15916) - No such process
Instead of it, you can use next commands:
$ powershell kill -f 15916
$ netstat -ano | grep ':8080' | awk '{print $5}' | xargs powershell kill -f
$ netstat -ano | grep ':8080' | awk '{print $5}' | while read pid; do powershell kill -f $pid; done;
$ netstat -ano | grep ':8080' | awk '{sub(/\r/,"",$5) ; print $5}' | while read pid; do taskkill /F /PID $pid; done;
SUCCESS: The process with PID 15916 has been terminated.