This question already has answers here:
Make a Bash alias that takes a parameter?
(24 answers)
Closed 7 years ago.
Trying to create:
alias mcd="mkdir $1; cd $1"
Getting:
$ mcd foo
usage: mkdir [-pv] [-m mode] directory ...
-bash: foo: command not found
What am I doing wrong?
An alias can only substitute the first word of a command with some arbitrary text. It can not use parameters.
You can instead use a shell function:
mcd()
{
test -e "$1" || mkdir "$1"
cd "$1"
}
Related
This question already has answers here:
Aborting a shell script if any command returns a non-zero value
(10 answers)
Closed 3 months ago.
I want to write a shell script in which am going to use cmake build:
cmake ..
cmake --build .
After these 2 statements more statements are to be added in the script.
But if the cmake build fails the shell script should exit.
Is there any way to achieve this?
try add set -e in front of the script
bash manual: https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html
but it should also work with sh/zsh
#!/usr/bin/sh
set -e
echo foo
false
echo bar
Seeing very strange behavior where when when using gnu parallel to distribute export jobs using bcp from mssql-tools. It appears that when using the -q option for parallel, strings are interpreted differently on local host than on remote hosts.
Running only as a loop through files on local host, the bcp processes throws no errors
However, distributing the file exports with parallel, the bcp processes executing on the local host throw
/opt/mssql-tools/bin/bcp: unknown option
errors, while those executing on remote hosts (via a --sshloginfile param) finish successfully. The basic code being run looks like...
# setting some vars to pass
TO_SERVER_ODBCDSN="-D -S MyMSSQLServer"
TO_SERVER_IP="-S 172.18.54.22"
DB="$dest_db" #TODO: enforce being more careful with this value
TABLE="$tablename" # MUST exist beforehand, case matters
USER=$(tail -n+1 $source_home/mssql-creds.txt | head -1)
PASSWORD=$(tail -n+2 $source_home/mssql-creds.txt | head -1)
DATAFILES="/some/path/to/files/"
TARGET_GLOB="*.tsv"
RECOMMEDED_IMPORT_MODE='-c' # makes a HUGE difference, see https://stackoverflow.com/a/16310219/8236733
DELIMITER="\\\t" # (currently not used) DO NOT use format like "'\t'", nested quotes seem to cause hard-to-catch error, want "\t" literal
....
bcpexport() {
filename=$1
TO_SERVER_ODBCDSN=$2
DB=$3
TABLE=$4 # MUST exist beforehand, case matters
USER=$5
PASSWORD=$6
RECOMMEDED_IMPORT_MODE=$7 # makes a HUGE difference, see https://stackoverflow.com/a/16310219/8236733
DELIMITER=$8 # not currently used
WORKDIR=$9
LOGDIR=${10}
....
/opt/mssql-tools/bin/bcp "$TABLE" in "$localfile" \
$TO_SERVER_ODBCDSN \
-U $USER -P $PASSWORD \
-d $DB \
$RECOMMEDED_IMPORT_MODE
-t "\t" \
-e ${localfile}.bcperror.log
}
export -f bcpexport
parallelization_pernode=5
parallel -q -j $parallelization_pernode \
--sshloginfile $source_home/parallel-nodes.txt \
--env bcpexport \
bcpexport {} "$TO_SERVER_ODBCDSN" $DB $TABLE $USER $PASSWORD $RECOMMEDED_IMPORT_MODE $DELIMITER $workingdir $logdir \
::: $DATAFILES/$TARGET_GLOB #from hdfs nfs gateway
Looking at the bash interpretation of the processes (by running ps -aux | grep bcp on the hosts that parallelis given in the --sshloginfile) for the remote hosts we see...
/bin/bash -c bcpexport() { ... /opt/mssql-tools/bin/bcp "$TABLE" in "$localfile" $TO_SERVER_ODBCDSN -U $USER -P $PASSWORD -d $DB $RECOMMEDED_IMPORT_MODE; -t "\t" -e ${localfile}.bcperror.log; ...
for the local host, the bash interpretation is...
/bin/bash -c bcpexport() { ... /opt/mssql-tools/bin/bcp "$TABLE" in "$localfile" $TO_SERVER_ODBCDSN -U $USER -P $PASSWORD -d $DB $RECOMMEDED_IMPORT_MODE; -t "\t" -e ${localfile}.bcperror.log; ...
that is, they look the same.
My current thought is that the "\t" in the bcp command is being interpreted in a problematic way. Debugging parallel without vs with the -q option we see...
$ parallel -j 5 --sshloginfile ./parallel-nodes.txt echo "Number {}: Running on \`hostname\`: \t" ::: 1 2 3 4 5
Number 4: Running on HW04.ucera.local: t
Number 1: Running on HW04.ucera.local: t
Number 2: Running on HW03.ucera.local: t
Number 5: Running on HW03.ucera.local: t
Number 3: Running on HW02.ucera.local: t
$ parallel -q -j 5 --sshloginfile ./parallel-nodes.txt echo "Number {}: Running on \`hostname\`: \t" ::: 1 2 3 4 5
Number 1: Running on `hostname`:
Number 4: Running on `hostname`:
Number 3: Running on `hostname`: \t
Number 2: Running on `hostname`: \t
Number 5: Running on `hostname`: \t
The bcp command needs the "\t" literal not the "t" literal (and I suspect several other similar string corruptions (also I do believe that \t is the default for bcp anyway, but this is just an example and want to keep \t for code clarity)), but not sure how to get this for both local and remote nodes or even why this behavior differs by remote vs local.
Basically, need the the strings to be exactly the same for both local and remote hosts even if strings have spaces or escape characters in them (note, I think this used to not be the case (have older script on other machines that don't have this problem))
Not sure if this is counts more as a parallel problem or a bcp problem (currently thinking something is going wrong with the -q option in parallel, but not sure). Anyone have any debugging suggestions or fixes? Ideas of what could be happening?
Firstly, the reason why hostname is not expanded is due to -q. It quotes the ` so that it does not expand.
Secondly, I think what you see is the different behaviours in built-in echo and /bin/echo. Built-in echo depends on the shell. Here I compare echo \\\\t in different shells:
$ parallel --onall --tag -S sh#lo,bash#lo,csh#lo,tcsh#lo,ksh#lo,zsh#lo echo \\\\t ::: a
bash#lo \t a
tcsh#lo a
sh#lo a
ksh#lo \t a
zsh#lo a
csh#lo \t a
That does not, however, get you closer to a solution. If I were you I would use env_parallel to copy the environment variables. And if the login shell on the remote systems are not the same as your shell, then set PARALLEL_SHELL to force using that shell.
So:
#!/bin/bash
env_parallel --session
# setting some vars to pass
TO_SERVER_ODBCDSN="-D -S MyMSSQLServer"
:
:
PARALLEL_SHELL=bash env_parallel -q -j $parallelization_pernode ...
(no need to use neither --env nor 'export -f' when using 'env_parallel --session')
# Cleanup (not needed if this is the last line in the script)
env_parallel --end-session
I need to pass a sql script as an argument to a bash file. My bash file is as below
#!/bin/bash
if [ $# -lt 3 ]; then
echo "Usage: databasename, tablename, script"
exit 1
fi
export DB=$1
export PWD=absc
/research/util/Unload.py -e -d "|" -t $2 -c "MM/DD/YYYY" -s $3
I execute the bash file in this way:
./UnloadToInfx.sh "db_name" "mytable" "select * from mytable where city='LA' and date='05/22/2018'"
It prop up error regrading to $3 which is the sql script. How can I resolve this problem?
Change your script like this (encapsulate bare $1,2,3 references to "$1", "$2" etc.) The reason is that encapsulating them will present them as a unit for the program that you give the argument to.
Otherwise a string with spaces will be presented as multiple arguments and any glob characters like *? will be expanded to filenames in your current directory, which you don't want.
Even so, take care that your arguments don't start with a (-) hyphen, because then they might still be interpreted as option switches .
#!/bin/bash
if [ $# -lt 3 ]; then
echo "Usage: databasename, tablename, script"
exit 1
fi
export DB="$1"
export PWD=absc
/research/util/Unload.py -e -d "|" -t "$2" -c "MM/DD/YYYY" -s "$3"
This question already has answers here:
CMake's execute_process and arbitrary shell scripts
(3 answers)
Closed 5 years ago.
I wanted to extract a string from a header in an execute_process.
But there is a bug with the command and I try a lot of things, always the same error.
execute_process(
COMMAND cat $(version_h) | grep -a "define myVersion " | cut -d " " -f3 | cut -d '"' -f2`
OUTPUT_VARIABLE _Version)
If I write the command in the console line, there is no problem.
The error says: "Parse error. Function missing ending ")". Instead found unterminated string with text "
"
execute_process() only deals with processes and their arguments, i.e. there is no shell involved.
So, you have two main options:
execute_process(COMMAND bash -c "..." OUTPUT_VARIABLE _Version)
execute_process(COMMAND cat ... COMMAND grep ... COMMAND cut ... COMMAND cut ... OUTPUT_VARIABLE _Version)
In the second version, the standard output and standard inputs of the commands chain together.
If you want to do anything more complex, you'll have to create a separate script and invoke it in a process-, not shell-, oriented way, i.e. option 1.
The problem was that I need to remove a quote character and I guess there is a confusion with Cmake and the bash command.
execute_process(COMMAND cat ... COMMAND grep ... COMMAND cut ... COMMAND cut -c2- COMMAND rev COMMAND cut -c2- COMMAND rev OUTPUT_VARIABLE _Version)
This question already has answers here:
Why does sudo change the PATH? [closed]
(18 answers)
Closed 3 years ago.
I'm trying to execute a script from a different server using ssh
Here's the command I'm using from server 1 to launch a query on server 2:
ssh -t user#xxx.xxx.xxx.xx "cd /var/www/vhosts/xxxxx/subdomains/preprod/; sudo ./replace.sh";
but the problem is that when I do sudo the $home = /root while the script is under: /var/www/vhosts/xxxxx/subdomains/preprod/
How can i tell sudo to preserve the environment?
I tried sudo -P - , sudo -H, without any luck.
That's what I got from the man page.
sudo -E
-E The -E (preserve environment) option will override the env_reset
option in sudoers(5)). It is only
available when either the matching
command has the SETENV tag or the
setenv option is set in sudoers(5).