Modify zsh commands to forward errors - error-handling

I would like to modify one of my recent Bash aliases to forward errors. Here is the alias:
alias makecclip=
"make |& tee >(sed \"s,\x1B\[[0-9;]*[a-zA-Z],,g\" |
egrep \":[0-9]+:[0-9]+: error\" | cut -d : -f1,2,3 |
head -n 1 | xargs -0 echo -n | xclip -selection clipboard &&
xclip -selection clipboard -o)
This code displays the results of a C++ compilation, and then removes formatting and displays and adds to the clipboard the first error location (if there is any).
However, I would like to use this code like this:
makecclip && bin/someexecutablecreated
This though ruins the && operator, since it always runs bin/someexecutablecreated even when there is a compilation error present. How can I add modifications to the code to set the error flag, when the error list (the things saved to clipboard and echoed) is not empty?

You can address your issue by using the PIPESTATUS internal variable (this variable has other names in non-bash shells). This allows to have an history of exit statuses of commands passed by pipe.
You precised in the comments that you didn't use bash, but used zsh instead. As such, some of the syntax of my solution has to be changed, as they handle the PIPESTATUS variable differently.
In bash, you use ${PIPESTATUS[0]}, whereas you'll use ${pipestatus[1]} in zsh.
A first approach, using your existing alias, could be as follow :
makecclip && [ "${pipestatus[1]}" -eq "0" ] && echo "ok"
This runs the echo command only if "${pipestatus[1]}" is equal to 0 (no errors during make)
A more convenient solution would be to use a function instead of an alias for makecclip. In your ~/.bashrc file, you could write :
makecclip () {
make |& tee >(sed "s,\x1B\[[0-9;]*[a-zA-Z],,g" | egrep ":[0-9]+:[0-9]+: error" | cut -d : -f1,2,3 | head -n 1 | xargs -0 echo -n | xclip -selection clipboard && xclip -selection clipboard -o)
return "${pipestatus[1]}"
}
Now, makecclip && echo "ok" will work as expected.
Test cases :
#!/bin/zsh
#do not run this test if there is an existing makefile in your current directory
rm -f makefile
makecclip () {
make |& tee >(sed "s,\x1B\[[0-9;]*[a-zA-Z],,g" | egrep ":[0-9]+:[0-9]+: error" | cut -d : -f1,2,3 | head -n 1 | xargs -0 echo -n | xclip -selection clipboard && xclip -selection clipboard -o)
# this part is only present to check the pipestatus values during the tests.
# In the real function, I wrote 'return ${pipestatus[1]}' instead.
a=(${pipestatus[#]})
echo ${a[#]}
return ${a[1]}
}
echo "# no makefile"
makecclip && echo "ok"
echo -e "\n# empty makefile"
touch makefile
makecclip && echo "ok"
echo -e "\n# dummy makefile entry"
echo -e 'a:\n\t#echo "inside makefile"' > makefile
makecclip && echo "ok"
echo -e "\n# program with error makefile"
echo -e "int main(){error; return 0;}" > target.cc
echo -e 'a:\n\tgcc target.cc' > makefile
makecclip && echo "ok"
Output :
$ ./test.sh
# no makefile
make: *** No targets specified and no makefile found. Stop.
2 0
# empty makefile
make: *** No targets. Stop.
2 0
# dummy makefile entry
inside makefile
0 0
ok
# program with error
gcc target.cc
target.cc: In function ‘int main()’:
target.cc:1:12: error: ‘error’ was not declared in this scope
int main(){error; return 0;}
^
makefile:2: recipe for target 'a' failed
make: *** [a] Error 1
target.cc:1:12
2 0

Related

String manipulation in .gitlab-ci variables

I'm trying to set up my ci file with some variables. I'm able to generate a variable like so;
...
variables:
TARGET_PROJECT_DIR: "${CI_PROJECT_NAME}.git"
However, I don't seem to be able to do this;
...
variables:
PROJECT_PROTOCOL_RELATIVE_URL: "${CI_PROJECT_URL//https:\/\/}.git"
If I run that in bash, I get the expected output which is gitlab.com/my/repo/url.git with the 'https://' removed and the '.git' appended.
My workaround has just been to export it in the 'script' section, but it feels a lot neater to add this to the variables section, since this is part of a template that is being inherited by the actual jobs. Is it possible?
There are several more useful variables defined in the GitLab CI environment.
CI_PROJECT_PATH gives you the <namespace>/<project name> (or just <project name> if you have no extra namespace) string and
CI_SERVER_HOST gives you the server name, so you could do
variables:
PROJECT_PROTOCOL_RELATIVE_URL: ${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git
I have similar setups (also without quotes).
I'm not sure if that will work for you, since my runners and my server are under my control and I don't run pipelines with external projects.
But you can get all available variables displayed in the job log by running a job like this:
stages:
- env
show-env:
stage: env
script:
- env
Also always helpful is https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
After looking around for similar challenges I found your not answered question. Here are my suggestions:
stages:
- todo
todo-job:
stage: todo
only:
- master
script:
#your question / example
- echo ${CI_PROJECT_URL}
- echo ${CI_PROJECT_URL:8:100}.git
#Because you have the word manipulation in the title, I have some more examples:
#Return substring between the two '_'
- INPUT="someletters_12345_moreleters.ext"
- SUBSTRING=`expr match "$INPUT" '.*_\([[:digit:]]*\)_.*' `
- echo $SUBSTRING
#Store a substring in a new variable and create an output
- b=${INPUT:12:5}
- echo $b
#Substring using grep with regex (more readable)
- your_number=$(echo "someletters_12345_moreleters.ext" | grep -E -o '[0-9]{5}')
- echo $your_number
#Substring using variable and 'grep' with regex (more readable)
- your_number=$(echo "$INPUT" | grep -E -o '[0-9]{5}')
- echo $your_number
#split a string and return a part using 'cut'
- your_id=$(echo "Release V14_TEST-42" | cut -d "_" -f2 )
- echo $your_id
#split the string of a variable and return a part using 'cut'
- VAR="Release V14_TEST-42"
- your_number=$(echo "$VAR" | cut -d "_" -f2 )
- echo $your_number
Gitlab output looks like:
$ echo ${CI_PROJECT_URL}
https://gitlab.com/XXXXXXXXXX/gitlab_related_projects/test
$ echo ${CI_PROJECT_URL:8:100}.git
gitlab.com/XXXXXXXXXX/gitlab_related_projects/test.git
$ INPUT="someletters_12345_moreleters.ext"
$ SUBSTRING=`expr match "$INPUT" '.*_\([[:digit:]]*\)_.*' `
$ echo $SUBSTRING
12345
$ b=${INPUT:12:5}
$ echo $b
12345
$ your_number=$(echo "someletters_12345_moreleters.ext" | grep -E -o '[0-9]{5}')
$ echo $your_number
12345
$ your_number=$(echo "$INPUT" | grep -E -o '[0-9]{5}')
$ echo $your_number
12345
$ your_number=$(echo "Release V14_TEST-42" | cut -d "_" -f2 )
$ echo $your_number
TEST-42
$ VAR="Release V14_TEST-42"
$ your_number=$(echo "$VAR" | cut -d "_" -f2 )
$ echo $your_number
TEST-42
Cleaning up project directory and file based variables
00:01
Job succeeded

Linux simple while loop

I have log file keep updating for 30 minutes and I implement script which will check that log file till it has "success" message written in it.So far I have implemented below.Any help or correction would be appreciated.
while [ "($cat R12TECH2.log | grep 'success')" != " " ]
do
echo "Please wait...devccm Adautoconfig is still running..."
sleep 5
done
echo "Status of devccm adautoconfig"
cat R12TECH2.log | grep 'success'
exit
Replace
while [ "($cat R12TECH2.log | grep 'success')" != " " ]
With:
while ! grep -q 'success' R12TECH2.log
The while statement does not require a [...] statement. It will work with any command that provides a satisfactory exit code. grep is one such command. Since we don't care about the output of the grep command, we use -q to silence it.
Grep and exit codes
Consider the test file:
$ cat R12TECH2.log
line 1
success
line 3
This grep command returns success (0):
$ grep -q 'success' R12TECH2.log; echo code=$?
code=0
We, however, want the while loop to run only if success is not in the file. Thus, provide a leading ! which tells the shell to negate the exit code:
$ ! grep -q 'success' R12TECH2.log; echo code=$?
code=1

AccuRev: how do you list the managed files?

I need to see which files have been added or removed between two streams. The most obvious way would be "git lsfiles" in each stream. Except this is not GIT and I do not see an analogous command. So for today:
for f in $(find * -type f);do
accurev stat "$f"
done | \
fgrep -v '(external)' | \
awk '{print $1}' > .list
If there is a better way, it should be clear and easy to find here:
http://www.accurev.com/download/docs/5.7.0_books/AccuRev_5_7_User_CLI.pdf
but it is not. Help? Thank you.
If you want to see the difference between two streams, run the following command: accurev diff -a -v "Stream1" -V "Stream2"
As the command line question has been answered, here's how to do the same via the AccuRev GUI.
Select one dynamic stream, workspace or snapshot.
Right click and select "Show Diff By Files"
Select a different dynamic stream, workspace or snapshot.
You'll be presented with a list of files different between the two choices, and yes you can mix-and-match between dynamic streams, workspaces and snapshots.
You can then select any file and select "Show Difference" to see differences between the two files.
Since neither of the two answers addressed the question, I eventually worked out a script to do what is really needed. "accurev lsfiles" is sorely needed.
#! /bin/bash
declare -r progpid=$$
declare -r progdir=$(cd $(dirname $0) >/dev/null && pwd)
declare -r prog=$(basename $0)
declare -r program="$progdir/$prog"
declare -r usage_text=' [ <directory> ... ]
If no directory is specified, "." is assumed'
die() {
echo "$prog error: $*"
exec 1>/dev/null 2>&1
kill -9 $progpid
exit 1
} 1>&2
usage() {
test $# -gt 0 && {
exec 1>&2
echo "$prog usage error: $*"
}
printf "USAGE: $prog %s\n" "$usage_text"
exit $#
}
init() {
shift_ct=$#
tmpd=$(mktemp -d ${TMPDIR:-/tmp}/ls-XXXXXX)
test -d "$tmpd" || die "mktemp -d does not work"
exec 4> ${tmpd}/files
trap "rm -rf '$tmpd'" EXIT
prune_pat=
while :
do
test $# -eq 0 && break
test -f "$1" && break
[[ "$1" =~ -.* ]] || break
case "X$1" in
X-p )
prune_pat+="${2}|"
shift 2 || usage "missing arg for '-p' option"
;;
X-p* )
prune_pat+="${1#-p}"
shift
;;
X-x* )
set -x
tput reset 1>&2
PS4='>${FUNCNAME:-lsf}> '
shift
;;
X-* )
usage "unknown option: $1"
;;
* )
break
;;
esac
done
(( shift_ct -= $# ))
test ${#prune_pat} -gt 0 && \
prune_pat='(^|/)('${prune_pat%|}')$'
}
chkdir() {
declare list=$(exec 2> /dev/null
for f in "$#"
do ls -d ${f}/* ${f}/.*
done | \
grep -v -E '.*/\.\.*$' )
for f in $(accurev stat ${list} | \
grep -v -F '(external)' | \
awk '{print $1}' | \
sed 's#^/*\./##')
do
test ${#prune_pat} -gt 0 && [[ $f =~ ${prune_pat} ]] && continue
if test -d "$f"
then chkdir "$f"
elif test -f "$f" -o -L "$f"
then echo "$f" 1>&4
fi
done
}
init ${1+"$#"}
(( shift_ct > 0 )) && shift ${shift_ct}
test $# -eq 0 && set -- .
chkdir "$#"
exec 4>&-
sort -u ${tmpd}/files
It is a bit over-the-top, but I have a boilerplate I always use for my scripts.

shell script not working

I have a shell script to do the following things
sudo as a user (johnsmith) and perform few things
Exit from that user and check url status
If status is not equal to 1 , ssh to one more server and execute a
script.
But when I am running it, the lines inside 'ENDBASH' are not getting executed at all.
#!/bin/ksh
echo "Outside ENDBASH ${###*/}"
sudo -u johnssmith bash <<'ENDBASH'
echo "Inside ENDBASH ${###*/}"
#Obtaining the new version file
for file in "${###*/}"
do
if echo "$file" | grep -E "abc_cde_efg"; then
echo "Version found: $file"
else
echo "Version not found"
fi
done
exit
ENDBASH
urlArray=('http://server:port/servicename1/services/servicename1?wsdl' 'http://server:port/servicename2/services/servicename2?wsdl')
status=0
for url in "${urlArray[#]}"
do
result=`curl -s $url`
if (echo $result | grep '<?xml' >/dev/null 2>&1); then
service=$(echo $url | cut -d"/" -f4)
echo "$service is Running"
else
service=$(echo $url | cut -d"/" -f4)
echo "$service is not Running"
status=1
fi
done
if [ $status != 1 ] ; then
ssh -t username#hostname /home/dev_was/test1.sh
fi
You need to explicitly pass the arguments received by your script to the internal script:
sudo -u johnssmith bash -s "$#" <<'ENDBASH'

Whats wrong with this C shell script?

I am trying to write a C shell equivalent script for the bash script mentioned here.
This is what I have :
#! /bin/tcsh
set now=`date +%Y%m%d%H%M.%S`
if (( ! -f "./cache" ) || (-n "`find ./monme -newer ./cache`" ))
then
touch cache -t "$now"
echo "new files added" | mail -s "new build" myemail#myserver.com
endif
and this is the error I get
$ ./scr
if: Badly formed number.
$
This page mentions that "Numbers in the C-shell must be integers", so I tried
set now=`date +%Y%m%d%H%M`
but I get the same error still.
I cut down your script to this:
#! /bin/tcsh
if ( -n "`find ./monme -newer ./cache`" ) then
echo hello
endif
This gives the same error. I think the culprit is
-n "`find ./monme -newer ./cache`"
What is -n supposed to do? I think it wants a number, but gets something else...
Update: -n in bash means "length of string is non-zero". In my version of tcsh it is as easy to replace as to use == "" like this:
if (( ! -f "./cache" ) || ("`find ./monme -newer ./cache`" != ""))
then
touch cache -t "$now"
echo "new files added" | mail -s "new build" myemail#myserver.com
endif
Try that and see if it works.