I use Jenkinsefile file to run the Stages.
It is in Jenkins pipeline installed on windows, Declarative pipeline.
On the begining I do:
pipeline {
agent { label 'master'}
environment {
My_build_result = 7
}
....
Than
stage('Test') {
steps {
echo 'Testing..'
bat """
cd Utils
"C:\\Program Files\\MATLAB\\R2019b\\bin\\matlab.exe" -wait -nodisplay -nosplash -nodesktop -r "run('automatic_tests\\run_test.m');"
echo %errorlevel%
set /a My_build_result_temp = %errorlevel%
set My_build_result = %My_build_result_temp%
"""
script {
My_build_result = bat(returnStatus:true , script: "exit (2)").trim()
echo "My_build_result ${env.My_build_result}"
if (My_build_result != 0) {
echo "inside if"
}
}
}
}
The variable My_build_result get value 7 at the begining
Inside the bat section, it suppose to get value 0 from %errorlevel%
Inside the script section it suppose to get value 2
BUT
in the echo "My_build_result ${env.My_build_result}" I get print of 7
(and it goes inside the if sentense)
How do I define variable that can be set value in bat"""
"""
and in script """
"""
section of the stage
and also be familiar in another stages and in the post { always { .. }} at the end ???
BTW: add env.before My_build_result (env.My_build_result ) does not work
Thanks a lot
In the first bat call, you are setting the environment variable only inside of the batch script environment. Environment variable values that are assigned through set don't persist when the script ends. Think of these like local variables. Simply use returnStatus: true to return the last value of ERRORLEVEL. There is no need to use %ERRORLEVEL% in the batch script here.
steps {
script {
My_build_result = bat returnStatus: true, script: """
cd Utils
"C:\\Program Files\\MATLAB\\R2019b\\bin\\matlab.exe" -wait -nodisplay -nosplash -nodesktop -r "run('automatic_tests\\run_test.m');"
"""
// My_build_result now has the value of ERRORLEVEL from the last command
// called in the batch script.
}
}
In the 2nd bat call the 1st mistake is to call the trim() method. Result type of bat step is Integer, when returnStatus: true is passed. The trim() method is only available when returnStdout: true is passed in which case the result type would be String. The 2nd mistake is to use brackets around the exit code value. The fixed code should look like:
My_build_result = bat returnStatus: true, script: "exit 2"
// My_build_result now equals 2
I have 3 grep runs and results send to my email:
((grep -irl 'abc' /usr/www/users/FTPUSERNAME/*) && (grep -irl 'xyz' /usr/www/users/FTPUSERNAME/*) && (grep -irl 'xXx' /usr/www/users/FTPUSERNAME/*)) | mail me#mywebsite.com
However this creates a combined list of all files.
Is there a way to separate the results by inserting a title to each grep
So the email I get looks something like this:
Title for files with abc
/Path-To-File/filename.php
/Path-To-File/filename.php
/Path-To-File/filename.php
Title for files with xyz
/Path-To-File/filename.php
/Path-To-File/filename.php
/Path-To-File/filename.php
Title for files with xXx
/Path-To-File/filename.php
/Path-To-File/filename.php
Thanks for the help,
Amit
FWIW I wouldn't do 3 separate greps just for that and I also wouldn't use grep to find files when there's a tool with a very obvious name for that (find), e.g. with GNU awk for IGNORECASE and true multi-dimensional arrays:
find /usr/www/users/FTPUSERNAME/ -type f -exec awk '
BEGIN {
IGNORECASE = 1
split("abc xyz xXx",res)
}
{
for (i in res) {
re = res[i]
if ($0 ~ re) {
hits[re][FILENAME]
}
}
}
END {
for (i=1; i in res; i++) {
re = res[i]
printf "Title for files with %s\n", re
for (file in hits[re]) {
print file
}
print ""
}
}
' {} + |
mail ...
Note that given the above if you need to add a 4th regexp you just add it to the split() call, it only searches for files once instead of 3 times, you're only opening each found file once instead of 3 times, and you can format the output of the search however you like. There's various ways it could be optimized for speed if necessary.
You should be able to achieve this by pre-pending each of your grep commands with an echo. Here is your command slightly modified :
$ ((echo "Title for files with abc" && grep -irl 'abc' /usr/www/users/FTPUSERNAME/*) && ( echo "Title for files with xyz" && grep -irl 'xyz' /usr/www/users/FTPUSERNAME/*) && ( echo "Title for files with xXx" && grep -irl 'xXx' /usr/www/users/FTPUSERNAME/*)) | mail me#mywebsite.com
I would like to call different other scripts, depending on whether a paramter is given or not:
"paramtest": "if [ -z $1 ]; then echo Foo $1; else echo Bar; fi",
npm run paramtest
should give "Bar".
npm run paramtest -- whatever
should give "Foo whatever".
However in practice I only get: (The parameter is added to the whole line, not 'passed in')
> if [ -z $1 ]; then echo Foo; else echo Bar; fi "whatever
sh: 1: Syntax error: word unexpected
What can I do better?
Essentially I am after running full test suite / only individual test with the same command...
"test" : "if [ -z $1 ]; then mocha ./test/**/*.test.js; else mocha $1
Wrapping it in a shell function should do the trick:
"test": "f() { if [ $# -eq 0 ]; then mocha './test/**/*.test.js'; else mocha -- \"$#\"; fi; }; f"
Note that I changed the if condition and the else branch slightly so you can specify multiple file arguments if necessary.
A more succinct method:
"test": "f() { mocha -- \"${#:-./test/**/*.test.js}\"; }; f"
Using a shell function this way might look familiar, as the same technique is often used for git aliases.
Detailed Explanation
Let's use this script for demonstration:
"scripts": {
"myscript": "if [ \"$1\" = one ]; then printf %s\\\\n \"$#\"; else echo false; fi"
}
Here if the first argument is "one", we print all the arguments, and otherwise we print "false". We are of course assuming that npm run-script is using an sh-like shell, and not, e.g., Windows' cmd.exe.
I can't see anything in the npm documentation specifically detailing how arguments are passed to the script, so let's take a look at the source code (npm v6.14.7 at the time of writing). It seems that the script is joined with its arguments here and is then executed here. Essentially, npm run myscript -- one two three becomes
sh -c 'if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi "one" "two" "three"'
Our arguments one two three are simply quote-escaped and concatenated to the script command. In terms of the shell grammar, this means that they are ending up as arguments to fi. sh of course rejects this because fi is just a builtin to end if and takes no arguments.
Our goal is something more like
sh -c 'if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi' sh "one" "two" "three"
Here one, two, and three are arguments to sh itself and thus become the argument variables $1, $2, and $3 in the given script. npm doesn't let us do this directly, but we can accomplish the same thing by wrapping our script in a shell function:
"scripts": {
"myscript": "f() { if [ \"$1\" = one ]; then printf %s\\\\n \"$#\"; else echo false; fi; }; f"
}
The script here ends with an invocation of the function, so npm will end up concatenating the arguments to this invocation, ultimately calling the function as f "one" "two" "three":
sh -c 'f() { if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi; }; f "one" "two" "three"'
ok so i have have this
{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}
and at the moment i'm using this shell command to decode it to get the string i need,
echo $x | grep -Po '"utterance":.*?[^\\]"' | sed -e s/://g -e s/utterance//g -e 's/"//g'
but this only works when you have a grep compiled with perl and plus the script i use to get that JSON string is written in perl, so is there any way i can do this same decoding in a simple perl script or a simpler unix command, or better yet, c or objective-c?
the script i'm using to get the json is here, http://pastebin.com/jBGzJbMk and if you want a file to use then download http://trevorrudolph.com/a.flac
How about:
perl -MJSON -nE 'say decode_json($_)->{hypotheses}[0]{utterance}'
in script form:
use JSON;
while (<>) {
print decode_json($_)->{hypotheses}[0]{utterance}, "\n"
}
Well, I'm not sure if I can deduce what you are after correctly, but this is a way to decode that JSON string in perl.
Of course, you'll need to know the data structure in order to get the data you need. The line that prints the "utterance" string is commented out in the code below.
use strict;
use warnings;
use Data::Dumper;
use JSON;
my $json = decode_json
q#{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}#;
#print $json->{'hypotheses'}[0]{'utterance'};
print Dumper $json;
Output:
$VAR1 = {
'status' => 0,
'hypotheses' => [
{
'utterance' => 'hello how are you',
'confidence' => '0.96311796'
}
],
'id' => '7aceb216d02ecdca7ceffadcadea8950-1'
};
Quick hack:
while (<>) {
say for /"utterance":"?(.*?)(?<!\\)"/;
}
Or as a one-liner:
perl -lnwe 'print for /"utterance":"(.+?)(?<!\\)"/g' inputfile.txt
The one-liner is troublesome if you happen to be using Windows, since " is interpreted by the shell.
Quick hack#2:
This will hopefully go through any hash structure and find keys.
my $json = decode_json $str;
say find_key($json, 'utterance');
sub find_key {
my ($ref, $find) = #_;
if (ref $ref) {
if (ref $ref eq 'HASH' and defined $ref->{$find}) {
return $ref->{$find};
} else {
for (values $ref) {
my $found = find_key($_, $find);
if (defined $found) {
return $found;
}
}
}
}
return;
}
Based on the naming, it's possible to have multiple hypotheses. The prints the utterance of each hypothesis:
echo '{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}' | \
perl -MJSON::XS -n000E'
say $_->{utterance}
for #{ JSON::XS->new->decode($_)->{hypotheses} }'
Or as a script:
use feature qw( say );
use JSON::XS;
my $json = '{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}';
say $_->{utterance}
for #{ JSON::XS->new->decode($json)->{hypotheses} };
If you don't want to use any modules from CPAN and try a regex instead there are multiple variants you can try:
# JSON is on a single line:
$json = '{"other":"stuff","hypo":[{"utterance":"hi, this is \"bob\"","moo":0}]}';
# RegEx with negative look behind:
# Match everything up to a double quote without a Backslash in front of it
print "$1\n" if ($json =~ m/"utterance":"(.*?)(?<!\\)"/)
This regex works if there is only one utterance. It doesn't matter what else is in the string around it, since it only searches for the double quoted string following the utterance key.
For a more robust version you could add whitespace where necessary/possible and make the . in the RegEx match newlines: m/"utterance"\s*:\s*"(.*?)(?<!\\)"/s
If you have multiple entries for the utterance confidence hash/object, changing case and weird formatting of the JSON string try this:
# weird JSON:
$json = <<'EOJSON';
{
"status":0,
"id":"an ID",
"hypotheses":[
{
"UtTeraNcE":"hello my name is \"Bob\".",
"confidence":0.0
},
{
'utterance' : 'how are you?',
"confidence":0.1
},
{
"utterance"
: "
thought
so!
",
"confidence" : 0.9
}
]
}
EOJSON
# RegEx with alternatives:
print "$1\n" while ( $json =~ m/["']utterance["']\s*:\s*["'](([^\\"']|\\.)*)["']/gis);
The main part of this RegEx is "(([^\\"]|\\.)*)". Description in detail as extended regex:
/
["'] # opening quotes
( # start capturing parentheses for $1
( # start of grouping alternatives
[^\\"'] # anything that's not a backslash or a quote
| # or
\\. # a backslash followed by anything
) # end of grouping
* # in any quantity
) # end capturing parentheses
["'] # closing quotes
/xgs
If you have many data sets and speed is a concern you can add the o modifier to the regex and use character classes instead of the i modifier. You can suppress the capturing of the alternatives to $2 with clustering parenthesis (?:pattern). Then you get this final result:
m/["'][uU][tT][tT][eE][rR][aA][nN][cC][eE]["']\s*:\s*["']((?:[^\\"']|\\.)*)["']/gos
Yes, sometimes perl looks like a big explosion in a bracket factory ;-)
Just stubmled upon another nice method of doing this, i finaly found how to acsess the Mac OS X JavaScript engine form commandline, heres the script,
alias jsc='/System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc'
x='{"status":0,"id":"7aceb216d02ecdca7ceffadcadea8950-1","hypotheses":[{"utterance":"hello how are you","confidence":0.96311796}]}'
jsc -e "print(${x}['hypotheses'][0]['utterance'])"
Ugh, yes i came up with another answer, im strudying python and it reads arrays in both its python format and the same format as a json so, i jsut made this one liner when your variable is x
python -c "print ${x}['hypotheses'][0]['utterance']"
figured it out for unix but would love to see your perl and c, objective-c answers...
echo $X | sed -e 's/.*utterance//' -e 's/confidence.*//' -e s/://g -e 's/"//g' -e 's/,//g'
:D
shorter copy of the same sed:
echo $X | sed -e 's/.*utterance//;s/confidence.*//;s/://g;s/"//g;s/,//g'
This is similar to another issue, but I only want make to prompt for a value if I'm running a specific target and a mandatory variable has not been specified.
The current code:
install-crontab: PASSWORD ?= "$(shell read -p "Password: "; echo "$$REPLY")"
install-crontab: $(SCRIPT_PATH)
#echo "#midnight \"$(SCRIPT_PATH)\" [...] \"$(PASSWORD)\""
This just results in the following output and no prompt:
Password: read: 1: arg count
#midnight [...] ""
The important point here is that I have to ask only when running this target, and only if the variable has not been defined. I can't use a configure script, because obviously I shouldn't store passwords in a config script, and because this target is not part of the standard installation procedure.
Turns out the problem was that Makefiles don't use Dash / Bash-style quotation, and that Dash's read built-in needs a variable name, unlike Bash. Resulting code:
install-crontab-delicious: $(DELICIOUS_TARGET_PATH)
#while [ -z "$$DELICIOUS_USER" ]; do \
read -r -p "Delicious user name: " DELICIOUS_USER;\
done && \
while [ -z "$$DELICIOUS_PASSWORD" ]; do \
read -r -p "Delicious password: " DELICIOUS_PASSWORD; \
done && \
while [ -z "$$DELICIOUS_PATH" ]; do \
read -r -p "Delicious backup path: " DELICIOUS_PATH; \
done && \
( \
CRONTAB_NOHEADER=Y crontab -l || true; \
printf '%s' \
'#midnight ' \
'"$(DELICIOUS_TARGET_PATH)" ' \
"\"$$DELICIOUS_USER\" " \
"\"$$DELICIOUS_PASSWORD\" " \
"\"$$DELICIOUS_PATH\""; \
printf '\n') | crontab -
Result:
$ crontab -r; make install-crontab-delicious && crontab -l
Delicious user name: a\b c\d
Delicious password: e f g
Delicious backup path: h\ i
no crontab for <user>
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
$ DELICIOUS_PASSWORD=foo make install-crontab-delicious && crontab -l
Delicious user name: bar
Delicious backup path: baz
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
#midnight "/usr/local/bin/export_Delicious" "bar" "foo" "baz"
This code:
treats all input characters as literals, so it works with spaces and backslashes,
avoids problems if the user presses Enter without writing anything,
uses environment variables if they exist, and
works whether crontab is empty or not.
l0b0's answer helped me with a similar problem where I wanted to exit if the user doesn't input 'y'. I ended up doing this:
#while [ -z "$$CONTINUE" ]; do \
read -r -p "Type anything but Y or y to exit. [y/N] " CONTINUE; \
done ; \
if [ ! $$CONTINUE == "y" ]; then \
if [ ! $$CONTINUE == "Y" ]; then \
echo "Exiting." ; exit 1 ; \
fi \
fi
I hope that helps someone. It's hard to find more info about using user input for an if/else in a makefile.