I have a pipeline in which I am trying to pass the Build Number.
When I print the Build Number its prints fine however, I am having issues passing it to the a shell ansible command inside a stage.
pipeline {
agent { label 'Prod_Slave' }
stages {
stage('Ansible stuff') {
parallel {
stage('APP') {
steps {
echo "${env.BUILD_NUMBER}"
println "${env.BUILD_NUMBER}"
sh 'ansible-playbook $ansible_playbook/site.yml -e Latest_Build_Number4=${env.BUILD_NUMBER}'
}
}
}
}
}
}
Output:
[Pipeline]
echo (hide)
16 [Pipeline]
echo 16 [Pipeline]
echo 16
/script.sh: line 1: Latest_Build_Number4=${env.BUILD_NUMBER}: bad
substitution
When i change to:
sh 'ansible-playbook $ansible_playbook/site.yml -e Latest_Build_Number4="${env.BUILD_NUMBER}"'
Output:
#2#tmp/durable-5ed077ca/script.sh: line 1: ${env.BUILD_NUMBER}: bad
substitution
Finally tried this:
sh 'ansible-playbook $ansible_playbook/site.yml -e Latest_Build_Number4="$env.BUILD_NUMBER"'
Output:
-e Latest_Build_Number4=.BUILD_NUMBER
Single quotes won't allow substitution. Try double quotes, escaping the dollar sign:
sh "ansible-playbook \$ansible_playbook/site.yml -e Latest_Build_Number4=${env.BUILD_NUMBER}"
Related
I tried to run my nextflow script and the first two precess worded fine, but the third process Conbinevcf reported an error, showing that the variable prefix was not found.
process Annovar_genebased {
publishDir "${params.output}/annovar", mode: 'copy'
input:
path 'snp_anatation' from anatation1.flatMap()
val humandb
val refgene
output:
path "*.exonic_variant_function" into end
"""
prefix=\$(basename \$(readlink snp_anatation) .avinput)
perl $refgene -geneanno -dbtype refGene -out \${prefix}.anatation -buildver hg19 $snp_anatation $humandb -hgvs
rm *.log
rm *.variant_function
"""
}
process Annovar {
publishDir "${params.output}/annovar", mode: 'copy'
input:
path 'snp_anatation' from anatation2.flatMap()
val annovar_table
val humandb
output:
path "*.csv" into end1
"""
prefix=\$(basename \$(readlink snp_anatation) .avinput)
perl $annovar_table $snp_anatation $humandb -buildver hg19 -out \${prefix}.anatation -remove -protocol refGene,cytoBand,exac03,clinvar_20200316,gnomad211_exome -operation g,r,f,f,f -nastring . -csvout -polish
"""
}
I got stuck on this process
process Combinevcf {
publishDir "${params.output}/combinevcf", mode: 'copy'
input:
path 'genebased' from end.flatMap()
path 'allbased' from end1.flatMap()
output:
path "*_3.csv" into end3
"""
prefix=\$(basename \$(readlink genebased) .exonic_variant_function)
prefix1=\$(basename \$(readlink allbased) .csv)
cat ${prefix}.exonic_variant_function | tr -s ‘[:blank:]’ ‘,’ | awk 'BEGIN{FS=",";OFS="," }{ print \$3,\$13,\$22}' | awk ' BEGIN { OFS=", "; print "refGene", "refGene", "refGene", "refGene", "refGene", "Zogysity","chr", "filter" } { print \$0, "" } ' > ${prefix}_1.csv
awk 'BEGIN{FS=",";OFS="," }{ print \$1,\$2,\$3,\$4,\$5,\$6,\$7,\$8,\$9,\$10,\$15,\$21,\$24,\$25}' ${prefix1}.csv > ${prefix1}_2.csv
paste ${prefix}_1.csv ${prefix1}_2.csv > ${prefix}_3.csv
"""
}
I am not sure what went wrong, any help would be appreciated.
You need to escape your ${prefix} with backslashes to tell nextflow that the variable prefix is in the script block scope, and not in the nextflow scope.
See https://www.nextflow.io/docs/latest/process.html#script for more info:
Since Nextflow uses the same Bash syntax for variable substitutions in strings, you must manage them carefully depending on whether you want to evaluate a Nextflow variable or a Bash variable
I use Jenkinsefile file to run the Stages.
It is in Jenkins pipeline installed on windows, Declarative pipeline.
On the begining I do:
pipeline {
agent { label 'master'}
environment {
My_build_result = 7
}
....
Than
stage('Test') {
steps {
echo 'Testing..'
bat """
cd Utils
"C:\\Program Files\\MATLAB\\R2019b\\bin\\matlab.exe" -wait -nodisplay -nosplash -nodesktop -r "run('automatic_tests\\run_test.m');"
echo %errorlevel%
set /a My_build_result_temp = %errorlevel%
set My_build_result = %My_build_result_temp%
"""
script {
My_build_result = bat(returnStatus:true , script: "exit (2)").trim()
echo "My_build_result ${env.My_build_result}"
if (My_build_result != 0) {
echo "inside if"
}
}
}
}
The variable My_build_result get value 7 at the begining
Inside the bat section, it suppose to get value 0 from %errorlevel%
Inside the script section it suppose to get value 2
BUT
in the echo "My_build_result ${env.My_build_result}" I get print of 7
(and it goes inside the if sentense)
How do I define variable that can be set value in bat"""
"""
and in script """
"""
section of the stage
and also be familiar in another stages and in the post { always { .. }} at the end ???
BTW: add env.before My_build_result (env.My_build_result ) does not work
Thanks a lot
In the first bat call, you are setting the environment variable only inside of the batch script environment. Environment variable values that are assigned through set don't persist when the script ends. Think of these like local variables. Simply use returnStatus: true to return the last value of ERRORLEVEL. There is no need to use %ERRORLEVEL% in the batch script here.
steps {
script {
My_build_result = bat returnStatus: true, script: """
cd Utils
"C:\\Program Files\\MATLAB\\R2019b\\bin\\matlab.exe" -wait -nodisplay -nosplash -nodesktop -r "run('automatic_tests\\run_test.m');"
"""
// My_build_result now has the value of ERRORLEVEL from the last command
// called in the batch script.
}
}
In the 2nd bat call the 1st mistake is to call the trim() method. Result type of bat step is Integer, when returnStatus: true is passed. The trim() method is only available when returnStdout: true is passed in which case the result type would be String. The 2nd mistake is to use brackets around the exit code value. The fixed code should look like:
My_build_result = bat returnStatus: true, script: "exit 2"
// My_build_result now equals 2
I am trying to convert a “sed” script I use on my FreeBSD machine to one using “Powershell” on Windows 10.
This is the sed script. It is used to strip a header from an email plus the immediately following line and send the output to “email_1.txt”. The file is fed to the script on the command line; i.e. “COMMAND file”
sed '/Received: by 2002:a17:90a:3566:0:0:0:0/,/^/d' <$1> email_1.txt
I cannot find a way to get this to work with “PowerShell”.
Thanks!
You have a couple of options.
Install sed -
Something like scoop might be helpful here.
Write a pure powershell solution.
This will be very similar to what you would write if you were to try to do the same thing in "pure" bash. Here is an attempt to do so:
--
function Delete-TargetLines {
[cmdletbinding()]
param(
[String]$needle,
[int]$count = [int]1,
[parameter(ValueFromPipeline)]
[string[]]$haystack
)
Begin {
[int]$seen = 0
}
Process {
if ($seen -gt 0) {
$seen -= 1
} elseif ( $haystack -match $needle ) {
$seen = 1
} else {
$haystack
}
}
}
And an example of running it:
> #("Pre-line", "This is a test", "second line", "post line") | Delete-TargetLines -needle "test"
Pre-line
post line
> Get-Content $myfile | Delete-TargetLines -needle 'value' > $outfile
I would like to call different other scripts, depending on whether a paramter is given or not:
"paramtest": "if [ -z $1 ]; then echo Foo $1; else echo Bar; fi",
npm run paramtest
should give "Bar".
npm run paramtest -- whatever
should give "Foo whatever".
However in practice I only get: (The parameter is added to the whole line, not 'passed in')
> if [ -z $1 ]; then echo Foo; else echo Bar; fi "whatever
sh: 1: Syntax error: word unexpected
What can I do better?
Essentially I am after running full test suite / only individual test with the same command...
"test" : "if [ -z $1 ]; then mocha ./test/**/*.test.js; else mocha $1
Wrapping it in a shell function should do the trick:
"test": "f() { if [ $# -eq 0 ]; then mocha './test/**/*.test.js'; else mocha -- \"$#\"; fi; }; f"
Note that I changed the if condition and the else branch slightly so you can specify multiple file arguments if necessary.
A more succinct method:
"test": "f() { mocha -- \"${#:-./test/**/*.test.js}\"; }; f"
Using a shell function this way might look familiar, as the same technique is often used for git aliases.
Detailed Explanation
Let's use this script for demonstration:
"scripts": {
"myscript": "if [ \"$1\" = one ]; then printf %s\\\\n \"$#\"; else echo false; fi"
}
Here if the first argument is "one", we print all the arguments, and otherwise we print "false". We are of course assuming that npm run-script is using an sh-like shell, and not, e.g., Windows' cmd.exe.
I can't see anything in the npm documentation specifically detailing how arguments are passed to the script, so let's take a look at the source code (npm v6.14.7 at the time of writing). It seems that the script is joined with its arguments here and is then executed here. Essentially, npm run myscript -- one two three becomes
sh -c 'if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi "one" "two" "three"'
Our arguments one two three are simply quote-escaped and concatenated to the script command. In terms of the shell grammar, this means that they are ending up as arguments to fi. sh of course rejects this because fi is just a builtin to end if and takes no arguments.
Our goal is something more like
sh -c 'if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi' sh "one" "two" "three"
Here one, two, and three are arguments to sh itself and thus become the argument variables $1, $2, and $3 in the given script. npm doesn't let us do this directly, but we can accomplish the same thing by wrapping our script in a shell function:
"scripts": {
"myscript": "f() { if [ \"$1\" = one ]; then printf %s\\\\n \"$#\"; else echo false; fi; }; f"
}
The script here ends with an invocation of the function, so npm will end up concatenating the arguments to this invocation, ultimately calling the function as f "one" "two" "three":
sh -c 'f() { if [ "$1" = one ]; then printf %s\\n "$#"; else echo false; fi; }; f "one" "two" "three"'
In this README I give instructions for a quick CL for testing the released tool. I think it would be much better if I provided a .bat and unix script which executed the commands in one click/command. At the same time, unlike a compiled program, it's transparent and users can open the script with the editor and inspect the commands executed.
Can I in a bat save a file?
This is what I'd like it to execute.
$ vim Test.java (windows: notepad Test.java)
class T {
private static void p(int i, Double d, String... s){}
}
public class Test{
#com.dp4j.InjectReflection public void t() {
T.p(1,new Double(2),"hello", "reflection");
}
}
$ ls Test.class T.class (windows: dir Test.class T.class)
ls: Test.class: No such file or directory ls: T.class: No such file or directory
$ javac -cp dp4j-1.0-jar-with-dependencies.jar Test.java
$ ls Test.class T.class (windows: dir Test.class T.class)
ls Test.class T.class
Yes.
You can put each line in an echo command piped to the file:
echo class T { > MyFile.java
echo private static void p(int i, Double d, String... s){} >> MyFile.java
echo } >> MyFile.java
echo ... >> MyFile.java
> creates a file; >> appends to it.
You can then compile it normally.
Um, perhaps this?
// 2>NUL&GOTO :START
//Just ignore the above line, it's for the batch script.
class T {
private static void p(int i, Double d, String... s){}
}
public class Test{
#com.dp4j.InjectReflection public void t() {
T.p(1,new Double(2),"hello", "reflection");
}
}
/*We start the batch script here.
:START
#CLS&ECHO OFF
START NOTEPAD %0
IF EXIST Test.class GOTO :EXISTS
IF EXIST T.class GOTO :EXISTS
JAVAC -cp dp4j-1.0-jar-with-dependencies.jar %0
GOTO :END
:EXISTS
ECHO There is a preexisting class file. Aborting.
:END
REM We end the batch script here.*/
By the way, here the batch script and java source are the same file.
This is a unix exectable script that does what I want. I don't know if it works with cygwin on windows:
#!/bin/sh
v=1.1
test_file="Test10.java"
jar_file="dp4j-$v-jar-with-dependencies.jar"
cmd="curl -O --fail -L http://downloads.sourceforge.net/project/dp4j/$v/$jar_file"
echo $cmd
$cmd
echo
# Start
cat > $test_file << __EOF__
class T10 {
private static void p(int i, Double d, String... s){}
}
public class Test10{
#com.dp4j.InjectReflection
public void t() {
T10.p(1,new Double(2),"hello", "reflection");
}
}
__EOF__
cmd="cat $test_file"
echo $cmd
$cmd
echo
cmd="javac -Averbose=true -cp $jar_file $test_file"
echo $cmd
$cmd
echo
echo "TEST PASSED: $test_file was compiled with Reflection Injected."
echo "When JUnit/TestNG.jar is in the classpath you may use #Test in lieu of #InjectReflection."
echo "Javadoc, sources, and other artifacts maybe downloaded from http://repo2.maven.org/maven2/com/dp4j/dp4j/"$v"/"
Here's one line command to download it and execute it:
wget http://sourceforge.net/projects/dp4j/files/1.2/TESTDRIVE ; chmod +x TESTDRIVE ; ./TESTDRIVE