Is it possible to pass inputs to a gradle task? For example, you can pass inputs to a java or a c program. Can I do something similar with a gradle task?
If you mean, that you need to pass some value via command line arguments, you can do it like:
customTask {
if ( project.hasProperty("someArg") ) {
println project.property("someArg")
}
}
Then run your application with this argument
gradle customTask -PsomeArg=PassedValue
Or you can make a task and get the values on execution phase, like:
def username
def password
task getCridentials << {
def console = System.console()
if (console) {
username = console.readLine('username: ')
password = console.readPassword('password: ')
} else {
logger.error "Can't get console."
}
}
Related
I just involved the new project for API test for our service by using Gatling. At this point, I want to search query, below is the code:
def chnSendToRender(testData: FeederBuilderBase[String]): ChainBuilder = {
feed(testData)
exec(api.AdvanceSearch.searchAsset(s"{\"all\":[{\"all:aggregate:text\":{\"contains\":\"#{edlAssetId}_Rendered\"}}]}", "#{authToken}")
.check(status.is(200).saveAs("searchStatus"))
.check(jsonPath("$..asset:id").findAll.optional.saveAs("renderedAssetList"))
)
.doIf(session => session("searchStatus").as[Int] == 200) {
exec { session =>
printConsoleLog("Rendered Asset ID List: " + session("renderedAssetList").as[String], "INFO")
session
}
}
}
I declared the feeder already in the simulation scala file:
class GVRERenderEditor_new extends Simulation {
private val edlToRender = csv("data/render/edl_asset_ids.csv").queue
private val chnPostRender = components.notifications.notice.JobsPolling_new.chnSendToRender(edlToRender)
private val scnSendEDLForRender = scenario("Search Post Render")
.exitBlockOnFail(exec(preSimAuth))
.exec(chnPostRender)
setUp(
scnSendEDLForRender.inject(atOnceUsers(1)).protocols(httpProtocol)
)
.maxDuration(sessionDuration.seconds)
.assertions(global.successfulRequests.percent.is(100))
}
But Gatling test failed to run, showing this error: Exception in thread "main" java.lang.UnsupportedOperationException: There were no requests sent during the simulation, reports won't be generated
If I hardcode the #{edlAssetId} (put the real edlAssetId in that query), I will get result. I think I passed the parameter wrongly in this case. I've tried to print the output in console log but no luck. What's wrong with this code? I would appreciate your help. Thanks!
feed(testData)
exec(api.AdvanceSearch.searchAsset(s"{\"all\":[{\"all:aggregate:text\":{\"contains\":\"#{edlAssetId}_Rendered\"}}]}", "#{authToken}")
.check(status.is(200).saveAs("searchStatus"))
.check(jsonPath("$..asset:id").findAll.optional.saveAs("renderedAssetList"))
)
You're missing a . (dot) before the exec to attach it to the feed.
As a result, your method is returning the last instruction, ie the exec only.
Given the following nextflow.config:
google {
project = "cool-project"
region = "europe-west4"
lifeSciences {
bootDiskSize = "200 GB"
debug = true
preemptible = true
}
}
Is it possible to override one or more of those settings using command line arguments. For example, if I wanted to specify that no preemptible machines should be used, can I do the following:
nextflow run main.nf -c nextflow.config --google.lifeSciences.preemptible false
?
Overriding pipeline parameters can be done using Nextflow's command line interface by prefixing the parameter name with a double dash. For example, put the following in a file called 'test.nf':
#!/usr/bin/env nextflow
params.greeting = 'Hello'
names = Channel.of( "foo", "bar", "baz" )
process greet {
input:
val name from names
output:
stdout result
"""
echo "${params.greeting} ${name}"
"""
}
result.view { it.trim() }
And run it using:
nextflow run -ansi-log false test.nf --greeting 'Bonjour'
Results:
N E X T F L O W ~ version 20.10.0
Launching `test.nf` [backstabbing_cajal] - revision: 431ef92cef
[46/22b4f0] Submitted process > greet (1)
[ca/32992c] Submitted process > greet (3)
[6e/5880b0] Submitted process > greet (2)
Bonjour bar
Bonjour foo
Bonjour baz
This works fine for pipeline params, but AFAIK there's no way to directly override executor config like you describe on the command line. You can however, just parameterize these values and set them on the command line like described above. For example, in your nextflow.config:
params {
gc_region = false
gc_preemptible = true
...
}
profiles {
'test' {
includeConfig 'conf/test.config'
}
'google' {
includeConfig 'conf/google.config'
}
...
}
And in a file called 'conf/google.config':
google {
project = "cool-project"
region = params.gc_region
lifeSciences {
bootDiskSize = "200 GB"
debug = true
preemptible = params.gc_preemptible
}
}
Then you should be able to override these in the usual way:
nextflow run main.nf -profile google --gc_region "europe-west4" --gc_preemptible false
Note that you can also specify multiple configuration profiles by separating the profile names with a comma:
nextflow run main.nf -profile google,test ...
After running gradle tasks that include tests on my multi-module project, I want to see a summary of every test failure across all modules, e.g.
module 1:
testmodule1thing1 PASSED
testmodule1thing2 FAILED
results
2 tests 1 passed 1 failed
module 2:
testmodule2thing1 PASSED
testmodule2thing2 FAILED
results
2 tests 1 passed 1 failed
module 3:
testmodule3thing1 FAILED
results
1 tests 1 passed 1 failed
BUILD FAILED
========= I already have everything above this line
test failures:
testmodule1thing1
testmodule2thing2
testmodule3thing1
========= I want everything between the last line and this line
Is this possible? If so, how? If a full task summary isn't possible, I can live with a per-module summary
This can also be accomplished directly within the test task itself:
tasks.withType(Test) {
// a collection to track failedTests
ext.failedTests = []
afterTest { descriptor, result ->
if (result.resultType == TestResult.ResultType.FAILURE) {
String failedTest = "${descriptor.className}::${descriptor.name}"
logger.debug("Adding " + failedTest + " to failedTests...")
failedTests << [failedTest]
}
}
afterSuite { suite, result ->
if (!suite.parent) { // will match the outermost suite
// logs each failed test
if (!failedTests.empty) {
logger.lifecycle("Failed tests:")
failedTests.each { failedTest ->
logger.lifecycle("${failedTest}")
}
}
}
}
}
You can use a testlistener in combination with the buildFinished hook for that. A very simple solution can look like this first draft:
allprojects {
// add a collection to track failedTests
ext.failedTests = []
// add a testlistener to all tasks of type Test
tasks.withType(Test) {
afterTest { TestDescriptor descriptor, TestResult result ->
if(result.resultType == org.gradle.api.tasks.testing.TestResult.ResultType.FAILURE){
failedTests << ["${descriptor.className}::${descriptor.name}"]
}
}
}
// print out tracked failed tests when the build has finished
gradle.buildFinished {
if(!failedTests.empty){
println "Failed tests for ${project.name}:"
failedTests.each { failedTest ->
println failedTest
}
println ""
}
}
}
Another option to have better visibility for your failed tests is maybe using gradle build scans (https://plugins.gradle.org/plugin/com.gradle.build-scan).
I have the following code in my build.gradle
Contents in version.properties are:
buildVersion=1.2.3
Value of $v variable during the Gradle build is coming as: 1.2.3
Value of $artifactoryVersion variable in JENKINS build is coming as: 1.2.3.1, 1.2.3.2, 1.2.3.x ... and so on where the 4th digit is Jenkins BUILD_NUMBER available to gradle build script during Jenkins build.
BUT, when I'm running this build.gradle on my desktop where I dont have BUILD_NUMBER variable available or set in my ENVIRONMENT variables, I get an error saying trim() can't work on null. (as there's no BUILD_NUMBER for Desktop/local build).
I'm trying to find a way i.e.
What should I code in my script so that if BUILD_NUMBER is not available, then instead of gradle build processing failing for an error, it'd set jenkinsBuild = "0" (hard coded) otherwise, pick what it gets during Jenkins build.
For ex: in Bash, we set a variable var1=${BUILD_NUMBER:-"0"} which will set var1 to a valid Jenkins BUILD number if it's available and set to a value, otherwise if it's NULL, then var1 = "0".
I DON'T want to have each developer/user set this BUILD_NUMBER in some property file. All I want is, if this variable doesn't exist, then the code should put "0" in jenkinsBuilds variable and doesn't error out during desktop builds. I know during Jenkins build, it's working fine.
// Build Script
def fname = new File( 'version.properties' )
Properties props = new Properties()
props.load( new FileInputStream( fname ) )
def v = props.get( 'buildVersion' )
def env = System.getenv()
def jenkinsBuild = env['BUILD_NUMBER'].trim()
if( jenkinsBuild.length() > 0 ) {
artifactoryVersion = "$v.$jenkinsBuild"
}
All you need is some regular Java/Groovy code:
def jenkinsBuild = System.getenv("BUILD_NUMBER") ?: "0"
The code above uses Groovy's "elvis" operator, and is a shorthand for the following code, which uses Java's ternary operator:
def buildNumber = System.getenv("BUILD_NUMBER")
def jenkinsBuild = buildNumber != null ? buildNumber : "0"
Here's the answer to using a Java plain object (JDK8):
public class Sample {
private String region;
private String fruit;
public Sample() {
region = System.getenv().getOrDefault("REGION", null);
fruit = System.getenv().getOrDefault("FRUIT", "apple");
}
}
With the Env-Inject plugin you can get and set build parameters.
For example, under "Inject environment variables to the build process", add a Groovy script such as:
def paramsMap = [:]
def build = Thread.currentThread().executable
def my_var = build.getEnvVars()["MY_PARAM"]
if (!my_var) paramsMap.put("MY_PARAM", "default value")
// Return parameters map
out.println("Injecting parameters:\n" + paramsMap)
return paramsMap
The following is a program that dynamically calls all the getXXX methods on an object of CLASS, where the CLASS-name gets passed via command-line. And it works just fine.
// Program: callAllMethods.groovy
// Invoke this program as: groovy callAllMethods Date
args.each { arg ->
println "Methods of ${arg} ..."
def code = """
x = new ${arg}()
x.class.methods.each { f ->
if (f.name.startsWith("get")) {
print "new ${arg}()." + f.name + ": " + f.invoke(x)
println ''
}
}
"""
evaluate("$code")
println ''
}
However, when I try the simpler style of dynamic-method invocation (that does not use METHOD.invoke(OBJECT) but rather OBJECT."METHOD-NAME"()), like so,
// Program: callAllMethods.groovy
// Invoke this program as: groovy callAllMethods Date
args.each { arg ->
println "Methods of ${arg} ..."
def code = """
x = new ${arg}()
x.class.methods.each { f ->
if (f.name.startsWith("get")) {
result = x."${f.name}"()
println "new ${arg}().${f.name}: ${result}"
}
}
"""
evaluate("$code")
println ''
}
... I get the following error:
$ groovy callGetMethods.groovy Date
Methods of Date ...
Caught: groovy.lang.MissingPropertyException: No such property: f for class: callGetMethods
groovy.lang.MissingPropertyException: No such property: f for class: callGetMethods
at callGetMethods$_run_closure1.doCall(callGetMethods.groovy:13)
at callGetMethods.run(callGetMethods.groovy:10)
I cannot understand why! Groovy version that I'm using:
$ groovy -version
Groovy Version: 2.1.3 JVM: 1.6.0_43 Vendor: Sun Microsystems Inc. OS: Linux
It happens because when you use the reflection based one (the x.class.methods.each one), you concatenate and generate the code upon GString evaluation, which resolves only one variable against the current scope, which is arg, and that is ok. If you print the code, it outputs a perfectly runnable Groovy code:
x = new Date()
x.class.methods.each { f ->
if (f.name.startsWith("get")) {
print "new Date()." + f.name + ": " + f.invoke(x)
println ''
}
}
In your second version, the GString variables are resolved against the scope they were created, which is the script binding. So it tries to fetch the f variable from that scope, and not from the code variable. And that is why it crashes at the ${f} variable.
If you change the code variable into a plain string (single quotes), it won't resolve the variable arg, and thus you will need to tweak a bit around it to create a new class from it. Even so, it will fail, unless you pass as argument groovy callAllMethods java.util.Date, which is not groovy (pun intended).
So, to use your code in that way, the GString shouldn't be resolved in declaration time, but rather at evaluate() time. Still, the arg variable needs to be resolved in declaration time, thus, you need to concat it. Here is the result:
args.each { arg ->
println "Methods of ${arg} ..."
def code = '''
x = new '''+arg+'''()
x.class.methods.each { m ->
if (m.name.startsWith("get")) {
result = x."${m.name}"()
println "new '''+arg+'''().${m.name}: ${result}"
}
}
'''
evaluate code
println ''
}
Which, in my box (jdk7, groovy 2.1.3), outputs:
new Date().getDay: 0
new Date().getTimezoneOffset: 180
new Date().getDate: 2
new Date().getHours: 10
new Date().getMinutes: 39
new Date().getMonth: 5
new Date().getSeconds: 56
new Date().getTime: 1370180396136
new Date().getYear: 113
new Date().getClass: class java.util.Date
If you just want to output the properties from an object, may i suggest the object.properties?
args.each { arg ->
println "Methods of ${arg} ..."
def code = '''
x = new '''+arg+'''()
x.properties.each {
println "new '''+arg+'''().${it.key}: ${x[it.key]}"
}
'''
evaluate code
println ''
}
It outputs a bit more stuff for Date, though :-).