We are using Cucumber + Selenium framework.
I have implemented the rerun plugin which captures the failed cases.
Now to execute those failed ones I need to have one extra test runner where I can pass the failed cases path which was created using first test runners execution.
Here I want the solution where I can run both these test runners sequentially so that the entire suite gets executed in one and any failed scripts are got executed in the second runner.
Test Runner1:-
#RunWith(ExtendedCucumber.class)
#CucumberOptions(format = { "json:target/cucumber.json", "pretty" }, features = ".",
glue = {"" }, tags = { "#TestSuite" }, plugin = {"rerun:target/rerun.txt" })
Test Runner2:-
#RunWith(ExtendedCucumber.class)
#CucumberOptions(format = { "json:target/cucumber.json","pretty" },features = "#target/rerun.txt",glue = {"" },plugin = {"rerun:target/rerun.txt" })
I want to execute only one instance and it should execute both above but Test Runner 1 and Test Runner 2 sequentially as Test Runner 2 has input dependency.
Related
I would like to use github secrets to inject a secret API key into my unit testing functions to prevent having ay API key within the source:
My (simplfied) test is defining the APIKEY as string constant.
const APIKEY: &str = "ddbe2befdeadbeefa2d20f09b4a694";
#[test]
fn test_apikey() {
assert!( usekey(APIKEY).is_ok() );
}
Now I would like to get the API-key from the arguments which the test executable gets when I run:
> cargo test -- ddbe2befdeadbeefa2d20f09b4a694";
But while I tried to use the following code to get the first argument with:
let args: Vec<String> = env::args().collect();
let apikey = &args[1];
I realized that my tests won't be processed any more whenever I give an argument to cargo test --.
Here is the call without giving arguments:
▶ cargo test -v --
[..]
Finished test [unoptimized + debuginfo] target(s) in 0.05s
Running `/home/pat/git/myprg/target/debug/deps/myrg-7e0bdeadbeef95ff`
running 7 tests
test tests::test_cityid ... ok
test tests::test_coordinate ... ok
test tests::test_apikey ... ok
test tests::test_city ... ok
test tests::test_language ... ok
test tests::test_units ... ok
test tests::test_cities ... ok
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
But no test will be processed when I try to give the apikey in an argument to the tests:
▶ cargo test -v -- ddbe2befdeadbeefa2d20f09b4a694
[..]
Finished test [unoptimized + debuginfo] target(s) in 0.05s
Running `/home/pat/git/myprg/target/debug/deps/myprg-7e0bdeadbeef95ff ddbe2befdeadbeefa2d20f09b4a694";`
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 7 filtered out
One can see that the test was correctly called with the apikey attached.
I do not understand why this happens and diving into rust tests documentation did not lead me to something useful.
Do you have any tips?
Seems that giving secrets through program arguments is not how it's been made.
Instead use a method like this:
fn apikey() -> String {
match std::env::var("APIKEY") {
Ok(key) => key,
Err(_r) => {
eprintln!("error: set API-key with environment valiable APIKEY");
"".to_string()
}
}
}
...I can inject the API key into my tests in github actions with:
env:
APIKEY: ${{ secrets.APIKEY }}
jobs:
build:
steps:
- name: Run tests
run: cargo test --verbose
Thanks for the help!
I've recorded a test using Selenium IDE and am submitting the generated .side file to selenium-side-runner to run on a Selenium Grid built using Zalenium. Is it possible to run a command that calls driver.manage().addCookie() from the test that was submitted to selenium-side-runner? I want to do this to send messages back to Zalenium with test progress and status
I added a command executeScript to the Selenium IDE editor with a target of driver.manage().addCookie({name: 'test', value: 'test'})
I see that the command that selenium-side-runner generated in commons.js was
await driver.executeScript(`driver.manage().addCookie({name:'test', value: 'test'});`);
Doing this causes the browser to report an error JavascriptError: javascript error: driver is not defined
I think what I need is the code to be generated without the driver.executeScript wrapper. Is there a way to accomplish this without exporting my Selenium IDE test to NUnit?
I was able to make this functionality work by crudely modifying the selenium-side-runner package on my Windows dev machine
In file ~\node_modules\selenium-side-runner\node_modules\selianize\dist\selianize.cjs.js
Change
function generateScript(script, isExpression = false) {
return `await driver.executeScript(\`${isExpression ? `return (${script.script})` : script.script}\`${script.argv.length ? ',' : ''}${script.argv.map(n => `vars["${n}"]`).join(',')});`;
}
to
function generateScript(script, isExpression = false) {
if (script.script.indexOf('zalenium') > -1)
{
return script.script;
} else
{
return `await driver.executeScript(\`${isExpression ? `return (${script.script})` : script.script}\`${script.argv.length ? ',' : ''}${script.argv.map(n => `vars["${n}"]`).join(',')});`;
}
}
Now when running a test with selenium-side-runner, calling "executeScript" with any value that contains zalenium will generate the command verbatim in the test script
I have a main feature file where I have included a "setup" feature file that should add some test data. This setup feature file has an annotation that I have called #ignore. However, following the instructions in this Can't be enable to #ignore annotation for the features SO answer, but I am still seeing the setup feature file being run outside of the main test feature.
Main feature file, unsubscribe_user.feature:
Feature: Unsubscribe User
Background:
* def props = read('properties/user-properties.json')
* url urlBase
* configure headers = props.headers
* def authoriZation = call read('classpath:basic-auth.js') { username: 'admin', password: 'admin' }
* def testDataSetup = call read('classpath:com/meanwhileinhell/app/karate/feature/mockserver/testDataSetup.feature') { data1: #(props.data1), data2: #(props.data2) }
Scenario: Unsubscribe user
...
...
Scenario: Remove test data
* def testDataTearDown = call read('classpath:com/meanwhileinhell/app/karate/feature/mockserver/testDataTearDown.feature') { data1: #(props.data1), data2: #(props.data2) }
...
testDataSetup.feature file
#ignore
Feature: Add data to REST Mock Server
Background:
* url mockServerUrlBase
Scenario: Add data
* print 'Adding test data'
Given path 'mapping'
And request { data1: '#(data1)', data2: '#(data2)' }
When method post
Then status 201
Now from my Java runner class, I have added #KarateOptions(tags = "~#ignore").
import org.junit.runner.RunWith;
import com.intuit.karate.KarateOptions;
import com.intuit.karate.junit4.Karate;
import cucumber.api.CucumberOptions;
#RunWith(Karate.class)
#CucumberOptions(features = "classpath:com/meanwhileinhell/app/karate/feature/unsubscribe_user.feature")
#KarateOptions(tags = "~#ignore")
public class KarateTestUnSubscribeUserRunner {
}
However, I can still see my print statement in my setup class being called, and two POSTs being performed. I have also tried running my suite with the following cmd options, but again, still see the feature file run twice.
./gradlew clean test -Dkarate.env=local -Dkarate.options="--tags ~#ignore" --debug
I am following this wrong somewhere? Is there something I can add to my karate-config.js file? I am using Karate version 0.9.0.
Annotations only work on the "top level" feature. Not on "called" features.
If your problem is that the features are being run even when not expected, you must be missing something, or some Java class is running without knowing it. So please follow this process and we can fix it: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
EDIT: I think I got it - please don't mix CucumberOptions, in fact we deprecated it, use only KarateOptions. Even that is not recommended in 0.9.5 onwards and you should move to JUnit 5.
Read the docs: https://github.com/intuit/karate#karate-options
I'm using selenium-server version 3.0.1 and nightwatch version ^0.9.12 on node 8.9.0. My e2e test do run, clicks work, and reading the DOM works, but setValue just doesn't.
For example, the following test:
browser
.url("...")
.waitForElementPresent('select[name=foo]', 5000)
.click('select[name=foo] option:nth-child(2)')
.waitForElementPresent('input[name=bar]', 5000)
.setValue('input[name=bar]', "hello world")
.getValue('input[name=bar]', function(input) {
this.assert.equal(input.value, "hello world");
})
.end();
will open the url, wait for foo and click the second option. It will wait for bar, then fails:
Running: test
✔ Element <select[name=foo]> was present after 24 milliseconds.
✔ Element <input[name=bar]> was present after 28 milliseconds.
✖ Failed [equal]: ('' == 'hello world') - expected "hello world" but got: ""
at Object.<anonymous> (/test/e2e/specs/test.js:49:21)
FAILED: 1 assertions failed and 2 passed (4.692s)
_________________________________________________
TEST FAILURE: 1 assertions failed, 2 passed. (4.9s)
✖ test
- run through apply process (4.692s)
Failed [equal]: ('' == 'hello world') - expected "hello world" but got: ""
If I replace the setValue with a delay and enter a value by hand, the test will pass, so getValue is working.
This does run, and pass, on other systems, but I can't get it working on my own so I think it's a selenium-server issue.
I've tried a lot of the 101 fixes, clearing the npm cache, re-running an npm install, etc. But with no errors other than the failure, how do I debug this?
Assuming you are trying to test using Chrome, you need to update your ChromeDriver.
Chrome 65 was released recently and older ChromeDriver versions are apparently incompatible with it.
Download the latest one from the downloads page.
Make Nightwatch use it, nightwatch.json -
{
...
"selenium": {
...
"cli_args": {
"webdriver.chrome.driver": "path/to/chromedriver.exe"
Assuming Nightwatch uses it (you can see which one it uses using the Windows Task Manager - assuming you are using Windows - look for the command line of chromedriver.exe), setValue should now work again.
The error says it all :
Failed [equal]: ('' == 'hello world') - expected "hello world" but got: ""
In your code block you have induced wait for the WebElement identified as 'input[name=bar]' and next invoked setValue() which is successful as follows :
.waitForElementPresent('input[name=bar]', 5000)
.setValue('input[name=bar]', "hello world")
Now the JavaScript associated with this WebElement identified as 'input[name=bar]' will require some time to render the value within the HTML DOM. But in you code block you are trying to access the entered value (in the previous step) too early. Hence your script finds <null> as the value.
Solution
You need to induce a waiter i.e. expect clause for the value to be rendered within the HTML DOM through the associated JavaScript as follows :
.waitForElementPresent('input[name=bar]', 5000)
.setValue('input[name=bar]', "hello world")
.expect.element('input[name=bar]').to.have.value.that.equals('hello world');
.getValue('input[name=bar]', function(input) {
this.assert.equal(input.value, "hello world");
})
The 'else' value of the elvis/ternary operator in my gradle build file is not setting the property value if I do not run gradle with the "-P" option.
Here's the root project's build.gradle
defaultTasks 'loadConfiguration'
task loadConfiguration << {
def profile = hasProperty('profile') ? profile : 'dev'
ext.profile = profile
def configFile = file('profile.properties')
def config = new ConfigSlurper(profile).parse(configFile.toURL())
ext.config = config
}
configure (subprojects) {
profile = profile //inject property into sub-project
println profile
task buildear << {
ear
}
}
The sub-project 'ear' is in the settings.gradle file.
Below are the results of a build attempt-
With external property set:
$ gradle -Pprofile=wwww
Parallel execution is an incubating feature.
wwww
wwww
wwww
:loadConfiguration
BUILD SUCCESSFUL
Total time: 5.834 secs
With empty external property set:
$ gradle -Pprofile
Parallel execution is an incubating feature.
:loadConfiguration
BUILD SUCCESSFUL
Total time: 5.389 secs
With no external property set:
$ gradle
Parallel execution is an incubating feature.
FAILURE: Build failed with an exception.
* Where:
Build file '/home/robert/codingk/kazi/trunk2/mazama2/build.gradle' line: 13
* What went wrong:
A problem occurred evaluating root project 'mazama2'.
> Could not find property 'profile' on project ':ear'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 2.104 secs
Line 13 is the line "profile = profile //inject property into sub-project"
I cannot figure out why the 'profile' property is not getting set to 'dev'. A few things I've tried without success:
profile= "${profile}
def profile = project.hasProperty('profile') ? profile : 'dev'
project.ext.profile = profile
some remarks about your initial build script.
In your initial build script you declared a task (loadConfiguration) to set the profile. This logic is executed AFTER the configure block below
is executed. That's one reason why "profile" is always null in the configuration block
The second problem is, that you need to be careful about scoping. In the snippet
ext.profile = profile
you add a dynamic property to the loadConfiguration task, not to the project itself. That's why you can't reference
The profile property if you havn't passed it via commandline.
Maybe instead of doing the configuration loading in a seperate task, do it on the top level of your build:
ext.profile = hasProperty('profile') ? profile : 'dev'
def configFile = file('profile.properties')
def config = new ConfigSlurper(profile).parse(configFile.toURL())
ext.config = config
configure (subprojects) {
profile = profile //inject property into sub-project
println profile
task buildear << {
ear
}
}
Got it working with the following modifications. I may be wrong but I believe it had to do with task ordering; declaring loadConfiguration() as a default task did not cause it to be executed first which is what I needed
def loadConfiguration() {
def profile = hasProperty('profile') ? profile : 'dev'
ext.profile = profile
def configFile = file('profile.properties')
def config = new ConfigSlurper(profile).parse(configFile.toURL())
ext.config = config
}
configure (subprojects) {
loadConfiguration()
profile = profile //inject property into sub-project
println profile
task buildear << {
ear
}
}