Executing Specific Geb Tests according to environment - testing

I have a set of Spec tests I am executing within a Grails Project.
I need to execute a certain set of Specs when I am on local, and another set of Spec when I run the pre-prod environment.
My current config is executing all my specs at the same time for both environements, which is something I want to avoid.
I have multiple environments, that I have configured in my GebConfig:
environments {
local {
baseUrl = "http://localhost:8090/myApp/login/auth"
}
pre-prod {
baseUrl = "https://preprod/myApp/login/auth"
}
}

You could use a spock config file.
Create annotations for the two types of tests - #Local and #PreProd, for example in Groovy:
import java.lang.annotation
#Retention(RetentionPolicy.RUNTIME)
#Target([ElementType.TYPE, ElementType.METHOD])
#Inherited
public #interface Local {}
Next step is to annotate your specs accordingly, for example:
#Local
class SpecificationThatRunsLocally extends GebSpec { ... }
Then create a SpockConfig.groovy file next to your GebConfig.groovy file with the following contents:
def gebEnv = System.getProperty("geb.env")
if (gebEnv) {
switch(gebEnv) {
case 'local':
runner { include Local }
break
case 'pre-prod':
runner { include PreProd }
break
}
}
EDIT: It looks like Grails is using it's own test runner which means SpockConfig.groovy is not taken into account when running specifications from Grails. If you need it to work under Grails then the you should use #IgnoreIf/#Require built-in Spock extension annotations.
First create a Closure class with the logic for when a given spec should be enabled. You could put the logic directly as a closure argument to the extension annotations but it can get annoying to copy that bit of code all over the place if you want to annotate a lot of specs.
class Local extends Closure<Boolean> {
public Local() { super(null) }
Boolean doCall() {
System.properties['geb.env'] == 'local'
}
}
class PreProd extends Closure<Boolean> {
public PreProd() { super(null) }
Boolean doCall() {
System.properties['geb.env'] == 'pre-prod'
}
}
And then annotate your specs:
#Requires(Local)
class SpecificationThatRunsLocally extends GebSpec { ... }
#Requires(PreProd)
class SpecificationThatRunsInPreProd extends GebSpec { ... }

Related

How to run multiple test suites in another test suite with geb

I would like to run all my test suites. I want to call the test suites in one test suit called Allrun.
Can I call test suites in another test suite. I am looking for something like this:
class Myfirst extends GebReportingSpec {
def myfunction(){
when:''
at mypage1
and:''
element1.text() == "mytext1"
}
}
class Mysecond extends GebReportingSpec {
def mysecondFunction() {
when:''
at mypage2
and:''
element2.text() == "mytext2"
}
}
class AlltestSuites extends GebReportingSpec {
Myfirst myfirst = new Myfirst ()
Mysecond mysecond = new Mysecond ()
def allrun(){
myfirst.myfunction()
mysecond.mysecondFunction()
}
}
How can i do this? Does anyone has an idea
You can create a AllRun.groovy file like below to run all class files in the specified order [Myfirst.class, Mysecond.class, etc].
package geb.groovy
import org.junit.runner.RunWith
import org.junit.runners.Suite
import geb.groovy.scripts.Myfirst
import geb.groovy.scripts.Mysecond
#RunWith(Suite.class)
#Suite.SuiteClasses([Myfirst.class, Mysecond.class])
class CustomJUnitSpecRunner {
}
And in build.gradle systemProperties System.properties, exclude the individual Myfirst.class, Mysecond.class files. So that they run only once through AllRun file.
include 'geb/groovy/AllRun/**'
exclude 'geb/groovy/scripts/**'
Groups of specs can be run using the command below, where package is a directory under src/test/groovy/specs:
mvn test -Dtest=specs.package.*
If you are relying on specs to be run in a particular order, I would reconsider your approach. If you require data in a certain state then make use of setup() and setupSpec() methods. Also consider running related tests within a single spec and run these in the order the are written by using the #Stepwise annotation.
#Stepwise
class MyStepWiseTest extends GebReportingSpec {
def myfunction(){
when:''
at mypage1
and:''
element1.text() == "mytext1"
}
def mysecondFunction() {
when:''
at mypage2
and:''
element2.text() == "mytext2"
}
}

access environment variables in jenkins shared library code

When I use my new shared library I cannot access environment variables for any src class which is executed either directly by the Jenkinsfile or via a var/*.groovy script. This problem persists even when I add withEnv to the var/*groovy script.
What is the trick to get environment variables to propagate to jenkins shared library src class execution?
Jenkinsfile
withEnv(["FOO=BAR2"]) {
println "Jenkinsfile FOO=${FOO}"
library 'my-shared-jenkins-library'
lib.displayEnv()
Shared Library var/lib.groovy
def displayEnv() {
println "Shared lib var/lib FOO=${FOO}"
MyClass c = new MyClass()
}
Shared Library src/MyClass.groovy
class MyClass() {
MyClass() {
throw new Exception("Shared lib src/MyClass FOO=${System.getenv('FOO')")
}
}
** Run Result **
Jenkinsfile FOO=BAR
Shared lib var/lib FOO=BAR
java.lang.Exception: Shared lib src/MyClass FOO=null
...
It sure looks like the only way to handle this is to pass the this from Jenkins file down to the var/lib.groovy and harvest from that object
Jenkinsfile
withEnv(["FOO=BAR2"]) {
library 'my-shared-jenkins-library'
lib.displayEnv(this)
var/lib.groovy
def displayEnv(script) {
println "Shared lib var/lib FOO=${FOO}"
MyClass c = new MyClass(script)
}
src class
MyClass(def script) {
throw new Exception("FOO=${script.env.FOO}")
}
I believe you can populate the environment variable as below, where shared library can access.
Jenkisfile
env.FOO="BAR2"
library 'my-shared-jenkins-library'
lib()
vars/lib.groovy
def call(){
echo ("FOO: ${FOO}")
echo ("FOO:"+env.FOO)
}
Another method is use the "steps" variable:
In Jenkinsfile
mypackages.myclass.mymethod(steps)
In src
class myclass implements Serializable {
void mymethod(steps) {
String myEnvVar = steps.sh(returnStdout: true, script: "env | grep 'myVar' | cut -f 2- -d '='")
}
}
I stumbled upon this problem lately, so I'm gonna add my $0.02.
The basic template I use for var/*.groovy is:
// var/myMethod.groovy
import cool.package.Clazz
def call(Map m) {
m.put('env', env)
m.put('steps', steps)
new Clazz(m).call()
}
And the template for src/**/*.groovy
// src/cool/package/Clazz.groovy
class Clazz {
private String cool_field_1 = "default-value-1"
private int cool_value = 42
def env
def steps
def call() {
steps.echo("env.BUILD_TAG: ${env.BUILD_TAG}")
//...
}
}
In Jenkinsfile it is used standard way:
#Library('mylib#mybranch')
pipeline {
stages {
stage('St 1') {
steps { myMethod('cool_value': 43) }
}
}
}
Disclaimer: I don't do Groovy but since it looks similar to Java I can use it a little. Also using Map seems to give the advantage of quite flexible interface.
Not sure what the experts will say about the solution but I was able to access the variables defined in my Jenkinsfile from the shared library using evaluate.
Jenkinsfile
myVar = "abc"
vars/test.groovy
String myVar = evaluate("myVar")
For me this just works.
Jenkinsfile:
#Library('jenkins-library') _
pipeline {
agent any
environment {
FOO = 'bar'
}
stages {
stage('Build') {
steps {
script {
buildImage()
...
The library vars/buildImage.groovy:
def call() {
println(this.env.FOO)
println(env.FOO)
}
So to pass the environment to a class in the library, just use this in the vars/yourfunc.groovy.

phpunit: reusing dataprovider

I want to run multiple test cases against the content of a whole set of files. I could use a data provider to load my files and use the same provider for all the tests like this:
class mytest extends PHPUnit_Framework_TestCase {
public function contentProvider() {
return glob(__DIR__ . '/files/*');
}
/**
* #dataProvider contentProvider
*/
public function test1($file) {
$content = file_get_contents($file);
// assert something here
}
...
/**
* #dataProvider contentProvider
*/
public function test10($file) {
$content = file_get_contents($file);
// assert something here
}
}
Obviously that means if I have 10 test cases, each file is loaded 10 times.
I could adjust the data provider to load all files and return one big structure with all the contents. But since the provider is called separately for each test it would still mean each file is loaded 10 times and in addition it would load all files into memory at the same time.
I could of course condense the 10 tests into one test with 10 assertions, but then it would abort right after the first assertion fails and I really want a report of all things that are wrong with the file.
I know that data providers can also return an iterator. But phpunit seems to rerun the iterator separately for each test, still resulting in loading each file 10 times.
Is there a clever way to make phpunit run an iterator only once and pass the result to each test, before continuing?
Test dependencies
If some tests are dependents, you should use the #depends annotation to declare Test dependencies. The data returned by the dependency is used by the test declaring this dependency.
But, if a test declared as dependency failed, the dependent test is not executed.
Statically stored data
To share data between tests, it's common to setup fixtures statically.
You can use the same method with data providers:
<?php
use PHPUnit\Framework\TestCase;
class MyTest extends TestCase
{
private static $filesContent = NULL;
public function filesContentProvider()
{
if (self::$filesContent === NULL) {
$paths = glob(__DIR__ . '/files/*');
self::$filesContent = array_map(function($path) {
return [file_get_contents($path)];
}, $paths);
}
}
/**
* #dataProvider filesContentProvider
*/
public function test1($content)
{
$this->assertNotEmpty($content, 'File must not be empty.');
}
/**
* #dataProvider filesContentProvider
*/
public function test2($content)
{
$this->assertStringStartsWith('<?php', $content,
'File must start with the PHP start tag.');
}
}
As you can see, it's not supported out of the box. As the test class instance is destroyed after each test method execution, you have to store the initialized data in a class variable.

How to capture a screenshot with PHPUnit and Selenium2 when the test fails?

I'm using PHPUnit 4.6 and PHPUnit Selenium 1.4.2 with PhantomJS. I want capture a screenshot with the last page when selenium test fails.
In PHPUnit Manual there is a example for Selenium 1, but I'm trying use with Selenium 2, because I need use GhostDriver.
WebTestCase.php
class WebTestCase extends PHPUnit_Extensions_Selenium2TestCase
{
protected $captureScreenshotOnFailure = TRUE;
protected $screenshotPath = '/../../screenshots';
protected $screenshotUrl = 'http://localhost:8080/screenshots';
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
}
Test.php
class Test extends WebTestCase
{
public function testTitle()
{
$this->url('');
assertEquals($this->title(), "My App");
}
}
But this not capture a screenshot.
$ vendor/bin/phpunit
PHPUnit 4.6-ge85198b by Sebastian Bergmann and contributors.
Configuration read from /MyApp/phpunit.xml
F
Time: 231 ms, Memory: 5.50Mb
There was 1 failure:
1) Test::testTitle
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-''
+'My App'
/MyApp/tests/functional/Test.php:7
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
Hmm. The difference between SeleniumTestCase and Selenium2TestCase is not really good documented in the PHPUnit Manual. Also there is no clear separation and not enough usage examples for common cases on Selenium2.
$captureScreenshotOnFailure does not exist on
PHPUnit_Extensions_Selenium2TestCase.
Anyway, let's try putting this together:
<?php
class Test extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
public function testEnterText()
{
$this->url("/");
try {
$this->assertEquals($this->title(), "My App");
} catch (Exception $e) {
$this->screenshot( __DIR__.'/'.$this->getName().'-'.time(). '.png');
}
}
public function screenshot($file)
{
$filedata = $this->currentScreenshot();
file_put_contents($file, $filedata);
}
}
The try-catch-block: in the try part the assertion is done, if the assertion fails, the exception is caught. The catch-block gives us a chance to (grab details of the exception or re-throw it or) make a screenshot.
The main function is $this->currentScreenshot(), which was used in this test
https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCaseTest.php#L733
ScreenshotListener
Please note that there is a ScreenshotListener around, which might be worth looking at:
https://github.com/giorgiosironi/phpunit-selenium/blob/master/PHPUnit/Extensions/Selenium2TestCase/ScreenshotListener.php
With usage example over at https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCase/ScreenshotListenerTest.php
This might be a cleaner implementation to grab test failures and make shots.
Combining the solutions from #Jens A. Koch and #John Joseph, we get this:
<?php
class homepageTest extends PHPUnit_Extensions_Selenium2TestCase {
private $listener;
public function setUp() {
// Your screenshots will be saved in '/var/www/vhosts/screenshots/'
$screenshots_dir = '/var/www/vhosts/screenshots/';
$this->listener = new PHPUnit_Extensions_Selenium2TestCase_ScreenshotListener($screenshots_dir);
$this->setBrowser('firefox');
$this->setBrowserUrl('https://netbeans.org');
}
public function testNetbeansContainsHorses() {
$this->url('https://netbeans.org');
$this->assertContains('Equestrian', $this->title()); // Will fail on NetBeans page.
}
public function onNotSuccessfulTest($e) {
$this->listener->addError($this, $e, microtime(true));
parent::onNotSuccessfulTest($e);
}
}
A way of doing this across all your web tests is to override one of the test failure functions from the parent test case class, and capture your screenshot there.
Example:
class MyBaseWebTests
{
$this->directory = '/some_path_to_put_screenshots_in/';
// Override PHPUnit_Extensions_Selenium2TestCase::onNotSuccessfulTest
public function onNotSuccessfulTest(Exception $e)
{
$filedata = $this->currentScreenshot();
$file = $this->directory . get_class($this) . '.png';
file_put_contents($file, $filedata);
parent::onNotSuccessfulTest($e);
}
}
Now, after any of your web tests fail, they will dump a screenshot in that folder with the name of the web test class as the filename.
Use this to save screenshot..very useful in case of headless browser.
$fp = fopen('path/35.png', 'wb');
fwrite($fp, $this->currentScreenshot());
fclose($fp);
sleep(1);

Grails integration test failing with MissingMethodException

I'm attempting to test a typical controller flow for user login. There are extended relations, as with most login systems, and the Grails documentation is completely useless. It doesn't have a single example that is actually real-world relevant for typical usage and is a feature complete example.
my test looks like this:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
controller.login()
assert "/user/main" == response.redirectUrl
}
}
The controller does:
def login() {
if (!params.login) {
return
}
println("Email " + params.login.email)
Person p = Person.findByEmail(params?.login?.email)
...
}
which fails with:
groovy.lang.MissingMethodException: No signature of method: immigration.Person.methodMissing() is applicable for argument types: () values: []
The correct data is shown in the println, but the method fails to be called.
The test suite cannot use mocks overall because there are database triggers that get called, the result of which will need to be tested.
The bizarre thing is that if I call the method directly in the test, it works, but calling it in the controller doesn't.
For an update: I regenerated the test directly from the grails command, and it added the #Test annotation:
#TestFor(UserController)
class UserControllerTests extends GroovyTestCase {
#Test
void testLogin() {
params.login = [email: "test1#example.com", password: "123"]
Person.findByEmail(params.login.email)
controller.login()
}
}
This works if I run it with
grail test-app -integration UserController
though the result isn't populated correctly - the response is empty, flash.message is null even though it should have a value, redirectedUrl is null, content body is empty, and so is view.
If I remove the #TestFor annotation, it doesn't work even in the slightest. It fails telling me that 'params' doesn't exist.
In another test, I have two methods. The first method runs, finds Person.findAllByEmail(), then the second method runs and can't find Person.findAllByEmail and crashes with a similar error - method missing.
In another weird update - it looks like the response object is sending back a redirect, but to the application baseUrl, not to the user controller at all.
Integration tests shouldn't use #TestFor. You need to create an instance of the controller in your test, and set params in that:
class UserControllerTests extends GroovyTestCase {
void testLogin() {
def controller = new UserController()
controller.params.login = [email:'test1#example.com', password:'123']
controller.login()
assert "/user/main" == controller.response.redirectedUrl
}
}
Details are in the user guide.
The TestFor annotation is used only in unit tests, since this mocks the structure. In the integration tests you have access of the full Grails environment, so there's no need for this mocks. So just remove the annotation and should work.
class UserControllerTests extends GroovyTestCase {
...
}