PHPUnit manual say:
If you point the PHPUnit command-line test runner to a directory it will look for *Test.php files.
see: http://www.phpunit.de/manual/3.6/en/organizing-tests.html#organizing-tests.filesystem
This is wrong!
When i call:
phpunit --config myconfig.xml --bootstrap mybootstrap.php tests
It takes all php files.
First idea was to use blacklist or whitelist in the config xml, but then i realised, that these lists are filters for subject under tests and filters test classes.
Second thought was to use testsuites within the config xml. But at the moment the test suites can be defined only, but not executed via command line (not jet implemented in PHPUnit, ticket is open for more than 1 year).
Next thought was to use a test suite loader, but i can not find a documentation on how to use them and if a tsl is what i think it is.
When i run:
phpunit --config myconfig.xml --bootstrap mybootstrap.php --loader My_Testsuite_Loader tests
PHPUnit takes all php file in "tests/" and executes them. The file "My/Testsuite/Loader.php" will be included. PHPUnit checks if the class My_Testsuite_Loader exists. All fine so far.
I used the "PHPUnit/Runner/StandardTestSuiteLoader.php" as template for "My/Testsuite/Loader.php". It contains the methods "load()" and "reload()". Both methods are never called by the PHPUnit Framework. Why not? I thought to have a own testsuiteloder will give me the oportunity to implement a test suite exclude schema.
Sample file system of my project:
<root>
|-src/MyProject/Package/Object.php
|-tests/MyProject/Package/Object/TestTemplate.php
|-tests/MyProject/Package/Object/GetFooTest.php
|-tests/MyProject/Package/Object/GetBarTest.php
|-tests/phpunit.xml
|-tests/bootstrap.php
|-tests/My/Testsuite/Loader.php
As you can see i use one file for all tests about a sut (method under test). All these *Test.php files inherits from TestTemplate (TestCase). In TestTemplate.php is a setup which initializes the object (Object.php) and stores it in a private member var.
How to use the test suite loader / for what is it meant to be?
(How to exclude test classes that do not fit to the pattern: "*Test.php"?)
You need to take out the 'tests' argument like so
phpunit --config myconfig.xml --bootstrap mybootstrap.php
Then in your myconfig.xml
<testsuites>
<testsuite name="AllTests">
<directory>.</directory>
</testsuite>
<testsuites>
<filter>
<blacklist>
<directory suffix=".xml">.</directory>
<file>MyProject/Package/Object/TestTemplate.php</file>
</blacklist>
</filter>
Related
I am using Gradle 7.6, Karate 1.3.1, Java 17.0.5 and Junit 5.8.1.
I want to configure a Jenkin job for each feature to create a health check monitor. I need gradle commands to run feature files using tags #smoke, #regression, #featureName etc.,
I have tried with the following command, it worked earlier and stopped working recently.
./gradlew test -Dkarate.options="--tags #smoke" -Dtest.single=TestRunner#testTagsWithoutFeatureName
Where TestRunner is the following Java class
import com.intuit.karate.junit5.Karate;
public class TestRunner {
#Karate.Test
Karate testTagsWithoutFeatureName() {
return Karate.run().tags("#smoke").relativeTo(getClass());
}
}
My advice is use the Runner class, that is better designed for running tests in CI. The JUnit helpers are just for local-dev convenience: https://stackoverflow.com/a/65578167/143475
It should be possible to even pass a feature to karate.options as the last argument. Which might be more convenient than writing a Java class for every combinations. You should experiment.
Otherwise no suggestions, but if you feel there's a bug, follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
How to run the JUnit test suite containing a set of test cases(groovy based) from the command line. Following is the test suite class generated by eclipse.
package com.example.testclasses;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
#RunWith(Suite.class)
#SuiteClasses({ abc.class, xyz.class })
public class AllTests {
}
The above test suite works when I run the above test suite(AllTests)as JUnit from eclipse, however, I want to run the test suite(AllTests) from the command line. How do I do this?
Info: I am using Geb(Groovy) based testing where all the test cases(example: abc, def) are groovy based(having .groovy extension).
If you wish to run your tests from the command line I would suggest using a build system. My personal choice would be to use Gradle but you could probably also get away with using Maven.
The benefit of using a build system, apart from being able to run the tests from the command line, is that it will help you manage your dependencies and it will be easier to build the project for others working on the same codebase - they won't have to manually setup all the dependencies and their versions in the IDE.
Try this:
java -cp /path/to/groovy/embeddable/groovy-all-1.8.1.jar groovy.lang.GroovyShell AllTests.groovy
where 1.8.1 should be replaced with your version of groovy-all-*.jar
I guess my question is somewhat similar to Multiple JUnit XML results on Jenkins, publish with separate graph? but with another scope.
We need to test our software against multiple environments (e.g. windows/linux, different browsers, server environments). As far as I've seen this is a pretty common use case, but what I can't seem to find is how to have a clean report, where you can distinguish between these environments.
I tried the following simple pipeline:
node {
// dummy report
def testResultXml = '''
<testsuite name="ComponentTests" tests="3">
<testcase classname="foo1" name="ASuccessfulTest"/>
<testcase classname="foo2" name="AnotherSuccessfulTest"/>
<testcase classname="foo3" name="AFailingTest">
<failure type="NotEnoughFoo"> details about failure </failure>
</testcase>
</testsuite>
'''
stage('test run 1'){
writeFile file: 'test1.xml', text: testResultXml, encoding: 'UTF-8'
junit "test1.xml"
}
stage('test run 2'){
writeFile file: 'test2.xml', text: testResultXml, encoding: 'UTF-8'
junit "test2.xml"
}
}
and got a test report from Jenkins showing me the 2 failed tests. My problem is now, if only one fails I don't know which.
Our output is from a ant junit task that has the same output (except some properties) for all environments.
Even the article about declarative pipeline 1.2 shows multi-environment tests as a example:
https://jenkins.io/blog/2017/09/25/declarative-1/ but says nothing about the presentation of the "Run Tests On Linux" and "Run Tests On Windows" results.
Did I miss something or is there a plugin that can publish the missing information (simply having the directory of the read junit xml file would help)?
After some testing the issue seems to be fixed already in version 1.22 of the Jenkins JUnit plugin.
The related issues where:
https://issues.jenkins-ci.org/browse/JENKINS-27395
https://issues.jenkins-ci.org/browse/JENKINS-37598
How to run just one method of a test case for debugging with Gradle? I have tried:
gradle test -tests example.TestFoo#testMethod1 --debug-jvm
but it ends up with following error:
No tests found for given includes: example.TestFoo#testMethod1
The test TestFoo class has testMethod1(), testMethod2(), etc.
Use . instead # in your tests filter expression to point to a method name:
gradle test --tests example.TestFoo.testMethod1 --debug-jvm
You can find more examples on filtering tests in 48.14.3. Test filtering documentation section.
In the PHPUnit docs, it says that it's possible to get code coverage data:
PHPUnit_Extensions_SeleniumTestCase can collect code coverage information for tests run through Selenium:
Copy PHPUnit/Extensions/SeleniumTestCase/phpunit_coverage.php into your webserver's document root directory.
In your webserver's php.ini configuration file, configure PHPUnit/Extensions/SeleniumTestCase/prepend.php and PHPUnit/Extensions/SeleniumTestCase/append.php as the auto_prepend_file and auto_append_file, respectively.
In your test case class that extends PHPUnit_Extensions_SeleniumTestCase, use
protected $coverageScriptUrl = 'http://host/phpunit_coverage.php';
to configure the URL for the phpunit_coverage.php script.
I haven't been able to get this to output any coverage information. I am able to get code coverage info through normal unit tests.
For my app running at http://localhost/ts2_templates/ I've copied phpunit_coverage.php to http://localhost/phpunit_coverage.php.
I've added the following to php.ini:
auto_prepend_file = "/path/to/pear/share/pear/PHPUnit/Extensions/SeleniumTestCase/prepend.php"
auto_append_file = "/path/to/pear/share/pear/PHPUnit/Extensions/SeleniumTestCase/append.php"
... and verified they are being called with a die("yep it's me");.
Finally, I added the following to my test case:
<?php
class WebTest extends PHPUnit_Extensions_Selenium2TestCase
{
# added line below
protected $coverageScriptUrl = 'http://localhost/phpunit_coverage.php';
protected function setUp()
{
$this->setBrowser('firefox');
$this->setBrowserUrl('http://localhost/ts2_templates');
}
public function testTitle()
{
$this->url('http://localhost/ts2_templates');
$this->assertContains('test', $this->title());
}
}
?>
Here's the command for running the test with code coverage, generated by PHPStorm:
/Applications/MAMP/bin/php5.3/bin/php -dxdebug.coverage_enable=1 /private/var/folders/pp/0t4y41f95j5313qm_f8b42fw0000gn/T/ide-phpunit.php --coverage-clover /path/to/coverage/ts2_templates$WebTest.coverage --no-configuration WebTest /Users/Ian/php/ts2_templates/tests/WebTest.php
Heres the output of the coverage XML file:
<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1341015508">
<project timestamp="1341015508">
<metrics files="0" loc="0" ncloc="0" classes="0" methods="0" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="0" coveredstatements="0" elements="0" coveredelements="0"/>
</project>
</coverage>
The test itself passes.
I have verified there are no exit or die statements anywhere in the code.
Any ideas?
I also had some issues getting things to work. The following post in the YII forum by Samuel Goldstein helped me out:
I ended up moving the prepend.php and append.php into my project's document root.
I also found that the temporary file location made a difference - I originally was trying to save them to /tmp/ and PHP was silently failing. When I changed $GLOBALS['PHPUNIT_COVERAGE_DATA_DIRECTORY'] to myroot/protected/runtime/tmp and did a chmod 777 on that directory, it started working.
One thing that might frustrate you a bit is that code run through Ajax does not get flagged as being covered.
This appears to be a known problem with Selenium. Google "github sebastianbergmann phpunit-selenium issues" and track down closed issue #22 for more information.
It is definitely working. I have set up selenium tests in symfony measuring the coverage according to the documentation.
The biggest issue I had was, that the coverage data had the wrong path to the files in it, and therefore could not align the sources with the coverage data. This is because I executed tests from a different place, as the server kept the files in. Therefore I tweaked the append.php to rewrite the path to the place where my source files are.