I'm using the eclEmma plugin to test code coverage for my gwt application. I've written jUnit test classes for client code, such as testing get/set methods etc. as well as jUnit tests for rpc services. I used "syncproxy" to test my equivalent GreetService, GreetServiceAsync and GreetServiceImpl rpc services. For example I have a location service that gets a users location and this is part of my test class:
public class LocationServiceTest {
private static LocationService rpcService =
(LocationService) SyncProxy.newProxyInstance(LocationService.class,
"http://localhost:...", "location");
#Test
public void testAdministrativeAreaLevel2LocationService() {
String result = rpcService.getAddress("49.28839970000001,-123.1259316");
assertTrue((result != null) && (result.startsWith("Vancouver")));
}
The jUnit tests all pass, but when I run eclEmma on my project (I right click the project, select "Coverage as" then "jUnit test") I only get coverage results for client code, and 0% coverage for all my server code.
Any suggestions for how to get eclEmma to cover server code? Or for what I might be doing wrong?
EclEmma tracks coverage on code launched at the test jvm (the vm you launch when you run the test). You seem to be running your server before, so eclEmma "can't see" its coverage. You could try running the server inside your tests, with Cargo, for example.
Related
I am running IntelliJ IDEA 2018.3.1 and am attempting test a class with the integrated test runner. The test seems to compile but not run.
This is a multi-module Maven project, and other modules have tests that run. However, I have not been able to find any differences between the projects. The surefire plugin is specifically defined on this project, and <skipTests> is specifically set to false. I have reimported the project several times in case the maven configuration is affecting the built-in runner.
The image below is the only output I get. Debug/Breakpoints will not stop.
If anyone can help or throw possibilities at me, I would appreciate it.
Edit:
Here's a simplified version of the test I'm attempting to run:
package com.jason;
// imports
#RunWith(BlockJUnit4TestRunner.class)
public class MyTest {
private ClassUnderTest clazz;
private DaoClass dao;
#Before
public void setUp() {
// using Mockito to mock the DaoClass
// injecting the DAO into the ClassUnderTest
}
#Test
public void testMethod() {
Assert.assertTrue(true);
}
}
I attempt to run the test by right-clicking on the method annotated with #Test and clicking run. The option to run the test DOES appear in the context menu. When I do so, all that appears is the screenshot.
I have attempted to do the following to troubleshoot the issue:
In the pom.xml file for the appropriate module, I have manually specified the surefire plugin in the <build><plugins> section. I then did a reimport to pick up the changes.
I have put breakpoints in the code and run the test in debug mode.
I have attempted to log output, both with an slf4j logger and a System.out.println()
I have attempted to find any differences in the IDEA .iml file between a module where the tests run and this module where the tests do not run.
I have written a very simple test class, with a method annotated with #Test and containing the line Assert.assertTrue(true)
Edit 2
Attempting to run mvn test -Dcontrollername produces the following output:
Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project rma-svc: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
Edit 3
I've updated my Maven surefire plugin to 2.22.2 and am not seeing the forked JVM issue any longer. However, running mvn test -DskipTests=false outputs No tests were executed!
The title may be a bit confusing at this point; hopefully I can clear it up.
What I Have
I'm running Cucumber JVM with Selenium WebDriver to automate our system test cases. These test cases are currently stored in JIRA using the XRay Test Management plugin. XRay also provides APIs to fetch the feature files as well as upload the results back to JIRA.
I have created a custom JIRA utility class to download the tests as feature files and upload the test results from and to JIRA - as well as demonstrated that it does work. These are run in the #BeforeClass and #AfterClass in the Cucumber Runner class respectively.
I have also demonstrated that the developed test framework does work by manually running with feature files created on my computer.
What I Want
I want to be able to (eventually) run the automation test framework automatically with our CI tools. Along with this, it would pull the defined automation tests from JIRA and push the test results back to JIRA.
I do not want the feature files stored with the code. In my opinion, this defeats the purpose of it being dynamic as the tests we execute will change over time (in number executed and the steps themselves).
What Is Happening (Or More Specifically, Not Happening)
When I try to execute the Cucumber Runner class without any feature files in the framework, Cucumber says "No features found at [src/test/resources/features/]". This is understandable since there are no feature files (yet).
However, it does not run the #BeforeClass; thus it does not download the feature files to be run. I have tried this both with and without tags in the runner class.
Code
#RunWith(Cucumber.class)
#CucumberOptions(
tags={"#smoketests"},
features= {"src/test/resources/features/"},
plugin={"json:target/reports/cucumber.json"},
monochrome=true)
public class RunCucumberTest {
#BeforeClass
public static void executeBeforeTests() {
JiraUtil.getFeatureFiles();
//String browser = "firefox";
String browser = "chrome";
//String browser = "safari";
//String browser = "edge";
//String browser = "ie";
DriverUtil.getInstance().setDriver(browser);
}
#AfterClass
public static void executeAfterTests() {
DriverUtil.getInstance().resetDriver();
JiraUtil.uploadTestResults();
}
}
Back To My Question
How can I execute the JIRA Util code so I can download the feature files?
Is it possible to achieve what I want? Or do I have to admit defeat and just have all the feature files stored with the code?
This is the expected behavior when using JUnit. A test suite will not invoke the #BeforeClass, #AfterClass or #ClassRule when there are no tests in the suite or if all tests are ignored[1]. This avoids the execution of a potentially expensive setup for naught.
This does mean you can't use a class rule to bootstrap your tests. Nor should you attempt to do so. In a build process it is a good practice to fetch all sources and resources prior to compilation.
If you are using maven could write a maven instead and attach it to the generate-test-sources phase[2]. Creating a maven plugin is a bit more involved then a JUnit Rule but not prohibitively so. Check the Guide to Developing Java Plugins.
I assume there are similar options for Gradle.
I am using ASP.NET 5 RC1 and I need to write integration tests ...
So on the Test project, ASPNET5_WEB_TEST, I have the following:
public class IntegrationTests {
private readonly TestServer _server;
private readonly HttpClient _client;
public IntegrationTests() {
_server = new TestServer(TestServer.CreateBuilder().UseStartup<Startup>());
_client = _server.CreateClient();
}
// Test methods ...
}
The Startup class is from the ASP.NET 5 project I am testing: ASPNET5_WEB
When I run the test I get the following error:
The configuration file 'C:\Projects\ASPNET5_TEST\config.json' was not found and is not optional.
I know I get this error because on Startup I have:
builder
.AddJsonFile("config.json", false)
.AddJsonFile($"config.{environment.EnvironmentName}.json", true);
To fix this error I need to copy, at least, config.json from my web project, ASPNET5_WEB, to my test project, ASPNET5_WEB_TEST. But this means I will need to maintain duplicate config.json or at least copy it every time I make a change.
Can't I tell TestServer to use Startup of the web project and also its config.*.json files?
And can I have a config.testing.json and set on the TestServer the environment to Testing so the Startup code uses config.json and config.testing.json?
I assume you're using the TestServer from aspnet, if so, it wasn't built to support the way you're config files are read. The TestServer is used to run simple integration tests for their "hosting engine" but not for integrations tests for a website.
Their ApplicationDeployerFactory class is what you can use however. Refer to this as an example of how to run an "integration" server. I've used selenium in conjunction with that to run integration tests against the project I'm working on atm.
Yes, you can.
Take a look at this issue https://github.com/aspnet/Mvc/issues/3410 and The mentioned package.
Basically you need to implement your own IApplicationEnvironment
Normally, I would start Jetty by constructing a Server instance, setting a connector, a handler, and LifeCycleListener, followed by a call to start() on the Server instance. I haven't the foggiest idea how to make this happen with the jettyRun task in Gradle. The documentation is confusing to me, and I have yet to find an example of how this task works, other than page after page of gradle jettyRun.
This task is appealing to me because it allegedly returns immediately after execution. This is helpful for running Selenium tests after my webapp is running from Jenkins. I tried to do this via a JavaExec task, but this won't work since the JavaExec task does not terminate until the underlying JVM terminates as well.
It sounds like you want to start Jetty for in-container integration tests. Besides having a look at the source code these two posts should get you started:
War and Jetty plugins with http integration tests
Right way to do basic web integration testing?
The key feature you are looking for, starting Jetty in the background, is jettyRun.daemon = true.
What I'm using for integration test in build.gradle is looks like below. I think this code is simple and intuitive.
test {
exclude '**/*IntegrationTest*'
}
task integrationTest(type: Test) {
include '**/*IntegrationTest*'
doFirst {
jettyRun.httpPort = 8080 // Port for test
jettyRun.daemon = true
jettyRun.execute()
}
doLast {
jettyStop.stopPort = 8091 // Port for stop signal
jettyStop.stopKey = 'stopKey'
jettyStop.execute()
}
}
I want to add a suite of Selenium tests as part of a global PHPUnit test suite for an application. I have hooked the suite of Selenium tests into the global AllTests.php file and everything runs fine whilst the Selenium server is running.
However, I would like the script to skip the Selnium tests if the Selenium server isn't running so other developers aren't forced to install Selenium server in order for the tests to run. I would normally try to connect within the setUp method of each testcase and mark the tests as skipped if this failed, but this seems to throw a RuntimeException with message:
The response from the Selenium RC server is invalid: ERROR Server Exception: sessionId should not be null; has this session been started yet?
Does anyone have a method for marking the Selenium tests as skipped in this scenario?
You could use test dependencies that were introduced in PHPUnit 3.4.
Basically
write a test that checks whether Selenium is up.
If not, call $this->markTestAsSkipped().
Make all your selenium requiring tests depend on this one.
My preferred selenium / PHPUnit Configuration:
Maintaining integration (selenium) tests can be a lot of work. I use the firefox selenium IDE for developing test cases, which doesn't support exporting test suites to PHPUnit, and only supports individual test cases. As such - if I had to maintain even 5 tests, that'd be a lot of manual work to re-PHPUnit them every time they needed to be updated. That is why I setup PHPUnit to use Selenium IDE's HTML Test files! They can be reloaded & reused between PHPUnit & selenium IDE
<?php
class RunSeleniumTests extends PHPUnit_Extensions_SeleniumTestCase {
protected $captureScreenshotOnFailure = true;
protected $screenshotPath = 'build/screenshots';
protected $screenshotUrl = "http://localhost/site-under-test/build/screenshots";
//This is where the magic happens! PHPUnit will parse all "selenese" *.html files
public static $seleneseDirectory = 'tests/selenium';
protected function setUp() {
parent::setUp();
$selenium_running = false;
$fp = #fsockopen('localhost', 4444);
if ($fp !== false) {
$selenium_running = true;
fclose($fp);
}
if (! $selenium_running)
$this->markTestSkipped('Please start selenium server');
//OK to run tests
$this->setBrowser("*firefox");
$this->setBrowserUrl("http://localhost/");
$this->setSpeed(0);
$this->start();
//Setup each test case to be logged into WordPress
$this->open('/site-under-test/wp-login.php');
$this->type('id=user_login', 'admin');
$this->type('id=user_pass', '1234');
$this->click('id=wp-submit');
$this->waitForPageToLoad();
}
//No need to write separate tests here - PHPUnit runs them all from the Selenese files stored in the $seleneseDirectory above!
} ?>
You can try skipWithNoServerRunning()
For more information follow this link