Scenario: Below negative scenario to be tested during Integration test. currently the test case getting failed due exit and not reaching to the test method.
example :
private void method1(int a){
try{
if(a == 0){
throw exception();
}else{
---
}
}catch(exceptionclass e){
System.exit(1);
}
}
Sound like a bad smell to me that calling a method on a object can cause JVM to exist. Normally it should be done in the main method.
So I would refactor your codes such that your testing object will throw a kind of Exception to indicate that some kind of fatal error happens such that the main method can catch it and terminate the JVM.
Then you can simply test that if it will throw this Exception from your test case.
Related
In my framework - , Test & PageObjModel classes are there. incase any method is not executed because of wrong xpath or other any issue the Test is terminating but in the Report-Console it is showing all the Testcase are PASS.
Any issue in the Action methods of PageObjModel, some times it will skip that particular line of code and continue execution. sometimes entire test is terminating. in both the case Report-Console it is showing all the Testcase are PASS i.e No failures of Testcases.
Please have a look into below example of code and suggest me to fail the TC incase of failures.
I tried to include Assert.fail(), Assert.true(fail) in the "catch" block, though it is showing PASS results.
PageObjModel:
class A{
// list of xpaths
public static fibal String Login_xpath = "//*[#id='login'];
public void login(){
try{
// All Action methods are define here
}catch (Exception e){
e.printStackTrace();
//Assert.assertTrue(false);
Assert.fail();
} } }
Test class:
#Test
public void loginTest(){
//calling methods
xx.login();
}
This is just sample code. please help me where can I put Assertions to fail the TC when failure occurs in the code
If we are using try-catch in your code, please add throw as well. otherwise, it won't inform testng about the failure.
catch (AssertionError e) {
e.printStackTrace();
takeSnapShot(action);
throw e;
}
The following test succeeds with Process finished with exit code 0. Note, this test does print the exception to the logs, but does not fail the test (which is the behavior I want).
#Test
fun why_does_this_test_pass() {
val job = launch(Unconfined) {
throw IllegalStateException("why does this exception not fail the test?")
}
// because of `Unconfined` dispatcher, exception is thrown before test function completes
}
As expected, this test fails with Process finished with exit code 255
#Test
fun as_expected_this_test_fails() {
throw IllegalStateException("this exception fails the test")
}
Why do these tests not behave the same way?
Compare your test with the following one that does not use any coroutines, but starts a new thread instead:
#Test
fun why_does_this_test_pass() {
val job = thread { // <-- NOTE: Changed here
throw IllegalStateException("why does this exception not fail the test?")
}
// NOTE: No need for runBlocking any more
job.join() // ensures exception is thrown before test function completes
}
What happens here? Just like the test with launch, this test passes if you run it, but the exception gets printed on the console.
So, using launch to start a new coroutine is very much like using thread to start a new thread. If it fails, the error gets handled by uncaught exception handler in thread and by CoroutineExceptionHandler (see it in the docs) by launch. Exceptions in launch are not swallowed, but are handled by the coroutine exception handler.
If you want exception to propagate to the test, you shall replace launch with async and replace join with await in your code. See also this question: What is the difference between launch/join and async/await in Kotlin coroutines
UPDATE: Kotlin coroutines had recently introduced the concept of "Structured Concurrency" to avoid this kind of exception loss. The code in this question does not compile anymore. To compile it, you'd have to either explicitly say GlobalScope.launch (as in "I confirm that it Ok to loose my exceptions, here is my signature") or wrap the test into runBlocking { ... }, in which case exception is not lost.
I was able to create an exception throwing CoroutineContext for tests.
val coroutineContext = Unconfined + CoroutineExceptionHandler { _, throwable ->
throw throwable
}
Though this would probably not be suitable for production. Maybe need to catch cancellation exceptions or something, I'm not sure
A custom test rule so far seems to be the best solution.
/**
* Coroutines can throw exceptions that can go unnoticed by the JUnit Test Runner which will pass
* a test that should have failed. This rule will ensure the test fails, provided that you use the
* [CoroutineContext] provided by [dispatcher].
*/
class CoroutineExceptionRule : TestWatcher(), TestRule {
private val exceptions = Collections.synchronizedList(mutableListOf<Throwable>())
val dispatcher: CoroutineContext
get() = Unconfined + CoroutineExceptionHandler { _, throwable ->
// I want to hook into test lifecycle and fail test immediately here
exceptions.add(throwable)
// this throw will not always fail the test. this does print the stacktrace at least
throw throwable
}
override fun starting(description: Description) {
// exceptions from a previous test execution should not fail this test
exceptions.clear()
}
override fun finished(description: Description) {
// instead of waiting for test to finish to fail it
exceptions.forEach { throw AssertionError(it) }
}
}
I'm hoping to improve it via this post though
UPDATE: just use runBlocking - like Roman suggests.
I have a unit test that tests if method throws an exception when condition is present, and method does throws exception as expected.
- (void)testMethodThrowsWhenConditionIsPresent {
XCTAssertThrows([Foo methodWithCondition: condition], #"Condition is true, method should throw exception");
}
Here is the exception source:
- (void)methodWithCondition:(someType)condition {
if (condition) {
[NSException raise: #"condition is true!" format: #"condition is true!"];
}
}
Why does the test stop at the line the exception is thrown? The test does not go on, it stops at that line, when I expect it to continue and return 1 from XCTAssertThrows(), making the test succeed. The test instead stops with Xcode bringing me to the line it was thrown, with a green `Thread 1: breakpoint 1.1' and the debugger appearing in the console.
Why does the test stop when the execution is thrown?
Because you have a breakpoint, which stops execution.
Why, after removing the breakpoint, does my application crash when the exception is thrown?
Because you have an unhandled exception. Unhandled exceptions cause your program to crash.
How can I handle an exception so it won't crash my program?
The easy answer to this question is to simply NOT throw an exception. In other programming languages, like Java, this is perfectly standard. But in Objective-C, we don't really do exceptions. In Objective-C, exceptions should be saved for TRULY exceptional behavior.
With that said, and a strong suggestion for you to find another way to handle whatever it is you're trying to handle, this is how you handle an exception in Objective-C:
#try {
// code that could throw an exception
}
#catch (NSException *e) {
// handle the exception...
}
#finally {
// post try-catch code, executed every time
}
I have a few NUnit tests that run Selenium.
There are some prerequisite to some tests. An example for this would be logging in to our website.
We use a standard test user for test A, but if that user doesn't exist for whatever reason, we'll get a test failure with nothing useful (Selenium will just report it couldn't find the element at line 50). So I planned to check for the user's existence before we try to run the test - in the TextFixtureSetUp method.
I have a check to ensure the user exists, and if not, throw a helpful error message.
For example:
[TestFixtureSetUp]
public void SetUp()
{
bool userExists = userManager.GetUserByEmailAddress("someuser#fish.com") != null;
if (!userExists)
{
throw new Exception("Test user someuser#fish.com doesn't exist.");
}
}
vs
[TestFixtureSetUp]
public void SetUp()
{
bool userExists = userManager.GetUserByEmailAddress("someuser#fish.com") != null;
if (!userExists)
{
Assert.Fail("Test user someuser#fish.com doesn't exist.");
}
}
My question is this a good idea? Should I throw an exception or use Assert.Fail()? Am I thinking about this in the wrong way, or is it something doesn't matter really.
Reason to throw exception - you can catch it later on and try to use another user.
Reason to fail asserrt - when user is not found, it means end to the testmodel.
If you go the exception way - think about GetUserByEmailAddress will be throwing it if it does not find the right user...
Instead of Assert.Fail() raising AssertionException, I would rather use Assert.Inconclusive() or better Assume.That(userManager.GetUserByEmailAddress("someuser#fish.com"), Is.Not.Null) to raise InconclusiveException in this case.
You may want to watch it here: https://docs.nunit.org/articles/nunit/writing-tests/Assumptions.html
In below code I want to neutralize the throw and continue the method - Can it be done ?
public class TestChild extends TestParent{
private String s;
public void doit(String arg) throws Exception {
if(arg == null) {
Exception e = new Exception("exception");
throw e;
}
s=arg;
}
}
The net result should be that, in case of the exception triggered (arg == null)
throw e is replaced by Log(e)
s=arg is executed
Thanks
PS : I can 'swallow' the exception or replace it with another exception but in all cases the method does not continue, all my interventions take place when the harm is done (ie the exception has been thrown)
I strongly doubt that general solution exists. But for your particular code and requirements 1 and 2:
privileged public aspect SkipNullBlockAspect {
public pointcut needSkip(TestChild t1, String a1): execution(void TestChild.doit(String))
&& this(t1) && args(a1) ;
void around(TestChild t1, String a1): needSkip(t1, a1){
if(a1==null) //if argument is null - doing hack.
{
a1=""; //alter argument to skip if block.
proceed(t1, a1);
t1.s=null;
a1=null; //restore argument
System.out.println("Little hack.");
}
else
proceed(t1, a1);
}
}
I think that generally what you want makes no sense most cases because if an application throws an exception it has a reason to do so, and that reason almost always includes the intention not to continue with the normal control flow of the method where the exception was thrown due to possible subsequent errors caused by bogus data. For example, what if you could neutralise the throw in your code and the next lines of code would do something like this:
if(arg == null)
throw new Exception("exception");
// We magically neutralise the exception and are here with arg == null
arg.someMethod(); // NullPointerException
double x = 11.0 / Integer.parseInt(arg); // NumberFormatException
anotherMethod(arg); // might throw exception if arg == null
Do you get my point? You take incalculable risks by continuing control flow here, assuming you can at all. Now what are the alternatives?
Let us assume you know exactly that a value of null does not do any harm here. Then why not just catch the exception with an after() throwing advice?
Or if null is harmful and you know about it, why not intercept method execution and overwrite the parameter so as to avoid the exception to begin with?
Speculatively assuming that the method content is a black box to you and you are trying to do some hacky things here, you can use an around() advice and from there call proceed() multiple times with different argument values (e.g. some authentication token or password) until the called method does not throw an exception anymore.
As you see, there are many ways to solve your practical problem depending on what exactly the problem is and what you want to achieve.
Having said all this, now let us return to your initial technical question of not catching, but actually neutralising an exception, i.e. somehow avoiding its being thrown at all. Because the AspectJ language does not contain technical means to do what you want (thank God!), you can look at other tools which can manipulate Java class files in a more low-level fashion. I have never used them productively, but I am pretty sure that you can do what you want using BCEL or Javassist.