I have several Cucumber tests that are running fine (I closely watch them), all pass, and everything works good. But Event Log in Rubymine keeps showing all the tests that pass in this way:
Failed to start: 0 passed, 19 not started
Besides, the Run tab (which shows all tests as Passed) also says this after (otherwise successful) run:
Done: Scenarios 0 of 19
Can someone please help me understand why could it happen?
Screenshot:
My setup: Rubymine / Ruby / Selenium / Cucumber.
Ok, apparently it is a bug that exists in the current release build.
I contacted them, and they said it is related to this issue.
They fixed it in EAP build 145.184.8.
I was able to confirm the fix after I installed that build.
Related
We have currently about 200 test features. We start to face something strange, most of the times tests are just stuck and would not proceed when we run mvn test command as the following:
mvn clean test -Dcucumber.options="--tags $tags" -Dtest=TestRunner -Dkarate.env=$env
Some tests would run as it was perfectly fine. But at some point the rest will just stuck as it it hangs.
We run the tests in parallel using 10 threads.
It stucks like this
Anybody experienced similar things? Any ideas what could possibly went wrong?
Thanks
This should be fixed in 0.9.5.RC3 - it is stable to use for API testing, so I recommend you upgrade.
If anyone faces this problem for any other version of Karate, please understand that the best (and possibly only) way to troubleshoot or solve this - is to follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
I actually have the same problem as you but I can't comment because of reputation, my project works with Gradle and I'm using IntelliJ IDEA and JDK1.8(at another moment before all this I tried Jetbrains SDK11 but had the same problem, I downgraded to java 8 and everything worked again) on this ocassion I did as peter said and upgraded to 0.9.5.RC4 but still when I execute some of my features they never end, for example, I'm currently working on a very simple feature that calls another feature for login, it works for many other features but for this one it appears to get to the end of its execution and never go back to the caller feature, as I was running out of options I made a new simple project copied the resources folder I store my features in and my run parallel class and tried again but it behaves in the same way, the execution never ends.
I'll upload an image with my screen while it executes as you can see it's been executing for 15 minutes
projectView
We have currently about 200 test features. We start to face something strange, most of the times tests are just stuck and would not proceed when we run mvn test command as the following:
mvn clean test -Dcucumber.options="--tags $tags" -Dtest=TestRunner -Dkarate.env=$env
Some tests would run as it was perfectly fine. But at some point the rest will just stuck as it it hangs.
We run the tests in parallel using 10 threads.
It stucks like this
Anybody experienced similar things? Any ideas what could possibly went wrong?
Thanks
This should be fixed in 0.9.5.RC3 - it is stable to use for API testing, so I recommend you upgrade.
If anyone faces this problem for any other version of Karate, please understand that the best (and possibly only) way to troubleshoot or solve this - is to follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
I actually have the same problem as you but I can't comment because of reputation, my project works with Gradle and I'm using IntelliJ IDEA and JDK1.8(at another moment before all this I tried Jetbrains SDK11 but had the same problem, I downgraded to java 8 and everything worked again) on this ocassion I did as peter said and upgraded to 0.9.5.RC4 but still when I execute some of my features they never end, for example, I'm currently working on a very simple feature that calls another feature for login, it works for many other features but for this one it appears to get to the end of its execution and never go back to the caller feature, as I was running out of options I made a new simple project copied the resources folder I store my features in and my run parallel class and tried again but it behaves in the same way, the execution never ends.
I'll upload an image with my screen while it executes as you can see it's been executing for 15 minutes
projectView
We have e2e test integrated with Jenkins system. For a few weeks this test successfully ran both, locally and from Jenkins (as a part of the build pipeline).
At the end of Sprint, I modified the script to reflect Sprint changes and made sure it passed locally. Then, I merged the changes with master. Now, e2e runs from Jenkins are failing 100% of the time, while when I locally connect to QA envs there is no problem.
The error I am getting is - Element is not clickable at point (x, y) which I cannot reproduce locally.
The server doesn't have a real screen so I cannot go out there and see what's going on. Resolutions are perfectly matching. I have other people running this test locally and there is no problem.
What could possibly cause these failures and how do I troubleshoot this problem?
Thanks for your help!
Its a question from 1000ft and pretty difficult to identify where exactly the issue could be but I listed down some probable causes/debugging tips that could help you
1.Whats your checkout strategy from source code repository? Check job workspace and it should have the most recent code and check if its indeed the latest one.
May be configure Job to always pick a new version instead of 'update'
2.Add a reporter based on the test framework you are using especially the ones which provide screenshots. Refer my blog for more details -
3.Check the stack trace of your error from Jenkins console report and verify the exact trigerring point
I go to run an acceptance test, but it only outputs
Codeception PHP Testing Framework v2.0.12
Powered by PHPUnit 4.5.1 by Sebastian Bergmann and contributors.
Then it quits with out any error message. No tests run, no error message.
Browser tests were working fine, and only acceptance tests were broken.
I have all ready solved the problem, but I want to create a record for next time I or any one else runs into this problem.
If you codeception is quitting without any errors or fail messages, it means that there is an error in your code somewhere. I found the error in me Acceptance Helper file, where I had a duplicate of a function. Functions can not have the same name in php, so everything fails. But codeception does not output any error messages.
In order to solve this problem, you need to look through your Helper functions to find a syntax error, or it could be in your actual tests.
The reason it fails is codeception hits an error in the php code, and dies, not throwing any errors. Leaving you confused and frustrated. Now you can find this question and get back to doing what you're doing.
YAY!
There is a syntax error somewhere in your testing code, find it and get rid of it, and it will work. I'd bet the error is in the AcceptanceHelper.php or Browser or what ever suit won't run tests.
If you're seeing this behaviour (codeception quits without any error messages) there is likely a fatal PHP error happening somewhere, but it's not necessarily in your own code or in your codeception generated files.
Depending on your PHP configuration, these errors should show up in an error log, even if they're not output to the console.
For example, if you're using MAMP on Mac, the error log is here by default: /Applications/MAMP/logs/php_error.log
Clear the PHP error log, run your (non-working) test, and check the log again. It should give more insight.
In my case, it was a matter of running codeception 4 on Laravel 5.5, and hence missing classes from the symfony/service-contracts package. Installing this package with composer moved past the "invisible" problem (though I ultimately had to downgrade to composer 3, since there doesn't seem to be a compatible set of symfony/Laravel 5.5/composer 4 packages).
Let me describe a simple use-case:
Running all tests in our project may take up to 10 minutes.
Sometimes I see an obvious bug in my code after the first failed test, so I want to stop running all tests, fix the bug and re-run them. Unfortunately, I can either re-run all tests from the beginning, or re-run failed tests only.
Is there a plugin for IDEA which allows me to re-run failed tests AND tests, which weren't yet executed when I pressed "STOP"?
Atlassian has the solution for your problem: Clover. But it is commercial.
This goes against the idea of a test suite. Normally you want to run all your tests specifically so you know you haven't broken anything somewhere unexpected. If you change the code and then run a subset of the tests, the possibility exists that you broke something and one of the skipped tests would have failed. This is a case of not getting your cake and eating it too.
If you find a bug in an early test, by all means stop the suite. Fix the bug but then run the suite from the beginning.