This is not about pinning down the hang as such, its more about wondering if there is a way to make testcafe decide the test must be a failure after a given period of time.
Our test works fine on local machines, but occasionally when run in CI using a docker image on semaphore one of the selectors causes it to hang. I am fine ivestigating why that is, but it would be good if I could tesll the test to give up after say 5 minutes (it should finish in under 3) and exit gracefully so we can have the stack trace and video - if we have to kill the test run we don;t get either, so we end up having to debug by the logging we emit from the test code.
Currently, this functionality is not implemented in TestCafe. Please track the https://github.com/DevExpress/testcafe/issues/6096 issue to check our progress.
Related
So we are doing distributed testing of our web-app using JMeter. For that you need to have the jmeter-server.bat file running in background as it acts as sort of a listener. The problem arises when one of the slave machine out of 4 restarts due to the load and the test is effectively stuck right there as the master machine expects some output from the 4th machine. Currently the automation is done via ansible-playbooks which are called in Jenkins. There are more or less 15 tests that are downstream to one another. So even if one test is stuck, the time is wasted until someone check on the machines.
Things I've tried so far:
I've tried using the Windows Task Scheduler and kept the jmeter-server.bat to run without any user loggin in, but it starts the bat file in background which in-turn spawns all the child processes in the background as well i.e. starts Selenium Chrome in headless mode.
I've tried adding the jmeter-server.bat in startup and configuring the system to AutoLogon without any password to trigger a session which will call the startup file. But unfortunately the idea was scrapped by IT for being insecure.
Tried using the ansible playbook by using the win_command but it again gets stuck as the batch file never returns anything.
Created a service as well for the bat file, but again the child processes started in background.
The problem arises when one of the slave machine out of 4 restarts due to the load
Instead of trying to work around the issue I would rather recommend finding the root cause and fixing it.
Make sure to follow JMeter Best Practices
Configure Java to take heap dump on failure
Inspect Windows PerfMon and operating system/application logs
Check presence of .hprof files in the "bin" folder of your JMeter installation and see what do they say
In general using Selenium for conducting the load is not recommended, I would rather suggest using JMeter's HTTP Request samplers for that, given you properly configure JMeter to behave like a real browser from the system under test perspective there won't be any difference whether the load comes from HTTP Request samplers or from the real browser.
The same states documentation on the WebDriver Sampler
Note: It is NOT the intention of this project to replace the HTTP Samplers included in JMeter. Rather it is meant to compliment them by measuring the end user load time.
I try to debug a little test client side with
testcafe --debug-mode --dev Safari test1.ts
When the testcafe debug controls are shown I can inspect variables etc all fine.
However if I put a breakpoint in client side code and inspect variables if the browser execution is stopped testcafe restarts my test after some minutes and interrupts my debug session.
Is there a way to instruct testcafe to have infinite testing time for this scenario?
(I thought --debug-mode would imply that )
At this moment, you cannot configure this behavior in TestCafe. However, TestCafe already has an issue with this suggestion, so you can vote for it. With more votes, there is a bigger chance that this feature will be implemented in the near future. Right now, you can debug your application under the TestCafe core using http-playground.
I am developing a backend service using meanjs.org scaffold. I have written around 1700 tests and they were working fine. But now, tests just started to act weird. Suddenly a lot of tests fails with error:
Error: timeout of 10000ms exceeded. Ensure the done() callback is being called in this test.
Callback done() is being called so this is not the problem. The tests were working all the time but now they don't want to work any more. I tried to drop the mongo database, to stop and start mongod process but nothing works.
Did anyone have the same problem of tests not working from time to time?
I answered a similar post maybe it will be helpfull, as the solution is the same.
mocha timout
The problem is that your connection to mongodb probably exceeds the 2000ms timout from mocha.
When I run my selenium tests using a chrome browser all my tests cases run fine. When using the phantomjs browser it would appear that the browser session does not get reset after each test case. In my tests cases, I log in as a user to then navigate to certain pages and then logout. A problem occurs when a test case happens to fail. The browser session is not reset so when the next test case begins, the test that failed was unable to logout. This causes all test cases after a single failure to fail.
When searching the internet for a solution to this issue it been known sine 2013. I can't seem to find anything regarding this issue that's recent. Is there any up to date workarounds?
Manually trying to delete the cookies before or after each test case does not appear to work. webDriver.manage().deleteAllCookies();
I'm using phantomjs ver 2.1.1.
First of all PhantomJS is dead, you are better off switching to Headless Chrome or Headless Firefox.
Secondly PhantomJS is a port of Webkit which is not thread safe. This means that if you try and run more than one test in parallel you will see threading issues, to fix this you would need to start multiple instances of PhantomJS and have each GhostDriver instance connect to a different instance of PhantomJS.
The particular problem that you are seeing is that PhantomJS doesn't clear itself down properly, again the only solution would be to kill the initial PhantomJS instance you are running after your test finished and then start up a clean new one, unfortunately that is not supported by GhostDriver.
The final problem is that GhostDriver is dead as well, there was no point in continuing development when PhantomJS died.
TLDR; Use Chrome/Firefox Headless mode instead.
I'm using the karma-script-launcher to open a virtual machine/browser/url but when the tests are complete I don't know how to close everything.
If I leave the browser open in the virtual machine it eventually stops trying to reconnect - no error message just fails to reconnect. I'm not sure if this is a bug or not. I can't see why you wouldn't want the browser indefinitely trying to reconnect? What ever the reason it leaves me needing to close and re-open the browser each time I run the tests. But if I'm using the script launcher I don't see how this is possible.
Thanks for any help
not quite sure if this covers your answer but I stumbled over your question while looking for a solution to a similar issue, so let me explain - in case someone else encounters this issue.
Was running karma unit tests on codeship ci environment for a nodejs app.
Tests were successful, however the test would keep watching and wouldn't shut down, thus setting the tests to 'passed' - instead it would idle and after a while timeout, setting the tests to 'failed'.
Setting the option singleRun to true within the karma config helped.
Hope this helps anyone who's having the same issue.
Sorry if not topical enough :S