I am using TestCafe for some time now and never had a problem. A few days ago TestCafe started to run my tests twice, producing a lot of problems. I couldn't find the issue on https://github.com/DevExpress/testcafe/issues and wanted to ask if anyone else experienced these problems in the last time.
I am always starting the tests with: 'testcafe "chrome --start-fullscreen" tests/tests --speed 0.5 --skip-js-errors'.
Thank you so much and best regards from Germany
Stefan
I face this issue, whenever i am on same terminal with multiple interrupted run. Close your terminal
Related
I'm running a big campaign with about 500 tests targeting the API of an orchestration tool using both a parallel and a sequential runner for some of the tests which need to be executed in sequence.
This works fine using the Karate version 0.9.6.
As soon as I upgrade the Karate version to anything higher than v0.9.6, the results of the tests in the parallel runner always consists of a significant portion of failing tests. The sequential runner does not have any issues. Also the parallel runner seems to work fine only if I set it's threadCount = 1.
If the threadCount is higher than 1, the runner seems always to start well but after some time there are quite some failed transactions without any further details like this: http call failed after 55 milliseconds for url: http://...
This is all the error log I have as there is not more than this entry for each failed test (around 40% of the full campaign).
As soon as one thread starts with an issue like that, basically all other threads follow to do the same and make their tests fail for the same reason (only with a slightly different number of milliseconds). While investigating the problem I was not able to identify a common pattern (like always starting with the same test failing, etc.).
Did anybody else face similar issues with Multithreading using a higher version than v0.9.6?
Is there a way to get more detailed logs? I use the value DEBUG in the logback-test.xml
Is there any recommendation of what to try to make it work?
Please don't hesitate in case you need more information.
I would be happy for any kind of help regarding this as I would like to benefit of the new karate-gatling facilities for Performance Testing which is only available in the latest versions of Karate.
Many Thanks for your reply!
we run 8 threads on karate 1.1.0 with no issues at all. Do you have some extra logs? So you don't get same timeout error http call failed after 55 milliseconds for url: http://... in lower version the running multithreads ?
Shot in the dark here, but do some of your scenarios have the same title? If so, try making them all unique and see if that solves the issue.
More logs will be useful. Its very little information here.
I am not sure of the exact error but I faced similar issues sometime ago but that was caused due to race conditions especially if you are trying to use variables set by java methods. Are you using Java interop in your tests? If so worth checking how these are being used. A test may be trying to execute with a variable/parameter that is still in use by other tests when running in parallel. This wouldn't be an issue in sequential.
Browser: All
Testcafe: v0.23.0
node.js: v8.12.0
OS: macOS, Windows 10
Since the update (0.21.0 -> 0.23.2), the execution time of the tests tripled.
Have any of you experienced this since the update?
Test Scenario: I want to launch a general regression test suite
Current Behaviour: - Some of the tests fail because of timeout-related issues
- Each action takes ~ 1 second more, the whole suite takes ~ 35 minutes
Expected Behaviour: - 100% tests passed in ~ 10 minutes
We have not experienced these kinds of performance issues. Moreover, we haven't noticed any performance slowdown in our tests. Thus I recommend you create a bug report in our github repository. Please use the special template for creating a bug. I kindly ask you to fill all fields and provide as many details as possible. If you use any CI system, specify this in the ticket as well. It could happen that the cause of the issue is in CI unstable behavior.
You could also check the 0.23.3-alpha.1 version. It contains fixes which can affect the performance (eg https://github.com/DevExpress/testcafe/issues/3070).
I plan to make an evolutionary program to make redcode warriors for CoreWars. However, I have absolutely no idea how to run the code generated without having to manually open the program and put in the Warriors. Since I hope to have the evolutionary program run through several warriors per minute at least, I'd rather not have to play the role of administrator that much. I'm using the ARES simulator, but as for my research on running warriors through it with a script, I haven't found anything.
I'm really just looking for something like:
SomeSimulator.exe --warrior1 megalordthedestroyer.red --warrior2 tinathebabybunny.red
So I found a python module that supports both parsing redcode and running pmars simulations. With a little bit of patching I'm able to hook this up to my evolutionary factory and automate it all.
The link: https://github.com/rodrigosetti/corewar
Currently experiencing very slow debug start up times (between 25-45 seconds) with idea CE 14.1.5. To reproduce:
Make change in code that will cause javac to be invoked.
Hit debug.
Status bar shows "Make" and entire IDE hangs.
I'm using a 2015 Macbook pro and Java 1.8.0_60 64bit.
After profiling with advice from above, I tracked it down to this issue:
IntelliJ freezes for about 30 seconds before debugging
Which in turn points to an issue with InetAddress.getLocalHost() on *nix. Following the advice in this linked post certainly helped it go away:
InetAddress.getLocalHost() throws UnknownHostException
I had similar problems of starting debugging application and tried different workaround tips to figure it out, but in my case i had a lot of forgotten breakpoints marked in application and removing them all application starts and performs lightening fast.
So, my advice remove all the breakpoints from the code at least at application startup.
You should assign more memory. This almost always reduces lagging / hanging issues. You can find a guide how to change this setting in IntelliJ manual.
We are writing integration tests for our Grails 2.0.0 application with the help of the Fixtures and Buid-Test-Data plugins.
During testing, it was discovered that the integration test fail at certain times, and pass at other times. Running 'test-app' sometimes results in all tests passing, and sometimes results in some of our tests failing.
When the tests fail, they are caused by a unique constraint being violated during the insert of an instance of a domain class. This would indicate that there are still records in the test DB. I am running the H2 db, and have definitely got 'dbCreate = "create-drop"' in my DataSource.groovy.
Grails 2.0 integration test pollution? seems to indicate there is a significant test-pollution problem in Grails. Are there any solutions to this? Have I hit Grails-8530?
[Edit] the test-pollution seems to be caused by the unit tests. We have sort-of proved this by deleting the unit tests and successfully running 'test-app' repeatedly.
When I run into errors like this I like to try and find the unit test(s) that is causing the problem. This might be kinda tricky since yours seem to only be failing on occasion.
1) I'd look at unit tests that were recently added. If this problem just started happening then that's a good place to look.
2) Metaclassing seems to be good at causing these type of errors so I'd look for metaclassing that isn't setup/torn down properly. Not as much of an issue with 2.0 as with <= 1.3.7 but could be the problem.
3) I wrote a plugin that executes your tests in a random order. Which might not help you solve your current problem. But what might help you is it prints out all of your tests so you can take what it gives you and run grails test-app <pasted list of unit tests> IntegrationTestThatIsFailing then start removing unit tests to find the culprit(s). ( http://grails.org/plugin/random-test-order). I found a bug in this with 2.0 that I haven't had time to fix yet (integration tests fail when asserting on rendered view name) but it should still print out your test names for you (which is better than doing it yourself :)
The fact integration tests fail with a constraint violation due to existing records reminds me of a situation I once encountered with functional tests (selenium) executing in unpredictable order, some of them not cleaning up the database properly. Sure, the situation with functional tests is different, since it is more difficult to restore the database state (The testcase cannot rollback a transaction in another jvm).
Although integration tests usually roll back transactions, it is still possible to break this behavior if your code controls transactions (commits) explicitly.
First, I would try forcing execution order as mentioned by Jarred in 3). Assuming you can then reproduce the behavior, I would then check transactional behaviour next. Setting the logging level of org.hibernate.transaction to debug should show you where transaction boundaries are.
Sorry, don't yet have a good explanation why wiping out the unit tests helps getting rid of the symptoms besides a general "possibly metaclassing issues". :)