I'm searching for the equivalent of mocha:
mocha test.spec.js -w
opening nigtwatch manually on every change burns a lot of time during test writing.
any ideas of this?
I've checked nightwatch docs and google, nothing came up
From what I've seen, there isn't a --watch mode or command for nightwatch. This is only an answer in that what I've seen confirms that this doesn't seem to exist.
Here are the sources that led me to this conclusion, including possible leads for other solutions:
I've found an issue in the nightwatch repo asking about that in 2016: https://github.com/nightwatchjs/nightwatch/issues/1061. The answer then was
'No, but you can use something like the grunt-contrib-watch module.'
I found a stackoverflow question in 2018 that talks about how nightwatch, and other things using selenium, always return exit code 1: Nightwatch.js always returns exit code 1. That leads me to believe that nightwatch always exits its tests. The question was asking about integrate Nightwatch.js tests in a Jenkins Job. A comment offers a possible solution for that, so maybe it's possible to make a wrapper for nightwatch this way:
This happens with other selenium based libraries as well. I think I ran into this issue with protractor in the past. The way we tackled it was by parsing the output instead of using the exit code and looked for any failures. If none were found then we returned 0 explicitly... you can check for failed tests using browser.currentTest.results.failed and then figure out a way to pass or fail your build based on that.
Related
I am having an issue in Github Actions.
I am using Cypress to test the frontend of my app, and everything is working perfectly in the Cypress app.
But when I am pushing everything on master in GitHub and run the test via Github Actions, every now and then, the same flaky test is failing with he following error.
Timed out retrying after 5000ms: Expected to find element "xxx" but never found it.
This line is the problematic one:
cy.purposeElement("delete_user_dialog").should('be.visible')
The element we are talking about takes 0.2s to appear with a fade-in animation.
My guess is that the page was slow to respond, while Cypress was fast to act.
How can I avoid solve a flaky test like that?
I could use a cy.wait but it is not recommended and I do not want to increase the time of the test.
Plus, the result is as flaky.
It may be as simple as increasing the timeout, especially if the test works perfectly when running on the local machine.
cy.purposeElement("delete_user_dialog", {timeout:20_000}).should('be.visible')
To be 100% sure, do a burn test which is described here: Burning Tests with cypress-grep
I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!
Using C#, I have
driver = new ChromeDriver();
It gives:
System.ComponentModel.Win32Exception: The specified executable is not a valid application for this OS platform.
It worked before I changed something.
This answer specifies that to improve undetectability of Selenium, you have to run a certain PERL script. I ran it like so:
perl -pi -e 's/cdc_/dog_/g' C:\Users\user\source\repos\SleeveAce\packages\Selenium.WebDriver.ChromeDriver.91.0.4472.10100\driver\win32\chromedriver.exe
What do I do now? Before, the C# app was working perfectly, now it has that error. When I open the .exe as a notepad++ and search for cdc_, I find multiple instances, so I suppose the script is not working? I might be wrong though if the script was not supposed to do that.
EDIT: I have just tried changing all the cdc_ values in a hex editor as well with dog_. They were modified successfully. However, the error is still there.
EDIT2: Solved! See comments for explanation.
Additional info
Using Perl 5 v32
After running the script, nothing was shown (no
errors too!)
When using the APIs defined by Protractor & Jasmine (the default/supported runner for Protractor), the tests will always work okay on individual developer laptops. For some reason when the test runs on the Jenkins CI server, they will fail (despite being in the same docker containers on both hosts, and that was wildly frustrating.)
This error occurs: A Jasmine spec timed out. Resetting the WebDriver Control Flow.
This error also appears: Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
Setting getPageTimeout & allScriptsTimeout to 30 seconds had no effect on this.
I tried changing jasmine.DEFAULT_TIMEOUT_INTERVAL to 60 seconds for all tests in this suite, once the first error appears then every test will wait the full 60 seconds and time out.
I've read and reread Protractor's page on timeouts but none of that seems relevant to this situation.
Even stranger still, it seems like some kind of buffer issue - at first the tests would always fail on a particular spec, and nothing about that spec looked wrong. While debugging I upgraded the selenium docker container from 2.53.1-beryllium to 3.4.0-einsteinium and the tests still failed but they failed a couple specs down - suggesting that maybe there was some optimization in the update and so it was able to get more done before it gave out.
I confirmed that by rearranging the order of the specs - the specs that had failed consistently before were now passing and a test that previously passed began to fail (but around the same time in the test duration as the other failures before the reorder.)
Environment:
protractor - 5.1.2
selenium/standalone-chrome-debug - 3.4.0-einsteinium
docker - 1.12.5
The solution ended up being simple - I first found it on a chrome bug report, and it turned out it was also listed right on the front page of the docker-selenium repo but the text wasn't clear as to what it was for when I'd read it the first time. (It says that selenium will crash without it, but the errors I was getting from Jasmine were just talking about timeouts, and that was quite misleading.)
Chrome apparently utilizes /dev/shm, and apparently that's fairly small in docker. There are workarounds for chrome and firefox linked from their README that explain how to resolve the issue.
I had a couple test suites fail after applying the fix but all the test suites have been running and passing for the last day, so I think that was actually the problem and that this solution works. Hope this helps!
we are considering to use FitNesse/Slim.
But is there a way to start all written Tests without browsing the webpage and starting each manually ?
It would be sufficient if there is a one-start-all kind of button somewhere to click.
So either starting all tests from command line (with a report of course) or with on button from the webpage.
Is this doable ?
Thanks in advance
There is a very easy way to do that. You can run FitNesse tests from the command line. You do this by using the following command line:
java -jar lib/fitnesse.jar -c "FrontPage?suite&format=text"
This will run all tests under the FrontPage and show the results as they happen in a command line friendly format. If you change FrontPage to FrontPage.MainSuite, it will run only the tests under that page.
If you have tests that are in different states. Maybe some of them are started but are not done yet. You can add a Suite Tag to the tests that must run, then you can filter the tests that are run. that would look like the following:
java -jar lib/fitnesse.jar -c "FrontPage?suite&suiteFilter=MustBeGreen&format=text"
This is also possible to do using an ANT java task.
Assuming you have a current version of fitnesse and have it running on port 8080, the following link will take you to a page with more details: http://localhost:8080/FitNesse.UserGuide.ControllingFitnesseFromTheCommandLine
Dan has it correct; but the User Guide is also posted online at:
http://fitnesse.org/FitNesse.UserGuide.ControllingFitNesseFromTheCommandLine