Slow running acceptance test with php + codeception + phantomjs - testing

I am moving to codeception 2.0.3 for doing some tests in various web platforms I am developing. I started doing some acceptance testing. Mainly check for pages ok and doing some form completions (sign in, sign up, nothing fancy).
I have been monitoring tests with firefox browser and we are moving now tests to a dedicated server so I switched to phantomjs as my testing browser.
Configuration in acceptance.suite.yml
WebDriver:
url: 'localUrl'
browser: phantomjs
window_size: 'maximize'
capabilities:
phantomjs.cli.args: ['--ignore-ssl-errors=true']
The thing is that with this headless configuration, tests are running very slowly. I mean, the test I wrote is checking that four links are OK (no error or exception message) without any fancy assertions (something which I can check in less than 20 secs) and it is taking more than a minute a half.
Am I missing something in the configuration of the testing stack? I read that phantomjs testing in this way is supossed to be fast and reliable something that can be integrated while developing but I don't seem to get it working right. I have been doing TDD in Smalltalk and maybe I am a little biased with the way that things work in that environment so maybe my expectations are too high but I had hopes that this could be a little more responsive and easy going.
I am using codeception 2.0.3 with phantomjs 1.9.7 on a linux box with php 5.5.
Any suggestion is welcomed. Thanks!!!

I had very slow running tests and started using strict locators. That sped up things a lot.
Instead of writing:
$I->fillField('username, 'john');
which will try and fail many locator types before actually working, specify the locator you are using and write:
$I->fillField(['id' => 'username'], 'john');
or
$I->fillField(['class' => 'username'], 'john');
or
$I->fillField(['css' => 'input .username'], 'john');
Read here: http://codeception.com/docs/04-AcceptanceTests#Click

Could be problem with phantomjs, usually headless with Codeception PhpBrowser goes very fast, try to switch to it.
For in-browser try to use Codeception WebDriver + latest selenium 2 standalone.
Also if you are developing with some framework, you can check if it is present in Codeception modules list. If so you can use it, it runs much faster than PhpBrowser since it does not need any server and works with symfony dom - crawler and browser - kit.

The size of the database sql file also matters. In my case import of some big tables caused more than minute of additional time.

Related

How does the elixir community recommend setting up integration tests with chromedriver?

I am using Hound (https://github.com/HashNuke/hound) for integration testing a Phoenix application. I have chrome and chrome headless working. To get it working I have another terminal window running chromedriver (installed via brew). This feels odd to me. Is there a library or test setup that would feel more "integrated" into the application? What's the Elixir way of doing this?
In the Ruby world there's the webdrivers gem (https://github.com/titusfortner/webdrivers). As far as I know it downloads a specified driver (lets say chromedriver) to $HOME. Then with every test run, the test uses the driver downloaded to that destination to execute the tests.
Before the webdrivers gem there was chromedriver-helper gem. Before that it was phantomjs. These implementations made it so running integration tests required 1: downloading the driver 2: running the test
In Elixir (with Hound) I have my tests working by first running chromedriver --verbose in a terminal split, and in the other screen I run mix test. This works fine but feels disjointed. This adds extra steps, 1: download the driver 2: start the driver 3: run the test 4: stop driver
I could write a script manually to run chromedriver in the background, and stop it after the tests are run.
I am new to the Elixir community and so I've researched a lot. It's still not clear to me if there is a "traveled path" I should go down vs just hooking everything up manually.
Have I missed a recommended abstraction? Is this intentional? Is this "just not created, yet"?
Thank you
Have you checked out wallaby? See https://github.com/keathley/wallaby

MicrosoftWebDriver 16299, 15063 doesn't work when I minimise Edge browser window

When I execute tests using MicrosoftWebDriver for Edge all works fine, but when I minimise the window all tests become failed. Does Edge or MicrosoftWebDriver have any options to avoid that behavior?
As you have been trying to minimise the Browser Window while your Test Execution is In Progress it will be against all the Best Practices. At this point it is worth to mention that as Selenium mocks the User Actions hence Selenium needs Browser focus. If the focus is lost Selenium won't be able to execute the lines of code. Consider the following steps while your Test Execution :
Browser Maximize : While you execute your tests always keep the Web Browser maximized so majority of the elements are within the Viewport
As per best practices, you should try to execute your Test Scripts / Automation Framework in an isolated Test Environment away from Manual Intervention with all the required Software and Hardware configurations and setup.
You can find a detailed discussion in How to run chrome driver in background using selenium with Ruby for Mac OSx?
You also have an option to use Google Chrome or Mozilla Firefox in Headless version.
For Firefox you can find a detailed discussion in How to make firefox headless programatically in Selenium with python?
For Chrome you can find a detailed discussion in selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH error with Headless Chrome
For Ghost Browser you can find a detailed discussion in Which drivers support “no-browser”/“headless” testing?

phantomjs custom header in robot framework

We are using phantomjs browser in robot script and have a requirement to set the phantomjs custom header as a part of capability. Below is the code snippet we have tried
${dc} Evaluate sys.modules['selenium.webdriver'].DesiredCapabilities.PHANTOMJS sys, selenium.webdriver
Set To Dictionary ${dc} phantomjs.page.customHeaders.Authorization=Basic <Credentials>
${service args}= Create List --proxy=localhost:8080 --web-security=false --ignore-ssl-errors=yes --ssl-protocol=ANY --load-images=yes
Create Webdriver    PhantomJS   service_args=${service args} executable_path=/usr/sap/ljs/webapps/s4c/WEB-INF/classes/WebDriver/phantomjs desired_capabilitie=${dc}
This is code is failing to launch the browser. We didn't find concrete documentation for setting capability to phantomjs in robot. If anyone tried setting custom header to phantomjs in robot framework please suggest us.
Perhaps stating the obvious: none of your end users are using the system under test with PhantomJS which significantly lowers confidence of the achieved results. So for that reason I strongly suggest putting effort in getting the right environment setup so that the test mimic end user situation as close as possible.
It is very likely that you're using a modern version of Selenium that no longer supports PhantomJS as it is Deprecated as of Selenium 3.6.0 and removed as of 3.7.0. Releases.
So if your pip list shows a Selenium Module version of 3.6.0 or higher you may want to downgrade.

How to clear session using selenium and phantomJS

I have a series of automated tests that use selenium, specifically Geb with Spock. When I run all my tests cases one after another using a chromedriver, my tests run fine. Each test method gets a new session and therefore start with a fresh browser each time. This is not the case when using phantomJS. If a test method happens to fail when using phantomJS, all following test cases will fail because the session was not reset. I can see this happening because the screenshots taken at the end of each test method.
I have tried researching this issue and have discovered it has been a known issue for a couple of years now. Some say it was fixed in phantomjs 2.1 but I'm currently using version 2.1.1, which is the most recent version.
I have also tried using driver.manage().deleteAllCookies(); with no luck, as other threads have mentioned.
Is there a workaround anyone is aware of?
have you tried webdriver's quit method?

Selenium testing machine

--- TL;DR
At this point I suggest everyone to tied their Continuous Integration server/service to https://ghostinspector.com/
OLD QUESTION
after three days googling and testing I gave up, and I need help.
My objective is allow my co-workers to record one or more tests with Selenium IDE. Export them, upload them into a server, and get this server running these tests using the webdriver with htmlunit. As we build or fix the app, we will upload the tests to make out test library.
Record a test with Selenium IDE is okay. But getting it running is the problem. The machine we intend to let the tests is an linux amazon server. No front-end stuff, no kde, gtk, so no firefox, chrome, etc... This is why I've specified the htmlunit driver.
So far I wasn't able to get this task running even into my machine - Ubuntu 12.04 x86_64.
I downloaded the selenium-server tarball, and tried running:
java -jar selenium-server.jar -htmlSuite "*webdriver" "our.site.org" "/path/to/testsuite1.html" "/path/to/report1.html"
No success. Even changing the "*webdriver" (using other pops-up a browser screen).
I've tried running the server and the standalone server and connecting via browser.
I've tried PHP bindings by facebook.
I've tried PHPUnit and Testing Selenium classes - along with their respectives exported scripts from Selenium Formatters.
I really do not know where I'm slipping. Can anyone give me a safe direction, tutorial, etc, to follow with?
--- EDIT
Okay, my question may be resumed to:
What si the command line that would allow me to run selenese scripts with selenium-server, using the HtmlUnit driver?
Are you using Continuous Integration?
If so, you should consider getting a plugin to have your CI software run the Selenium tests. Works like a charm for me with Jenkins.
Considering your particular setup, you could both have the amazon linux server run the tests with HTMLUnitDriver, and declare other machines (with a GUI and proper browser) as "nodes" to run your test on other browsers.
Link to a tutorial
Have you read this blog post by David Burns (Automated Tester):
http://www.theautomatedtester.co.uk/tutorials/selenium/selenium_rc_setup.htm
He describes the way to run selenese tests using HTMLSuite.
We are going to use the HTMLSuite commands of the Selenium Remote
Control. This allows you run your Selenese Test Suites as is. The
command should look like java -jar selenium-servre.jar -htmlsuite
. Browser
could be : -*firefox
-*chrome
-*iexplore
-*iehta
-*safari
-*custom /path/to/browser
The path to the test suite and the results file should be a full path.
Here is an example command; java -jar selenium-server.jar -htmlsuite
*iexplore http://www.theautomatedtester.co.uk c:\testsuite\testsuite.html c:\testsuite\results.html
I would point out that htmlunit does not seem to be a supported option so I would expect to use -*custom and provide a path to htmlunit.
This is legacy functionality so there is a chance it doesn't work as expected any more. HTMLSuite expects the tests to be in Selenese (HTML table) format, you mention trying with the PHP binding, I would not expect this to work. If you do want to use some PHP bindings I would suggest using Adam Saunter's fork of the facebook bindings, they are the most up to date and best supported.
https://github.com/Element-34/saunter.php
With Selenium WebDriver you can point to start a HtmlUnit in a already started node
In Java you'll do something like this:
IWebDriver driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), DesiredCapabilities.HtmlUnit());
To start the node just make sure to set browserName to 'htmlunit'.