I'm using Capybara to test features on a progressively enhanced website. Let's say my feature is to navigate around a hierarchy of locations. The non-javascript version involves getting a new version of the page when we click around on different locations. The enhanced javascript version opens up hidden elements, or loads up new information via Ajax.
I start by writing a test for the non javascript version, which looks something like this:
When I visit the page for "UK"
And I click "London"
Then I should see the information for "London"
Using the default mechanize driver, the test fails, I develop the feature, then the test passes.
I then create an identical test for the javascript version, flagged up with #javascript. It runs the test with the javascript driver, and that test passes because the feature has been implemented. (It's running through the non-js flow). However, I want the javascript version of the test to fail at this point because the feature has not yet been enhanced with Javascript.
So I'm looking for a good strategy for determining whether or not a whole new page has come from the server, and making sure both versions of the feature work. (I plan on integrating this with pushState so testing for a changed URL won't do)
I'm interested to hear other peoples opinions on this - I'm not convinced Cucumber is the right tool for the job, since you're describing features from the perspective of user interaction, and it sounds like your implementation of progressive enhancement will result in essentially the same user interaction.
That aside, I think you may want to consider building in some kind of testing hook to the page itself to help with this. Hard to say what without knowing your exact situation, but maybe one of:
The script-enhanced version of the page could add some element to the DOM, indicating that the enhancements are active, or indicating that the data came from an AJAX request rather than page load.
You could generate a random page identifier (from the server) on every load (e.g. new GUID), embed this into the DOM and assert that it hasn't changed after the interaction (on the enhanced version). This would be a very simple way of achieving your stated goal ("determining whether or not a whole new page has come from the server")
Why does your javascript "enhanced" version work the same as your non-enhanced? Your cucumber tests using #javascript should be testing to ensure the enhancements work.
For instance,
* if the javascript opens a modal dialog instead of following a link to a new page, test for that.
* if the javascript submits a form and updates a value, test for that.
These tests would fail if run without javascript support.
Related
I have been using the firefox plugin version of Selenium so far to enter information into a form and submit it. The form gets posted to a database and the info shows up on another URL that I check. I need to be able to check the info I submitted in the original form against the info that shows up on the second page the "info submitted" page.
Does the firefox plugin version of selenium support a way of storing the information for the fields I submit to the initial form as variables so I can check those values against the "submitted info" page later?
I am new to selenium in general.
I found following FireFox plugin to check variables you use in test, so it seems it's possible with Selenium IDE.
On the other hand, I would recommend using Selenium WebDriver as it is more up to date and allow you much more than IDE due to everything what programming languages like Java or C# provides.
In your case, you could for example, open first page in fist browser tab, post form, then move to second tab with another page and check values correctness. Easy to implement and quite effective.
You can find numerous tutorials on the internet, or you can check my own tutorial which I started to post recently under my own project - Test Automation 101.
I am looking for creating a functional training package for a web based product (more like a screen capture).
However, the requirement is not just to have screen capture, but to have a test mode for the training where we can prompt the user to click on the screen and check if they have done proper flow.
for e.g. A banker will first go through the screen capture to see "how to open an account" in the core banking application . Later user will be presented with a test where user has to click through all the controls and flow.
I have seen selenium being used for authoring good UI automation test cases, so I wanted to explore the possibility of using selenium for creating these training modules
These training modules needs to be launched from intranet application. Is it possible to launch selenium authored UI automation test cases from the browser? I want to stay away from writing any extensions or plugins.
I haven't used selenium and I might be completely off tangent here, so any other suggestion to achieve this using open source tools are welcome.
Without writing any scripts, you may consider downloading Selenium IDE. It has a recording and replay function that may meet your need.
For a fast tutorial, please follow this link.
https://www.youtube.com/watch?v=gsHyDIyA3dg
Hope it helps.
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
Now I hava a requirement about Web GUI TA
I want to simulate some users(20-30) click a button at the same time and evaluate the performance of Web GUI at that time.
I use RobotFrameWork + Selenium library to do the Web Gui TA before, but as far as I know. selenium library only can handle one broswer at one time, so i dont know how to do now.
Can you give me some advice? need use another library or framework?
Like mentioned by other, what you want to do in this case is not UI testing but rather stress/load testing. You should be able to try easily Gatling. First you record the http request associated with the click on your button. Then, you write a simple scenario that launches this request 20 times at once. Something like:
setUp(scn.inject(atOnce(20 users)))
.protocols(httpConf)
Selenium has a "grid" option you can use to configure many instances running many browsers.
http://www.seleniumhq.org/projects/grid/
http://www.seleniumhq.org/docs/07_selenium_grid.jsp
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to
run the tests against a vast combination of browsers / OS. minimize
the maintenance time for the grid by allowing you to implement custom
hooks to leverage virtual infrastructure for instance.
In short, you create a "hub" that manages things, then each "node" can perform tests as required by the hub.
Consider, however, this may not be the best route to go down. Something like multi-mechanize might be more useful: http://testutils.org/multi-mechanize/
That will allow you to have many users "clicking" the button, but not via a browser but via direct HTTP calls. That might be more suitable for multi user simultaneous "headless" load testing, which is what I think you are attempting to do.
I'm slightly confused at this question:
Are you wanting to test the GUI? If it's something like "This button makes a dropdown menu appear", then it doesn't matter how many users do it at the same time, it'll either always work, or never work.
Are you wanting to test the server under load? If so, then Selenium will work, but there are better tools. I have used JMeter with success, but there is a really good listing of all of them here: http://performance-testing.org/
Finally, are you wanting to press 20 different buttons on the same page on the same browser at the same time? If so, this isn't possible with selenium, and it isn't a standard use case either.
I'm trying to figure out an ideal way to test my views, written in Jade. They contain some logic. For example on the home page if a cookie has been set it should show a button to link to the main console and have option to logout versus if no cookie is set it should show login.
Any advice is appreciated.
Have a look at selenium
http://seleniumhq.org/
And of course the node adapter
https://github.com/LearnBoost/soda
I think that are not hard bugs to debug(You should see that in the browser window if it happens). But if you really want to I think you have following options:
Write the tests the way TJ did looking at his tests. He has written tests for almost everything.
Doing headless testing use for example tobi or zombie.js. There is also phantom.js, but this product is not developed in node.js.
selenium(not developed in node.js) with soda like deltanovember pointed out.
You could use js-test-driver to test all your browsers(client-side javascript). You could also use YETI(using YUI test framework) to test all your code in all the browsers.