Are there any tools available that allow user input to be synchronised between browsers during manual cross browser testing?
For example, mouse clicks, data entry into a form etc.
Information:
This sort of functionality was described by Mads Kristensen in a recent DotNetRocks podcast. He outlined a BrowserLink extension he had created (around 11:00) that demonstrated this during a recent talk. However I'm unable to find any other reference to it, and would like to know if any other alternatives exist.
Could you explain how you mean 'synchronise'? As in, it needs to happen on each browser at exactly the same time? Or you just want a consistent set of data used and the timing to be the same for each test?
If the latter, I'd suggest looking at something like Selenium WebDriver. I know this is more automated, but you can't get something exactly consistent if you're doing it manually.
Related
I am writing a web app where many distinct users will be logged in and interacting in real time. Is it possible to use TestCafe to test such a scenario?
I don't know your specific needs, there might be some problems that are difficult to foresee now. However, TestCafe supports multiple browser windows, so it should be possible to use multiple windows where there are different users logged in each window.
Some details about that could be found in the documentation here.
I always recommend trying out the chosen technology first on a few cases before going with it for good. You can pick out a few test frameworks like TestCafe and see what suits you best in your context.
I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.
Is there any way for me to make selenium more human-like so I don't trigger CAPTCHAs?
Hopefully not.
You are scraping, i.e. you are developing a bot, and if you try to avoid being identified as a bot, it will just be a question of time until the captcha gets improved to detect your strategy.
Don´t do it. The captcha is there for a reason, which is: to detect and lockout bots!
Better check if the page you want to scrape supports an API that allows computer-to-computer communication. If there is one, use it. If there is none, suggest one, but depending on whether the web page owner wants to support your goals, or not, he might say "no".
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
Now I hava a requirement about Web GUI TA
I want to simulate some users(20-30) click a button at the same time and evaluate the performance of Web GUI at that time.
I use RobotFrameWork + Selenium library to do the Web Gui TA before, but as far as I know. selenium library only can handle one broswer at one time, so i dont know how to do now.
Can you give me some advice? need use another library or framework?
Like mentioned by other, what you want to do in this case is not UI testing but rather stress/load testing. You should be able to try easily Gatling. First you record the http request associated with the click on your button. Then, you write a simple scenario that launches this request 20 times at once. Something like:
setUp(scn.inject(atOnce(20 users)))
.protocols(httpConf)
Selenium has a "grid" option you can use to configure many instances running many browsers.
http://www.seleniumhq.org/projects/grid/
http://www.seleniumhq.org/docs/07_selenium_grid.jsp
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to
run the tests against a vast combination of browsers / OS. minimize
the maintenance time for the grid by allowing you to implement custom
hooks to leverage virtual infrastructure for instance.
In short, you create a "hub" that manages things, then each "node" can perform tests as required by the hub.
Consider, however, this may not be the best route to go down. Something like multi-mechanize might be more useful: http://testutils.org/multi-mechanize/
That will allow you to have many users "clicking" the button, but not via a browser but via direct HTTP calls. That might be more suitable for multi user simultaneous "headless" load testing, which is what I think you are attempting to do.
I'm slightly confused at this question:
Are you wanting to test the GUI? If it's something like "This button makes a dropdown menu appear", then it doesn't matter how many users do it at the same time, it'll either always work, or never work.
Are you wanting to test the server under load? If so, then Selenium will work, but there are better tools. I have used JMeter with success, but there is a really good listing of all of them here: http://performance-testing.org/
Finally, are you wanting to press 20 different buttons on the same page on the same browser at the same time? If so, this isn't possible with selenium, and it isn't a standard use case either.
I'm using Capybara to test features on a progressively enhanced website. Let's say my feature is to navigate around a hierarchy of locations. The non-javascript version involves getting a new version of the page when we click around on different locations. The enhanced javascript version opens up hidden elements, or loads up new information via Ajax.
I start by writing a test for the non javascript version, which looks something like this:
When I visit the page for "UK"
And I click "London"
Then I should see the information for "London"
Using the default mechanize driver, the test fails, I develop the feature, then the test passes.
I then create an identical test for the javascript version, flagged up with #javascript. It runs the test with the javascript driver, and that test passes because the feature has been implemented. (It's running through the non-js flow). However, I want the javascript version of the test to fail at this point because the feature has not yet been enhanced with Javascript.
So I'm looking for a good strategy for determining whether or not a whole new page has come from the server, and making sure both versions of the feature work. (I plan on integrating this with pushState so testing for a changed URL won't do)
I'm interested to hear other peoples opinions on this - I'm not convinced Cucumber is the right tool for the job, since you're describing features from the perspective of user interaction, and it sounds like your implementation of progressive enhancement will result in essentially the same user interaction.
That aside, I think you may want to consider building in some kind of testing hook to the page itself to help with this. Hard to say what without knowing your exact situation, but maybe one of:
The script-enhanced version of the page could add some element to the DOM, indicating that the enhancements are active, or indicating that the data came from an AJAX request rather than page load.
You could generate a random page identifier (from the server) on every load (e.g. new GUID), embed this into the DOM and assert that it hasn't changed after the interaction (on the enhanced version). This would be a very simple way of achieving your stated goal ("determining whether or not a whole new page has come from the server")
Why does your javascript "enhanced" version work the same as your non-enhanced? Your cucumber tests using #javascript should be testing to ensure the enhancements work.
For instance,
* if the javascript opens a modal dialog instead of following a link to a new page, test for that.
* if the javascript submits a form and updates a value, test for that.
These tests would fail if run without javascript support.