I have been using Selenium a lot lately (testing an ExtJs app) and while in many ways it is wonderful, there are three things that cause me a lot of grief:
Inability to directly click on elements other than buttons. This has led me to write a bunch of Robot code to move the mouse around. That works but is a bit fragile, plus if you touch your mouse while a test is running, you are screwed. I tried the Selenium forums to see if there was a better way, and got nowhere. I think (but am not sure) that this is a fundamental limitation of Selenium's JS injection technique.
Inability in many cases to control what the 'id' attribute get sets to. This happens inside ExtJs and some elements let you set it, some don't, and some do but the attribute ends up where you don't expect it. You end up having to use XPath in some cases. Using XPath with ExtJs is kind of horrible as ExtJs creates massive levels of nested DIV's. You can also sometimes use CSS locators (which are also inconsistently controllable in ExtJs). (BTW, this is obviously not a Selenium problem per se).
The time that Selenium takes to fire up FF is too long... way longer than a normal human FF startup, about 2 seconds per test, which translates into tests that last several minutes, way too long.
I briefly looked at Watij, BadBoy and a couple of other web functional testing apps but none of them looked anywhere near as good as Selenium. (The way Selenium tests can be written in Java and run through jUnit is really, really sweet). There are also a few commercial alternatives but they are beyond my budget and there is no assurance that they would work any better anyway.
Any thoughts or suggestions appreciated.
About 3:
Selenium copies at startup the firefox profile to a temporary folder. If you don't specify a custom profile, Selenium propably uses the default profile which propably is bloated with addons you don't need for the test. Startup Firefox with '-p' and create a new profile for Selenium and copy it to a location you can point selenium to. This should speed up the test a bit.
Update:
Firefox Profile location / Windows: %APPDATA%\Mozilla\Firefox\Profiles
My thoughts:
There are some cases that click isn't enough, but you certainly shouldn't be limited to buttons. You might want to experiment further with your locators.
Why do you need to set the id attribute of elements? I have experience with testing ExtJS applications, and the problem is usually locating elements that have dynamically set ids. In my opinion this is an issue with ExtJS and not Selenium. Using smart XPath techniques using contains, starts-with, and substring can make your locators much more reliable. CSS locators are also often helpful as you mentioned.
As amarsuperstar says, you don't need to start Firefox before every test. If you do, you might want to consider using the browserSessionReuse command to speed up launching the browser. Alternatively you can use Selenium Grid to run tests in parallel.
Finally, it's well worth looking into the WebDriver API that will be in the soon to be released alpha of Selenium 2. In my experience Firefox launch times are reduced, and commands such as click are much improved.
I am not sure about the first two points, but as for the third, I don't think you need to start the browser for every test. You can use the Seleium server (a jar in the selenium directory somewhere) then point your tests to that e.g. localhost:6554 and that will only open the browser once.
With that you can have the steps in your script to start server -> run all test -> terminate server and you will only have one browser session across your tests.
My experiences (hopefully useful ;)
I've never had this problem, even with ExtJS. I have not used it with ExtJS 3.x, though. Is it possible that you are experiencing some thing as a result of your environment rather than Selenium?
UPDATE: As Dave Hunt reminded me, sometimes I've had to use mousedown/mouseup actions in lieu of "click"
I've found many clever ways to navigate using CSS locators (selenium supports most of CSS 3). In addition, you can use xpath like xpath=id('myid')//div['#class='foo'] (the ID part is crucial).
I've also never experienced this. Perhaps you can give some details about your environment?
Thanks for all the answers guys, I really appreciate it. I spent all day yesterday on this stuff and I wanted to add a couple of observations:
The way ExtJS lays stuff out can really make it hard to locate elements. For example, quite often the 'id' that you specified appears on an element that is 2-3-4 items in the DOM above the one you really want. The actual behaviour seems to vary greatly depending on the type of element. I have half a mind to write all this up for the benefit of future Se-ExtJs testers, as it all seems very trial-and-error and tedious. But ultimately it seems like it can be made to work nicely. And, of course, this is in no way a reflection on Se. Whether it reflects poorly on ExtJs, I'm not really sure, but it is amazing how many DIV tags even fairly simple projects create.
This is probably a 'well OBVIOUSLY' but to anyone else doing this, I'd recommend getting comfy with XPath. It seems a bit obtuse at first but after a few hours, as noted above, you can find almost anything with it, and almost always in ways that are not overly brittle.
Happy Holidays!
Selenium is one of the backends to the TestPlan testing framework and it can address a few of these issues for you.
We don't seem to have any problems here. Our front-end uses many different locator syntaxes to locate any elements on the screen. Though if your page is complicated and Selenium truly can't do it then ours won't help you either.
We base everything on XPath, so after a while you just get used to it. There are all sorts of shortcut syntax that you can use in XPath that may help. In TestPlan scripts however you can also use variable expansion in XPaths which makes them much easier to maintain.
TestPlan caches browser sessions when possible and not otherwise requested. This helps the speed a little bit, but only so much since normally you want a fresh session for each test anyway.
TestPlan
Related
So, I am trying to create some sort of bot to interact with Twitter. I am familiar with Cypress and wouldn´t be a problem at all learn how to deal with Selenium, but my point is: for what I´ve been reading, they´re both specific for testing, and what I want to do is not exactly testing, but keep an algorithm running for as long as I want.
I can obviously make a loop on the Cypress or the Selenium code, but I don´t see it as the easier way to deal with the problem.
So my question is: is there some ideal framework to keep pressing buttons or typing stuff for a few days on a webpage, not for testing, but to automatically interact with other accounts?
What you need is what is called an autoclicker.
Although this can be done with Selenium, there are simpler and faster alternatives like this Chrome extension: Auto Click/Auto Fill.
A search for autoclickers can help you figure out the best alternatives for your use case and there are plenty of approaches in many different languages.
I am a novice in testing.
I am working on Linux.
I was reading about testing in headless mode and came across two things. One was X virtual frame buffer which does graphical operations in memory. So, no output is displayed. The implementation details I found in this link http://www.seleniumtests.com/2012/04/headless-tests-with-firefox-webdriver.html.
The other one that I came across was HtmlUnitDriver. This also does not open any browser while running the test. I wrote a basic sample code using HtmlUnitDriver and the assertions seem to work fine.
I understand that HtmlUnitDriver doesn't work too well with javascript. But apart from this, are there any major differences to choose one over the other?
I am going to be testing a web application that does have some abount of javascript in it.
I am a novice in this field. So, any answers, suggestions, etc. will be appreciated.
Thank you in advance
From my experience with both approaches:
HtmlUnit will in most practical cases be faster than a real browser with xvfb -- simply because it doesn't spend time rendering the pages. (A data point: 17 secs. HtmlDriver vs. 62 secs. FirefoxDriver for a specific test suite I'm using now).
It is easier to run several tests concurrently -- and it consumes a lot less resources -- using HtmlUnit. This can be very important if you have a large number of tests and you need them to finish fast (e.g. you want to follow the 10-minute-build rule).
As you said, HtmlUnit has its own quirks with JavaScript and the DOM. Not better or worse than any other browser (Firefox, Safari, IE, Chrome, ... -- they all have their own quirks), but one on which it is very questionable to spend time fixing bugs. I also find such bugs very difficult to diagnose, but that may be only my ignorance.
One advantage of real browsers + xvfb is that you can always use the exact same tests without xvfb and see what's going on -- possibly even use a console to run some JavaScript to diagnose issues. I sometimes feel quite blind when working with HtmlUnit, and because of above-mentioned quirks you can't always use the same exact tests code in both environments.
So, in summary, unless total test duration is important and you're ready to spend some time fighting HtmlUnit, it's just easier to go with a regular browser + xvfb.
I also like using xvnc, which has the added benefit of allowing you to connect to the screen of a running test and see what's going on (not sure whether you can do that with xvfb).
Hi I want differences between architectures of selenium RC and webDriver.
I read lot of functional differences but did not find anything for architecture differences.
If anyone can send/post links of differences that will really appriciate.
Thanks in advance
The difference is quite substantial.
Selenium RC works only using JavaScript for its every command. That means that everything you write is eventually translated into Javascript and run in the browser. This approach has several pros and cons - it should work the same way in every browser, but is limited to "same origin policy" and to JavaScript limitations.
http://seleniumhq.org/docs/05_selenium_rc.html#how-selenium-rc-works
WebDriver actually uses each browser's own and native API's to work with them. That means that it sometimes needs direct help from browser development team, sometimes fails on a new browser version release, sometimes behaves slightly differently on different browsers (but hey, on IE, js was also not the most reliable thing), but is a much stronger tool overall. It should be faster, it should allow for much more complex work without any js limitations.
http://seleniumhq.org/docs/03_webdriver.html#how-does-webdriver-drive-the-browser-compared-to-selenium-rc
Note that while Selenium RC has been oficially deprecated, the WebDriver is now being developed rapidly and it still suffers from several child-illnesses and is not in its full strength. That said, using WebDriver, you can do anything Selenium RC can do. And sometimes more. With an occasional minor bug.
here you can Selenium Webdriver Architecture
http://qeworks.com/selenium-webdriver-architecture/
you can also find diff. between RC and webdriver below
http://qeworks.com/difference-between-selenium-rc-and-selenium-webdriver/
Though there are many advantages of Page Project Model, some of them worth mentioning are :
Simple and clear page classes with sensible method names.
You can actually give the customize names to you methods. Like above so that you need not to keep anything in mind.
Just looking into the method name gives you all idea about the capabilities of the method.
Makes tests more readable. In comparison to above commands of selenium , where in you need to add all the commands in the tests scripts . in page object model you need to put method names. The methods which you have created according to your understanding of application so these methods name are more readable and easy to understand.
Stay [DRY]
Page object model believes in the principle of Do not repeat yourself.
Good support of tests, because everything is stored in one place.
Easy creation of new tests. In fact, tests can be created by a person not knowing the features of automation tools.
As I have actually implemented it in my project so definitely there are some flaws:
All locators should be kept in page class file.
And this abstraction leads to some chaos with in the Page Class file.
So you need to implement something like key word driven on top of Page Object Model so as to fully take the advantages.
My duty is to pick up and continue developing PHP website for a small sized business client. Project has no testing code. I want to quickly establish at least very basic regression testing for the backend of the site.
I need to test the full contents of the web page char to char. Must see the diff of failed tests.
I need to be able to set up cookies and GET/POST data.
Once every few days I am updating the local database from production database. I would like to then have an overview of failed tests and very quickly update my test suits so that everything is passing again.
Is using WatiN or Selenium a good idea? My local environment is Linux.
About Selenium (and only Selenium as I don't know WatiN) - it can only do what you can do in your broswer. It can click, type in fields, submit forms, take screenshots (that's a very good one), set up cookies (so yes to this one). You can always set up GET data through the URL. But I am not aware of any technique in Selenium that would allow you to set up POST data in any other way than navigating in a browser. Also, because the tests are in your browser, they are not particularly fast. E.g, on our product, a single thorough test with ~250 steps takes about 10 minutes on my computer to complete. Of course, you can always divide that between many computers using Selenium Grid. It's just more work.
To conclude it - I'd say yes, Selenium is good for your needs as there are so many ways to write a good test in it that everyone finds his style. It is good for quick checks, functionality affirmations, but also for full-scale tests etc. But if you want to do some really advanced stuff, then that's a job for a long time. Selenium offers so much functionality in so many different ways that it is definitely a full-time job to understand them and know how to use them.
Try Selenium-IDE for 20 minutes. It is just an addon for Firefox that can record your actions and then replay them. If you like what you see, go for it. If no, hire someone who will.
I'm not sure if I am too late here, but in regards to WatiN it is only IE based, so if you plan to use any other browser you are better off with Selenium WebDriver (though WatiN has some Firefox support). From what I have found (I have used both WatiN and Selenium) Selenium can achieve more low level interactions (also see Selenium Grid), but really I think it is dependant on what you are looking to achieve and personal preference. If you have time to write your own wrapper to interact with WatiN/Selenium you will find the tests themselves are rather quick to run. Also, the beauty of Automation is that once these tests are written you can run them and walk away while they complete.
My team has started to use selenium to do IE browser testing. One major sticking point is that selenium has a hard (read: seemingly impossible) time figuring out when a page has fully finished loading. We use a lot of javascript, a fair amount of AJAX, and a smattering of third party controls. Thus, it's hard to know when all of that activity has completed and the page is ready to test. Much of this code is legacy, so simply rewriting or refactoring the page is not an option -- it's a very large enterprisey app.
My main question is this: Is Watin any better in this regard? Does it have any sort of built-in mechanism for detecting when a page has fully loaded?
We unfortunately can't just wait for an element to become visible, which is the generally accepted practice for selenium. Often we are dealing with a scrolled list of dynamic elements and need to wait for the javacript to finish, as there is an indeterminate (to the test) number of elements on which we must wait.
Switching to another browser is simply not possible, we must find a solution that works with IE. I'm trying to decide if it's worth our time to investigate watin, or if we would just be trading one set of limitations for another.
I used WatiN for quite a while and have more recently made the switch to Selenium because I feel it actually handles the situation you explain better than WatiN does (has a 'wait' that can be created on the driver in Selenium WebDriver). However, I think it really ends up being a matter of opinion.
WatiN will only work well with IE, so you are limited to this and can't branch out into other browsers. Due to this, WatiN is made for IE, suggesting that you might have better success with it for IE.
As a side note, have you tried using Windows calls to check if a browser has finished loading? I believe this is possible? An the addition of XPath in Selenium can be rather useful, though slow with IE.
EDIT: Alternatively, couldn't you call some javascript to see when the javascript on the page has finished?
EDIT EDIT: Also this post covers a lot of comparisons and opinions.
Well as far as my work with WatiN goes it's easier than Selenium. We too have a lot of javascript, ajax - my solution for that is using
WebBrowser.Current.Div(ids.waitForBagGridToLoadIdentifier).WaitUntil("style", "display: none;");
to wait for a particular javascript to finish it's work. There is probably a similar approach for Selenium.