I love Selenium IDE; mainly because it enables me to automate many regression tests without me having to write any code at all!
Now, I want to do load testing on my website i.e execute the same quantum of testing activity concurrently with many browser windows putting load on the website to check if it holds integrity. I want to stick to the GUI (Selenium IDE) and still want to accomplish this.
I searched and found Selenium Grid can help me accomplish this, however, running Grid involves exporting test cases to Java TestNG WebDriver or something similar and then setting up test suites. This will definitely involve some complexity in terms of coding.
The question is can I avoid this and stick to Selenium IDE (the GUI) to perform this task?
Many thanks in advance!
The only way (that i can think of) is running the same script on multiple computers at the same time. I would run the script as fast as you can without having it crash in normal circumstances. Grab a few volunteers and get everyone to click RUN at the same time.
Alternatively you could run multiple VM clients if you don't have access to multiple computers. Your internet connection may be the only barrier to entry with the VM method
Related
I'd like to know if this is feasible: I'm doing manual testing on a website but I'd like to test the website on Chrome and Firefox at the same time, so is there a way to mimic my mouse clicks and keyboard input on from Chrome to Firefox using webdriver or any other technology?
Edit:
More info as asked in the comment.
I work as a tester, and I've written some E2E tests using webdriver.io and I know that I can run my test on multiple browsers concurrently using multiremote.
I have tried tests recorders too since a long time ago and didn't find much luck, and what I'm trying to do can be partially accomplished with some tests recording services, by recording the tests and then running them on multiple environments.
However what I'm trying to find is a way to read the actions of the user from one browser say Chrome, and apply them on another, so I imagine this will involve finding the selector and the action applied and applying them in the other browser.
I know I can write the tests or record them, but it was just an idea, and it would be great to have it for exploratory testing on multiple browsers at the same time, when I googled it all I got was writing scripts or recording, so I wanted to ask experienced people about their experience, and I think I could have posted this in https://sqa.stackexchange.com/ however it's more related to programming than to QA, so I needed programmers.
P.S: I'll try editing the question more to make it more readable. Thank you.
I have a group that has some non-technical people creating tests and test suites using the Selenium IDE. I'd like for that group to be able to work independently, yet after the fact be able to run a series of those suites with minimal button clicks. There are a lot of reasons why I'm not just writing tests using some 'native' language (groovy or java) and making this easy to use for the team will help adoption of testing.
So, I would like to be able to just instruct the members of the team to open a single 'suite' (or equivalent) and run it and it would then run each of the suites that I have designated as part of the 'master suite' (if you follow me).
I know that I could just maintain a list of the suites that are part of our automated tests, but it would be easier for me to sell if it was possible to just open up a single file and click 'go' and then walk away and see the results after coming back from getting a cup of coffee or something.
If your reason for not going with a native language is because of your non-technical people, then your automation strategy will fail.
Sorry for being blunt, but there is a reason why there is the IDE, and there is the native language support. They both serve very different purpose, and if you don't approach automation with the respect that it's a programming exercise, then your automation strategy will fail.
Selenium IDE is extremely limited. You are unable to string multiple test suites together. You only have the option of creating one huge a** list of test cases in one suite, or loading your suites in one at a time.
Go WebDriver - everything that you want to do, is extremely limited, if even possible using the IDE.
Yes, I wrote a framework that does that. You can record as many "Selenium Builder" scripts as you like and they will be ran by my framework in a multi-threaded fashion as a group. Just fork your own copy of my framework and then modify it to your uses.
Why can't we just record all of our test cases in Selenium IDE, export it to Java/WebDriver and run it in WebDriver with Eclipse?
I need a clear explanation as I am very much confused in using WebDriver!
And can anyone please explain why IDE recorded scripts fail in WebDriver?
why cant we just record all of our test cases in IDE, export it to java/webdriver and run it in webdriver
Great question, and here is the answer:
Selenium IDE is a Record and Playback tool, which is very easy to use, but it's very unreliable. Record and playback is typically a frowned upon in web applications. Since web applications are frequently changed, the IDE is not an ideal solution for a production environment, because of the maintenance nightmare that may arise.
Let me give you a practical example. You record your test, and you find an element with a dynamic ID. Sure we can import it into eclipse, but what happens when that test starts failing down the road? why not simply make your test agile and independent to catch these in the first place.
It also boils down to your principles of test automation. Test automation in MY opinion (and several other professionals), believe that test automation should be approached from a programming perspective. Programmers should write the tests, and maintain the tests. Ideally, your quality assurance personnel should be trained to write and maintain their own tests.
So again, back to your question, the IDE is designed to be a quick solution to automation, NOT a solution to a full regression suite.
And can anyone please explain why IDE recorded scripts fail in Webdriver?
I haven't used the IDE in a while, but the reason they fail, is because the scripts that are exported, are simply the steps, not an entire java file. This also is because Selenium IDE exportations are supposed to be agnostic when it comes to how to run your test. Say I'm a user of jUnit.. what if Selenium IDE exported it to TestNG all the time? That wouldn't be fair.. honestly i'd rather create my own tests than changing that one line every single time i create my test file.
You may read the full text of a research conducted, called Why do Record/Replay Tests
of Web Applications Break?
Why can't we just record all of our test cases in Selenium IDE, export it to Java/WebDriver and run it in WebDriver with Eclipse.
You can actually do this with Selenium IDE quite easily. Record your test case / test suite in Selenium IDE, export to "Java / JUnit 4 / Webdriver" to a .java file. This will generate a JUnit test that you can import and run from Eclipse (with the correct version of JUnit of course).
It's not 100% reliable, and you may need to make some manual changes/corrections, but in general it works pretty well. Start with a single small testcase and work from there.
Why not just record in the IDE and play the recording with WebDriver???
The IDE Does Not Know What to Wait For
Suppose Bob is performing manual checks and recording his interaction with Selenium IDE. He performs an operation that takes a while to update the GUI, and when the operation has finished, he clicks a button. Bob knows that the GUI has finished updating because a spinner that was show when the operation started is removed when the operation is finished. How is the IDE going to capture the fact that the operation must have finished before clicking on the button? (Note here this is not a case where the button is disabled until the operation is finished.) This is not a hypothetical: when you test dynamic tables (like those managed by DataTables), the user can change anything at any time.
This can be one of the reasons a sequence of commands created with the IDE would fail with WebDriver. It just does not know what to wait for.
It is possible to add waits manually in the IDE but if you are doing this, then you are no longer "just record[ing] all of our test cases". And what you are doing becomes more like writing code for WebDriver.
Users are Inefficient
Not long ago there was a Selenium question in which the user wanted to get Selenium to click on the last record in a table that had multiple pages of records. So the question was, how can I page through the table, all the way down to the last page and then click the last record? Someone (maybe me, maybe someone else) pointed out that if the table is sortable, it could be sorted in reverse order and then the Selenium code could click on the first record. That's 2 operations rather than p+1 operations: clicking p times, where p is the number of pages, plus 1 time for the click on the record.
When we write code for WebDriver, we have the opportunity to write the tests to avoid taking the scenic route to get the results we want. See the next point for the technical details as to why it matters.
Selenium Operations Can Be Costly
If the software that runs your Selenium commands is local, and your browser is local, you may not feel that Selenium operations can be costly but if you run your commands on one machine and the browser is remote, then you will notice a significant slowdown. For instance, if you spawn your browser in a Sauce Labs VM or in a BrowserStack VM to run a test suite, network delays are going to add significant time to how long it takes the suite to complete. For a complete application test suite, this can mean many minutes more.
The IDE produces a sequence of commands that each require a round-trip between the Selenium script and the browser. Each round-trip adds up. Suppose I want to check that two elements contain the same text. I'm not going to use Selenese because I don't usually use the IDE but using WebDriver code in Python, the script could be:
a = driver.find_element_by_id("a")
b = driver.find_element_by_id("b")
assert_equal(a.text, b.text)
This code requires 4 round-trips: one round-trip per find_element... and one per access to the text field. The same test could be written:
a_text, b_text = driver.execute_script("""
var a = document.getElementById("a");
var b = document.getElementById("b");
return [a.textcontent, b.textContent];
""");
assert_equal(a_text, b_text);
This needs only one round-trip. When you use the IDE to record actions, the sequence of commands is like the earlier snippet: lots of round-trips. When you write your code for WebDriver, you have the opportunity to optimize as you code.
This does not mean that the Selenium IDE has no use but I would never ever think of just recording tests with it and then playing those recordings with WebDriver.
If you are a newbie, you can actually use 80% of your IDE script as your webdriver JAVA script, u just have to slightly improvise your exported IDE script and it will work fine.
But when you start testing complex functionalities you have to learn some of the basic java methods to work it out!
I beg to differ. I see no reason in having to maintain extra code when a simple record and playback can achieve same results in a fraction of the time.
I do agree with the impl of the recorded script has to be agile. It can't just be as simple as the build in commands and one has to make it agile and even inplement their own macros (aka commands/aliases) specific to the app under test.
Still, if implemented right and with the help of few Selenium IDE plugins like flow control it becomes a breeze to maintain such complex test suits and have them played back via the HTML Runner. I've used Selenium IDE HTML scripts to fully cover complex sites and reused the same script for all responsive modes of the app.
Maintanance is simple as the IDE will help you quickly adjust a failed cmd and AJAX waits can easily be accounted for correctly by waiting for DOM content to change on the page. I'm still not convinced of this AJAX excuse for dismissing Selenium IDE.
For a Maven integration of flow controls command/aliases/macros used by Selenium IDE and Maven's SureFire test plugin see:
https://github.com/paulbors/sideflow
Feel free to use it in your product and improve as needed.
webdriver allows you to execute your test cases captured with Selenium IDE in a Selenium server or via RC to multiple servers. This includes options to integrate with Selenium-enabled providers (see link at the bottom).
For example, if you've captured your test case in IDE, you could easily integrate with your own selenium server, or a provider like Saucelabs to handle the running of your test cases in Firefox on Ubuntu, Safari on OSX and IE10 on Windows 8. As with any test case, you can run it from your IDE (Eclipse / IDEA / etc ) your builder (maven / gradle /), and / or via your CI system.
Check out https://saucelabs.com/selenium for some examples. We've also used our selenium test cases for load testing with Soasta - WebDriver allows simple IDE tests to be used in many different contexts.
Because when the complexity of your code will increase, it will be difficult for you to manage it using IDE. Maintenance of your scripts will be difficult using IDE.
Also Exporting test cases from IDE to webdriver is not 100% reliable.
Other people have said a lot of helpful answers. One thing that is missing is that if you write code instead of using the IDE you have the power to make changes to the database. It depends on the website you are testing how useful this is, but here are some useful examples I've experienced. In one case the website had multiple sections of a class. One of our tests was merging these sections into one. The IDE couldn't do this. But with webdriver we can add all of our test data into the database, perform the merge, and then remove the data from the database.
Another advantage that webdriver has is Selenium Grid. With Grid you can run your tests in parallel and on multiple browsers. This makes your testing run faster and you can more thoroughly test the code.
My duty is to pick up and continue developing PHP website for a small sized business client. Project has no testing code. I want to quickly establish at least very basic regression testing for the backend of the site.
I need to test the full contents of the web page char to char. Must see the diff of failed tests.
I need to be able to set up cookies and GET/POST data.
Once every few days I am updating the local database from production database. I would like to then have an overview of failed tests and very quickly update my test suits so that everything is passing again.
Is using WatiN or Selenium a good idea? My local environment is Linux.
About Selenium (and only Selenium as I don't know WatiN) - it can only do what you can do in your broswer. It can click, type in fields, submit forms, take screenshots (that's a very good one), set up cookies (so yes to this one). You can always set up GET data through the URL. But I am not aware of any technique in Selenium that would allow you to set up POST data in any other way than navigating in a browser. Also, because the tests are in your browser, they are not particularly fast. E.g, on our product, a single thorough test with ~250 steps takes about 10 minutes on my computer to complete. Of course, you can always divide that between many computers using Selenium Grid. It's just more work.
To conclude it - I'd say yes, Selenium is good for your needs as there are so many ways to write a good test in it that everyone finds his style. It is good for quick checks, functionality affirmations, but also for full-scale tests etc. But if you want to do some really advanced stuff, then that's a job for a long time. Selenium offers so much functionality in so many different ways that it is definitely a full-time job to understand them and know how to use them.
Try Selenium-IDE for 20 minutes. It is just an addon for Firefox that can record your actions and then replay them. If you like what you see, go for it. If no, hire someone who will.
I'm not sure if I am too late here, but in regards to WatiN it is only IE based, so if you plan to use any other browser you are better off with Selenium WebDriver (though WatiN has some Firefox support). From what I have found (I have used both WatiN and Selenium) Selenium can achieve more low level interactions (also see Selenium Grid), but really I think it is dependant on what you are looking to achieve and personal preference. If you have time to write your own wrapper to interact with WatiN/Selenium you will find the tests themselves are rather quick to run. Also, the beauty of Automation is that once these tests are written you can run them and walk away while they complete.
My project is compatible only with Internet Explorer. I want the test scripts to get generated automatically as it is done in Selenium IDE.
Can i use Selenium RC to test my application? I could not use Selenium IDE as it can be used only with Mozilla Firefox.
seleniumrc works with IE. You can specify the browser and the path to it within the config file.
It can be easily integrated into night builds via ant.
stick to writing the testcase in java.
Selenium RC and Selenium Grid are both really good at running tests against IE. You can see all the browsers that are supported by Selenium here and Selenium is Designed to write for one browser and work in the rest. THere are a few little quirks that wont work in every browser but 99% of the time it will.
Selenium RC works with IE, but is very buggy with IE 6 (to the point of being unusable). Generating the scripts is not trivial and there are many methods of doing it. We have created a Firefox extension that examines objects via introspection to make click recording easy. There are many options out there but your best bet is to write your tests with Firefox/Firebug (or Chrome). They will make object location much simpler and if you are careful the locator strings should still work in IE.
There could be two answer to you question:
Besides Selenium, though it has ample of advantages, I am reading about another tool which uses same API which Selenium use. The only changes in API I have seen so far is it reduces the complexity of functions thus making it more easier and simpler for user who is learning.
The tool is called 'Helium' and it has 50% (and more) less complex functions and code as Selenium has.
The only problem with this tool is it is paid tool for learning purpose and for implementing not-so-big scale project you can use it. But yeah after some time its gonna cost you.
I have implemented some code on Helium. Please let me know , if you face any issue initially or you are thinking to implement it.
Other being, you can use Selenium Builder(http://khyatisehgal.wordpress.com/2014/05/26/selenium-builder-exporting-and-execution/) which is an advanced form of Selenium IDE. It imports your command in different languages and does work more effectively and efficiently as Selenium IDE does(http://khyatisehgal.wordpress.com/2014/05/25/selenium-builder/)
Please let me know , if you have any doubt in any of the tool.
I know Watin is compatible with IE and Firefox. If you want to generate the test code you can use the Watin Test Recorder
This of course is implying that you are using .Net
... Or you could just use the .net bindings that comes along with the latest couple of versions, then you can just run 'em through nUnit.
For ex. Selenium IDE users Katalon Recorder might be a good match. Supports different browsers.