I am working at a company that uses Redmine as the issue tracking tool and for user stories.
I need to design the QA for them. However I didn't find a section where someone can tie a user story or an issue with a test case. I've used to have this functionality on Rally.
Is there a free tool that integrates with Redmine or something like a plugin?
I want the following (or the more the better!)
Description
test steps (input-output)
Status (pass-fail)
Suite name (the suite the test is part of)
Attached issues
Version of SW under test
Thanks in advance
There is a "test case management" section in the Redmine Third Party tools documentation.
thanks for the support.A former colleague brought this to my attention:
https://bitbucket.org/bugzinga/redcase/wiki/Home
One thing I often see is people trying to adapt tools to do things that they were not originally designed for. Redmine is a good tool for issue tracking and is designed mainly to be a project management tool, although it is possible to adapt redmine, I would suggest in this case looking for a dedicated test management tool that has been designed to do that specific job.
I would suggest looking at TestLodge test management tool which I have worked on and is a designed to help you manage and execute your tests but at the same time integrates with Redmine and does things like automatically create tickets whenever a test fails.
By doing this, you are going to benefit from a range of things such as an interface designed for testing that will make you and your team a lot more productive along with a series of reports that will allow you to spot trends within your testing.
Related
We are three testers and going to prepare automation project with selenium and java code so what are the steps for environment setup , scripts integration and running the testcases and getting the results for the whole project suits
So there are a few things we have to use in order to allow multiple engineers to work on the same framework.
Step 1) Creating the framework, assuming you know how to do this already, you have working tests you can skip this stage, however if not please follow the tutorial i link below.
http://toolsqa.com/selenium-webdriver/
Step 2) Creating a REPO, my preference is GitHub, you can use any git repo however i will post the guide to set one up with GitHub, its a similar process for all. This will allow you to merge code properly without causing conflicts.
https://help.github.com/articles/create-a-repo/
Step 3) Source Control program - to push, pull and fetch from your GitHub Repo, you can do this from Command Prompt however i find cloning the repo into a program like 'SourceTree' is really easy, so i've posted that below.
https://confluence.atlassian.com/get-started-with-sourcetree
If you follow these 3 guides, you will be able to have your automation test scripts on GitHub by the end of the day.
If you have any more questions please do not hesitate to ask.
All the best, Jack
The easiest and most logical way to do this would be to create one branch in your CVS (git or SVN, etc) and have each person setup the dev environment in the same way. Work exactly like developers and pull code before you check-in/commit (this will ensure that one small error does not break your framework) and swear to resolve conflicts during merge (to ensure you don't step on each others' toes).
Also, before you kick off, agree on a standard of coding (including package naming, design pattern usage, filename and methodname usage) and if this is in sync with the dev coding standards in your company, even better.
There will be a few hiccups along the way, but experience is the best way to create a process for your development and check-in practices.
Good luck with your new project and happy coding!
You have asked two questions, in my opinion the answer of your questions is.
how multiple automation testers work in same selenium project - You can use any version control system, Git Hub is the best option which gives you a lot of facilities. You all three can work on same project at same time or you can go for any centralized version control system like tortoise svn which is not much likely used now a days. I will suggest Git Hub for that.
what are the steps for environment setup , scripts integration and running the test cases and getting the results for the whole project suits - It depends on various factors like application and the kind of framework you want to use, there are many frameworks which are widely used for automation testing like Modular Framework, Data Driven, Keyword Driven, BDD, Cucumber, TestNg etc or if you have bandwidth and time you can design your custom framework as per the needs.
I hope I put some glimpse on your queries.
Thanks
Could someone explain what an automation test is and why I would use it. I read from the wiki page that a tester would create a automation script? What kind of scripting language can be used to do this?
Automation tests are carried out to check the behavior of an application against expected behavior. Normally used in regression testing where you validate that a newer version of the application doesn't hinder any of the previous version's features. These might also be carried along with manual testing.
Coming to the scripting language part, this might help you:- https://softwareengineering.stackexchange.com/questions/19292/best-language-or-tool-for-automating-tedious-manual-tasks
In simple words, If you are doing regression test or testing same piece of code over and over you can automate that manual process. That's called automation testing.
You can use several different scripting languages to achieve this and it's depends on which tool you are using. Some popular automation tools are Selenium, QTP, Loadrunner, Jmeter, SOAP UI etc.
You want to check your login with more than 1000 of users how much time you will spend to run this test case ?
In the same way you want to test you mobile API's before it used by developer how will you test?
There are lots of thing for that you have to go for automation In small application, sites you can work as an tester after that when those app's sites will grow will large data than those product owner will move for automated test cases
I am in a new project that is looking to include an end to end testing framework.
We want something flexible, I've used Fitnesse before and I think we need something similar to it.
We are also using Hudson CI and are looking for something that would integrate easily with it.
Is there a clear winner?
UPDATED: The system has many components, some of them are web services running on tomcat, there are a couple of NoSQL databases too, but no UI testing is required for the moment.
Please add a comment if further clarification on the project details is needed.
The robot framework is a good keyword driven testing framework that we use for end to end integration testing.
http://code.google.com/p/robotframework/
There is a plugin for jenkins (a fork of hudson) that is very useful. It reportedly works with hudson as well.
Great to read you have decided start using Robot Framework. Its Hudson plugin already has recently got much better (trend charts, available from Hudson directly, ...) and moved to new place:
http://wiki.hudson-ci.org/display/HUDSON/Robot+Framework+Plugin
Can you be more specific, when you say, you want to have an end-to-end testing framework? What kind of application are you developing (standalone, web based, web service, ...)? What language do you program in?
I would also look for previous knowledge in your team or around your team (e.g. other teams). It might be sufficient, to use TestNG or JUnit (I have seen an old paper on UI testing using JUnit). Almost all test frameworks should be fine for Hudson, as long as they can be started in a headless/scripted mode and if they provide the results in a format that Hudson understands (in the worst case HTML is sufficent, though Hudson will only be able to display the pages). So if you have IBM Monitoring tools available, you might want to look at Rational Functional Tester or Rational Performance Tester.
FitNesse is a good tool for functional testing. The business contacts can create the test cases by themselfs (developers have to provide some connector code), which will create more visibility and removes some work from the test engineers. Drawback is, that you can't really do UI testing. If you need a open source plattform for UI, have a look at seam. Even though I like the OSS philosophi, you might need longer ramp up time (higher costs) and possibly more maintainace time for OSS (not true for all OSS). In any case, check how easy it is to get support for your framework. You might even consider paid support for your framework (regardless if open or closed source). Also don't forget, even though you can fix OSS source yourself, you might not have the expertise or time for fixing bugs or adding features.
So give us more info and we can give you better advices.
My experience with jBehave is very good. Recommend.
It's based on BDD - Behavior Driven Development.
I have been working on Test Automation from last few months and have been using the tool named "Testcomplete". But I have noticed that the tools do not matter a lot in the field of automation. Only thing you expect from an automation tool is the ability of the tool to spit out the recognition strings for the different controls used in the test application.
Apart from this, you will always have to build a automation framework which will serve your needs writing code.
So my question is, Is my thinking that automation tools do not matter a lot in the field of automation correct? In the sense, you can use any tools to get your automation running. Or Do the tools really matter? (Please ignore the costs factor of the tools). Also if I need to learn a new automation tool, then what do I concentrate on? Or how do I go about learing the tool? In short, what exactly does "learning a tool" mean?
My 3 best reasons for choosing which tool to use:
it works. This is important, not all tools work in all scenarios i.e. flash, silver light, adobe air, legacy apps with no automation support, etc.
whole team skills. This includes not only testers, but also developers. Test automation shouldn't be an isolated effort, developers should also collaborate on it. This is far easier when dev and test are using the same language/platform.
price. Doesn't have to be free (but it could), but of course its an important factor.
Personally we use the same test runner as the one for the unit tests. That along with extra third party automation pieces that do the plumbing for you.
Some additional thoughts on why the tool is important:
Community - What's the user community like? Are there a lot of user-generated resources out there to help?
Support - (if vendor) What's customer support like? Do they fix problems quickly? Is it easy to find solutions to common problems?
Extensibility - Often in test automation, you'll need to roll your own or code work-arounds, if the tool does not support a particular type of object in your application. How easy is it to extend the product? What programming language does the tool use? What kind of support do you get from the IDE?
An other piece of advice: sometimes you'll need wrapper classes around certain frameworks. We were using WatiN, which was really good at its time, but it lacked Chrome (it had a small percentage that time) support. The thing that killed WatiN for us was the lack of coping with new FireFox releases: FireFox 8 was out, and we had to run on our tests on FireFox 3.6...
Selenium was the solution, but it had a totally different logic and we already had more than a 1000 tests.
So we had to create a wrapper class around Selenium to "fake" it was WatiN. We had some issues, but we had to rewrite only some special cases... And not all tests.
The point is, sometimes, support for frameworks just cease to exist. But with an own framework focusing on what the test actually does instead of how it works would save you in this situation.
Variety of Test Automation Frameworks and Test Automation Tools are already available in the market. Thus, I would not recommend the built your own Test Automation Framework at all.
As far as selection of automation tools is a concern, I would say it does matter on the following basis:
Support: How much level of support you have when you are choosing an automation tool for your project.
Community: How big community is using that tool and how responsive that community is about sharing knowledge?
Pricing: (Proprietary or Open Source) Last but not the least is the pricing of the automation tool that you are planning to introduce in your project.
QA teams' expertise also matters sometimes. For example, in case your QA team does not have a developer or semi-developer skills vs Not-Technical QA Team, etc.
Regarding the Automation framework, there are many automation frameworks also available in the market already, therefore no need to reinvent the wheel. and selection of automation framework mostly depends on your selection of scripting language.
For example, if you choose python as your scripting language, then you have option to choose UnitTest, PyTest etc. as an automation framework.
In case of Java, you have option of JUnit and TestNG as an automation framework.
and so on, the base on your selection of scripting language.
Finally, when it comes to structuring your automation framework, it solely depends on many things as the following:
Your nature of the project
Single product vs multiple products
and many more...
Check an example of multiple product automation project directory structure. https://github.com/pancht/python-selenium-framework
I hope, in some way, I would have helped you out in giving an answer to your question.
Thanks,
Panchdev Singh Chauhan
My goal is:
Our customers could generate new web-tests.
Our continuous integration server makes a test-environment deployment; it should execute the tests against it
The test could also be run against some other environment.
(Final acceptance tests should be made by the customer, to test fonts etc, but this would be a great pre-acceptance check for our test-environment. Customers could focus on other things than now.)
Usually some property (like text field id) has changed or something and the tests will break in a few weeks. It seems that recorded tests broke often, so it's better to easily record a new one than trying to maintenance and modify an old test.
Now, I found a whole new approach. Maybe recording is not the right way.
How about, if our customers could make use cases in a human readable own language which the machine would understand and compile to web-recording (with Domain Specific Language, DSL).
This is not sci-fi, it has been already made, so read on. :-)
I have tried to use these automatic web testing frameworks:
Visual Studio web test (Customers can't execute)
Selenium (Works only with Firefox, our customers have IE)
WatiN (.NET version of Watir, recorder seems to be a bit buggy)
HP Quick Test Pro (Not easy enough to make new tests)
None of these have provided actually what I need... but Selenium is the closest one.
Our customers speak Finnish, so in the beginning of a software project, in specification phase, user writes a use case like:
Avaa "OmaLomake"
Syötä "Tuomas" kohtaan "nimi"
Paina "Seuraava"
Translation:
Open "MyForm"
Insert "Tuomas" into field "name"
Press "Next"
Now... This is a human-readable use case, but also it can be compiled to automatic web acceptance test. Open, Insert, into field and Press are keywords, others are values.
What kind of DSL tool would be good for this?
Microsoft is making a new DSL-making-tool in their Oslo-project called MGrammar. It means that you can make a custom language to make it easy for non-technical people to work with machines. (The same basic idea that was (and failed) with Cobol and Visual Basic.)
I found that someone has already made this kind of DSL with MGrammar, but it is for Watin, not Selenium:
http://www.codinginstinct.com/2008/11/creating-watin-dsl-using-mgrammar.html
So the continuous integration server process will be:
Fetch a new version from source control (as usual).
Build, run unit tests and analyze the code (as usual).
Make an installation package and tag version in version control (as usual).
Compile use cases to web tests
Run web tests
Accept/Reject the software :-)
Running a web-test in the continuous integration server usually means a lot of configuration work. So, before I try this, I'm curious, what do you think?
Have you used same kind of setup, and what are your experiences? (What exact environment?)
How about DSL, will it have enough power for use cases or will it be another endless development task? Will the customers ever generate the tests?
First of all, Selenium does work with IE and other browsers as well as Firefox; cross browser support is one of its strengths. Here's the list of supported browsers.
However, if you want a human language-based DSL for writing your tests, take a look at Cucumber - the syntax is almost exactly like your example above. Cucumber already has Finnish language support - see the examples at this link.
Fitnesse and Selenium Integration tools such as Selenesse(http://github.com/marisaseal/selenesse) or Fitnium(http://www.magneticreason.com/tools/fitnium/fitnium.html) can also serve your purpose. However, you need to find answer for who will put the element locators in the test cases written by customers. If customers put the locators using the recorders, it may not be possible to maintain. If customers write the steps and a automation tester/developer can put those locators using regex, custom location strategy, this approach may work.
The TestPlan software uses a specialize language for writing tests. It is highly domain specific and works very well in web environments. It supports the Selenium backend so you gain that compatibility, plus it can run without a browser, for even faster tests. I have used it on some fairly large web projects in the type of setup you are looking for.
Your example script might look like this:
GotoURL /SomePage
Click MyForm
SubmitForm with
%Params:name% Tuomos
%Submit% value:Next
end
That's it. It nicely describes what the user wants to do and is a functioning test. You can combine scripts into units and have custom function as well. So if you really wanted you could write the Finish equivalents to the names.