In my project, test cases located in Jira Zephyr. I have some troubles with Zephyr as you know ( lack of preconditions for test cases, native support for BDD ...). So I decided to switch test management tool to Xray.
I haven't found any solution to migration my test cases from Zephyr to Xray.
After that new issue is:
Is there any problem if i install Xray to Jira account while Zephyr is staying on?
Both plugins uses the issue type "Test", so that's a problem. But you can configure anohter name for the issue type of one plugin, i.e. "zephyr test".
Those 2 tools are two different plugins for JIRA, so they can co-exists and be used in parallel (by the same or different teams).
In order to migrate data you have 3 ways:
Migrate by yourself (complicated)
Ask Xray to help you with the
migration
Ask a 3-rd party consultant/vendor company to help you
with this. Here are a few
https://www.servicerocket.com/
http://www.moveworkforward.com/
.. many more (just google)
Ask a consultant (me or somebody else)
Related
I'm running Jest integration tests on Jenkins and I want to integrate them with TestRail in order to automatically put test results to TestRail. In this way I will know easily how many tests are passed/failed?
Does anyone tried that?
I guess you are looking for something like this one https://github.com/zeljkosimic95/Jest-2-Testrail . Although your question is too old but it might help someone else. This is not official plugin but it may help you.
Adding a suggestion here, we should choose the tool after analyzing all the requirements in your software testing services and product. Because there is no official plugin for this except this https://gitlab.com/craydent/jest-testrail (haven't tried). But you still can do this without plugin with this library/code https://github.com/zeljkosimic95/Jest-2-Testrail
You can probably have a look at Agiletestware Pangolin solution which allows you to export results of your tests into TestRail automatically from popular CI systems.
In order to be able to upload test results, Pangolin requires you to create report in JUnit format which can be done by using https://github.com/jest-community/jest-junit
Disclaimer: I'm a developer of Agiletestware Pangolin
I'm looking for a solution on how to deal with API versions in ANYpoint API manager. At the moment it is possible to create a new version of an API. But it is not possible to distinguish between different OTAP environment. In my situation it could be possible that a test environment has a newer API version than production. Do anyone recognize this issue and how did you solve it?
Currently, there are no environment promotion capabilities per se in Anypoint Platform. Having said that, there are a number of things that you can do that can help in that regard, for example:
- You can export an API from Organization A and import it on Organization B.
- You may define different sub organizations, to reflect your OTAP environment structure.
In general, it is not strictly bad that on QA, Stage or UAT environment, you have a newer API version than in Production, if you plan to implement such version in there.
Just my 2c, Nahuel.
As far as i understand if we want are going to create multiple apis then RAML versioning should be followed. what i do i can share with others.
development raml version= 0.1.0
First major release version=1.0.0
if there is any minor change then we can do 1.0.1
if there is any major contract breaking change then we can use 2.0.0.
QUESTIONS:
Do you have any direct or indirect experience with Web Test Manager by Sela Software Labs?
Positive or negative experience is fine. I’m just looking for some facts to base production decisions from.
How risky is it to install Sela "Web Test Manager" to our Production Server
WTM is a TFS Web Access extension. It extends website capabilities to include editing test steps and running tests.
Any other alternatives to executing tests and logging test results in TFS that we should consider?
Scenario: I have 2-3 Developers starting to run test cases as early as this week. We have 3 MTM (Microsoft Test Manager) licenses we use for testing (2 testers, one dev with VS2010 Ultimate). Purchasing another two full copies of Microsoft Test Professional for each of our VS2010 Pro/Premium (not Ultimate) devs just for periodically running test cases and doing light test case editing is not reasonable. We do not need trace listeners for general test pass runs.
Option #1: Sela Software Labs (Sela Group, Sela International) developed Web Test Manager several months ago but there is very little product reviews or customer feedback publicly available.
Sela WTM website: http://www.sela.co.il/alm/products_WTM.html
Single review posted on Microsoft partner marketing site: http://pinpoint.microsoft.com/en-us/applications/web-test-manager-wtm-is-the-only-tool-that-enables-to-test-with-tfs-2010-directly-from-the-browser-now-for-the-first-time-you-can-manage-your-tests-12884914644
Option #2: Developers track their results in individual spreadsheets which Testers then re-enter using MTM. This is not appealing at all and introduces several tedious failure risks.
Other options?
Are you talking about automated tests or manual tests? For doing automated testing, I believe you don't need additional VS licenses..
Fancy version: Set up a test machine with the Visual Studio Test Agent (and Build/Workflow agent if you need to do deployment), a TFS Test Controller, and trigger test runs from a dev machine. If you do it right, results get automatically published to TFS and attached to the correct TestCase workitems and TFS build objects. Check out Visual Studio Test and Lab Management for more info on that (definitely some extra overhead in building out the infrastructure, but it's really slick once you've made the investment). You should be able to trigger the Build-Deploy-Test workflows from any VS license that has TFS access, I think.
Less fancy, but still doable: Still install the Test Agent, but don't worry about actually wiring it up to a Test Controller. The Test Agent installation will at least give you MSTest and the ability to /publish the results of test execution up to TFS for reporting/result storage.
If you're looking to do manual test execution/reporting, I unfortunately don't have a lot of suggestions.. Most of my team's investment has been on automation, so I don't have a ton of experience working with the manual testing interfaces. :/
I'm testing WTM to use it in our company. We have common reasons mentioned by you. For these issues WTM seems to be a good and the only one option.
You are right, unluckely there are almost no reviews. So why I want to share my experience with WTM here.
Installation is easy and quick. Got no problems.
There is not much documenatation stuff on the web page. It's a pity. Hope Sela would make it better in the future (s. VisualAssist X Extension as an example for good public page). At least there is a good pdf documentation in the WTM folder after installing it.
Technically the extension is very good. But there are still some limitations (listed in documentation) and enhancements to be done (i.e. filtering, test step editing for customized TestCase TFS Work Item)
I think, for now it is a good option for testers who don't really need all MTM features.
I created an open source tool called the TFS Test Steps Editor. Originally it was developed to work around MTM's lack of ability to insert line breaks in test steps. I just released a new version that has the ability to publish test results for manual tests.
MTM is a pretty big pain to use: it's slow and buggy, and worst of all, it will sometimes lose data while attempting to publish a result. My tool saves all of your in-progress test execution to disk as you work, and you can export a .ZIP of your results for backup or to re-open on another machine and resume testing.
We are looking for a software to run our test cases automatically.
We want a software which will run on our server (or a commercial), which automatically gets the newest commit on github. Then compiles the commit of the project with CMake and run Ctest on our test cases. The results should then be visualized on a nice website.
I had a look at CDash, but as the documentation is so bad I did not even get it to get the latest commit from github.
So my questions are:
Is there a good tutorial to CDash? Except the bad wiki page.
What software is available for running tests on new commits to github, what are their advantages and drawbacks?
In answer to your second question, Jenkins is a robost and extensible continuous integration tool that can be integrated tightly with GitHub using a plug-in (or loosely using standard Git support). It also supports CMake via a plug-in. Whether it has disadvantages that will make it less useful for you depends on your organization and build process, but I've found it to be highly customizable to a wide variety of processes. I recommend taking a look at it.
There's also a third-party Ctest plugin available for Jenkins.
CDash works in pair with CTest. If you are already using CMake then it should be fairly easy to submit your testing results to CDash. I'd recommend reading the CTest documentation:
http://www.vtk.org/Wiki/CMake_Testing_With_CTest
You can either install your own CDash server or use Kitware's hosted server at my.cdash.org. You can test your server with a sample project available at:
http://www.cdash.org/cdash/resources/software.html
We have our source code stored in Kiln/Mercurial repositories; we use MSBuild to build our product and we have Unit Tests that utilize MSTest (Visual Studio Unit Tests).
What solutions exist to implement a continuous integration machine (i.e. Build machine).
The requirements for this are:
A build should be kicked of when necessary (i.e. code has changed in the Repositories we care about)
Before the actual build, the latest version of the source code must be acquired from the repository we are building from
The build must build the entire product
The build must build all Unit Tests
The build must execute all unit tests
A summary of success/failure must be sent out after the build has finished; this must include information about the build itself but also about which Unit Tests failed and which ones succeeded.
The summary must contain which changesets were in this build that were not yet in the previous successful (!) build
The system must be configurable so that it can build from multiple branches(/Repositories).
Ideally, this system would run on a single box (our product isn't that big) without any server components.
What solutions are currently available? What are their pros/cons? From the list above, what can be done and what cannot be done?
Thanks
TeamCity, from JetBrains, the makers of ReSharp, will do all of that. You will have to configure it for what specifically it means to "build your product", but you can configure up everything you specified with it.
The software can alert you to failed builds, even down to alerting only the person responsible for checking in code that broke the build. It even comes with handy web pages you can view to see only your own changes, which builds they've been through successfully, which ones are pending, and which ones are currently being executed.
Since it is a distributed product, you can make it grow with your organization and product. If at some point you discover that you're waiting for the build to complete too much, because a lot of builds are being queued up, you can add more build agents. The build agents are basically separate client programs you install on additional machines, that execute the actual build configurations.
It comes in two flavors, the professional version and the enterprise version. The professional version is free, can contain up to 20 build configurations, 20 users, and 3 build agents. The enterprise version has unlimited users and build configurations, and you can also use LDAP based security (think domain verified users.) There's also some other bonuses from the enterprise version. You can also buy licenses for more build agents if you need more than the initial 3.
Now, if "no server components" means you don't want it to act like a web server, you're going to be hard pressed to find something that will react to your commits.
However, if you mean that you don't want to have to install a server OS, then TeamCity can work on workstation versions of Windows as well. That isn't to say that you shouldn't consider setting up a proper server for it, but it will run on a workstation if that is what you require.
Our product BuildMaster does all of the things you listed by design and there is a free, somewhat limited edition (e.g. you can only have a limited number of issue tracking providers integrate with it, the database change script packaging tool isn't included in the free version, etc.) for 5 users or fewer.
What you've described is the basics of a CI Tool, so every CI Tool should be OK.
I use Cruise Control.NET but it is bugged with Mercurial and is not very straightforward at first glance. I am nevertheless happy with it. Other tools that come in my mind are Hudson, Team Build (from TFS) and TeamCity.
I have not tried other tools but you can see pros/cons here :
TeamCity vs CC.net
Hudson vs CC.net, Link 1 and Link 2
CC.net vs TFS
EDIT : I forgot to mention that Hudson and Cruise Control.net are Open Source project, you can easily write plugins and patches to your install.
EDIT² : Mercurial bugs seem to be fixed in the upcoming 1.6 version of ccnet (changes commited to the trunk this week).
There's always BuildBot which I like (and have contributed some code to ). It's fairly easy to set-up and run on any OS, and to do simple tasks like that you say, and remarkably flexible if you need it.
What you might find missing is batteries-included log-scrapers and/or report generators that other more commercial CI-servers comes with, especially for Enterprise-y frameworks.
It scales pretty well too, Mozilla and Chromium use it, amongst others.