I'm running Jest integration tests on Jenkins and I want to integrate them with TestRail in order to automatically put test results to TestRail. In this way I will know easily how many tests are passed/failed?
Does anyone tried that?
I guess you are looking for something like this one https://github.com/zeljkosimic95/Jest-2-Testrail . Although your question is too old but it might help someone else. This is not official plugin but it may help you.
Adding a suggestion here, we should choose the tool after analyzing all the requirements in your software testing services and product. Because there is no official plugin for this except this https://gitlab.com/craydent/jest-testrail (haven't tried). But you still can do this without plugin with this library/code https://github.com/zeljkosimic95/Jest-2-Testrail
You can probably have a look at Agiletestware Pangolin solution which allows you to export results of your tests into TestRail automatically from popular CI systems.
In order to be able to upload test results, Pangolin requires you to create report in JUnit format which can be done by using https://github.com/jest-community/jest-junit
Disclaimer: I'm a developer of Agiletestware Pangolin
I'm new to this idea but i was just wondering if there is a way or tool to automate certain tasks before deploying my website. i would like to do the following tasks with one go:
minify all JS used
minify the CSS
minify all HTML and PHP
create these as a copy of the files before actually doing these so that they don't destroy the original source
There's a whole slew of tools, frameworks and packages that deal with problems you described. In general, they fall into a category of build automation, so you might start with that.
Minification can be easily performed during a build process by a build server.
Deployment however is a huge topic in and of itself and there are various tools as well to aid you with Capistrano being seemingly the most popular one.
While you are at it, take a look at my guide on writing your first Capistrano recipe:
http://guides.beanstalkapp.com/deployments/deploy-with-capistrano.html
I hope you will find it useful!
The tasks you describe have more to do with general file processing and backup tasks rather than deployment. Any good scripting language will do for that (Ruby, Perl, etc.).
For actually automating deployment of web applications (which can also incorporate your file processing tasks), take a look at Capistrano:
https://github.com/capistrano/capistrano
QUESTIONS:
Do you have any direct or indirect experience with Web Test Manager by Sela Software Labs?
Positive or negative experience is fine. I’m just looking for some facts to base production decisions from.
How risky is it to install Sela "Web Test Manager" to our Production Server
WTM is a TFS Web Access extension. It extends website capabilities to include editing test steps and running tests.
Any other alternatives to executing tests and logging test results in TFS that we should consider?
Scenario: I have 2-3 Developers starting to run test cases as early as this week. We have 3 MTM (Microsoft Test Manager) licenses we use for testing (2 testers, one dev with VS2010 Ultimate). Purchasing another two full copies of Microsoft Test Professional for each of our VS2010 Pro/Premium (not Ultimate) devs just for periodically running test cases and doing light test case editing is not reasonable. We do not need trace listeners for general test pass runs.
Option #1: Sela Software Labs (Sela Group, Sela International) developed Web Test Manager several months ago but there is very little product reviews or customer feedback publicly available.
Sela WTM website: http://www.sela.co.il/alm/products_WTM.html
Single review posted on Microsoft partner marketing site: http://pinpoint.microsoft.com/en-us/applications/web-test-manager-wtm-is-the-only-tool-that-enables-to-test-with-tfs-2010-directly-from-the-browser-now-for-the-first-time-you-can-manage-your-tests-12884914644
Option #2: Developers track their results in individual spreadsheets which Testers then re-enter using MTM. This is not appealing at all and introduces several tedious failure risks.
Other options?
Are you talking about automated tests or manual tests? For doing automated testing, I believe you don't need additional VS licenses..
Fancy version: Set up a test machine with the Visual Studio Test Agent (and Build/Workflow agent if you need to do deployment), a TFS Test Controller, and trigger test runs from a dev machine. If you do it right, results get automatically published to TFS and attached to the correct TestCase workitems and TFS build objects. Check out Visual Studio Test and Lab Management for more info on that (definitely some extra overhead in building out the infrastructure, but it's really slick once you've made the investment). You should be able to trigger the Build-Deploy-Test workflows from any VS license that has TFS access, I think.
Less fancy, but still doable: Still install the Test Agent, but don't worry about actually wiring it up to a Test Controller. The Test Agent installation will at least give you MSTest and the ability to /publish the results of test execution up to TFS for reporting/result storage.
If you're looking to do manual test execution/reporting, I unfortunately don't have a lot of suggestions.. Most of my team's investment has been on automation, so I don't have a ton of experience working with the manual testing interfaces. :/
I'm testing WTM to use it in our company. We have common reasons mentioned by you. For these issues WTM seems to be a good and the only one option.
You are right, unluckely there are almost no reviews. So why I want to share my experience with WTM here.
Installation is easy and quick. Got no problems.
There is not much documenatation stuff on the web page. It's a pity. Hope Sela would make it better in the future (s. VisualAssist X Extension as an example for good public page). At least there is a good pdf documentation in the WTM folder after installing it.
Technically the extension is very good. But there are still some limitations (listed in documentation) and enhancements to be done (i.e. filtering, test step editing for customized TestCase TFS Work Item)
I think, for now it is a good option for testers who don't really need all MTM features.
I created an open source tool called the TFS Test Steps Editor. Originally it was developed to work around MTM's lack of ability to insert line breaks in test steps. I just released a new version that has the ability to publish test results for manual tests.
MTM is a pretty big pain to use: it's slow and buggy, and worst of all, it will sometimes lose data while attempting to publish a result. My tool saves all of your in-progress test execution to disk as you work, and you can export a .ZIP of your results for backup or to re-open on another machine and resume testing.
We are looking for a software to run our test cases automatically.
We want a software which will run on our server (or a commercial), which automatically gets the newest commit on github. Then compiles the commit of the project with CMake and run Ctest on our test cases. The results should then be visualized on a nice website.
I had a look at CDash, but as the documentation is so bad I did not even get it to get the latest commit from github.
So my questions are:
Is there a good tutorial to CDash? Except the bad wiki page.
What software is available for running tests on new commits to github, what are their advantages and drawbacks?
In answer to your second question, Jenkins is a robost and extensible continuous integration tool that can be integrated tightly with GitHub using a plug-in (or loosely using standard Git support). It also supports CMake via a plug-in. Whether it has disadvantages that will make it less useful for you depends on your organization and build process, but I've found it to be highly customizable to a wide variety of processes. I recommend taking a look at it.
There's also a third-party Ctest plugin available for Jenkins.
CDash works in pair with CTest. If you are already using CMake then it should be fairly easy to submit your testing results to CDash. I'd recommend reading the CTest documentation:
http://www.vtk.org/Wiki/CMake_Testing_With_CTest
You can either install your own CDash server or use Kitware's hosted server at my.cdash.org. You can test your server with a sample project available at:
http://www.cdash.org/cdash/resources/software.html
At home, I use CTRL+SHIFT+B or F7 or whatever key sequence initiates the build for my build tool. At work, this doesn't quite cut it.
At my first job (an internship) we used a product called Visual Build, which I have come to like very much. It's the best build tool I've ever worked with. The down side here is that it's not free.
At my latest job, I came in knowing literally nothing about Ant. Now, unfortunately, I've become deeply involved in our build processes, and cannot extricate myself. It works, yes, but after coming from Visual build, it seems like it's fighting me every step of the way. Yes, it's free, but we're not trying to be a free-software-only development company or anything.
I've never looked in to make or any other build tools, so I don't really know what else is out there.
Has anybody ever seen or had experience with Visual Build? Mostly I'm fond of a few key things:
it has a GUI
it runs arbitrary VBScript without the need of a compiled class
you can step through the build process, or start from anywhere in the middle.
Are there any free build tools that have this? Is there any way to convince people that it's worth it to move on? It's 2008. We use IDEs to develop, why not (IBEs) to build?
Edit: I'm mostly looking for an answer to my last questions; Is there a solution with a built-in GUI that I can use for free?
Not very sophisticated, but we use a set of batch files. And that works great.
We use FinalBuilder - I think it's very similar to VisualBuild, though I've not used the latter.
It does run from the command line, and you can integrate it with CC.Net if you want.
For Java projects we use Teamcity, sort of cruise control like, but you can also do a remote run, i.e. you send your changes to the server, it builds and does unit tests, if everything works ok, THEN you checkin, very nice build tool and free for up to 20 build configurations.
For our Visual Studio 2005 projects including packaging the final exes and dlls with InstallShield and putting them up on a shared server we use Final Builder, it's not free, but it is very easy to use and get started with.
We also telnet out (from FinalBuilder) to a number of other platforms (Unix/Linux/OpenVMS) and start remote builds by running makefiles there.
We do not use continous build, but there is a FinalBuilder Server which handles that and comes free with the FinalBuilder Professional license.
We are very happy with FinalBuilder, it's quite easy to get up to speed with and powerful enough to solve most problems.
CMake. Generates build file for KDevelop, Eclipse, Makefiles and Visual Studio (and XCode), and it really works. You can easily extend it with macros, although the programming capabilities are rather limited. It's easy to learn, and porting an existing application from Visual Studio to it is pretty easy. However, you are limited to C++/C and IIRC Fortran code.
KDE is also using CMake now, so it seems to scale very well (i.e. generation time for the projects/dependency checking is not too bad).
I am not sure this is exactly what you are looking for, but I LOVE CruiseControl.NET. I have it build my projects using the MSBuild task. It doesn't have a GUI exactly, but there is a web interface to view the results of your builds and a System Tray resident program which will alert you of the build status.
UppercuT. It's free.
UppercuT uses NAnt to build and it is the insanely easy to use Build Framework.
Automated Builds as easy as (1) solution name, (2) source control path, (3) company name for most projects!!!
http://code.google.com/p/uppercut/
Some good explanations here: UppercuT
Going back to the keystrokes thing for a sec, I found Hoekey which the CTO loves. I don't use it myself, but as a way of assign keystrokes to things, it's pretty good.
I know nothing of Visual Build, but from your description it sounds like it is tied to Windows and doesn't run from the command line.
If you are building Java software (I assume you are since you are using Ant), it's preferable to have a cross-platform tool. If you can run the tool from the command-line, then it is scriptable which is extremely important for automation.
Ant is also extensible and a de facto standard. Many tools that you may use (Cobertura, TestNG, etc.) provide Ant tasks so that they can easily be intergrated with your build.
I use Ant for all Java projects. Some people prefer Maven, but I'm not one of them. Ant is far from perfect (the XML syntax is a bit clunky) but it is well documented, extremely stable and pretty straightforward.
If you use a standard tool, such as Ant or Maven, you will be able to take advantage of any number of Continuous Integration products. I doubt you will find many that work with Visual Build.
Most IDEs support Ant, so they give you a GUI of sorts and your CI server will give you a web interface for doing builds.
NAnt (.NET port of Ant). Works great and is easily extensible.
For small projects I do use post-build scripts and with the support of 7z, Nsis and similar CLI tools it's doing the job perfectly for me.
TeamCity and CuriseControl works well for any projects,but here is why you would like to choose TeamCity:-
Ease of setup: During setup we found TeamCity easier to setup and use especially compared to CruiseControl. We did not need to edit XML files or massively configure individual build machines like CruiseControl.
Ease of extensibility:TeamCity stands out in its ease of extensibility too. If we find that builds are waiting in the queue too long, we can add more computers as agents. The only additional work on our end is registering the new computers with the TeamCity server and installing msbuild and subversion.
Interaction with Subversion: One can check how many and what changes were committed to subversion since the last build, who started a build etc.
I've grown very fond of scons for building C++ files. It's very straightforward and the build scripts are written in Python (which is much better than some hacked together DSL IMO).
Ant or Maven are great little build tools.
And if you want to automate the build process there are some great tools like TeamCity and Bamboo.
Personally I use Makefiles for pretty much everything because they are simple as hell. But in my work, I'm forced to use ant.
The main problem I have against ant is that XML makes it hard to read and understand, even with the correct indentation. On the other hand, the verbosity of XML can help when reading someone else's ant file, but still makes it a PITA when the file is more than a few tens of lines.
As for having a GUI to build... I've always felt that's a minus rather than a plus.
Maven is the best for me because it handles the project dependencies