Apologies, if this has already been asked, but please bare with me, i am new to restSharp and API testing. I have ready many posts,articles and responses. But due to my limitation, I was wondering if anyone can just guide me through, on how to achieve it.
What i would like to achieve, is to run the same test scenario in different test environments(dev, uat, pro etc..) for the same Gherkin scenarios, if possible by making use of the tags e.g #dev #uat etc..
Currently, we have 3 runsetting files for each of these environments, (dev.runsetting, uat.runsetting etc..in each we pass different parameters), to change to different environment, i do this through C# and select the necessary test setting files
If I understood you correctly, you want to control via tags which scenario is executed against which environment.
The SpecFlow+ Runner has a feature called Targets that could help you. With this, you can define multiple targets and their configuration, for your scenarios.
You can find an example for this feature here: https://github.com/SpecFlowOSS/SpecFlow.Plus.Examples/tree/master/SeleniumWebTest
In this example, we are using the targets to test against different browsers.
In your case, I would change the browser to your environments.
In your CI/CD pipeline, you could then use the filtering in the SpecFlow+ Runner to filter out for your different environments. In Azure DevOps the filter criteria is Name~\(target: <TARGETNAME>\).
Documentation for it is here: https://docs.specflow.org/projects/specflow-runner/en/latest/Profile/Filter.html
Full disclosure: I am one of the maintainers and developers on SpecFlow and SpecFlow+
Related
Azure Pipelines has a way of displaying test results after builds, described here. This seems to be very limited, however, and doesn't display much of the stuff we want.
We're using NUnit 3 with a .NET Core project, and NUnit outputs a bunch of stuff for test results. We specifically want to display FQN of the test at a minimum, but being able to easily view tests by category would suffice as well. This doesn't seem to be possible with the default Tests tab.
My question: Does DevOps support doing something like this or do I need to make my own site/API for viewing test results? I'm open to pretty much anything, we just want to avoid anything that requires manual offline processing.
As per the Microsoft comment to the question, this is not currently possible. I added a suggestion on their tracker here.
We are three testers and going to prepare automation project with selenium and java code so what are the steps for environment setup , scripts integration and running the testcases and getting the results for the whole project suits
So there are a few things we have to use in order to allow multiple engineers to work on the same framework.
Step 1) Creating the framework, assuming you know how to do this already, you have working tests you can skip this stage, however if not please follow the tutorial i link below.
http://toolsqa.com/selenium-webdriver/
Step 2) Creating a REPO, my preference is GitHub, you can use any git repo however i will post the guide to set one up with GitHub, its a similar process for all. This will allow you to merge code properly without causing conflicts.
https://help.github.com/articles/create-a-repo/
Step 3) Source Control program - to push, pull and fetch from your GitHub Repo, you can do this from Command Prompt however i find cloning the repo into a program like 'SourceTree' is really easy, so i've posted that below.
https://confluence.atlassian.com/get-started-with-sourcetree
If you follow these 3 guides, you will be able to have your automation test scripts on GitHub by the end of the day.
If you have any more questions please do not hesitate to ask.
All the best, Jack
The easiest and most logical way to do this would be to create one branch in your CVS (git or SVN, etc) and have each person setup the dev environment in the same way. Work exactly like developers and pull code before you check-in/commit (this will ensure that one small error does not break your framework) and swear to resolve conflicts during merge (to ensure you don't step on each others' toes).
Also, before you kick off, agree on a standard of coding (including package naming, design pattern usage, filename and methodname usage) and if this is in sync with the dev coding standards in your company, even better.
There will be a few hiccups along the way, but experience is the best way to create a process for your development and check-in practices.
Good luck with your new project and happy coding!
You have asked two questions, in my opinion the answer of your questions is.
how multiple automation testers work in same selenium project - You can use any version control system, Git Hub is the best option which gives you a lot of facilities. You all three can work on same project at same time or you can go for any centralized version control system like tortoise svn which is not much likely used now a days. I will suggest Git Hub for that.
what are the steps for environment setup , scripts integration and running the test cases and getting the results for the whole project suits - It depends on various factors like application and the kind of framework you want to use, there are many frameworks which are widely used for automation testing like Modular Framework, Data Driven, Keyword Driven, BDD, Cucumber, TestNg etc or if you have bandwidth and time you can design your custom framework as per the needs.
I hope I put some glimpse on your queries.
Thanks
I am working at a company that uses Redmine as the issue tracking tool and for user stories.
I need to design the QA for them. However I didn't find a section where someone can tie a user story or an issue with a test case. I've used to have this functionality on Rally.
Is there a free tool that integrates with Redmine or something like a plugin?
I want the following (or the more the better!)
Description
test steps (input-output)
Status (pass-fail)
Suite name (the suite the test is part of)
Attached issues
Version of SW under test
Thanks in advance
There is a "test case management" section in the Redmine Third Party tools documentation.
thanks for the support.A former colleague brought this to my attention:
https://bitbucket.org/bugzinga/redcase/wiki/Home
One thing I often see is people trying to adapt tools to do things that they were not originally designed for. Redmine is a good tool for issue tracking and is designed mainly to be a project management tool, although it is possible to adapt redmine, I would suggest in this case looking for a dedicated test management tool that has been designed to do that specific job.
I would suggest looking at TestLodge test management tool which I have worked on and is a designed to help you manage and execute your tests but at the same time integrates with Redmine and does things like automatically create tickets whenever a test fails.
By doing this, you are going to benefit from a range of things such as an interface designed for testing that will make you and your team a lot more productive along with a series of reports that will allow you to spot trends within your testing.
I'm looking for an junit report visualization tool. I have a set of xml reports generated by a custom test-suite tool and I want to be able to visualize their history. A plugin for jenkins would be ideal, but a standalone application is also ok.
The one thing I have found so far that seems to fit the spec:
http://junitth.sourceforge.net/
Though I am a bit wary of using this as development seems to have stopped a year or two ago.
Any suggestions welcome.. thanks
There's a new tool called Allure. This is not exactly what you described because it uses its own XML files which can be automatically generated during test run via provided JUnit RunListener. However it supports a lot of test-related features like custom test description, grouping by features and stories, adding attachments, etc. You could try to adapt your custom XML to their schema and then generate the visualization with standalone tool or plugin.
Does anyone know of any good internationalization testing tools or plugins that can be automated and triggered by jenkins?
Thanks in advance.
Lingoport's Globalyzer ( http://www.lingoport.com/globalyzer ) is a tool which does static code analysis to identify internalization issues and help mitigate them. It run the analysis in two modes, interactive with a UI, or from the command line.
The command line can be integrated in a continuous build environment. There is configuration work to adapt how the static analysis is performed on specific projects by languages (Java, JavaScript, etc.) and tune what should be identified and what should be filtered, as it is different for each project.
See http://www.globalyzer.com/gzserver/help/commandline.htm and related help files may give you a more in-depth answer to your question.