I have automated testcase in Test Manager. This testcase was executed several times in different builds (It is situated in several test runs). I can see history of test execution through Test Manager UI (Test Manager -> Analyze Test Runs -> Open Test Run -> View Results for Testcase -> Result History table).
How to get same data using TFS API?
I would do it this way:
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.TestManagement.Client;
var tfsCollection = new TfsTeamProjectCollection(
new Uri(#"http://<yourTFS>:8080/tfs/<your collection>"),
new System.Net.NetworkCredential(<user who can access to TFS>,<password>));
tfsCollection.EnsureAuthenticated();
ITestManagementService testManagementService = tfsCollection.GetService<ITestManagementService>();
var testRuns = testManagementService.QueryTestRuns("SELECT * FROM TestRun WHERE TestRun.TestPlanId=<your test plan ID>");
IEnumerable<ITestCaseResult> testResultHistoryYouWant = from testRun in testRuns
from testResult in testRun.QueryResults()
where testResult.TestCaseId == <your test case ID>
select testResult;
Related
I want to setup a connection to the database so that i can test 2 methods in 2 repositories, i want to test getAllGamesasync() method which returns a list of all entities from the database, and the getGamesByNameasync() method which returns games by their names.
I am running docker to run the db and has populated rows with dummy data, i want to connect to it and run the test, So the question is What connection string to configure so code can talk to the SQL Server instance running in docker.
The methods work fine i have tested using a In-Memory DB to manually insert entities and test them against the methods using the below unit test. So the unit test for Get_GamesByName looks like this:
public async Task Get_GamesByName()
{
var options = new DbContextOptionsBuilder<GamesDbContext>()
.options.UseSqlServer(Configuration.GetConnectionString("GamesDbContext")));
using (var context = new GamesDbContext(option))
{
GamesRepository gamesRepository = new GamesRepository (context);
var result = await gamesRepository.GetGamesByNameAsync("Witcher");
Assert.Equal(2, result.count);
I have multiple Environment and a lot of test cases, but not all test cases are needed to be run in all environment. Is there a way to run only an specific test cases from a test suite based on the selected Environment.
For Example
If I select Environment1, it will run the following test cases
TC0001
TC0002
TC0003
TC0004
TC0005
If I select Environment2, it will run only the following test cases
TC0001
TC0003
TC0005
There can be different solution to achieve this since you have multiple environments i.e., pro software being used.
I would achieve the solution using Test Suite's Setup Script:
Create Test Suite level custom property. Use the same name as your environment name. For instance, DEV is the environment defined, use the same as test suite property name and provide the list of values separated by comma as value for that property, say TC1, TC2 etc.,
Similarly defined other environments and its values as well.
Copy the below script in Setup Script for the test suite and execute the script which enables or disables the test cases according to the environment and property value
Test Suite's Setup Script
/**
* This is soapui's Setup Script
* which enables / disables required
* test cases based on the user list
* for that specific environment
**/
def disableTestCase(testCaze) {
testCaze.disabled = true
}
def enableTestCase(testCaze) {
testCaze.disabled = false
}
def getEnvironmentSpecificList(def testSuite) {
def currentEnv = testSuite.project.activeEnvironment.NAME
def enableList = testSuite.getPropertyValue(currentEnv).split(',').collect { it.trim()}
log.info "List of test for enable: ${enableList}"
enableList
}
def userList = getEnvironmentSpecificList(testSuite)
testSuite.testCaseList.each { kase ->
if (userList.contains(kase.name)) {
enableTestCase(kase)
} else {
disableTestCase(kase)
}
}
Other way to achieve this is using Event feature of ReadyAPI, you may use TestRunListener.beforeRun() and filter the test case whether to execute or ignore.
EDIT:
If you are using ReadyAPI, then you can the new feature called tag the test cases. A test case can be tagged with multiple values and you can execute tests using specific tags. In this case, you may not needed to have the setup script as that is for the open source edition. Refer documentation for more details.
This solution is only specific to Pro software and Open Source edition does have this tag feature.
I am trying to build an automation framework on Rally. I am using Rally RESTful API and Pyral. The ranking method of the workspace I am working in is DnD method. I am able to copy the test set and its test cases but unable to maintain the original ranking of test cases within test set.
I tried to GET the test cases of a specific test set with order = 'DragAndDropRank' but it only gives me the same order as FormattedID of the test case. Ex:
query_criteria = 'TestSets = "%s"' % str(ts._ref)
response = self.rally.get('TestCase', fetch=True, query=query_criteria, order='DragAndDropRank')
for tc in response:
....
Let's say there are three test cases TC1, TC2, TC3 within the test set TS1. In the "Iteration Test Case Status" view of Rally, the order of test cases in TS1 is: TC3, TC1, TC2. However, the order of tc generated from code above is always TC1, TC2, TC3.
How can I get the original relative order of test cases within the test set in the "Iteration Test Case Status" view of Rally?
It is a defect that was first mentioned in this post.
Unfortunately the defect is still open.
I am working with a school project where I am going to analyse a companies defect database.
They are using Microsoft Foundation Server (TFS).
I am all new to TFS and the TFS api.
I have some problem in getting the right data from TFS using the TFS Client Object Model
.
I can retrieve all Test Plans, their respective test suites and every test case that a specific test suite uses, but the problem comes when I want to see in which test suite I have a specific test result from a test case. Since more than one the suite can use the same test cases, I cant see in which suite the result came from.
I am doing this way in getting test cases from suites:
foreach (ITestSuiteEntry testcase in suiteEntrys)
{
Console.WriteLine("\t \t Test Case: {0}", testcase.Title+", "+"Test case priority: "+ testcase.TestCase.Priority+"Test case Id: "+testcase.TestCase.Id);
//Console.Write(", Test Case: {0}", testcase.ToString());
//get each test result:
getTestResult(testcase,proj);
}
private void getTestResult(ITestSuiteEntry testcase, ITestManagementTeamProject proj)
{
var testResults = proj.TestResults.ByTestId(testcase.TestCase.Id);
foreach (ITestCaseResult result in testResults)
{
Console.WriteLine("\t \t \t"+result.DateCreated+","+result.Outcome);
}
}
So my question is, how can I see the Test Suite which was used to execute the test?
Have a look at following code snippet.
It shows you how to get Test Results for a specific Test Suite using Test Points Ids.
You can use similar approach to achieve your goal.
var tfsCollection = new TfsTeamProjectCollection(
new Uri(tfsUrl),
new System.Net.NetworkCredential(<user>, <password>));
tfsCollection.EnsureAuthenticated();
var testManagementService = tfsCollection.GetService<ITestManagementService>();
var teamProject = testManagementService.GetTeamProject(projectName);
var testPlan = teamProject.TestPlans.Find(testPlanId);
// Get all Test Cases belonging to a particular Test Suite.
// (If you are using several Test Configurations and want to take only one of them into account,
// you will have to add 'AND ConfigurationId = <your Test Configuration Id>' to the WHERE clause.)
string queryForTestPointsForSpecificTestSuite = string.Format("SELECT * FROM TestPoint WHERE SuiteId = {0}", suiteId );
var testPoints = testPlan.QueryTestPoints(queryForTestPointsForSpecificTestSuite);
// We are going to use these ids when analyzing Test Results
List<int> testPointsIds = (from testPoint in testPoints select testPoint.Id).ToList();
var testResults = teamProject.TestResults.ByTestId(testCaseId);
var testResultsForSpecificTestSuite = testResults.Where(testResult => testPointsIds.Contains(testResult.TestPointId));
This blog post will help you when creating queries: WIQL for Test
We want to dynamically trigger integration tests in different downstream builds in jenkins. We have a parametrized integration test project that takes a test name as a parameter. We dynamically determine our test names from the git repo.
We have a parent project that uses jenkins-cli to start a build of the integration project for each test found in the source code. The parent project and integration project are related via matching fingerprints.
The problem with this approach is that the aggregate test results doesn't work. I think the problem is that the "downstream" integration tests are started via jenkins-cli, so jenkins doesn't realize they are downstream.
I've looked at many jenkins plugins to try to get this working. The Join and Parameterized Trigger plugins don't help because they expect a static list of projects to build. The parameter factories available for Parameterized Trigger won't work either because there's no factory to create an arbitrary list of parameters. The Log Trigger plugin won't work.
The Groovy Postbuild Plugin looks like it should work, but I couldn't figure out how to trigger a build from it.
def job = hudson.model.Hudson.instance.getJob("job")
def params = new StringParameterValue('PARAMTEST', "somestring")
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
hudson.model.Hudson.instance.queue.schedule(job, 0, causeAction, paramsAction)
This is what finally worked for me.
NOTE: The Pipeline Plugin should render this question moot, but I haven't had a chance to update our infrastructure.
To start a downstream job without parameters:
job = manager.hudson.getItem(name)
cause = new hudson.model.Cause.UpstreamCause(manager.build)
causeAction = new hudson.model.CauseAction(cause)
manager.hudson.queue.schedule(job, 0, causeAction)
To start a downstream job with parameters, you have to add a ParametersAction. Suppose Job1 has parameters A and C which default to "B" and "D" respectively. I.e.:
A == "B"
C == "D"
Suppose Job2 has the same A and B parameters, but also takes parameter E which defaults to "F". The following post build script in Job1 will copy its A and C parameters and set parameter E to the concatenation of A's and C's values:
params = []
val = ''
manager.build.properties.actions.each {
if (it instanceof hudson.model.ParametersAction) {
it.parameters.each {
value = it.createVariableResolver(manager.build).resolve(it.name)
params += it
val += value
}
}
}
params += new hudson.model.StringParameterValue('E', val)
paramsAction = new hudson.model.ParametersAction(params)
jobName = 'Job2'
job = manager.hudson.getItem(jobName)
cause = new hudson.model.Cause.UpstreamCause(manager.build)
causeAction = new hudson.model.CauseAction(cause)
def waitingItem = manager.hudson.queue.schedule(job, 0, causeAction, paramsAction)
def childFuture = waitingItem.getFuture()
def childBuild = childFuture.get()
hudson.plugins.parameterizedtrigger.BuildInfoExporterAction.addBuildInfoExporterAction(
manager.build, childProjectName, childBuild.number, childBuild.result
)
You have to add $JENKINS_HOME/plugins/parameterized-trigger/WEB-INF/classes to the Groovy Postbuild plugin's Additional groovy classpath.
Execute this Groovy script
import hudson.model.*
import jenkins.model.*
def build = Thread.currentThread().executable
def jobPattern = "PUTHEREYOURJOBNAME"
def matchedJobs = Jenkins.instance.items.findAll { job ->
job.name =~ /$jobPattern/
}
matchedJobs.each { job ->
println "Scheduling job name is: ${job.name}"
job.scheduleBuild(1, new Cause.UpstreamCause(build), new ParametersAction([ new StringParameterValue("PROPERTY1", "PROPERTY1VALUE"),new StringParameterValue("PROPERTY2", "PROPERTY2VALUE")]))
}
If you don't need to pass in properties from one build to the other just take the ParametersAction out.
The build you scheduled will have the same "cause" as your initial build. That's a nice way to pass in the "Changes". If you don't need this just do not use new Cause.UpstreamCause(build) in the function call
Since you are already starting the downstream jobs dynamically, how about you wait until they done and copy the test result files (I would archive them on the downstream jobs and then just download the 'build' artifacts) to the parent workspace. You might need to aggregate the files manually, depending if the Test plugin can work with several test result pages. In the post build step of the parent jobs configure the appropriate test plugin.
Using the Groovy Postbuild Plugin, maybe something like this will work (haven't tried it)
def job = hudson.getItem(jobname)
hudson.queue.schedule(job)
I am actually surprised that if you fingerprint both jobs (e.g. with the BUILD_TAG variable of the parent job) the aggregated results are not picked up. In my understanding Jenkins simply looks at md5sums to relate jobs (Aggregate downstream test results and triggering via the cli should not affect aggregating results. Somehow, there is something additional going on to maintain the upstream/downstream relation that I am not aware of...
This worked for me using "Execute system groovy
script"
import hudson.model.*
def currentBuild = Thread.currentThread().executable
def job = hudson.model.Hudson.instance.getJob("jobname")
def params = new StringParameterValue('paramname', "somestring")
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
hudson.model.Hudson.instance.queue.schedule(job, 0, causeAction, paramsAction)