Does anybody know of an integration between Rally ALM and robotframework?
I'm looking for something that would log test results in robotframework back to Rally test cases.
With the pyral rally module for Python, seems like it could be fairly straightforward.
As far as I can tell there is nothing out there to do this-- but its pretty easy to do, only needing about 50 lines of python code for a simple integration that logs robot framework to Rally test case results.
In my case, I have it log results for any test who's name starts with a Rally test case id: (e.g. "TCXXXX My Test Name").
The trick is to user the RobotFramework listener API (See: Elapsed time and result of a test in variables) and pyral, the Rally python API. Key for my needs was to define an "end_test" listener:
def end_test(self, name, attrs):
match = re.search('^(TC\d+)\s*(.*)', name)
tcId = match.group(1)
testName = match.group(2)
if tcId:
tcr = self.__logTestCaseResultToRally(tcId, testName, attrs)
self.__cleanTestCaseState()
In robotframework, I include this listener file, which also has some additional methods to add attachments and other information like notes to a test result (these can be directly called as libraries in your robotframework file):
def addAttachment(self, attachment):
if os.path.isfile(attachment) and os.access(attachment, os.R_OK):
self.attachments.append(attachment)
This method simply saves the attachment path in the listener object so that when end_test() is called, it has access to the file names to attach to the rally test case. __cleanTestCaseState() zeros these out so they are cleared before the next test cast starts.
Actually, I've never used Rally!
But in my opinion, With robot framework, I like using Testlink for testcase management system & jenkin for CI control system :)
You can search in the internet for installation.
Hope it useful :)
Related
I am writing UI automation script using karate DSL. In this at certain point I need to get value from network call in chrome. I want to interact with one of the webservice call in chrome devtools network tab and get the json response of that webservice.
I need this because I have to extract the value from that particular call and pass it on to the next step in my automation script.
I have seen the question related to sessionStorage(Is there a way of getting a sessionStorage using Karate DSL?) but I wonder how to do the same for network call using script command or any other way?
The first thing I would recommend is don't forget that Karate is an API testing tool at its core. Maybe all you need to do is manually make that call and get the response. You should be able to scrape the HTML and get the host and parameters needed.
That said - there's a new feature (only for Chrome) which is documented here: https://github.com/intuit/karate/tree/develop/karate-core#intercepting-http-requests - and is available in 0.9.6.RC2
It may not directly solve for what you want, but in a Karate mock, you should be able to set a value for use later e.g. by using a Java singleton or writing to a temp-file.
If there is something oddly more specific you need, please contribute code to Karate. Finally, there is an experimental way in which you can actually make raw requests to the Chrome DevTools session: https://github.com/intuit/karate/tree/develop/examples/ui-test#devtools-protocol-tips - it is for advanced users, but maybe you are one :)
I have some test scenarios and cases written in Specflow/Selenium in Visual Studio, usin MsTest. I just want to associate them to Microsoft Test Manager, so a test case written there is associated to an automated test.
Is that possible? How?
More Data: test were created by using Scenario Outline with some lines of examples.
You can associate testcases to a workitem in TFS/MTM, but we found it to cumbersome to do: It is a manual action in MTM that references the TestMethod by name. But because the TestMethod is generated by specflow by combining the title of the Scenario Outline and the first column of your Examples table, it is difficult to maintain:
Whenever a Scenario Outline title is changed, or the term in the first column of the examples table is changed, you have to re-couple the TestMethods to the workitems
When you add new Examples or Scenarios to your feature, you have to remember to link them to the workitem, one-by-one
To find the correct TestMethod in the dll is nearly undoable when you approach the thousandish scenarios.
What we did was using the WorkItem attribute in the Feature to connect (parts of) the feature to a workitem like #Workitem:42 . This is a little unnoticed feature in SpecFlow:
MsTest: Support for MSTest's [Owner] and [WorkItem] attributes with tags like #owner:foo #workitem:123 (Issue 162, Pull 161)
and it creates a WorkItemAttribute attached to the method that is connected to that tagged Scenario (Outline) or Feature.
Then, we imported all testcases into MTM with the Test Case Management tool and ran a custom made tool (making use of the TeamFoundation namespace and the TestManagement and WorkItemTracking Client) that connected each imported testcase to the correct workitem.
Whenever a test did run we could see the results in MTM, but also from the perspective of the connected workitem.
I've started to use Galen framework to test the layout of my website pages and I also have my other test, written in Selenium, integrated into Jenkins.
I'm using Java+JUnit+Maven and I would like to know if anyone has managed to integrate the Galen reporting into Jenkins and how.
Because for the moment I am using something like:
assertThat(layoutReport.errors(), is(0));
which tells me if there were errors in the tests but not where.
Thanks!
P.S. If someone with reputation could make the tag galen-framework so that we can group these type of questions, it would be great :D
In your case you could use Galen for generating HTML reports as it normally does when you run tests with it. Though you will have to manage the creation of GalenTestInfo objects.
Here is how the HTML report generation works. Imagine you have somewhere obtainAllTests method defined which returns a list of all executed tests.
List<GalenTestInfo> tests = obtainAllTests();
new HtmlReportBuilder().build(tests, "target/galen-html-reports");
Somewhere in your code you could create a GalenTestInfo and add it to some collection:
GalenTestInfo testInfo = GalenTestInfo.fromString("Here goes the name of your test");
Once you have done the layout checking and obtained LayoutReport object you could add this report to the report of the test. Here is how you can do it:
LayoutReport layoutReport = Galen.checkLayout(driver,
specPath, includedTags, null,
new Properties(), null);
testInfo.getReport().layout(layoutReport, "A title for your layout check");
You can find more insights in this project https://github.com/galenframework/galen-sample-java-tests. It has a basic setup for Galen tests in Java + TestNG + Maven. The reports in it are collected in a singleton GalenReportsContainer. There is also a reporter implemented in GalenReportingListener which takes all those tests from GalenReportsContainer and generates HTML reports.
You can see a full example for Java (TestNG and JUnit) and JavaScript here:
https://github.com/hypery2k/galen_samples.
I use the HTML Plugin together with Jenkins, see example here
I'm a QA who decided to use SpecFlow for my test automation after some consideration. I think it's brilliant, but missing one feature which I did use often with other test runners such as NUnit - something similar to the TestCaseSource property from NUnit to specify a potentially dynamic set of data for tests to be ran against at run time.
I would often have different data in each environment the test should run in, so cannot specify hardcoded values for test parameters. A trivial example is for checking that each type of user account is able to login, the user account credentials can be retrieved using a DB query to populate each test case dynamically in NUnit:
public List<User> GetTestData()
{
List<User> testData = new List<User>();
testData = MyDatabase.GetAllUsersInfo().ToList();
return testData;
}
[Test, TestCaseSource("GetTestData")]
public void CallLoginService(User user)
{
var response = LoginController.TryLogin(User.UserName, User.Password);
if (response.Error != null)
{
Assert.Fail("Failed to Login: {0}", response.Error);
}
Assert.AreEqual("Logged in ok", response.Message, "Login message not as expected");
}
Obviously this is a simple example of that feature, but I think it describes it well enough. I know we have the ability in SpecFlow to use a Scenario Outline and table of test run input data, but that is still static, so doesn't fit the bill.
I've been looking for a while and have not found anything in SpecFlow like this yet, does anybody know of anything similar to the above which can be used (or planned if anyone who works on the project reads this)?
Thanks :)
I have no idea if anything like this is planned but for now the problem is that there is a background code generation step when you edit your feature file via Visual Studio.
When it is saved in Visual Studio it is parsed and converted into the feature.cs file and that is the one that is compiled and used for testing.
So your process would become
edit your data source
export to feature file
get specflow's VS plugin to convert to feature.cs
run msbuild
run tests via Nunit or similar
I wouldn't do this. Instead I'd focus on getting my tests to be better examples. It sounds like you are to trying to exhaustively cover every possibility. Don't come up with examples to cover every possible case, but instead cover as much logic as possible with fewer tests.
HI
I'm testing a web application using watin. I need to pass multiple inputs to the application for doing this each time i need to change inputs in the code. So is it possible in watin to accept inputs from excel file.
This tool might help you: WAX
Wax allows users to create automated WatiN tests using Microsoft Excel
WatiN offers no (native) way for this. You should solve this with the test runner you are using:
NUnit, MBUnit, XUnit all offer some kind of RowTest functionality which allows you to read data from excel and use that as input for your WatiN tests.
HTH,
Jeroen
Lead dev WatiN
The approach that we use is to use SQL Express for our test data, we then generate a DAL layer using subsonic, and pass a string around as a key, and then get the data from the database as a "test data object".
The one line of code that we write in the DAL in a partial class is similar to
public partial class Project
{
public static Project GetProjectDetails(string ProjectName)
{
return new Select().From<Projects>().Where("ProjectName").IsEqualTo(ProjectName).ExecuteSingle<Project>();
}
}
Our test code then looks something like this:
TestData.Project project = TestData.Project.GetProjectDetails(projectName);
domContainer.TextField( ... project name field ...).Value = Project.ProjectName;
domContainer.TextField( ... project type field ...).Value = Project.ProjectType;
The other way is the nUnit feature Jeroen is refering to the nUnit row test extension.
If you were really motivated you could write a WatiN adapter for FitNesse or RobotFramework.
WatiN tests are written in .Net. .Net provides functionality to access Excel spreadsheets as needed. Just write them into the test code.