In my Jenkinsfile i'm counting all test results as
AbstractTestResultAction testResultAction = currentBuild.rawBuild.getAction(AbstractTestResultAction.class)
if (testResultAction != null) {
def total = testResultAction.totalCount
def failed = testResultAction.failCount
def skipped = testResultAction.skipCount
def passed = total - failed - skipped
But I also want to display all names of failed tests for slack message.
So far, i've tried to generate it with def failedTests = testResultAction.getResult().getFailedTests() but it returns not specific name like hudson.tasks.junit.CaseResult#37e0fb97.
Is there anyway to display full name of test? I am using Selenium + TestNG.
You can get description using .getTitle() method:
def failedTests = testResultAction.getResult().getFailedTests().collect { it.getTitle() }
Related
I was tryiny to solve dynamic test data problem using dynamic scenario outline as mentioned in the documentation https://github.com/karatelabs/karate#dynamic-scenario-outline
It worked perfectly fine when I passed something like this in Example section
Examples:
|[{'entity':country},{'entity':state},{'entity':district},{'entity':corporation}]]
But I tried to generate this json object programatically , I am getting aa strange error
WARN com.intuit.karate - ignoring dynamic expression, did not evaluate to list: users - [type: MAP, value: com.intuit.karate.ScriptObjectMap#2b8bb184]
Code to generate json object
* def user =
"""
function(response){
entity_type_ids =[]
var entityTypes = response.entityTypes
for(var i =0;i<entityTypes.length;i++ ){
object = {}
object['entity'] = entityTypes[i].id
entity_type_ids.push(object)
}
return JSON.stringify(entity_type_ids)
}
"""
I am trying to extract the content of the [Documentation] section as a string for comparision with other part in a Python script.
I was told to use Robot framework API https://robot-framework.readthedocs.io/en/stable/
to extract but I have no idea how.
However, I am required to work with version 3.1.2
Example:
*** Test Cases ***
ATC Verify that Sensor Battery can enable and disable manufacturing mode
[Documentation] E1: This is the description of the test 1
... E2: This is the description of the test 2
[Tags] E1 TRACE{Trace_of_E1}
... E2 TRACE{Trace_of_E2}
Extract the string as
E1: This is the description of the test 1
E2: This is the description of the test 2
Have a look at these examples. I did something similar to generate testplans descritio. I tried to adapt my code to your requirements and this could maybe work for you.
import os
import re
from robot.api.parsing import (
get_model, get_tokens, Documentation, EmptyLine, KeywordCall,
ModelVisitor, Token
)
class RobotParser(ModelVisitor):
def __init__(self):
# Create object with remarkup_text to store formated documentation
self.text = ''
def get_text(self):
return self.text
def visit_TestCase(self, node):
# The matched `TestCase` node is a block with `header` and
# `body` attributes. `header` is a statement with familiar
# `get_token` and `get_value` methods for getting certain
# tokens or their value.
for keyword in node.body:
# skip empty lines
if keyword.get_value(Token.DOCUMENTATION) == None:
continue
self.text += keyword.get_value(Token.ARGUMENT)
def visit_Documentation(self,node):
# The matched "Documentation" node with value
self.remarkup_text += node.value + self.new_line
def visit_File(self, node):
# Call `generic_visit` to visit also child nodes.
return self.generic_visit(node)
if __name__ == "__main__":
path = "../tests"
for filename in os.listdir(path):
if re.match(".*\.robot", filename):
model = get_model(os.path.join(path, filename))
robot_parser = RobotParser()
robot_parser.visit(model)
text=robot_parser._text()
The code marked as best answer didn't quite work for me and has a lot of redundancy but it inspired me enough to get into the parsing and write it in a much readable and efficient way that actually works as is. You just have to have your own way of generating & iterating through filesystem where you call the get_robot_metadata(filepath) function.
from robot.api.parsing import (get_model, ModelVisitor, Token)
class RobotParser(ModelVisitor):
def __init__(self):
self.testcases = {}
def visit_TestCase(self, node):
testcasename = (node.header.name)
self.testcases[testcasename] = {}
for section in node.body:
if section.get_value(Token.DOCUMENTATION) != None:
documentation = section.value
self.testcases[testcasename]['Documentation'] = documentation
elif section.get_value(Token.TAGS) != None:
tags = section.values
self.testcases[testcasename]['Tags'] = tags
def get_testcases(self):
return self.testcases
def get_robot_metadata(filepath):
if filepath.endswith('.robot'):
robot_parser = RobotParser()
model = get_model(filepath)
robot_parser.visit(model)
metadata = robot_parser.get_testcases()
return metadata
This function will be able to extract the [Documentation] section from the testcase:
def documentation_extractor(testcase):
documentation = []
for setting in testcase.settings:
if len(setting) > 2 and setting[1].lower() == "[documentation]":
for doc in setting[2:]:
if doc.startswith("#"):
# the start of a comment, so skip rest of the line
break
documentation.append(doc)
break
return "\n".join(documentation)
This question already has an answer here:
KARATE integration with SauceLabs
(1 answer)
Closed 1 year ago.
I am reading the json file which contains the array of configurations of browserstack browsers or devices on which I want run my UI automation tests and defining it in karate-config.js as a global variable, for ex: config.envrironments. That variable I am calling it in Examples Table as envrironments and using it in dynamic scenario outline to initialize driver for different browser/device sessions as envrironments[__num]. The execution is working fine as expected.
After execution, I want to call the browserstack api which updates the test scenario status as passed or failed with a reason in their dashboard or in the browserstack report integrated in jenkins for clear understanding. The api requires driver sessionId as api path param, status(passed or failed) and reason(if failed) as body.
I am able to get the status and reason from karate.info.errorMessage after configuring afterScenario for the feature. But my problem is to get the browserstack sessionId.
To get the driver.sessionId in afterScenario, i'm getting error as "driver" is not defined. I guess driver object is getting killed in between each execution of scenario outline and the afterScenario.
Is there any way to keep the driver alive until the afterScenario is completed?
or
any other alternatives like implementing ExecutionHook class or any other class through which we can get the driver details?
Please help. Thanks in advance.
UPDATE:
I'm using Java 8 and karate version 0.9.6
Below is my code:
Test Feature -
Feature: Test Feature
Background:
* def scenarioStatus =
"""
{
"status": "passed",
"reason": ""
}
"""
* configure afterScenario =
"""
function(){
if(karate.info.errorMessage){
scenarioStatus.status = 'failed';
scenarioStatus.reason = karate.info.errorMessage;
}
karate.log('session id : ' + karate.get('sessionId'));
// karate.call('browserstack.feature', scenarioStatus);
}
"""
#dummyMobileBrowser
Scenario Outline: Dummy Scenario
* def browserTestName = karate.info.scenarioName + ' - '
* call read('driver.feature#initializeDriver')
* def sessionId = driver.sessionId
* call read('loginPage.feature#login')
# I'm defining array of device or browser configs in karate-config.js
Examples:
| deviceConfigs |
driver.feature
Feature: Driver Related Feature
Background:
* def jsUtils = read('classpath:jsUtils.js')
* def getDriverConfig =
"""
function(){
if(browserstack == "yes"){
var configResult = karate.call('driver.feature#createBrowserStackConfig');
var browserstackConfig = configResult.browserstackConfig;
return browserstackConfig;
}else{
return deviceConfigs[__num];
}
}
"""
#initializeDriver
Scenario: Initialize Driver
* def driverConfig = getDriverConfig()
* configure driver = driverConfig
* print 'Driver Config: ', driverConfig
* driver envHost
# also tried to def here
# * def sessionId = driver.sessionId
# * eval scenarioStatus.browserstackSessionId = sessionId
# * karate.write(sessionId, 'classpath:browserstackSessionId.txt')
* driver.fullscreen()
#takeScreenshot
Scenario: Take screenshot
* driver.screenshot()
#createBrowserStackConfig
Scenario: Create Browserstack Config
* def deviceCapabilities = deviceConfigs[__num]
* def driverUrl = 'https://' + browserstackUsername + ':' + browserstackKey + '#' + browserstackUrl + '/wd/hub'
* eval commonCapabilities.build = (karate.match(typeof browserstackBuildName, 'undefined').pass) ? commonCapabilities.build + currentEpochTime : browserstackBuildName
* def desiredCapabilities = karate.merge(deviceCapabilities, commonCapabilities)
* def driverType = (karate.match(desiredCapabilities.browserName, "#notnull").pass) ? desiredCapabilities.browserName : desiredCapabilities.browser + 'driver'
* eval driverType = (karate.match(driverType, 'firefoxdriver').pass) ? 'geckodriver' : driverType
* eval desiredCapabilities.name = (karate.match(desiredCapabilities.browserName, "#notnull").pass) ? desiredCapabilities.browserName : desiredCapabilities.browser
* eval desiredCapabilities.name = browserTestName + desiredCapabilities.name
* def capabilities = karate.merge(deviceCapabilities, commonCapabilities)
* eval capabilities.name = (karate.match(capabilities.browserName, "#notnull").pass) ? capabilities.browserName : capabilities.browser
* eval capabilities.name = browserTestName + capabilities.name
* def browserSession = { desiredCapabilities: '#(desiredCapabilities)', capabilities: '#(capabilities)' }
* def browserstackConfig = { type: '#(driverType)', webDriverSession: '#(browserSession)', start: false, webDriverUrl: '#(driverUrl)' }
Even if the driver is not callable - you should be able to access variables, right ?
So if you did this within the normal test-flow:
* def sessionId = driver.sessionId
Then the sessionId variable should be directly accessible in the after-hooks. Do karate.get('sessionId') if needed, try it and please comment below if it works.
My apologies it seems repetitive question but it is really troubling me.
I am trying to call one feature file from another feature file along with variable values. and it is not working at all.
Below is the structure I am using.
my request json having variable name. Filename:InputRequest.json
{
"transaction" : "123",
"transactionDateTime" : "#(sTransDateTime)"
}
my featurefile1 : ABC.Feature
Background:
* def envValue = env
* def config = { username: '#(dbUserName)', password: '#(dbPassword)', url: '#(dbJDBCUrl)', driverClassName: "oracle.jdbc.driver.OracleDriver"};
* def dbUtils = Java.type('Common.DbUtils')
* def request1= read(karate.properties['user.dir'] + 'InputRequest.json')
* def endpoint= '/v1/ABC'
* def appDb = new dbUtils(config);
Scenario: ABC call
* configure cookies = null
Given url endpoint
And request request1
When method Post
Then status 200
Feature file from which I am calling ABC.Feature
#tag1
**my featurefile1: XYZ.Feature**
`Background`:
* def envValue = env
Scenario: XYZ call
* def sTransDateTime = function() { var SimpleDateFormat = Java.type('java.text.SimpleDateFormat'); var sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'+00:00'"); return sdf.format(new java.util.Date()); }
* def result = call read(karate.properties['user.dir'] + 'ABC.feature') { sTransDateTime: sTransDateTime }
Problem is,
While executing it, runnerTest has tag1 configured to execute.
Currently, it is ignoring entire ABC.feature to execute and also not generating cucumber report.
If I mention the same tag for ABC.feature (Which is not expected for me as this is just reusable component for me ) then it is being executed but sTransDateTime value is not being passed from XYZ.feature to ABC.feature. Eventually, InputRequest.json should have that value while communicating with the server as a part of the request.
I am using 0.9.4 Karate version. Any help please.
Change to this:
{ sTransDateTime: '#(sTransDateTime)' }
And read this explanation: https://github.com/intuit/karate#call-vs-read
I'm sorry the other part doesn't make sense and shouldn't happen, please follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Run the first.feature file successfully，however， call it from the second.feature failed without any clue to analysis. Do you have any idea help me find the root cause?
The source of my first.feature:
Feature: 采样管理-样本登记
Background: 读取随机生成的条形码、手机号、采样类型等作为入参
* url baseURL
* def randomData = Java.type('utils.RandomData')
* def barcode = randomData.getRandom(11)
* def randomPhone = randomData.getTelephone()
* def sampletype = randomData.getNum(0,1)
Scenario: 输入合法参数进行正常样本登记，确认能够登记成功
Given path 'iEhr/PersonSample'
# * header Content-type = 'application/x-www-form-urlencoded; charset=UTF-8'
* cookies { JSESSIONID: '#(jsessionID)',SESSION: '#(sessionID)', ACMETMP: '#(acmetmpID)'}
* def autoMotherName = "autoMname"+ barcode
# * def confData = {mothername: "#(autoMotherName)", barcode: "#(barcode)", mobile: '#(randomPhone)', sampletype:"#(sampletype)" }
# 设置sampletype为1，已被采样
* def confData = {mothername: "#(autoMotherName)", barcode: "#(barcode)", mobile: '#(randomPhone)', sampletype:"1" }
# 打印入参变量输出
* print confData
# 用例与数据分离
* def paramObj = read('classpath:mainFlow/sampleSaveReqTest.json')
* print paramObj
* form field param = paramObj
When method post
Then status 200
* json result = response[0].result
* def personId = result[0].personid
* def sampleid = result[0].sampleid
* print personId
* print sampleid
The source of my second.feature:
Feature: 提交递送样本
Background:
* def sampleResult = call read('classpath:mainFlow/first.feature')
* print sampleResult
I run the first.feature singly， it works. However, karate reports the error below after running the second.feature. Any idea how can I debug to find the root cause? I have no idea what's wrong with the second read. Many thanks!
* def sampleResult = call read('classpath:mainFlow/first.feature')
-unknown-:14 - javascript evaluation failed: read('classpath:mainFlow/first.feature'), null
Look for some issue with karate-config.js. As Babu said in the comments, it is very hard to make out what the problem is, I suggest you follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Also try if the latest preview version 0.9.3.RC2 is better at showing what the error is.
If you can replicate the problem as a small example, it will help us - because we really need to do better at showing more useful error logs, instead of just null.