Failed to integrate Bugzilla Testopia with Jenkins with the aid of Testopia Plugin - cannot execute any iterative build step - testing

I’m trying to integrate Bugzilla Testopia with Jenkins with the aid of the Testopia Plugin for Jenkins.
The general configuration is probably fine as the connection between Testopia and Jenkins is well established (Jenkins log says: ‘Connecting to Testopia to retrieve automated test cases’ and no error occurs then). However I’m unable to retrieve any information concerning Test Runs/Test Cases etc. from Testopia.
Moreover I cannot perform any of ‘Iterative Test Build Steps’. If I want to ‘Execute Shell’ in ‘Iterative Test Build Steps’ with Testopia Plugin no operation is carried out (even if I try: echo 12345 etc.). If I use ‘Single Test Build Steps‘ then shell command is executed.
My goal is to retrieve test class name from Testopia (it is stored in Testopia Test Case’s field - Automation/Scripts ) and then run maven build from Jenkins with this class name set as the parameter. Afterwards depending on the Jenkins build success or failure status I’d like to update the Test Case Status in Testopia.
How to fetch any information from Testopia into Jenkins?
Why any of ‘Iterative Test Build Steps’ is not executed?
Any clues? - Testopia Plugin site example wasn't too helpful for me.
Both Bugzilla and Jenkins are hosted on the same Ubuntu 14.04. I've got the latest stable versions of Jenkins, Bugzilla and Testopia.
Thanks in advance,
M.
EDIT:
Well this debugging does not work for me. I added a new log recorder with the 'ALL' level chosen and I cannot see any additional log neither in job's console output nor in the newly created logger output.
Maybe sth is wrong with my Testopia installation? Some more details concerning my configuration:
I've got Testopia installed on the same machine (as Jenkins) and usually I access it through: http://'ip_address'/bugzilla
- in Testopia plugin configuration my URL to Testopia installation is: http://'ip_address'/bugzilla/xmlrpc.cgi
- I've got only one bugzilla account - these admin's credentials I use in Jenkins
- sometimes in Jenkins I can see a warning concerning improper reverse proxy configuraiton - maybe it has sth to do with the problem
After job execution all Testopia's fields are 0 - Run Id, Build Id etc. - which obviously indicates that no information was successfully retrieved from Testopia.
Any ideas how to check why I cannot retrieve any information from Testopia?
EDIT 2:
In the meantime I think I've found a clue in the jenkins.log file in jenkins installation directory:
Exceptions like these occur:
INFO: TESTOPIA_TEST_SUITE_3 #13 main build action completed: SUCCESS
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
...
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get_test_cases' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
Shall I insert the full stacktrace?

It looks like that plugin logs a fair amount of information, though not all of it to the build console output itself.
To debug further, you could try adding a new log recorder for the logger jenkins.plugins.testopia (with log level "all"), run a build, then refresh the web page for the newly-created log recorder to see the output.
You should at least see "Filtering for automated test cases" after connection, information about each test case found, and then log output for each iterative build step as it's run on each test case.

I got the similiar problem when I use python xmlrpc to commnicate with my bugzilla-testopia server.
I chekc the error code and find XMLRPC.pm under my bugzilla install location : "./WebService/Server/XMLRPC.pm".
I know the error was thrown by this sub function, which checks login status and forward to modules where the moduel.function must be in PUBLIC_METHODS:
sub handle_login {
...
if (none { $_ eq $method } $class->PUBLIC_METHODS) {
ThrowCodeError('unknown_method', { method => $full_method });
}
...
}
I don't know why but TestCase.get could not be found in PUBLIC_METHODS, so I just marked them as comment and then it works. You can do this quick way to make sure your client setting is correct. Then you should solve the "PUBLIC_METHODS" problem next.

Related

Getting 'Could not find test files' Error when attempting to run TestCafe tests

I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!

IntelliJ, Cucumber, Java and Selenium - no tests are starting

First, important note: This is a project in which the Selenium/Cucumber test suite is working. It's working locally for the developers, and it's working in the Tekton environment.
Second: I have an identical IntelliJ environment as the developers. And other Selenium/Cucumber projects are running fine on my local machine.
When a try to run a fresh copy of the project in question locally, none of the tests start. And, there is no error message.
Normally, this will be cause by missing Glue parameters in the Run/Debug configuration, since IntelliJ for some reason often is unable to add these automatically. This is not the case here.
The test runner when I try to start a specific scenario:
Sorry for the language. The tests are all in Norwegian. But I don't think that matters for seeing the problem.
Normally, when there is an actual error regarding the test step - in the case "Gitt jeg er en vanlig søker" - there would be some info to the right of the steps overview. This is now blank, as can be seen. So, the test stes aren't even started.
The run/debug configuration:
e2e.cucumber.felles is the package where the Java file with the first step definition is located. So, that's not the reason.
Any ideas?

Global variable in Jenkins Repository URL

I am trying to use a global Jenkins variable in the Repository URL field:
Repository URL: ${BUILD-PEND-SRC}
BUILD-PEND-SRC is defined in Configure System and a value of a proper URL is set. If I do a shell execution job with echo ${BUILD-PEND-SRC} it does display the correct value.
However, when I run the job, I get
ERROR: Failed to check out ${BUILD-PEND-SRC}
org.tmatesoft.svn.core.SVNException: svn: E125002: Malformed URL '${BUILD-PEND-SRC}'
Which tells me that Jenkins did not resolve ${BUILD-PEND-SRC}.
I am summarizing the SO answer that solved it for git-based Jenkins pipeline jobs but also applies to svn-based jobs: https://stackoverflow.com/a/57065165/1994888 (credits go to #rupesh).
Summary
Edit your job config
go to the Pipeline section
go to the definition Pipeline script from SCM
uncheck Lightweight checkout
The issue seems to be with the scm-api-plugin (see the bug report in the Jenkins issue tracker), hence, it is not specific to a version control system.

Is it possible to somehow automatically create jobs in TestLink from Jenkins?

I'm doing a bit of reasearch on Jenkins and TestLink and I was wondering - is it possible to automatically generate new project/job in TestLink or does it always have to be done manually? The issue is that I would like to run tests on Jenkins and then all the test that failed should create a new job in TestLink.
Is it possible and how to do it?
With default TestLink Jenkins plugin you can't create new projects automatically on TestLink. You will end up in below type of error
FATAL: Test Project (name:TestSample1234) does not exist.
br.eti.kinoshita.testlinkjavaapi.util.TestLinkAPIException: Test Project (name:TestSample1234) does not exist.
at br.eti.kinoshita.testlinkjavaapi.BaseService.checkResponseError(BaseService.java:113)
at br.eti.kinoshita.testlinkjavaapi.BaseService.executeXmlRpcCall(BaseService.java:91)
at br.eti.kinoshita.testlinkjavaapi.TestProjectService.getTestProjectByName(TestProjectService.java:112)
at br.eti.kinoshita.testlinkjavaapi.TestLinkAPI.getTestProjectByName(TestLinkAPI.java:438)
at hudson.plugins.testlink.TestLinkBuilder.getTestLinkSite(TestLinkBuilder.java:309)
at hudson.plugins.testlink.TestLinkBuilder.perform(TestLinkBuilder.java:186)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.build(MavenModuleSetBuild.java:906)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:857)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1738)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
ERROR: Error communicating with TestLink. Check your TestLink configuration.
Finished: FAILURE
Maybe you can achieve this by customizing the Jenkin plugin, please have look in plugin code [1] and [2]
or asked from testlink forum
[1] https://github.com/jenkinsci/testlink-plugin
[2] https://github.com/kinow/testlink-java-api
Under Test Execution -> Test Reports and Metrics you will be able to filter out only the failed tests.
This can be exported, the icon is quite hidden on the top left, as an excel.
Possibly doing some work on that file you can be able to generate a test suite with only the failed tests

How to notify failure of ddl,dml scripts through jenkins?

We have configured a job to execute the .ddl & .dml files through jenkins.
We want to get notified when some code inside a dml or ddl file throws an error.
Can we do this through jenkins plugins. If Yes which plugin & how?
There are multiple approaches. Depending how you run the scripts, Jenkins will fail the build if there are non zero exit code.
Another appproach we use is Log Praser plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin . You can look for specific error string and fail with messsage. Then integrate this with Failure Cause Management plugin. It can given quick diagnostic message to the end user.