Been scratching my head for a while with this and I thought I would ask here. I am trying to run automated tests with tcm via command prompt. I have set up and configured a test controller and agent. Tcm is recognizing all the tests from tfs when I list etc. Created a test run and the results came back:
Total:128
Passed:0
Failed:128
Inconclusive:128
Does this have something to do with the Controller or Agent not being set up properly? Would tcm recognize the tests if they were not set up properly? I have set up a lab environment and I can access the controller PC.
Any help would be greatly appreciated.
Thanks
Russ
Based on the information that "The build directory of the test run either does not exist or access permission is required", from steps 6 and 7 of this MSDN documentation:
Create and Define a build definition
Copy a build directory with the relevant information into this solution
This may also have relevant information if the above doesn't work
Related
I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!
I Have created copy(Scrum Process Backup)of my current process(Scrum Process) being followed in TFS, I have added few fields and rules, before I make copied process(Scrum Process Backup)as default process. I want to test how it is working, Please let me know how to test and where to fine option to test
Thanks in Advance
I think you have to use some test environment:
Install second TFS, upload your process and use any data.
Create a new project collection (Manage project collections in Team Foundation Server).
I’m trying to integrate Bugzilla Testopia with Jenkins with the aid of the Testopia Plugin for Jenkins.
The general configuration is probably fine as the connection between Testopia and Jenkins is well established (Jenkins log says: ‘Connecting to Testopia to retrieve automated test cases’ and no error occurs then). However I’m unable to retrieve any information concerning Test Runs/Test Cases etc. from Testopia.
Moreover I cannot perform any of ‘Iterative Test Build Steps’. If I want to ‘Execute Shell’ in ‘Iterative Test Build Steps’ with Testopia Plugin no operation is carried out (even if I try: echo 12345 etc.). If I use ‘Single Test Build Steps‘ then shell command is executed.
My goal is to retrieve test class name from Testopia (it is stored in Testopia Test Case’s field - Automation/Scripts ) and then run maven build from Jenkins with this class name set as the parameter. Afterwards depending on the Jenkins build success or failure status I’d like to update the Test Case Status in Testopia.
How to fetch any information from Testopia into Jenkins?
Why any of ‘Iterative Test Build Steps’ is not executed?
Any clues? - Testopia Plugin site example wasn't too helpful for me.
Both Bugzilla and Jenkins are hosted on the same Ubuntu 14.04. I've got the latest stable versions of Jenkins, Bugzilla and Testopia.
Thanks in advance,
M.
EDIT:
Well this debugging does not work for me. I added a new log recorder with the 'ALL' level chosen and I cannot see any additional log neither in job's console output nor in the newly created logger output.
Maybe sth is wrong with my Testopia installation? Some more details concerning my configuration:
I've got Testopia installed on the same machine (as Jenkins) and usually I access it through: http://'ip_address'/bugzilla
- in Testopia plugin configuration my URL to Testopia installation is: http://'ip_address'/bugzilla/xmlrpc.cgi
- I've got only one bugzilla account - these admin's credentials I use in Jenkins
- sometimes in Jenkins I can see a warning concerning improper reverse proxy configuraiton - maybe it has sth to do with the problem
After job execution all Testopia's fields are 0 - Run Id, Build Id etc. - which obviously indicates that no information was successfully retrieved from Testopia.
Any ideas how to check why I cannot retrieve any information from Testopia?
EDIT 2:
In the meantime I think I've found a clue in the jenkins.log file in jenkins installation directory:
Exceptions like these occur:
INFO: TESTOPIA_TEST_SUITE_3 #13 main build action completed: SUCCESS
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
...
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get_test_cases' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
Shall I insert the full stacktrace?
It looks like that plugin logs a fair amount of information, though not all of it to the build console output itself.
To debug further, you could try adding a new log recorder for the logger jenkins.plugins.testopia (with log level "all"), run a build, then refresh the web page for the newly-created log recorder to see the output.
You should at least see "Filtering for automated test cases" after connection, information about each test case found, and then log output for each iterative build step as it's run on each test case.
I got the similiar problem when I use python xmlrpc to commnicate with my bugzilla-testopia server.
I chekc the error code and find XMLRPC.pm under my bugzilla install location : "./WebService/Server/XMLRPC.pm".
I know the error was thrown by this sub function, which checks login status and forward to modules where the moduel.function must be in PUBLIC_METHODS:
sub handle_login {
...
if (none { $_ eq $method } $class->PUBLIC_METHODS) {
ThrowCodeError('unknown_method', { method => $full_method });
}
...
}
I don't know why but TestCase.get could not be found in PUBLIC_METHODS, so I just marked them as comment and then it works. You can do this quick way to make sure your client setting is correct. Then you should solve the "PUBLIC_METHODS" problem next.
My team is working on setting up Jenkins to run our automation. This is working, now.
However, we would like to explore the possibility of passing an argument in through the command line, or adding a config file on the fly, that goes into the dll of our test solution and alters certain, limited values.
For example:
There are three tests, and our global login variable is pointing to our integration environment. However, for just one of the tests, we want to override the environment variable and point to our test environment instead.
Can this be done with Jenkins? I can't find anything detailing this in the Jenkins documentation.
Thanks for the replies, and sorry for missing the current answers to this question. To help others than stumble on this question/answer, look at the following:
[Yes, you can do this (1)][1]
[Yes (2)][2]
[Yes (2)][3]
[Here's the plugin to do this][4]
1: How to set environment variables in Jenkins?
2: Jenkins : Report results of intermediate [windows batch] build steps in email body
3: How to set an environment variable programmatically in Jenkins/Hudson?
4: https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin
I start a new job in jenkins ci configured as free-style. I add post-build action "qa-report" and "Publish JUnit test result report". But I dont get qa-report and test result trend chart. I dont have the prevelige to access master machine. Cases are worked restrictly on a node. I can access the node.
For qa-report, I get the information.
Run a build with qa-report plugin enabled to see results here
What should I do? Does free-style not enabled qa-report? Do I need to specify some files or paths which located on master? Is there something wrong with the edition of maven. I am using maven3X. What kind of path of the xml file should I input to get chart. Any idea and references would be appreciated.
Expected:
1 qa-report
2 test result chart