I'm running TeamCity 6.5 on a Windows Server, with a couple of build agents on the same server (all running as the system user as services). I had been building SilverLight projects and running the StatLight (v 1.4.4147) tests previously under Jenkins with no problems. On Jenkins, I called the StatLight test in a custom script as follows:
StatLight.exe -x="Tests.xap"
StatLight.exe -x="MoreTests.xap"
StatLight.exe -x="EvenMoreTests.xap"
... etc., but when I migrated my build jobs to TeamCity, I also changed these into a single command line step as follows:
StatLight.exe --teamcity -x="Tests.xap" -x="MoreTests.xap" -x="EvenMoreTests.xap"
This works about 50% of the time, but when it fails, there's no output in the build log to tell me why - I just get:
[11:41:18]: [MyProject\bin\Release\MoreTests.xap] Tests.ExtensionsTests.WatchObservableCollection
[11:41:18]: [MyProject\bin\Release\MoreTests.xap] Tests.SubscribingModelBaseTests.DisposeIsCalled
[11:41:18]: [MyProject\bin\Release\MoreTests.xap] --- Completed Test Run at: 28/09/2011 11:41:18. Total Run Time: 00:00:11.8125000
[11:41:19]: [MyProject\bin\Release\MoreTests.xap] Test run results: Total 6, Successful 6, Failed 0,
[11:41:19]: [Step 5/6] MyProject\bin\Release\EvenMoreTests.xap (9m:42s)
... and then nothing more. The time reported in that last line just goes up and up until I kill the the build job. Adding the --debug switch to StatLight doesn't improve the above output either.
Right now, I've switched the TeamCity build step to call each test individually as I was in Jenkins, but this is more of a workaround than a proper solution. And of course, I may still run into the above problem - I've yet to find out.
What I'd like to know is what steps I can take to debug this issue properly, or whether there are known issues that can cause the above behaviour?
There was one issue fixed in the 1.5 version relating to teamcity. http://statlight.codeplex.com/workitem/13654
I'm not sure it will fix your issue, but would you mind upgrading, trying and reporting back?
Related
I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!
First, important note: This is a project in which the Selenium/Cucumber test suite is working. It's working locally for the developers, and it's working in the Tekton environment.
Second: I have an identical IntelliJ environment as the developers. And other Selenium/Cucumber projects are running fine on my local machine.
When a try to run a fresh copy of the project in question locally, none of the tests start. And, there is no error message.
Normally, this will be cause by missing Glue parameters in the Run/Debug configuration, since IntelliJ for some reason often is unable to add these automatically. This is not the case here.
The test runner when I try to start a specific scenario:
Sorry for the language. The tests are all in Norwegian. But I don't think that matters for seeing the problem.
Normally, when there is an actual error regarding the test step - in the case "Gitt jeg er en vanlig søker" - there would be some info to the right of the steps overview. This is now blank, as can be seen. So, the test stes aren't even started.
The run/debug configuration:
e2e.cucumber.felles is the package where the Java file with the first step definition is located. So, that's not the reason.
Any ideas?
I am using VSTest to run tests via Azure DevOps. Tests run successfully, but using the option rerun failed tests: true, an error appears during the restart phase.
NUnit 3.12.0 ;
NUnit Adapter 3.16.1.0 (Checked with 4.0.0.0)
vstest.console.exe "C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1\Tests.Web.dll"
/Settings:"C:\agent2.172.2\_work\_temp\3utv233tymm.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1"
/TestCaseFilter:"FullyQualifiedName=Tests.Web.Tests._5.CourierModuleTest.N1_SendingTest.Id_5_1_01_TransferToCourierModule(Chrome)|FullyQualifiedName=Tests.Web.Tests._3.IssuanceOfDocuments.InformationOnTheApplicationIdentificationOfRecipient.N2_RecipientIdentificationTest.Id_3_2_13_RegisterAddressByFiasTest(Chrome)"
NUnit Adapter 3.16.1.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
How can I fix this error and successfully restart the tests in DevOps?
I tried googling for a similar issue but didn't find anything that would work. Any help is really appreciated
Here's similar discussion, according to comments from the Contributors of azure-pipelines-task repo:
1.You should update your VS and VSTest component within it to latest version since we don't support data driven tests for Rerun scenario. It will be available with VS 15.8 release and higher.
2.You should configure your VSTest task following this solution:
Recommended to give total number of your tests as input (Number of tests per batch).
I am creating a TeamCity build configuration with a few steps. One of those steps is running tests using MSTest. Currently, my tests are failing (by design so I can test the build process), but the build steps after the step for running the tests happen, even though I can see the test process exits with code 1 (ie, something other than code 0) in the build log. It does mark the build as failed, but I'd prefer it if the steps in the build stopped once failing tests were detected. Is that possible and I'm just missing something in the configuration?
When you create build step there are select Execute step for execution policy. Where you should select Only if all previous steps were successful.
Here is it:
But in your case there are bug in the MsTest and NUnit build steps. Here is workaround. And here is related issue.
nunit tests fails when run through cc.net saying process timeout. Process has been killed
All works fine when through nUNit or VS.
Also cc.net will then show the results of previous build even if the build is a clean one.
Any help plz.
The default timeout is 600 seconds. If your tests start to exceed that the build will fail with no indication. You may need to up the timeouts for your cc.net nunit task
If you are seeing the results from a previous build, it is probably because you are not deleting the results from your previous build.
For example, my NUnit test results are written to files with the name {foo}-results.xml:
<publishers>
<merge>
<files>
<file>bin\debug\*-results.xml</file>
</files>
</merge>
</publishers>
In my tasks, I have a step in my build file that deletes the entire "bin\debug" directory so that my results are always the current ones.
One possibility is that you have a permission issue. CruiseControl is perhaps running under a service account and has different permissions than your user account (which I'm assuming you use to manually run the tests.) Try logging into the machine as the service account, then see if you can run the unit tests through VS or NUnit.
I've seen this happen if a test has an assertion, e.g. Debug.Assert(something here). When this happens to me in CC.Net, the CC.Net build pops up a message box for the assertion. Since no one closes out the message box on the build server, the NUnit test times out.