I am trying to run a simple test against a number of cases. I am using VS Code on Windows with Karate extension and standalone karate.jar.
Here is my feature:
Feature: settings support paths
Background:
* def some_ids = read('some_ids.json')
Scenario Outline: migrated settings are OK
Given url 'https://someapi.myorg.net/settings/'
And path id, 'Settings/Blah'
When method get
Then status 200
And match response.settings !contains { DefaultCounty: '#number'}
Examples:
|some_ids|
The Json is something like
[
{ "id":"0023a832-c1f3-464e-9de7-ce2cd0e24413"},
// ... 300 more lines of ids
{ "id":"fff5a55e-e3a1-43d8-81ef-b590f388fe90"}
]
It all works well until the number of cases gets around 300 where it kind of freezes by the end of execution and never produces the summary in console.
With lower numbers it works just fine, and the summary always indicates threads: 1, which is also supported by elapsed time given that API responds in ~1 sec.
My question is, setting the freezing aside, can I run these tests in parallel using standalone JAR?
The doc says Karate can run 'examples' in parallel, but I did not find any specific instructions for standalone jar.
I am not using Java as main platform and have no experience with Java ecosystem to speak of, so ability to use Karate as standalone is a big win for me.
Yes, just add a -T option: https://github.com/intuit/karate/tree/master/karate-netty#parallel-execution
java -jar karate.jar -T 5 src/features
Having a CI pipeline that is using Teamcity and Octopus deploy and build scripts written with Cake I want to be able to display the error messages that generated by the build script.
Now the message displayed is:
Exit code 1 (new)
In order to be able to see the real error message one must view the build log and parse it.
So, even when using build script, I want to be able to display the build results in overview page and the list of errors like in the following picture:
I know that Cake provides support for integration with TeamCity, but the documentation and examples are not that straightforward.
Ca anyone provide some helpful information regarding this subject?
Cake implements a method to be able to write a build problem
TeamCityProvider​.BuildProblem(string, ​string)
Looking at the source code for this provider, I can determine that this will build up a string to output that conforms to the build script interaction specified in the TeamCity documentation, specifically reporting the build problem
##teamcity[buildProblem description='<description>' identity='<identity>']
by calling BuildProblem("Some message", "Some identity") this will output
##teamcity[buildProblem description='Some Message' identity='Some identity']
TeamCity should then fail the build and display the message as per the documentation;
To fail a build directly from the build script, a build problem has be reported. Build problems appear on the Build Results page and also affect the build status text.
You need to edit the cake build scripts to properly trap the exception and call the above method, so it will write to the output stream correctly.
I can replicate this behaviour using a PowerShell script to write the buildProblem message to the ouput stream
This will then show the very message in the build results on the overview page
Hope this helps
I was wondering if there is a service where I could send data and then group that data however I want and later display it in charts or similar?
What I want to do is - start protractor e2e tests and let them run forever. Since some of the tests randomly fail I can not rely on them if they were included in main build. What I'm going to do is run tests in each supported browser over and over again and then capture which tests failed in which browser, if they passed - reset failure counter and so on.. That way I can monitor how my system is doing without wasting my time waiting for build to run successfully.
My data could look something like that:
{
testName: 'Path to test + name',
status: 'failed/passed',
browser: 'firefox/chrome/safari/opera...'
}
Now when test fails in (let's say) chrome over and over again I will see that it's error counter is constantly increasing, that means something is either wrong with system or I need to fix test in chrome.
Does anyone know service that could consume my data and display aggregated info?
I just created a project with androidStudio, I added the structure of tests, and can see the test results with the format of xml/html.
My question is how could I see the test output in console?
You can check the project on https://github.com/kolyjjj/process_killer
It seems the output will show in console when you write Local unit tests. Located at module-name/src/test/java/.
But in Instrumented tests,Located at module-name/src/androidTest/java/,output did't show in the console,It will show in logcat.
I have seen people posting about this here and elsewhere, but I haven't found any solution that works. I am using XCode 4.4 and have a bunch of unit tests set up. I have ran them all before on this project, so I know that they do pass/fail when they are supposed to if they are actually ran.
I have about 15 test suites, and each one contains 1-7 tests. On most attempts, all of the test suites finished (and passed) except for 1 (FooTests). It gives the warning:
FooTests did not finish
testFoo did not finish
XCode will report that testing was successful, regardless of what happens in unfinished tests. Another thing to note, sometimes it is a different test that will not finish, and sometimes multiple suites will not finish. I have not noticed a case where all tests do finish, but judging by this seemingly random behaviour I believe that it is possible.
So, is this a bug in XCode? I can't think of any other reason that tests randomly don't finish and then cause XCode to report that everything was successful. Are there any solutions?
I am on XCode 4.5.2. For application unit test, if your test suites finish so quick that the main application is not correctly loaded before that, you will get the warning. You can simply avoid the problem by adding a sleep at the end of your test like following. It doesn't happen for logic unit test.
[NSThread sleepForTimeInterval:1.0];
I've just had this problem with XC4.5DP4.
I had a test which does some work in a loop, and does nothing else when it falls out of the loop, and I was getting the "did not finish" error.
In an attempt to prove that the test was finishing, I added this as the very last line:
NSLog(#"done");
Not only does "done" get printed to the output - now Xcode says that the test has finished.
Seems to be a workaround... go figure.
I'm using XCode46-DP3 and I've just resolved this problem in my tests. I've several tests that start a web server and then execute http call to it; at the end of the test the web server is stopped. In last week these tests have begun to have the warning 'did not finish'.
For me has been enough to add the following sleep at the end of these tests (precisely I've added it in the tearDown):
- (void)tearDown {
[self.httpServer stop];
[NSThread sleepForTimeInterval:1.0];
self.httpServer = nil;
self.urlComposer = nil;
}
The problem seems that your tests terminate too quickly for Xcode to receive and parse the log messages that indicate failure or success. Sleeping for 1 second in the last test case run worked reliably for me, where 0.0 didn't. To see which test case is the last test case, check the test action in the Scheme dialog.
I created a separate WorkaroundForTestsFinishingTooFast test case, with a single method:
- (void)testThatMakesSureWeDontFinishTooFast
{
[NSThread sleepForTimeInterval:1.0];
}
The problem is that as you add more test cases, you'll have to ensure that this test is the last method run. That means removing and adding this class from the Test action as reordering of test cases is not allowed. On the other hand, you're only delaying your entire test bundle by 1 second.
I had the same warnings in the Log Navigator. I fixed it for me.
In my project I have got 2 Schemes, one for running the project and one for the unit tests.
Go to Product --> Edit Scheme...
Select the UnitTest Scheme in the scheme picker
Select the "Test"-Icon on the Left
Change the debugger from LLDB to GDB and press OK
Tests should finish now. (For me it worked fine)
For me the solution was to slim down the logging output from the parts of the app that the tests were testing. I think xcode couldn't parse the tests output in time because of the other output I had throughout the app.
I had the same problem running with XCode 4.6, the reason for it, in my case, was inconsistency between the scheme and the actual unit tests in the test suits.
In the scheme I had some suits checked but in their .m file some unit tests were commented.
To solve the problem: either uncomment the test or deselected the file/suit in the scheme and all shall became green again :)
for people like me that forgot how to reach the scheme these are the required steps:
right click on the project name in the scheme section (right to the stop button)
choose edit scheme
choose the test debug
click on the triangle next to the unit test project and next to each file you have a check box
uncheck files that you placed unit test in comments
hope this helps