I can run all unit tests using the following command:
xcodebuild test -workspace Project.xcworkspace -scheme Scheme -sdk iphonesimulator7.0 -destination platform='iOS Simulator',OS=7.0,name='iPhone Retina (4-inch)'
Is there anything I can pass to this to run individual unit tests/classes in the same way that you can using the Xcode UI?
Cheers.
Not that I know of, and it's not on the man page either. But you have such option in the excellent xctool (-only SomeTestTarget:SomeTestClass/testSomeMethod).
For those who are using later versions of Xcode (8.0+) and may run across this post in the future (like myself), this is now possible using the -only-testing flag, which is documented in the man page for xcodebuild. The format for using the flag is as follows:
xcodebuild \
-destination "$DEST" \
-workspace "$WRKSP" \
-scheme "$SCHEME" \
-only-testing:Target/Class/Method test
You can also omit elements from the path after the -only-testing flag, e.g. -only-testing:Scheme/Class will run all these tests within that class.
Here is a related Stackoverflow question, touching on the new functionality:
How to use xcodebuild with -only-testing and -skip-testing flag?
Another way is to edit the schemer of your test file and disable/enable the tests as you please. Then you can rerun the same command through the CLI and it will only run the tests you specified above.
If you're using Kiwi, you should supposedly be able to do it using the KW_SPEC environment variable. I haven't been able to, though: Running a single KIWI spec with xctool.
Related
I'm trying to build some tests with googleTest which should test some MPI parallel code. Ideally I want our CI server to execute them through ctest.
My naive approach was to simply call ctest with MPI:
mpirun -n 3 ctest
However, this results in even trivial tests failing as long as at least two are executed.
Example of a trivial tests:
TEST(TestSuite, DummyTest1) {
EXPECT_TRUE(true);
}
TEST(TestSuite, DummyTest2) {
EXPECT_TRUE(true);
}
Am I supposed to launch ctest in some other way? Or do I have to approach this completely differently?
Additional info:
The tests are added via gtest_discover_tests().
Launching the test executable directly with MPI (mpirun -n 3 testExe) yields successful tests but I would prefer using ctest.
Version info:
googletest: 1.11.0
MPI: OpenMPI 4.0.3 AND MPICH 3.3.2
According to this CMake Forum post, which links to a Merge Requests and an issue in the CMake GitLab repository, gtest_discover_tests() currently simply doesn't support MPI.
A possible workaround is to abuse the CROSSCOMPILING_EMULATOR property to inject the wrapper into the test command. However, this changes the whole target and not only the tests so your mileage may vary.
set_property(TARGET TheExe PROPERTY CROSSCOMPILING_EMULATOR '${MPIEXEC_EXECUTABLE} ${MPIEXEC_NUMPROC_FLAG} 3')
gtest_discover_tests(TheExe)
See the forum post for a full minimal example.
I have more than 50 test cases in my UI-TEST scheme and I am running it via the below command
xcodebuild test -workspace tribo.xcworkspace -scheme triboUITests -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone X,OS=12.4'
what happens is the RAM of my Mac machine shoots up and the system hangs up , I have attached the activity monitor report tooenter image description here
can anyone help me in running all XCUITEST cases present in scheme from terminal
The console command itself seems legit.
The RAM problem also could be caused by inaccurate resource management in test code.
Consider inspecting your code for possible memory leakings and other bugs.
You can run a specific test or tests as part of the xcodebuild invocation.
From the xcodebuild help command line:
-only-testing:TEST-IDENTIFIER constrains testing by specifying tests to include, and excluding other tests
-skip-testing constrains testing by specifying tests to exclude, but including other tests
-skip-testing:TEST-IDENTIFIER constrains testing by specifying tests to exclude, but including other tests
Using these commands may help you understand which tests are causing the issue.
You could also use the Xcode Memory Graph Debugger (see this link:
https://medium.com/zendesk-engineering/ios-identifying-memory-leaks-using-the-xcode-memory-graph-debugger-e84f097b9d15
Finally, you if you are trying to run all your tests in parallel, you may have too many simulators running on the machine.
I am using fastlane snapshot tool for taking snapshot for app screens.
According to fastlane community, i need to run,
fastlane snapshot init
Then after configuring project ui test target, i need to run
fastlane snapshot
But if I want to provide some build parameters like, xcodebuild test test-only params, how can I do that. For example i want to build like,
xcodebuild test -workspace <path>
-scheme <name>
-destination <specifier>
-only-testing:TestBundleA/TestSuiteA/TestCaseA
-only-testing:TestBundleB/TestSuiteB
-only-testing:TestBundleC
I see,
fastlane snapshot --help
Then I added in Snapfile,
xcargs -only-testing:TestBundleB/TestSuiteB
But this gives error
(eval):36: syntax error, unexpected tSYMBEG, expecting keyword_do or
'{' or '(' only-testing:TestBundleB/TestSuiteB
How can i solve this error?
I am not familiar with running snapshot from the command line, so if you can, I would recommend creating a fastlane/Fastfile (or editing it if it already exists) to have a lane that calls snapshot with the options that you are looking for. You can call it with its various parameters as explained in the docs
The example shows you how snapshot could be called, and the Parameters table describes the other parameters you can pass to the fastlane Action.
To pass xcargs via the Snapfile, try xcargs "-only-testing:TestBundleB/TestSuiteB" in your Snapfile. See this Issue.
Recently I'm preparing UITesting for our app.
My XCode version is 7.3.1(7D1014).
When I had executed 'UITesting' in XCode as manual, I got the coverage report successfully as below.
UITesting executed by xcode
But I execute test that project with command line, created "Coverage.profdata" file size is different with xcode. And change that coverage data to 'cobertura' using 'slather', the result is zero.
Here is my xcodebuild statement.
xcodebuild test -project SampleTestApp.xcodeproj -scheme SampleTestApp -configuration Debug -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 5s,OS=9.3' -enableCodeCoverage YES
I think, I missed something. Is there anyone who experienced like me?
I need your help.
I looked log file furthermore. And I found something different between XCode testing and xcodebuild testing.
A. XCode Testing Log(~/Library/Developer/Xcode/DerivedData/MyProjectXXX/Logs/Test/XXXXX/Session-XXXXXX.log)
"wait_for_debugger" set 1 in Tests-Runner options
B. xcodebuild Testing Log(~/Library/Developer/Xcode/DerivedData/MyProjectXXX/Logs/Test/YYYYY/Session-YYYYY.log)
"wait_for_debugger" set 0 in Tests-Runner options
I wonder i can change this option value when executes test-runner.
How do I explicitly say with my go test command to run only tests for the main package and not others in my source directory.
At the moment it's working with $go test -v. But... I am using goconvey as well and it seems to be running recursively. According to this page https://github.com/smartystreets/goconvey/wiki/Profiles I have a file where I can pass arguments into the go test command. I know you can go test -v ./... for recursive or go test -c packagename/... but how do I just do it for the main?
Profiles is one to accomplish this, but you can also specify a 'depth' for the runner:
$ goconvey -depth=0
A value of 0 limits the runner to the working directory.
Run goconvey -help for details.