From: http://blog.carbonfive.com/2011/04/06/running-xcode-4-unit-tests-from-the-command-line/
Command line builds for Xcode 4 projects are a good first step but I really want to get my project’s tests running on a continuous integration server again. Since “test” isn’t a valid build action to pass to xcodebuild I’ve been looking for a configuration which would allow me to run tests in a headless environment.
I expect that GTM, GHUnit, and Cedar will all have reliable support for Xcode 4 projects eventually but I would like to start with just seeing some SenTestingKit tests pass.
I have been able to run test suites from the command line but only for “logic” tests. From the iOS Development Guide:
Xcode offers two types of unit tests: logic tests and application tests.
Logic tests.
These tests check the correct functionality of your code in a clean-room environment; that is, your code is not run inside an application. Logic tests let you put together very specific test cases to exercise your code at a very granular level (a single method in class) or as part of a workflow (several methods in one or more classes). You can use logic tests to perform stress-testing of your code to ensure that it behaves correctly in extreme situations that are unlikely in a running application. These tests help you produce robust code that works correctly when used in ways that you did not anticipate. Logic tests are iOS Simulator SDK–based; however, the application is not run in iOS Simulator: The code being tested is run during the corresponding target’s build phase.Application tests.
These tests check the functionality of your code in a running application. You can use application tests to ensure that the connections of your user-interface controls (outlets and actions) remain in place, and that your controls and controller objects work correctly with your object model as you work on your application. Because application tests run only on a device, you can also use these tests to perform hardware testing, such as getting the location of the device.
I am able to create a set of logic tests as a separate build target and scheme which will successfully run as part of an xcodebuild build.
- Create a new “LogicTests” unit test build target.
- Leave the “Test Host” build setting blank.
- Set the “Test After Build” build setting to “No”.
- Do not write tests which require an application to be present (ie no window or application delegate available).
- Create a new “LogicTests” scheme which includes the “LogicTests” build target in its run action.
- Run the “LogicTests” scheme using xcodebuild.
Scheme for running logic tests
> /Developer_Xcode4/usr/bin/xcodebuild -workspace WrappingScrollView.xcworkspace -scheme logictests -sdk iphonesimulator4.3 -configuration Debug clean build
…
Run test suite logictests
Test Suite ‘logictests’ started at 2011-04-06 07:44:57 +0000
Run test case testExample
Test Case ‘-[logictests testExample]‘ started.
/Users/Jonah/Desktop/WrappingScrollView/WrappingScrollViewDemo/logictests/logictests.m:30: error: -[logictests testExample] : Unit tests are not implemented yet in logictests
Test Case ‘-[logictests testExample]‘ failed (0.000 seconds).Test Suite ‘logictests’ finished at 2011-04-06 07:44:57 +0000.
Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.000) seconds
That’s not a great solution but it at least allows me to run some tests on a continuous integration server. Hopefully I can find a way to run application tests through xcodebuild as well.
In case it is of any help to other developers struggling with the difference between command line and Xcode builds. The “Run Script” build phase of a unit test build target just runs “${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests”. The RunUnitTests script in turn invokes a platform specific script like “${SYSTEM_DEVELOPER_DIR}/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests” which ultimately calls RunTestsForBundle in “${SYSTEM_DEVELOPER_DIR}/Tools/RunPlatformUnitTests.include”. Hopefully it will prove possible to use these scripts to match the behavior seen when running tests from within Xcode to automatically run application tests.
