RedGreen for iOS and Mac

A couple of weeks ago I released an extension for SenTestingKit with the aim of lowering the cognitive load of visually parsing the unit tests results in Xcode. I’d grown frustrated at the amount of time it took to identify which tests had failed and, more importantly, why they had failed. The standard format of the unit tests is quite difficult to work with, as can be seen in the example below:

Test Suite '/Users/neil/Code/.../RedGreenTests.octest(Tests)' started at 2013-05-11 10:32:02 +0000
Test Suite 'RedGreenTests' started at 2013-05-11 10:32:02 +0000
Test Case '-[RedGreenTests testBooleanAssertions]' started.
/Users/neil/Code/.../RedGreenTests.m:15: error: -[RedGreenTests testBooleanAssertions] : "true" should be false. 
    Expected true to be false.
/Users/neil/Code/.../RedGreenTests.m:16: error: -[RedGreenTests testBooleanAssertions] : "false" should be true. 
    Expected false to be true.
Test Case '-[RedGreenTests testBooleanAssertions]' failed (0.000 seconds).
Test Case '-[RedGreenTests testCanIProvideBetterOutput]' started.
Test Case '-[RedGreenTests testCanIProvideBetterOutput]' passed (0.000 seconds).
Test Case '-[RedGreenTests testDidIProvideBetterOutput]' started.
Test Case '-[RedGreenTests testDidIProvideBetterOutput]' passed (0.000 seconds).
Test Case '-[RedGreenTests testEqualityAssertions]' started.
/Users/neil/Code/.../RedGreenTests.m:30: error: -[RedGreenTests testEqualityAssertions] : '(null)' should be 
    equal to 'Invalid user credentials.' Unexpected error messsage.
Test Case '-[RedGreenTests testEqualityAssertions]' failed (0.000 seconds).
Test Case '-[RedGreenTests testEstimatesAreEqual]' started.
Test Case '-[RedGreenTests testEstimatesAreEqual]' passed (0.000 seconds).
Test Case '-[RedGreenTests testFails]' started.
/Users/neil/Code/.../RedGreenTests.m:45: error: -[RedGreenTests testFails] : You deliberately called STFail().
Test Case '-[RedGreenTests testFails]' failed (0.000 seconds).
Test Case '-[RedGreenTests testSenTestLogOutput]' started.
Test Case '-[RedGreenTests testSenTestLogOutput]' passed (0.000 seconds).
Test Case '-[RedGreenTests testSucceeds]' started.
Test Case '-[RedGreenTests testSucceeds]' passed (0.000 seconds).
Test Suite 'RedGreenTests' finished at 2013-05-11 10:32:02 +0000.
Executed 8 tests, with 4 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite '/Users/neil/Code/.../RedGreenTests.octest(Tests)' finished at 2013-05-11 10:32:02 +0000.
Executed 8 tests, with 4 failures (0 unexpected) in 0.001 (0.002) seconds

And that’s just the output for eight tests. You can imagine the difficulty of working with a test suite that runs into the hundreds of tests.

Digging into the information that was displayed, I realised the key data points where which test was bring run, did it pass or fail, where in the code did the failure occur and why did the test fail. I played around with rewriting the output in various formats with no loss of information fidelity and eventually came up with this:

Test Suite '/Users/neil/Code/.../RedGreenTests.octest(Tests)' started at 2013-05-11 10:33:17 +0000
Test Suite 'RedGreenTests' started at 2013-05-11 10:33:17 +0000
FAILED: -[RedGreenTests testBooleanAssertions] (0.00037s)
    Line 15: "true" should be false. Expected true to be false.
    Line 16: "false" should be true. Expected false to be true.
PASSED: -[RedGreenTests testCanIProvideBetterOutput] (0.00002s)
PASSED: -[RedGreenTests testDidIProvideBetterOutput] (0.00001s)
FAILED: -[RedGreenTests testEqualityAssertions] (0.00010s)
    Line 30: '(null)' should be equal to 'Invalid user credentials.' Unexpected error messsage.
PASSED: -[RedGreenTests testEstimatesAreEqual] (0.00005s)
FAILED: -[RedGreenTests testFails] (0.00009s)
    Line 45: You deliberately called STFail().
PASSED: -[RedGreenTests testSenTestLogOutput] (0.00002s)
PASSED: -[RedGreenTests testSucceeds] (0.00001s)
Test Suite 'RedGreenTests' finished at 2013-05-11 10:33:17 +0000.
Executed 8 tests, with 4 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite '/Users/neil/Code/.../RedGreenTests.octest(Tests)' finished at 2013-05-11 10:33:17 +0000.
Executed 8 tests, with 4 failures (0 unexpected) in 0.001 (0.002) seconds

It’s more compact with no loss of information and with careful use of whitespace it made identifying test failures significantly easier. I’d also played around with the excellent XcodeColors plugin for colouring console output, so it only seemed natural to add support for that into the formatter I was writing. The result can be seen below.

The unit test results have been gone from being a difficult-to-parse block of text to something you can immediately get a feel for without having to do any reading at all. Anything that helps so get to the source of a problem quicker is a good thing in my book.

If you find this useful or you have any feedback then, please, do let me know via Twitter.