XCTAssertEqual error: ("3") is not equal to ("3")

Question

NSMutableArray *arr = [NSMutableArray array];
[arr addObject:@"1"];
[arr addObject:@"2"];
[arr addObject:@"3"];

// This statement is fine.
XCTAssertTrue(arr.count == 3, @"Wrong array size.");

// This assertion fails with an error: ((arr.count) equal to (3)) failed: ("3") is not equal to ("3")
XCTAssertEqual(arr.count, 3, @"Wrong array size.");

What don't I understand about XCTAssertEqual? Why does the last assertion fail?

Solution 1

I have also had quite a bit of trouble with Xcode 5's tests. It still seems quite buggy with some odd behaviour - however I have found the definitive reason your particular XCTAssertEqual isn't working.

If we have a look at the test code we see it actually does the following (taken directly from XCTestsAssertionsImpl.h - it may be easier to view there):

#define _XCTPrimitiveAssertEqual(a1, a2, format...) \
({ \
    @try { \
        __typeof__(a1) a1value = (a1); \
        __typeof__(a2) a2value = (a2); \
        NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))]; \
        NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))]; \
        float aNaN = NAN; \
        NSValue *aNaNencoded = [NSValue value:&aNaN withObjCType:@encode(__typeof__(aNaN))]; \
        if ([a1encoded isEqualToValue:aNaNencoded] || [a2encoded isEqualToValue:aNaNencoded] || ![a1encoded isEqualToValue:a2encoded]) { \
                _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 0, @#a1, @#a2, _XCTDescriptionForValue(a1encoded), _XCTDescriptionForValue(a2encoded)),format); \
        } \
    } \
    @catch (id exception) { \
        _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 1, @#a1, @#a2, [exception reason]),format); \
    }\
})

Here's the problem:

What the test is actually doing is encoding the values into an NSValue and then comparing them. "Okay," you say, "but what's the problem with that?" I didn't think there was one either until I made my own test case for it. The problem is that NSValue's -isEqualToValue must also compare the NSValue's encoding type as well as its actual value. Both must be equal for the method to return YES.

In your case, arr.count is an NSUInteger which is a typedef of unsigned int. The compile-time constant 3 presumably degenerates into a signed int at runtime. Thus when the two are put into an NSValue object, their encoding types are not equal and thus the two CANNOT be equal according to -[NSValue isEqualToValue].

You can prove this with a custom example. The following code explicitly does exactly what XCTAssertEqual does:

// Note explicit types
unsigned int a1 = 3;
signed int a2 = 3;

__typeof__(a1) a1value = (a1);
__typeof__(a2) a2value = (a2);

NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))];
NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))];

if (![a1encoded isEqualToValue:a2encoded]) {
    NSLog(@"3 != 3 :(");
}

"3 != 3 :(" will appear in the log every time.

I hasten to add here that this is, in fact, expected behaviour. NSValue is supposed to check its type encoding when doing comparisons. Unfortunately it's just not what we were expecting when testing two ('equal') integers.

XCTAssertTrue, incidentally, has much more straightforward logic, and behaves generally as expected (again, see the actual source for how it determines whether the assertion fails).

Solution 2

I have had this problem, too. As @ephemera and @napier indicated, this is a type issue.

It can be solved by supplying a value of the correct type, using the c-literal modifiers.

XCTAssertEqual(arr.count, 3ul, @"Wrong array size.");

You can find the correct type by looking up the return type of the function used on the left hand side - ALT-click on arr.count:

- (NSUInteger)count;

Now ALT-click on NSUInteger to find its type:

typedef unsigned long NSUInteger;

Now find the c literal numeric format for unsigned long - google is a good friend but this page works:

http://www.tutorialspoint.com/cprogramming/c_constants.htm

As a quick hint here, you may need to use U (unsigned) L (long) or F (float), and make sure you write 1.0 instead of 1 to get a double. Lowercase also works, as in my example above.

Solution 3

In case someone else is looking for the issue lead by double comparing like me (solution above won't work for float & double), try:

XCTAssertEqualWithAccuracy(number.doubleValue, 12.34, 0.01);

Generates a failure when (difference between (\a expression1) and (\a expression2) is > (\a accuracy))).

Comments

[Jasper Blues]

Some other great matcher libraries are: OCHamcrest and Expecta. . there's also Kiwi and Cedar - fully fledged test frameworks with nice inbuilt matcher libraries. . (Just in case you haven't tried these yet).

[Rob Napier]

It's worth noting that the correct fix for this is to simply include the type information. XCTAssertEqual(arr.count, (NSUInteger)3, @"Wrong array size.");

[owenfi]

Thanks, an even easier way is: XCTAssertEqual(arr.count, 3U, @"Wrong array size.");

[zim]

Better to use (NSUInteger)3 rather than 3U because NSUInteger is typedef'ed differently for 64bit and 32bit compilation. For 64bit, NSUInteger is an unsigned long versus unsigned int for 32bit.

[ThomasW]

I don't think this works if you want your tests to work on both 32 and 64 bit. Using 3ul will cause a failure with 32 bit.

[Elise van Looij]

Or use XCTAssertEqualWithAccuracy(arr.count, 3, 0.000000001); which I learned from an answer given by @WayneHartman in link