XCTest testing for delegate methods being called

Question

I've been trying to test some classes I've created that perform networking actions using the NSNetServer class, among others. I'm having some issues ensuring that the delegate method is called.

I've tried numerous methods, including:

Using [NSThread sleepForTimeInterval:5.0f]; and [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:5.0f]]; to simply pause whilst the other actions are happening. The NSRunLoop method works the first time it is called (as shown in the example code below), but crashes on the second call. I understand neither are the "correct" way of doing things, but I don't know what the "correct" way is.

Using the NSCondition and NSConditionLock classes, which just seems to lock up the code and the callback is never called.

Using a while loop on a variable changed in the callback method, same as above.

Below is the code with a couple of extra comments and some of the tests removed for simplicity:

- (void)testCheckCredentials
{
    [self.server start];
    // Create a client
    self.nsb = [[NSNetServiceBrowser alloc] init];
    // Set the delegate to self
    self.nsb.delegate = self;
    // Search for the server
    [self.nsb searchForServicesOfType:self.protocol inDomain:@""];
    // Wait for the service to be found and resolved
    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:self.timeout]];
    XCTAssertTrue(self.serviceWasFound, @"Service was not found");
    // Open the connection to the server
    XCTAssertTrue([self.serverConnection open], @"Connection to server failed to open");
    // Wait for the client to connect
    /* This is where it crashes */
    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:self.timeout]];
    XCTAssertTrue(self.clientDidConnect, @"Client did not connect");
    /* Further, more class-specific tests */
}

- (void)netServiceBrowser:(NSNetServiceBrowser *)aNetServiceBrowser didFindService:(NSNetService *)service moreComing:(BOOL)moreComing
{
    NSLog(@"Found a service: %@ (%@)", service.name, service.domain);
    if ([self.serverName isEqualToString:service.name]) {
        self.serviceWasFound = YES;
    }
}

- (void)clientDidConnect:(RCFClientConnection *)client
{
    XCTAssertNotNil(client, @"Connected client is nil");
    self.clientConnection = client;
    self.clientDidConnect = YES;
}

I've also tried doing a lock on an NSCondition object:

[self.nsb searchForServicesOfType:self.protocol inDomain:@""];
// Wait for the service to be found and resolved
[self.lock lockWhenCondition:1];
XCTAssertTrue(self.serviceWasFound, @"Service was not found");

and

self.serviceWasFound = YES;
[self.lock unlockWithCondition:1]

When using the lock method, the netServiceBrowser:didFindService:moreComing: method is never called, same when I use:

while (!self.serviceWasFound) {};

I'm still learning Objective-C but I'm just totally stuck on this problem.

Solution 1

To handle testing components which call asynchronously executing methods and functions, XCTest has been enhanced in Xcode 6 to include the ability to handle blocks using new API and objects of class XCTestExpectation. These objects respond to new XCTest methods that allow the test method to wait until either the async call returns or a timeout is reached.

Here is the link for apple documentation for the above extract. Writing Tests of Asynchronous Operations

@interface sampleAPITests : XCTestCase<APIRequestClassDelegate>{
APIRequestClass *apiRequester;
XCTestExpectation *serverRespondExpectation;
}
@end

//implementation test class
- (void)testAPIConnectivity {
// This is an example of a functional test case.
serverRespondExpectation = [self expectationWithDescription:@"server responded"];
[apiRequester sendAPIRequestForMethod:nil withParams:nil];//send request to server to get tap info
apiRequester.delegate = self;
[self waitForExpectationsWithTimeout:1 handler:^(NSError *error) {
    if (error) {
        NSLog(@"Server Timeout Error: %@", error);
    }
   nslog(@"execute here after delegate called  or timeout");
}];
XCTAssert(YES, @"Pass");
}

//Delegate implementation
- (void) request:(APIRequest *)request didReceiveResponse:(NSDictionary *)jsonResponse success:(BOOL)success{
[serverRespondExpectation fulfill];
XCTAssertNotNil(jsonResponse,@"json object returned from server is nil");
}

Solution 2

A number of test libraries support testing assynchronous and threaded operations.

They all work essentially the same way:

• Wait x seconds for a condition to occur.
• Optionally perform a set of assertions after the first condition, failing if one of these is not met.
• Fail if the required condition does not occur within the specified (or default) time.

Here are some libraries that provide these features:

• The expecta matching library.
• The Kiwi test framework.
• The Typhoon DI framework has a utility for performing asynchronous integration tests.

As far as I know, all of these libraries use the run-loop approach, as others can result in deadlocks. For this approach to work reliably:

• When testing block-based callbacks you can create the block inline.
• For delegate callbacks you should create a separate stub (simplest possible implementation of a protocol) or use a mock ("magic" implementation created using a mocking library), and set this as the delegate. You may have issues if you set the test instance itself as the delegate.

Edit:

As of Xcode6 there's a new assertion XCTestExpectation for asynchronous testing.

Each of the above libraries can be installed using CocoaPods.

Comments

[Accatyyc]

There's no need to use a library for this. XCTest has XCTestException which was created for this purpose.

[Accatyyc]

XCTestExpectation was created to solve this problem. We use it ourselves and it works very well.

[Jasper Blues]

@Accatyyc Don't you mean XCTestExpectation? And XCTest is a library . . thanks for the info though - seems like a handy new feature.

[Jasper Blues]

Wow, at last. Thx for the info.

[Accatyyc]

Yes, that was a typo! It's a library. Maybe I should've said "third party library" instead since XCTest is built in

[Jasper Blues]

Yes and note that the answer above was written prior to Xcode 6, so it's timely new information in any case