Making HTTPS request in iOS 9 with self-signed certificate

Question

I want to make an HTTPS request to custom server with self-signed certificate. I'm using NSURLConnection class and processing authentication challenges, but always receive error message in a console:

NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9802)

then method "connection:didFailWithError:" gets called with the following error:

Error Domain=NSURLErrorDomain Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={NSURLErrorFailingURLPeerTrustErrorKey=<SecTrustRef: 0x150094100>, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802, NSErrorPeerCertificateChainKey=<CFArray 0x1500ddd90 [0x19f6dab68]>{type = immutable, count = 1, values = (
    0 : <cert(0x14e6fb370) s: (server certificate name) i: (custom CA name)>
)}, NSUnderlyingError=0x1504ae170 {Error Domain=kCFErrorDomainCFNetwork Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={NSErrorFailingURLStringKey=https://217.92.80.156:9090/(method name and parameters), NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFNetworkCFStreamSSLErrorOriginalValue=-9802, kCFStreamPropertySSLPeerCertificates=<CFArray 0x1500ddd90 [0x19f6dab68]>{type = immutable, count = 1, values = (
    0 : <cert(0x14e6fb370) s: (server certificate name) i: (custom CA name)>
)}, _kCFStreamPropertySSLClientCertificateState=2, kCFStreamPropertySSLPeerTrust=<SecTrustRef: 0x150094100>, NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made., _kCFStreamPropertySSLClientCertificates=<CFArray 0x14e5ee8e0 [0x19f6dab68]>{type = mutable-small, count = 2, values = (
    0 : <SecIdentityRef: 0x15012cd40>
    1 : <cert(0x15014aa70) s: (client certificate name) i: (custom CA name)>
)}, _kCFStreamErrorDomainKey=3, NSErrorFailingURLKey=https://217.92.80.156:9090/(method name and parameters), _kCFStreamErrorCodeKey=-9802}}, NSErrorClientCertificateChainKey=<CFArray 0x14e5ee8e0 [0x19f6dab68]>{type = mutable-small, count = 2, values = (
    0 : <SecIdentityRef: 0x15012cd40>
    1 : <cert(0x15014aa70) s: (client certificate name) i: (custom CA name)>
)}, NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made., NSErrorFailingURLKey=https://217.92.80.156:9090/(method name and parameters), NSErrorFailingURLStringKey=https://217.92.80.156:9090/(method name and parameters), NSErrorClientCertificateStateKey=2}

App receives two authentication challenges (NSURLAuthenticationMethodClientCertificate and NSURLAuthenticationMethodServerTrust) and processes them in a following manner:

- (void) connection:(NSURLConnection *)connection willSendRequestForAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge
{
    if(challenge.proposedCredential && !challenge.error)
    {
        [challenge.sender useCredential:challenge.proposedCredential forAuthenticationChallenge:challenge];

        return;
    }

    NSString *strAuthenticationMethod = challenge.protectionSpace.authenticationMethod;
    NSLog(@"authentication method: %@", strAuthenticationMethod);

    NSURLCredential *credential = nil;
    if([strAuthenticationMethod isEqualToString:NSURLAuthenticationMethodClientCertificate])
    {
        // get identity and certificate from p.12
        NSData *PKCS12Data = [NSData dataWithContentsOfFile:[[NSBundle mainBundle] pathForResource:@"client" ofType:@"p12"]];

        NSDictionary *optionsDictionary = [NSDictionary dictionaryWithObject:@"password" forKey:(__bridge id)kSecImportExportPassphrase];
        CFArrayRef items = CFArrayCreate(NULL, 0, 0, NULL);
        OSStatus securityError = SecPKCS12Import((__bridge CFDataRef)PKCS12Data,(__bridge CFDictionaryRef)optionsDictionary, &items);

        SecIdentityRef identity = NULL;
        SecCertificateRef certificate = NULL;
        if(securityError == errSecSuccess)
        { 
            CFDictionaryRef myIdentityAndTrust = CFArrayGetValueAtIndex(items, 0);
            identity = (SecIdentityRef)CFDictionaryGetValue (myIdentityAndTrust, kSecImportItemIdentity);

            CFArrayRef array = (CFArrayRef)CFDictionaryGetValue(myIdentityAndTrust, kSecImportItemCertChain);
            certificate = (SecCertificateRef)CFArrayGetValueAtIndex(array, 0);
        }

        credential = [NSURLCredential credentialWithIdentity:identity certificates:[NSArray arrayWithObject:(__bridge id)(certificate)] persistence:NSURLCredentialPersistenceNone];

        CFRelease(items);
    }
    else if([strAuthenticationMethod isEqualToString:NSURLAuthenticationMethodServerTrust])
    {       
        int trustCertificateCount = (int)SecTrustGetCertificateCount(challenge.protectionSpace.serverTrust);
        NSMutableArray *trustCertificates = [[NSMutableArray alloc] initWithCapacity:trustCertificateCount];
        for(int i = 0; i < trustCertificateCount; i ++)
        {
            SecCertificateRef trustCertificate =  SecTrustGetCertificateAtIndex(challenge.protectionSpace.serverTrust, i);
            [trustCertificates addObject:(__bridge id) trustCertificate];
        }            

        SecPolicyRef policyRef = NULL;
        policyRef = SecPolicyCreateSSL(YES, (__bridge CFStringRef) challenge.protectionSpace.host);

        SecTrustRef trustRef = NULL;
        if(policyRef)
        {
            SecTrustCreateWithCertificates((__bridge CFArrayRef) trustCertificates, policyRef, &trustRef);
            CFRelease(policyRef);
        }

        if(trustRef)
        {
//          SecTrustSetAnchorCertificates(trustRef, (__bridge CFArrayRef) [NSArray array]);
//          SecTrustSetAnchorCertificatesOnly(trustRef, NO);

            SecTrustResultType result;
            OSStatus trustEvalStatus = SecTrustEvaluate(trustRef, &result);
            if(trustEvalStatus == errSecSuccess)
            {
                // just temporary attempt to make it working.
                // i hope, there is no such problem, when we have final working version of certificates.
                if(result == kSecTrustResultRecoverableTrustFailure)
                {
                    CFDataRef errDataRef = SecTrustCopyExceptions(trustRef);
                    SecTrustSetExceptions(trustRef, errDataRef);

                    SecTrustEvaluate(trustRef, &result);
                }

                if(result == kSecTrustResultProceed || result == kSecTrustResultUnspecified)
                    credential = [NSURLCredential credentialForTrust:trustRef];
            }

            CFRelease(trustRef);
        }
    }
    else
    {
        DDLogWarn(@"Unexpected authentication method. Cancelling authentication ...");
        [challenge.sender cancelAuthenticationChallenge:challenge];
    }

    if(credential)
        [challenge.sender useCredential:credential forAuthenticationChallenge:challenge];
    else
        [challenge.sender cancelAuthenticationChallenge:challenge];
}

In CFNetwork diagnostic log i can see that Handshake procedure is about to be started. At least app sends "ClientHello" message, then server sends its "ServerHello" message and requires for authentication. And here app tries to send authentication response, but immediately receives error. (At the same time, in server logs i don't see any messages about handshake at all). Here is part of diagnostic log:

Sep 15 10:51:49  AppName[331] <Notice>: CFNetwork Diagnostics [3:49] 10:51:49.185 {
    Authentication Challenge
       Loader: <CFURLRequest 0x1501931c0 [0x19f6dab68]> {url = https://217.92.80.156:9090/(method name and parameters), cs = 0x0}
    Challenge: challenge space https://217.92.80.156:9090/, ServerTrustEvaluationRequested (Hash f9810ad8165b3620)
    } [3:49]
Sep 15 10:51:49  AppName[331] <Notice>: CFNetwork Diagnostics [3:50] 10:51:49.189 {
    Use Credential
        Loader: <CFURLRequest 0x1501931c0 [0x19f6dab68]> {url = https://217.92.80.156:9090/(method name and parameters), cs = 0x0}
    Credential: Name: server, Persistence: session
    } [3:50]
Sep 15 10:51:49  AppName[331] <Notice>: CFNetwork Diagnostics [3:51] 10:51:49.190 {
     touchConnection
              Loader: <CFURLRequest 0x1501931c0 [0x19f6dab68]> {url = https://217.92.80.156:9090/(method name and parameters), cs = 0x0}
    Timeout Interval: 60.000 seconds
    } [3:51]
Sep 15 10:51:49  AppName[331] <Notice>: CFNetwork Diagnostics [3:52] 10:51:49.192 {
    Response Error
    Request: <CFURLRequest 0x14e5d02a0 [0x19f6dab68]> {url = https://217.92.80.156:9090/(method name and parameters), cs = 0x0}
      Error: Error Domain=kCFErrorDomainCFNetwork Code=-1200 "(null)" UserInfo={_kCFNetworkCFStreamSSLErrorOriginalValue=-9802, kCFStreamPropertySSLPeerCertificates=<CFArray 0x1500ddd90 [0x19f6dab68]>{type = immutable, count = 1, values = (
                0 : <cert(0x14e6fb370) s: (server certificate name) i: (custom CA name)>
             )}, _kCFStreamPropertySSLClientCertificateState=2, kCFStreamPropertySSLPeerTrust=<SecTrustRef: 0x150094100>, _kCFStreamPropertySSLClientCertificates=<CFArray 0x14e5ee8e0 [0x19f6dab68]>{type = mutable-small, count = 2, values = (
                0 : <SecIdentityRef: 0x15012cd40>
                1 : <cert(0x15014aa70) s: (client certificate name) i: (custom CA name)>
             )}, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9802}
    } [3:52]

Our back-end instance can be installed on customer side, so i can't set any domain exception in Info.plist file. Also app may request server by IP address in IPv4 form, but not by domain name (as it is in my example).

What have i tried:

• used NSURLSession instead of NSURLConnection, but without any success;
• checked Apple's ATS requirements for server implementation here (back-end developer is sure that his implementation meets all of them);
• played with setting anchor certificates for trust validation in accordance with various solved issues from stackoverflow and Apple's developer forums;
• paid attention particularly to similar post and its related solution at developer forums;

I'm testing https request on iPad Air 2 with iOS 9 GM Seed (Build 13A340) and xCode 7 GM Seed (Build 7A218). Important note: this functionality works fine with iOS 8. Taking that into account i may assume, that problem is in our server, but our back-end developer assured me that there everything is fine.

Now i'm out of ideas. I would appreciate if anyone can give me a hint, or at least suggest some other diagnostic, which would reveal particular error, more specific than "fatal alert".

Thanks.

EDIT 1: SecTrustEvaluate always returns kSecTrustResultRecoverableTrustFailure, that is why i had to find some kind of workaround.

Solution 1

This issue was solved some time ago. It turned out to be invalid self-signed certificate. It didn't meet all requirements from Apple. Unfortunately i don't know, what exactly it was.

Solution 2

According to this: https://forums.developer.apple.com/message/36842#36842

The best approach to fix HTTP load failed (kCFStreamErrorDomainSSL, -9802) is to set an exception in the info.plist file as follows:

<key>NSAppTransportSecurity</key>
<dict>
  <key>NSExceptionDomains</key>
  <dict>
    <key>test.testdomain.com</key>
    <dict>
      <key>NSIncludesSubdomains</key>
      <true/>
      <key>NSExceptionAllowsInsecureHTTPLoads</key>
      <true/>
    </dict>
  </dict>
</dict>

The important point is that this is not less secure than iOS8, just not as secure as full ATS supported by iOS9.

Solution 3

Have you used nscurl to diagnose the connection issue? If you have a Mac running OS X v10.11 you can run something like this:

/usr/bin/nscurl --ats-diagnostics https://www.yourdomain.com

Alternatively, if you don't have 10.11, you can download the sample code here: https://developer.apple.com/library/mac/samplecode/SC1236/ and build it with XCode and run it like this (changing the path as appropriate for your machine):

/Users/somebody/Library/Developer/Xcode/DerivedData/TLSTool-hjuytnjaqebcfradighsrffxxyzq/Build/Products/Debug/TLSTool s_client -connect www.yourdomain.com:443

(To find the full path for the above, after you've built, open the Products group in your Project Navigator, right click on TLSTool, and "Show in Finder".)

You already linked to Apple's technote on this subject, https://developer.apple.com/library/prerelease/ios/technotes/App-Transport-Security-Technote/ but you didn't say if you ran nscurl or not.

Comments

[dlw]

Thanks - worked great after closing the dict tags.

[alfared]

Thanks a lot, but: 1. That allows ordinary HTTP download, and i need HTTPS (for safety reasons). 2. As i wrote, i can't not set any domain in exception list, because our customers often install back-end on their own servers. For example, it can be "server.company1.com" and "server.company2.com". Your answer assumes that I have to re-build app and add a new exception domain "server.company3.com" after we sell app to third company.

[spirographer]

You may be able to use a combination of information from Apple's documentation on ATS <developer.apple.com/library/prerelease/ios/technotes/…> as well as this blog post on self signed certificates #5 in particular <blog.httpwatch.com/2013/12/12/…> to solve your problem (you will probably need to iterate a few times until you get it right).

[Aaron]

@spirographer both your links are dead

[spirographer]

You are right (an extra > got captured in the URL), and I can't seem to edit the comment so, I'm adding them here again. developer.apple.com/library/prerelease/ios/technotes/… and also blog.httpwatch.com/2013/12/12/…

[ness-EE]

My VM's endpoint passes all the tests in /usr/bin/nscurl --ats-diagnostics https://mydomain.local:8888 but I still receive the error in the Simulator.

[new2ios]

This is not working for me (iOS 9.3.3). I have to install certificate manually, but that solution is not ok for me. I need to disable self-signed certificates in general.