Can't connect to SSL web service with WS-Security using PHP SOAP extension - certificate, complex WSDL

Question

Using the PHP5 SOAP extension I have been unable to connect to a web service having an https endpoint, with client certificate and using WS-Security, although I can connect using soapUI with the exact same wsdl and client certificate, and obtain the normal response to the request. There is no HTTP authentication and no proxy is involved. The message I get is 'Could not connect to host'. Have been able to verify that I am NOT hitting the host server. (Earlier I wrongly said that I was hitting the server.)

The self-signed client SSL certificate is a .pem file converted by openssl from a .p12 keystore which in turn was converted by keytool from a .jks keystore having a single entry consisting of private key and client certificate.

In soapUI I did not need to supply a server private certificate, the only two files I gave it were the wdsl and pem. I did have to supply the pem and its passphrase to be able to connect. I am speculating that despite the error message my problem might actually be in the formation of the XML request rather than the SSL connection itself.

The wsdl I have been given has nested complex types. The php server is on my Windows XP laptop with IIS.

The code, data values and WSDL extracts are shown below. (The WSSoapClient class simply extends SoapClient, adding a WS-Security Username Token header with mustUnderstand = true and including a nonce, both of which the soapUI call had required.)

Would so much appreciate any help. I'm a newbie thrown in at the deep end, and how! Have done vast amounts of Googling on this over many days, following many suggestions and have read Pro PHP by Kevin McArthur. An attempt to use classmaps in place of nested arrays also fell flat.

---

The Code

class STEeService
{


public function invokeWebService(array $connection, $operation, array $request)
 {
  try
   {  
    $localCertificateFilespec = $connection['localCertificateFilespec'];
$localCertificatePassphrase = $connection['localCertificatePassphrase'];

$sslOptions = array(
   'ssl' => array(
     'local_cert' => $localCertificateFilespec,
     'passphrase' => $localCertificatePassphrase,
     'allow_self-signed' => true,
     'verify_peer' => false
             )
          );  
$sslContext = stream_context_create($sslOptions);

$clientArguments = array(
    'stream_context' => $sslContext,
    'local_cert' => $localCertificateFilespec,    
    'passphrase' => $localCertificatePassphrase,
    'trace' => true,
    'exceptions' => true,   
    'encoding' => 'UTF-8',
    'soap_version' => SOAP_1_1
   );

$oClient = new WSSoapClient($connection['wsdlFilespec'], $clientArguments); 
$oClient->__setUsernameToken($connection['username'], $connection['password']);        

   return $oClient->__soapCall($operation, $request);      
   }
   catch (exception $e)
   {
    throw new Exception("Exception in eServices " . $operation . " ," . $e->getMessage(), "\n");
   }

 }
}

---

$connection is as follows:

array(5) { ["username"]=> string(8) "DFU00050" 
["password"]=> string(10) "Fabricate1" 
["wsdlFilespec"]=> 
string (63) "c:/inetpub/wwwroot/DMZExternalService_Concrete_WSDL_Staging.xml" 
["localCertificateFilespec"]=> string(37) 
"c:/inetpub/wwwroot/ClientKeystore.pem"
["localCertificatePassphrase"]=> string(14) "password123456" }

$clientArguments is as follows:

array(7) { ["stream_context"]=> resource(8) of type (stream-context) 
["local_cert"]=> string(37) "c:/inetpub/wwwroot/ClientKeystore.pem" 
["passphrase"]=> string(14) "password123456" 
["trace"]=> bool(true) ["exceptions"]=> bool(true) ["encoding"]=> string(5) "UTF-8" 
["soap_version"]=> int(1) }

$operation is as follows:

'getConsignmentDetails'

$request is as follows:

array(1) { [0]=> array(2) { ["header"]=> array(2) { 
["source"]=> string(9) "customerA" ["accountNo"]=> string(8) "10072906" } 
["consignmentId"]=> string(11) "GKQ00000085" } }

Note how there is an extra level of nesting, an array wrapping the request which is itself an array. This was suggested in a post although I don't see the reason, but it seems to help avoid other exceptions.

---

The exception thrown by ___soapCall is as follows:

    object(SoapFault)#6 (9) { ["message":protected]=> 
string(25) "Could not connect to host" ["string":"Exception":private]=> string(0) "" 
    ["code":protected]=> int(0) ["file":protected]=> string(43) "C:\Inetpub\wwwroot\eServices\WSSecurity.php" 
    ["line":protected]=> int(85) ["trace":"Exception":private]=> array(5) { [0]=> array(6) { 
    ["file"]=> string(43) "C:\Inetpub\wwwroot\eServices\WSSecurity.php" ["line"]=> int(85) ["function"]=> string(11) "__doRequest" 
    ["class"]=> string(10) "SoapClient" ["type"]=> string(2) "->" ["args"]=> array(4) { 
    [0]=> string(1240) " DFU00050 Fabricate1 E0ByMUA= 2010-10-28T13:13:52Z customerA10072906GKQ00000085 " 
    [1]=> string(127) "https://services.startrackexpress.com.au:7560/DMZExternalService/InterfaceServices/ExternalOps.serviceagent/OperationsEndpoint1" 
    [2]=> string(104) "/DMZExternalService/InterfaceServices/ExternalOps.serviceagent/OperationsEndpoint1/getConsignmentDetails" [3]=> int(1) } } 
    [1]=> array(4) { ["function"]=> string(11) "__doRequest" ["class"]=> string(39) "startrackexpress\eservices\WSSoapClient" 
    ["type"]=> string(2) "->" ["args"]=> array(5) { [0]=> string(1240) " DFU00050 Fabricate1 E0ByMUA= 2010-10-28T13:13:52Z customerA10072906GKQ00000085 " 
    [1]=> string(127) "https://services.startrackexpress.com.au:7560/DMZExternalService/InterfaceServices/ExternalOps.serviceagent/OperationsEndpoint1" 
    [2]=> string(104) "/DMZExternalService/InterfaceServices/ExternalOps.serviceagent/OperationsEndpoint1/getConsignmentDetails" [3]=> int(1) [4]=> int(0) } }
    [2]=> array(6) { ["file"]=> string(43) "C:\Inetpub\wwwroot\eServices\WSSecurity.php" ["line"]=> int(70) ["function"]=> string(10) "__soapCall" 
    ["class"]=> string(10) "SoapClient" ["type"]=> string(2) "->" ["args"]=> array(4) { [0]=> string(21) "getConsignmentDetails" [1]=> array(1) { 
    [0]=> array(2) { ["header"]=> array(2) { ["source"]=> string(9) "customerA" ["accountNo"]=> string(8) "10072906" } 
    ["consignmentId"]=> string(11) "GKQ00000085" } } [2]=> NULL [3]=> object(SoapHeader)#5 (4) { 
    ["namespace"]=> string(81) "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" ["name"]=> string(8) "Security" 
    ["data"]=> object(SoapVar)#4 (2) { ["enc_type"]=> int(147) ["enc_value"]=> string(594) " DFU00050 Fabricate1 E0ByMUA= 2010-10-28T13:13:52Z " } 
    ["mustUnderstand"]=> bool(true) } } } [3]=> array(6) { ["file"]=> string(42) "C:\Inetpub\wwwroot\eServices\eServices.php" 
    ["line"]=> int(87) ["function"]=> string(10) "__soapCall" ["class"]=> string(39) "startrackexpress\eservices\WSSoapClient" 
    ["type"]=> string(2) "->" ["args"]=> array(2) { [0]=> string(21) "getConsignmentDetails" [1]=> array(1) { [0]=> array(2) { 
    ["header"]=> array(2) { ["source"]=> string(9) "customerA" ["accountNo"]=> string(8) "10072906" } ["consignmentId"]=> string(11) "GKQ00000085" } } } } 
    [4]=> array(6) { ["file"]=> string(58) "C:\Inetpub\wwwroot\eServices\EnquireConsignmentDetails.php" ["line"]=> int(44) 
    ["function"]=> string(16) "invokeWebService" ["class"]=> string(38) "startrackexpress\eservices\STEeService" ["type"]=> string(2) "->" 
    ["args"]=> array(3) { [0]=> array(5) { ["username"]=> string(10) "DFU00050 " ["password"]=> string(12) "Fabricate1 " 
    ["wsdlFilespec"]=> string(63) "c:/inetpub/wwwroot/DMZExternalService_Concrete_WSDL_Staging.xml" 
    ["localCertificateFilespec"]=> string(37) "c:/inetpub/wwwroot/ClientKeystore.pem" ["localCertificatePassphrase"]=> string(14) "password123456" } 
    [1]=> string(21) "getConsignmentDetails" [2]=> array(1) { [0]=> array(2) { ["header"]=> array(2) { ["source"]=> string(9) "customerA" 
    ["accountNo"]=> string(8) "10072906" } ["consignmentId"]=> string(11) "GKQ00000085" } } } } } 
    ["previous":"Exception":private]=> NULL ["faultstring"]=> string(25) "Could not connect to host" ["faultcode"]=> string(4) "HTTP" }

---

Here are some WSDL extracts (TIBCO BusinessWorks):

            <xsd:complexType name="TransactionHeaderType">
            <xsd:sequence>
                <xsd:element name="source" type="xsd:string"/>
                <xsd:element name="accountNo" type="xsd:integer"/>
                <xsd:element name="userId" type="xsd:string" minOccurs="0"/>
                <xsd:element name="transactionId" type="xsd:string" minOccurs="0"/>
                <xsd:element name="transactionDatetime" type="xsd:dateTime" minOccurs="0"/>
            </xsd:sequence>
        </xsd:complexType>

---

       <xsd:element name="getConsignmentDetailRequest">
            <xsd:complexType>
                <xsd:sequence>
                    <xsd:element name="header" type="prim:TransactionHeaderType"/>
                    <xsd:element name="consignmentId" type="prim:ID" maxOccurs="unbounded"/>
                </xsd:sequence>
            </xsd:complexType>
        </xsd:element>
        <xsd:element name="getConsignmentDetailResponse">
            <xsd:complexType>
                <xsd:sequence>
                    <xsd:element name="consignment" type="freight:consignmentType" minOccurs="0" maxOccurs="unbounded"/>
                </xsd:sequence>
            </xsd:complexType>
        </xsd:element>

---

        <xsd:element name="getConsignmentDetailRequest">
            <xsd:complexType>
                <xsd:sequence>
                    <xsd:element name="header" type="prim:TransactionHeaderType"/>
                    <xsd:element name="consignmentId" type="prim:ID" maxOccurs="unbounded"/>
                </xsd:sequence>
            </xsd:complexType>
        </xsd:element>
        <xsd:element name="getConsignmentDetailResponse">
            <xsd:complexType>
                <xsd:sequence>
                    <xsd:element name="consignment" type="freight:consignmentType" minOccurs="0" maxOccurs="unbounded"/>
                </xsd:sequence>
            </xsd:complexType>
        </xsd:element>

---

    <wsdl:operation name="getConsignmentDetails">
        <wsdl:input message="tns:getConsignmentDetailsRequest"/>
        <wsdl:output message="tns:getConsignmentDetailsResponse"/>
        <wsdl:fault name="fault1" message="tns:fault"/>
    </wsdl:operation>

---

<wsdl:service name="ExternalOps">
    <wsdl:port name="OperationsEndpoint1" binding="tns:OperationsEndpoint1Binding">
        <soap:address location="https://services.startrackexpress.com.au:7560/DMZExternalService/InterfaceServices/ExternalOps.serviceagent/OperationsEndpoint1"/>
    </wsdl:port>
</wsdl:service>

---

And here in case it's relevant is the WSSoapClient class:

    <?PHP
namespace startrackexpress\eservices;
use SoapClient, SoapVar, SoapHeader;

class WSSoapClient extends SoapClient
{
 private $username;
 private $password;

/*Generates a WS-Security header*/
 private function wssecurity_header()
 {
  $timestamp = gmdate('Y-m-d\TH:i:s\Z');
  $nonce = mt_rand(); 
  $passdigest = base64_encode(pack('H*', sha1(pack('H*', $nonce).pack('a*', $timestamp).pack('a*', $this->password))));

  $auth = '
<wsse:Security SOAP-ENV:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
<wsse:UsernameToken>
    <wsse:Username>' . $this->username . '</wsse:Username>
    <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">' . 
 $this->password . '</wsse:Password>
    <wsse:Nonce>' . base64_encode(pack('H*', $nonce)).'</wsse:Nonce>
    <wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">' . $timestamp . '</wsu:Created>
   </wsse:UsernameToken>
</wsse:Security>
';
  $authvalues = new SoapVar($auth, XSD_ANYXML); 
  $header = new SoapHeader("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd", "Security",$authvalues, true);

  return $header;
 }

 // Sets a username and passphrase
 public function __setUsernameToken($username,$password)
 {
  $this->username=$username;
  $this->password=$password;
 }

 // Overwrites the original method, adding the security header
 public function __soapCall($function_name, $arguments, $options=null, $input_headers=null, $output_headers=null)
 {
  try
  {
    $result = parent::__soapCall($function_name, $arguments, $options, $this->wssecurity_header());
    return $result;
  }
  catch (exception $e)
  {
   throw new Exception("Exception in __soapCall, " . $e->getMessage(), "\n");
  }
 }
}
?>

Update:

The request XML would have been as follows:

<?xml version="1.0" encoding="UTF-8"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://startrackexpress/Common/Primitives/v1" xmlns:ns2="http://startrackexpress/Common/actions/externals/Consignment/v1" xmlns:ns3="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
    <SOAP-ENV:Header> <wsse:Security SOAP-ENV:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> 
<wsse:UsernameToken> <wsse:Username>DFU00050</wsse:Username> 
    <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">Fabricate1</wsse:Password> 
    <wsse:Nonce>M4FIeGA=</wsse:Nonce> 
    <wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2010-10-29T14:05:27Z</wsu:Created> 
    </wsse:UsernameToken> 
    </wsse:Security> </SOAP-ENV:Header>
    <SOAP-ENV:Body><ns2:getConsignmentDetailRequest>
    <ns2:header><ns1:source>customerA</ns1:source><ns1:accountNo>10072906</ns1:accountNo></ns2:header>
    <ns2:consignmentId>GKQ00000085</ns2:consignmentId>
    </ns2:getConsignmentDetailRequest></SOAP-ENV:Body>
    </SOAP-ENV:Envelope>

This was obtained with the following code in WSSoapClient:

public function __doRequest($request, $location, $action, $version)         {
    echo "<p> " . htmlspecialchars($request) . " </p>" ;    
    return parent::__doRequest($request, $location, $action, $version);
}

Solution 1

For anyone else struggling with the startrack API. Here is class I wrote to go via CURL instead.

Instructions: 

Add the attached file to:
Client/Executables 

Change line 28 from 

class WSSoapClient extends SoapClient

To:

require('SoapClientCurl.class.php');
class WSSoapClient extends SoapClientCurl

<?php

/**
 * Override to overcome problems with Startrack Self Signed SSL Certificates on
 * certain server configurations.
 *
 * The important options here that aren't available in the SoapClient options are
 * CURLOPT_SSLVERSION       - Forces the SSl Version to 3
 * CURLOPT_SSL_VERIFYHOST   - Tells ssl not to care that the Startrack SSL certificate is for a different domain
 * CURLOPT_SSL_VERIFYPEER   - Tells ssl not to care that the Startrack SSL certificate is from a bogus CA (I think)
 *
 */
class SoapClientCurl extends SoapClient
{
    /**
     *
     * @param string $request       - The XML SOAP request.
     * @param string $location      - The URL to request.
     * @param string $action        - The SOAP action.
     * @param int $version          - The SOAP version.
     * @param boolean $one_way      - If one_way is set to 1, this method returns nothing. Use this where a response is not expected.
     * @throws SoapFault
     * @return string|void
     */
    public function __doRequest($request, $location, $action, $version, $one_way = 0)
    {
        $handle = curl_init();

        curl_setopt($handle, CURLOPT_URL, $location);
        curl_setopt($handle, CURLOPT_HTTPHEADER, array(
                'Content-type: text/xml;charset="utf-8"',
                'Accept: text/xml',
                'Cache-Control: no-cache',
                'Pragma: no-cache',
                'SOAPAction: '.$action,
                'Content-length: '.strlen($request))
        );

        curl_setopt($handle, CURLOPT_RETURNTRANSFER,    true);
        curl_setopt($handle, CURLOPT_POSTFIELDS,        $request);
        curl_setopt($handle, CURLOPT_SSLVERSION,        3);
        curl_setopt($handle, CURLOPT_PORT,              443);
        curl_setopt($handle, CURLOPT_POST,              true );
        curl_setopt($handle, CURLOPT_SSL_VERIFYHOST,    false);
        curl_setopt($handle, CURLOPT_SSL_VERIFYPEER,    false);

        $response = curl_exec($handle);

        if(empty($response))
        {
            throw new SoapFault('CURL error: '.curl_error($handle), curl_errno($handle));
        }

        curl_close($handle);

        if(1 !== $one_way)
        {
            return $response;
        }
    }
}

Solution 2

When I ran the code under Apache (using XAMPP) it worked first time. My problem must have been somewhere in the configuration of IIS or PHP.

There was one typo but it wasn't the showstopper:

'allow_self-signed' => true

should be

'allow_self_signed' => true

Comments

[BillF]

Thanks Robin! No luck though unfortunately. I removed the stream context, no change. Then after restoring the stream context I removed the duplicate local_cert and passphrase from the SoapClient params, no luck. I had the duplicates in there only because in a posted working example it was that way. The cert does not seem to add to the security in the current service implementation (since it can be self-signed and no CSR is required) but at the moment the service requires it to be present, as I verified with soapUI.

[Robin]

I'm all out of technical suggestions! One thing to bear in mind is that the PHP SOAP libs are, ahem, not entirely of the very highest quality, so don't rule out the possibility that you've found a bug in the PHP SOAP implementation. Maybe the PHP SOAP client messes up the SSL handshake when using a client cert? If this is the case there might be some info in your service provider's error logs.

[BillF]

Thanks again, Robin. Following that up.