How to sign an Amazon web service request in .NET with SOAP and without WSE

Question

The Amazon Product Advertising API (formerly Amazon Associates Web Service or Amazon AWS) has implemented a new rule which is by August 15th 2009 all web service requests to them must be signed. They have provided sample code on their site showing how to do this in C# using both REST and SOAP. The implementation that I’m using is SOAP. You can find the sample code here, I’m not including it because there is a fair amount.

The problem I’m having is their sample code uses WSE 3 and our current code doesn’t use WSE. Does anyone know how to implement this update with just using the auto generated code from the WSDL? I’d like to not have to switch over to the WSE 3 stuff right now if I don’t have to since this update is more of a quick patch to hold us over until we can fully implement this in the current dev version (August 3rd they’re starting to drop 1 in 5 requests, in the live environment, if they aren’t signed which is bad news for our application).

Here’s a snippet of the main portion that does the actual signing of the SOAP request.

class ClientOutputFilter : SoapFilter
{
    // to store the AWS Access Key ID and corresponding Secret Key.
    String akid;
    String secret;

    // Constructor
    public ClientOutputFilter(String awsAccessKeyId, String awsSecretKey)
    {
        this.akid = awsAccessKeyId;
        this.secret = awsSecretKey;
    }

    // Here's the core logic:
    // 1. Concatenate operation name and timestamp to get StringToSign.
    // 2. Compute HMAC on StringToSign with Secret Key to get Signature.
    // 3. Add AWSAccessKeyId, Timestamp and Signature elements to the header.
    public override SoapFilterResult ProcessMessage(SoapEnvelope envelope)
    {
        var body = envelope.Body;
        var firstNode = body.ChildNodes.Item(0);
        String operation = firstNode.Name;

        DateTime currentTime = DateTime.UtcNow;
        String timestamp = currentTime.ToString("yyyy-MM-ddTHH:mm:ssZ");

        String toSign = operation + timestamp;
        byte[] toSignBytes = Encoding.UTF8.GetBytes(toSign);
        byte[] secretBytes = Encoding.UTF8.GetBytes(secret);
        HMAC signer = new HMACSHA256(secretBytes);  // important! has to be HMAC-SHA-256, SHA-1 will not work.

        byte[] sigBytes = signer.ComputeHash(toSignBytes);
        String signature = Convert.ToBase64String(sigBytes); // important! has to be Base64 encoded

        var header = envelope.Header;
        XmlDocument doc = header.OwnerDocument;

        // create the elements - Namespace and Prefix are critical!
        XmlElement akidElement = doc.CreateElement(
            AmazonHmacAssertion.AWS_PFX, 
            "AWSAccessKeyId", 
            AmazonHmacAssertion.AWS_NS);
        akidElement.AppendChild(doc.CreateTextNode(akid));

        XmlElement tsElement = doc.CreateElement(
            AmazonHmacAssertion.AWS_PFX,
            "Timestamp",
            AmazonHmacAssertion.AWS_NS);
        tsElement.AppendChild(doc.CreateTextNode(timestamp));

        XmlElement sigElement = doc.CreateElement(
            AmazonHmacAssertion.AWS_PFX,
            "Signature",
            AmazonHmacAssertion.AWS_NS);
        sigElement.AppendChild(doc.CreateTextNode(signature));

        header.AppendChild(akidElement);
        header.AppendChild(tsElement);
        header.AppendChild(sigElement);

        // we're done
        return SoapFilterResult.Continue;
    }
}

And that gets called like this when making the actual web service call

// create an instance of the serivce
var api = new AWSECommerceService();

// apply the security policy, which will add the require security elements to the
// outgoing SOAP header
var amazonHmacAssertion = new AmazonHmacAssertion(MY_AWS_ID, MY_AWS_SECRET);
api.SetPolicy(amazonHmacAssertion.Policy());

Solution 1

I ended up updating the code to use WCF since that's what it is in the current dev version I've been working on. Then I used some code that was posted on the Amazon forums, but made it a little easier to use.

UPDATE: new easier to use code that lets you still use the config settings for everything

In the previous code I posted, and what I've seen elsewhere, when the service object is created one of the constructor overrides is used to tell it to use HTTPS, give it the HTTPS url and to manually attach the message inspector that will do the signing. The downfall to not using the default constructor is you lose the ability to configure the service via the config file.

I've since redone this code so you can continue to use the default, parameterless, constructor and configure the service via the config file. The benifit of this is you don't have to recompile your code to use this, or make changes once deployed such as to maxStringContentLength (which is what caused this change to take place as well as discover the downfalls to doing it all in code). I also updated the signing part a bit so that way you can tell it what hashing algorithm to use as well as the regex for extracting the Action.

These two changes are because not all web services from Amazon use the same hashing algorithm and the Action might need to be extracted differently. This means you can reuse the same code for each service type just by changing what’s in the config file.

public class SigningExtension : BehaviorExtensionElement
{
    public override Type BehaviorType
    {
        get { return typeof(SigningBehavior); }
    }

    [ConfigurationProperty("actionPattern", IsRequired = true)]
    public string ActionPattern
    {
        get { return this["actionPattern"] as string; }
        set { this["actionPattern"] = value; }
    }

    [ConfigurationProperty("algorithm", IsRequired = true)]
    public string Algorithm
    {
        get { return this["algorithm"] as string; }
        set { this["algorithm"] = value; }
    }

    [ConfigurationProperty("algorithmKey", IsRequired = true)]
    public string AlgorithmKey
    {
        get { return this["algorithmKey"] as string; }
        set { this["algorithmKey"] = value; }
    }

    protected override object CreateBehavior()
    {
        var hmac = HMAC.Create(Algorithm);
        if (hmac == null)
        {
            throw new ArgumentException(string.Format("Algorithm of type ({0}) is not supported.", Algorithm));
        }

        if (string.IsNullOrEmpty(AlgorithmKey))
        {
            throw new ArgumentException("AlgorithmKey cannot be null or empty.");
        }

        hmac.Key = Encoding.UTF8.GetBytes(AlgorithmKey);

        return new SigningBehavior(hmac, ActionPattern);
    }
}

public class SigningBehavior : IEndpointBehavior
{
    private HMAC algorithm;

    private string actionPattern;

    public SigningBehavior(HMAC algorithm, string actionPattern)
    {
        this.algorithm = algorithm;
        this.actionPattern = actionPattern;
    }

    public void Validate(ServiceEndpoint endpoint)
    {
    }

    public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
    {
    }

    public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
    {
    }

    public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
    {
        clientRuntime.MessageInspectors.Add(new SigningMessageInspector(algorithm, actionPattern));
    }
}

public class SigningMessageInspector : IClientMessageInspector
{
    private readonly HMAC Signer;

    private readonly Regex ActionRegex;

    public SigningMessageInspector(HMAC algorithm, string actionPattern)
    {
        Signer = algorithm;
        ActionRegex = new Regex(actionPattern);
    }

    public void AfterReceiveReply(ref Message reply, object correlationState)
    {
    }

    public object BeforeSendRequest(ref Message request, IClientChannel channel)
    {
        var operation = GetOperation(request.Headers.Action);
        var timeStamp = DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ");
        var toSignBytes = Encoding.UTF8.GetBytes(operation + timeStamp);
        var sigBytes = Signer.ComputeHash(toSignBytes);
        var signature = Convert.ToBase64String(sigBytes);

        request.Headers.Add(MessageHeader.CreateHeader("AWSAccessKeyId", Helpers.NameSpace, Helpers.AWSAccessKeyId));
        request.Headers.Add(MessageHeader.CreateHeader("Timestamp", Helpers.NameSpace, timeStamp));
        request.Headers.Add(MessageHeader.CreateHeader("Signature", Helpers.NameSpace, signature));

        return null;
    }

    private string GetOperation(string request)
    {
        var match = ActionRegex.Match(request);
        var val = match.Groups["action"];
        return val.Value;
    }
}

To use this you don't need to make any changes to your existing code, you can even put the signing code in a whole other assembly if need be. You just need to set up the config section as so (note: the version number is important, without it matching the code will not load or run)

<system.serviceModel>
  <extensions>
    <behaviorExtensions>
      <add name="signer" type="WebServices.Amazon.SigningExtension, AmazonExtensions, Version=1.3.11.7, Culture=neutral, PublicKeyToken=null" />
    </behaviorExtensions>
  </extensions>
  <behaviors>
    <endpointBehaviors>
      <behavior name="AWSECommerceBehaviors">
        <signer algorithm="HMACSHA256" algorithmKey="..." actionPattern="\w:\/\/.+/(?&lt;action&gt;.+)" />
      </behavior>
    </endpointBehaviors>
  </behaviors>
  <bindings>
    <basicHttpBinding>
      <binding name="AWSECommerceServiceBinding" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" useDefaultWebProxy="true" maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536">
        <readerQuotas maxDepth="32" maxStringContentLength="16384" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384" />
        <security mode="Transport">
          <transport clientCredentialType="None" proxyCredentialType="None" realm="" />
          <message clientCredentialType="UserName" algorithmSuite="Default" />
        </security>
      </binding>
    </basicHttpBinding>
  </bindings>
  <client>
    <endpoint address="https://ecs.amazonaws.com/onca/soap?Service=AWSECommerceService" behaviorConfiguration="AWSECommerceBehaviors" binding="basicHttpBinding" bindingConfiguration="AWSECommerceServiceBinding" contract="WebServices.Amazon.AWSECommerceServicePortType" name="AWSECommerceServicePort" />
  </client>
</system.serviceModel>

Solution 2

Hey Brian, I'm dealing with the same issue in my app. I'm using the WSDL generated code -- in fact I generated it again today to ensure the latest version. I found that signing with an X509 certificate the most straightforward path. With a few minutes of testing under my belt, so far it appears to work okay. Essentially you change from:

AWSECommerceService service = new AWSECommerceService();
// ...then invoke some AWS call

To:

AWSECommerceService service = new AWSECommerceService();
service.ClientCertificates.Add(X509Certificate.CreateFromCertFile(@"path/to/cert.pem"));
// ...then invoke some AWS call

Viper at bytesblocks.com posted more details, including how to obtain the X509 certificate Amazon generates for you.

EDIT: as the discussion here indicates, this might not actually sign the request. Will post as I learn more.

EDIT: this doesn't appear to sign the request at all. Instead, it appears to require an https connection, and uses the certificate for SSL client authentication. SSL client authentication is an infrequently used feature of SSL. It would have been nice if the Amazon product advertising API supported it as an authentication mechanism! Unfortunately that doesn't seem to be the case. The evidence is twofold: (1) it's not one of the documented authentication schemes, and (2) it doesn't matter what certificate you specify.

Some confusion is added by Amazon still not enforcing authentication on requests even after their proclaimed the August 15 2009 deadline. This makes requests appear to pass correctly when the certificate is added, even though it might not add any value.

Look at Brian Surowiec's answer for a solution that works. I'm leaving this answer here to document the appealing but apparently failed approach, as I can still see it discussed in blogs and Amazon forums.