InvalidCiphertextException when calling kms.decrypt with S3 metadata

Question

I can add a client-side encrypted file via the Java SDK, and I can fetch the file too. I'm now trying to access it with boto3. (I know boto3 doesn't have support for this, but s3-wrapper does. This concerns boto3, though).

I'm getting the s3 metadata and then calling kms.decrypt like this:

object_info = s3.head_object(Bucket=bucket_name, Key=key_name)
metadata = object_info['Metadata'] # metadata is a dict with lots of x-amz-key, x-amz-iv, etc
ekey = kms.decrypt(CiphertextBlob=metadata,EncryptionContext=metadata)

# fails with:
# ParamValidationError:
# Parameter validation failed: Invalid type for parameter CiphertextBlob, value: .. type: <class 'dict'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object`

So, what if I should be passing in the key as the CiphertextBlob?

# this key looks like 'CiAlgoyM4k...
ekey = kms.decrypt(CiphertextBlob=metadata['x-amz-key-v2'],EncryptionContext=metadata)

# fails with:
# botocore.exceptions.ClientError: An error occurred (InvalidCiphertextException) when calling the Decrypt operation: None

Then I tried passing in a base64'd key:

# this key looks like b'\n %\x82\x8c\x8c\xe2ML...
cblob = base64.b64decode(metadata['x-amz-key-v2'])
ekey = kms.decrypt(CiphertextBlob=cblob,EncryptionContext=metadata)

# (still) fails with:
# botocore.exceptions.ClientError: An error occurred (InvalidCiphertextException) when calling the Decrypt operation: None

I then tried passing in the s3 contents as the blob.

full_object = s3.get_object(Bucket=bucket_name, Key=key_name)
ekey = kms.decrypt(CiphertextBlob=full_object['Body'].read(),EncryptionContext=metadata)

# Still fails with:
# botocore.exceptions.ClientError: An error occurred (InvalidCiphertextException) when calling the Decrypt operation: None

So, I guess my question is, what is a proper CiphertextBlob from the S3 metadata? Since I'm placing it with the Java SDK's EncryptedPutObjectRequest and AmazonS3EncryptionClient, which abstracts it away, I don't know what that blob should look like.

related links

• stackoverflow: client-side decryption in ruby; does not use KMS
• boto3 github issues: the start of kms decryption in boto3 for s3; does not work, perhaps because this is a "v2" key?

Solution

From your comments, I'm almost sure you encrypted the file using envelope encryption, and not a customer master key (# metadata is a dict with lots of x-amz-key, x-amz-iv, etc). Another issue is that you are passing an encryption context, but always making it be the entire dictionary. Is this your intention?

So, my suggestions are:

1. Make sure you call kms.decrypt on your envelope key, and then actually decrypt the data with the decrypted key (assuming my comment above is correct).
2. Double-check you're passing what you want on your encryption context.