How to retrieve Secret Manager data in buildspec.yaml

Question

Im working on creating the CodeBuild which is integrated with SonarQube, So I pass values and sonar credentials directly in my Buildspec.yaml

Instead of Hardcoding directly, I tried to retrieve using the below command from SecretManager as it is mentioned in the below link. But it is not getting the correct values. it throws an error.

Command : '{{resolve:secretsmanager:MyRDSSecret:SecretString:username}}'

Link: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-secretsmanager

Error [ERROR] SonarQube server [{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:********:secret:**********:SecretString:SonarURL}}] can not be reached

How I used echo '{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:***:secret:**************:SecretString:*******}}'

Note: All the * inside my commard are the secretname and secreturl

Solution 1

If you wish to retrieve secrets in your buildspec file, I would recommend to use Systems Manager Parameter Store which is natively integrated with CodeBuild. Systems Manager is a service in itself, search it from the AWS Console homepage, then Paramater Store is in the bottom left of the Systems Manager Console page.

Lets assume you want to include Access Key and Secret Key in buildspec.yml file:
- Create AccessKey/SecretKey pair for a IAM User
- Save the above keys in an SSM parameter store as secure string (e.g. '/CodeBuild/AWS_ACCESS_KEY_ID' and '/CodeBuild/AWS_SECRET_ACCESS_KEY')
- Export the two values in your build environment using the following buildspec directive(s):

version: 0.2
env:
    parameter-store:
        AWS_ACCESS_KEY_ID_PARAM: /CodeBuild/AWS_ACCESS_KEY_ID
        AWS_SECRET_ACCESS_KEY_PARAM: /CodeBuild/AWS_SECRET_ACCESS_KEY

phases:
    build:
        commands:
            - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID_PARAM
            - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY_PARAM
            # Your Ansible commands below
            - ansible-playbook -i hosts ec2-key.yml 

[1] Build Specification Reference for CodeBuild - Build Spec Syntax - https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax

Solution 2

CodeBuild just launched this today - https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codebuild-adds-support-for-aws-secrets-manager/

Solution 3

The dynamic reference syntax you are trying to use only works with the Cloud Formation (CFN) service. In some cases, CFN restricts where these dynamic references to secrets will expand. Specifically, they do not expand in places where the secrets might be visible in the console, such as in EC2 metadata.

If you are trying to setup Code Build via CFN, this may be what you are seeing. However, as shariqmaws mentioned, you can use parameter store and either store your secret there or use parameter store as a pass through to secrets manager (in case you want to use secrets manager to rotate your secrets or for other reasons).