Parsing secrets from AWS secrets manager using AWS cli

Question

I am retrieving secrets I have stored in AWS secrets manager with the AWS cli like this:

aws secretsmanager get-secret-value --secret-id secrets

Which returns

arn:aws:secretsmanager<ID>:secret:my_secrets <number> my_secrets {"API_KEY":"ABCDEFGHI"}       <UUID string>
VERSIONSTAGES   AWSCURRENT

Does anyone know how I only get the secret ("API_KEY": "ABCDEFGHI")? I need to move these secrets to my register-task-definition environment variables. The best way would be to store them in a file and delete it after us or store them in variable. It is running on a linux machine.

Solution 1

Use the --query option of the CLI to extract just the secret.

aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text

Solution 2

aws secretsmanager get-secret-value --secret-id secrets| jq --raw-output '.SecretString' | jq -r .API_KEY

using jq you can print.

Solution 3

Small addition to helloV answer. You can add the output parameter text to remove the quotes.

aws secretsmanager get-secret-value \
   --secret-id secrets \
   --query SecretString \
   --output text

Solution 4

When you have multiple secret and you get json return, you can use get the exact value of password by using

aws secretsmanager get-secret-value --secret-id <secret_bucket_name> | jq --raw-output '.SecretString' | jq -r .key_for_password

Solution 5

So I faced a bit of trouble in extracting what I needed, the value for my two variables that I stored in SecretsManager. So here is what worked for me.

NOTE: It's an example from the AWS SecretsManager doc.

I ran this

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS

The response of this query is:

{
  "ARN": "arn:aws:secretsmanager:us-west-2:123456789012:secret:MyTestDatabaseSecret-a1b2c3",
  "Name": "MyTestDatabaseSecret",
  "VersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE",
  "SecretString": "{\n  \"username\":\"david\",\n  \"password\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",
  "VersionStages": [
    "AWSPREVIOUS"
  ],
  "CreatedDate": 1523477145.713
}

Now I want to get the value of username or password to be precise

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS | jq --raw-output .SecretString | jq -r ."password"

Output

BnQw&XDWgaEeT9XGTT29

View more solutions

Comments

[Cale]

Is this no longer working? every secret is null for me

[Cale]

Ah SecretString selectes the entire secret string, it is not used to match a secret key

[Balaji Boggaram Ramanarayan]

jq is not a standard util

[committedandroider]

@caleb Just running the command resulted in that error or w -query did it work?

[Aseem Jain]

jq is json utility for shell command. it helps to parse the Json and pull the attribute from the file.

[crypdick]

this is a copy-paste of this answer stackoverflow.com/a/52921217/4212158

[Yann Stoneman]

And if your secret has a dash, you can use: aws secretsmanager get-secret-value --secret-id my-secret --query SecretString --output text | jq '."my-secret"'

[Yann Stoneman]

Correction, to remove quotes in addition to making it work for a secret with a dash: aws secretsmanager get-secret-value --secret-id my-secret --query SecretString --output text | jq -r '."my-secret"'

[GwenM]

you're missing a "\" before the last }, like tr -d \"\}

[Jon]

@GwenM not needed on GNU/Bash

[GwenM]

ah yeah you might be right, I was using ZSH on mac

[Auro Sarma]

In my case, The password has '&' symbol and is converted to '\u0026'. Is there any workaround for this?

[Ashish Karpe]

I want to list all secrets and then get all values and run grep against it

[Ashish Karpe]

> aws secretsmanager get-secret-value --secret-id * is not working for me

[thecloudguy]

what is the error message? Do you have access rights to all? Are you passing the version number in your query? Instead of using * wild character, can you make a list of secret-id and then try to run a loop. First, try to run it for 10 odd secrets and then check to scale it up.

[Ashish Karpe]

github.com/ashishkarpe/scripts_aws_cli/blob/main/… have written script which worked for me thanks

[thecloudguy]

Did it work? You tested. If yes, then great. :)

[Ashish Karpe]

yes I tested it and it worked

[Pierre]

a good answer which doesn't require jq