Docker and securing passwords

Question

I've been experimenting with Docker recently on building some services to play around with and one thing that keeps nagging me has been putting passwords in a Dockerfile. I'm a developer so storing passwords in source feels like a punch in the face. Should this even be a concern? Are there any good conventions on how to handle passwords in Dockerfiles?

Solution 1

Definitely it is a concern. Dockerfiles are commonly checked in to repositories and shared with other people. An alternative is to provide any credentials (usernames, passwords, tokens, anything sensitive) as environment variables at runtime. This is possible via the -e argument (for individual vars on the CLI) or --env-file argument (for multiple variables in a file) to docker run. Read this for using environmental with docker-compose.

Using --env-file is definitely a safer option since this protects against the secrets showing up in ps or in logs if one uses set -x.

However, env vars are not particularly secure either. They are visible via docker inspect, and hence they are available to any user that can run docker commands. (Of course, any user that has access to docker on the host also has root anyway.)

My preferred pattern is to use a wrapper script as the ENTRYPOINT or CMD. The wrapper script can first import secrets from an outside location in to the container at run time, then execute the application, providing the secrets. The exact mechanics of this vary based on your run time environment. In AWS, you can use a combination of IAM roles, the Key Management Service, and S3 to store encrypted secrets in an S3 bucket. Something like HashiCorp Vault or credstash is another option.

AFAIK there is no optimal pattern for using sensitive data as part of the build process. In fact, I have an SO question on this topic. You can use docker-squash to remove layers from an image. But there's no native functionality in Docker for this purpose.

You may find shykes comments on config in containers useful.

Solution 2

Our team avoids putting credentials in repositories, so that means they're not allowed in Dockerfile. Our best practice within applications is to use creds from environment variables.

We solve for this using docker-compose.

Within docker-compose.yml, you can specify a file that contains the environment variables for the container:

 env_file:
- .env

Make sure to add .env to .gitignore, then set the credentials within the .env file like:

SOME_USERNAME=myUser
SOME_PWD_VAR=myPwd

Store the .env file locally or in a secure location where the rest of the team can grab it.

See: https://docs.docker.com/compose/environment-variables/#/the-env-file

Solution 3

Docker now (version 1.13 or 17.06 and higher) has support for managing secret information. Here's an overview and more detailed documentation

Similar feature exists in kubernetes and DCOS

Solution 4

You should never add credentials to a container unless you're OK broadcasting the creds to whomever can download the image. In particular, doing and ADD creds and later RUN rm creds is not secure because the creds file remains in the final image in an intermediate filesystem layer. It's easy for anyone with access to the image to extract it.

The typical solution I've seen when you need creds to checkout dependencies and such is to use one container to build another. I.e., typically you have some build environment in your base container and you need to invoke that to build your app container. So the simple solution is to add your app source and then RUN the build commands. This is insecure if you need creds in that RUN. Instead what you do is put your source into a local directory, run (as in docker run) the container to perform the build step with the local source directory mounted as volume and the creds either injected or mounted as another volume. Once the build step is complete you build your final container by simply ADDing the local source directory which now contains the built artifacts.

I'm hoping Docker adds some features to simplify all this!

Update: looks like the method going forward will be to have nested builds. In short, the dockerfile would describe a first container that is used to build the run-time environment and then a second nested container build that can assemble all the pieces into the final container. This way the build-time stuff isn't in the second container. This of a Java app where you need the JDK for building the app but only the JRE for running it. There are a number of proposals being discussed, best to start from https://github.com/docker/docker/issues/7115 and follow some of the links for alternate proposals.

Solution 5

An alternative to using environment variables, which can get messy if you have a lot of them, is to use volumes to make a directory on the host accessible in the container.

If you put all your credentials as files in that folder, then the container can read the files and use them as it pleases.

For example:

$ echo "secret" > /root/configs/password.txt
$ docker run -v /root/configs:/cfg ...

In the Docker container:

# echo Password is `cat /cfg/password.txt`
Password is secret

Many programs can read their credentials from a separate file, so this way you can just point the program to one of the files.

View more solutions

Comments

[Luís Bianchin]

There is an open issue on Github requesting for best practices regarding Docker and secrets, the issue is here: github.com/docker/docker/issues/13490

[Petr Gladkikh]

As noted in other comments there will be 2 layers (after ADD and after first RUN) that contain .config file.

[gnoll110]

Yer, env variables seems the best way to go. I've been looking at this in the context of TDDing Dockerfile development.

[Lie Ryan]

According to Docker's ARG docs: "It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc"

[D. Visser]

You can also do this without a .env file, if you wish. Just use the environment property in your docker-compose.yml file . "Environment variables with only a key are resolved to their values on the machine Compose is running on, which can be helpful for secret or host-specific values."

[equivalent8]

give this man a cookie ! :) yep this is really good practice I just want to add that docs.docker.com/compose/env-file this should work automatically but in docker compose version 2 it seems you need to declare it as described in this answer.

[Tomato]

I'm concerned that if your password is an env variable, it appears in docker inspect.

[Henk Wiersema]

Just wondering why Docker disrecommends to use --build-arg var=secret for passing a SSH private key into an image, there is no rationale documented. Can anyone explain it?

[tudor -Reinstate Monica-]

@HenkWiersema Process information, logs and command history are insecure. Process information is available publicly and that includes all command line parameters. Frequently these calls end up in logs which can be public also. It's not uncommon for an attacker to inspect information on running processes and trawl public logfiles for secrets. Even when it's not public, it could be stored in your command history which would make it easy for someone to get secrets through a non-administrative account.

[ely]

What is the recommended way to supply credentials that are needed at build time? For example, an image that needs aws s3 access to fetch a large data set that will reside inside the image?

[GrandOpener]

A default installation of docker (on linux) requires sudoer privileges to run docker inspect. If the attacker can already sudo, snatching your password out of docker inspect is probably pretty low on your list of things that can now go wrong. This particular detail seems like acceptable risk to me.

[vee_ess]

I imagine the reason it is not recommended is because docker history exposes build-arg/ARG variables. One can pull any image, inspect it and see any secrets passed during build as a build-arg/ARG parameter.

[vee_ess]

@GrandOpener That only applies to the situation where there is an attacker using your system. If I push a docker image to a repository, and it's pulled by someone else, I don't care if they have sudo on their own system but I definitely care if they see secrets in env that are no longer supposed to be there.

[P.J]

Some useful commands from above links: docker secret create : create a secret docker secret inspect : display detailed information about a secret docker secret ls: view all secrets docker secret rm : remove a specific secret --secret flag for docker service create: create a secret during service creation --secret-add and --secret-rm flags for docker service update: update the value of a secret or remove a secret during service update task. Docker secrets are protected at rest on manager nodes and provisioned to worker nodes when during container startup.

[Heather QC]

Yes, you need to set up a swarm to use Docker secrets

[Jeff Lambert]

This is a good start on an answer, but needs much more information from that which is linked to appear in the answer itself.

[pawamoy]

Using environment variables is discouraged by the Docker team itself, as env var can be seen through /proc/<pid>/environ and docker inspect. It only obfuscate the way to get the credentials for an attacker who has gained root access. Of course credentials should never be tracked by the CVS. I guess the only way to prevent a root-user to get cred is to read the credentials from within the web app (hoping that it does not update its proc environ file) from an encrypted file, the decryption process securely asking for a password. I think I'm gonna try with a tomb: github.com/dyne/Tomb

[aug70co]

Do not use this method! It will leak sensitive information on docker history

[John Y]

Not sure this can be the accepted answer if it only works with swarms. Many people aren't using swarms, but still need to pass secrets.

[theUtherSide]

.gitignore so that the .env file with sensitive information does not get checked-in to GitHub. I'm pretty sure this won't work if you add it .dockerignore

[Groostav]

Isnt one of the problems with environment params that they intrinsically get copied to all descendent's, including grand-child processes? Thus a script or a plugin used in your docker container would have access to these passwords by accident?

[opensource-developer]

hi @theUtherSide, thanks for your answer, I had a question, when i dont checkin the .env file and I do a deployment to a server on premise, do you suggest to create the .env file again on server manually?

[theUtherSide]

I would not recommend manually creating a .env file on a server...or doing anything manually on server for that matter. There are many ways/tools for this type of thing.

[JakeCowton]

I appreciate the reference to the bible.

[fsevenm]

@theUtherSide it would be great if you can give us any examples or links of the "ways/tools" you meant.

[Swapnil Gangrade]

If we use vault then where to store the vault secret id and role id which is used by the application to fetch credentialsmemory. How can we pass that into docker?

[lnksz]

"but not at build-time (building the image)," and then your example is for build-time! If you would use the mounted secret-file in the CMD pragma, then it would make sense.

[Murmel]

@lnksz Thanks, I adapted the example

[Robin Dinse]

Docker Compose can manage secrets without a swarm. See this answer.

[Robin Dinse]

This only worked with a prefix like this: DOCKER_BUILDKIT=1 docker-compose up -d

[Sebi2020]

This is not very helpful as most of the out-of-the-box images expect passwords to be passed with environment variables

[Sebi2020]

This is a bad idea. especially typing in passwords into bash. All your secrets get stored into bash_history.

[Malvineous]

Yes that's true, but I was only using it as an example of what the file contents should look like. You can use a text editor or other program to create the file. However if you are running on a system where your .bash_history could be compromised, you shouldn't be putting your passwords in at all, whether in files, environment variables or on the command line. Use a secure system when dealing with passwords like this.