Docker compose installing requirements.txt

Question

In my docker image I am cloning the git master branch to retrieve code. I am using docker-compose for the development environment and running my containers with volumes. I ran across an issue when installing new project requirements from my python requirements.txt file. In the development environment, it will never install new requirements on dev environment because when re-building the image, the latest code is pulled from github.

Below is an example of my dockerfile:

FROM base

# Clone application
RUN git clone repo-url
# Install application requirements
RUN pip3 install -r app/requirements.txt

# ....

Here is my compose file:

myapp:
    image: development
    env_file: .env
    ports:
        - "8000:80"
    volumes:
        - .:/home/app

    command: python3 manage.py runserver 0.0.0.0:8000

Is there any way to install newly added requirements after build on development?

Solution 1

There are two ways you can do this.

By hand

You can enter the container and do it yourself. Downside: not automated.

$ docker-compose exec myapp bash
2912d2cd9eab# pip3 install -r /home/app/requirements.txt

Using an entrypoint script

You can use an entrypoint script that runs prep work, then runs the command.

Dockerfile:

COPY entrypoint.sh /entrypoint.sh
RUN chmod 755 /entrypoint.sh

# ... probably other stuff in here ...

CMD ["python3", "manage.py", "runserver", "0.0.0.0:8000"]
ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh:

#!/bin/sh

cd /home/app
pip3 install -r requirements.txt

# May as well do this too, while we're here.
python3 manage.py migrate

exec "$@"

The entrypoint is run like this at container startup:

/entrypoint.sh $CMD

Which expands to:

/entrypoint.sh python3 manage.py runserver 0.0.0.0:8000

The prep work is run first, then at the end of the entrypoint script, the passed-in argument(s) are exec'd. That's your command, so entrypoint.sh exits and is replaced by your Django app server.

UPDATE:

After taking comments to chat, it came up that it is important to use exec to run the command, instead of running it at the end of the entrypoint script like this:

python3 manage.py runserver 0.0.0.0:8000

I can't exactly recall why it matters, but I ran into this previously as well. You need to exec the command or it will not work properly.

Solution 2

The way I solved this is by running two services:

1. server: run the server depends on requirements
2. requirements: installs requirements prior to running server

And this is how the docker-compose.yml file would look like:

version: '3'

services:
  django:
    image: python:3.7-alpine
    volumes:
     - pip37:/usr/local/lib/python3.7/site-packages
     - .:/project
    ports: 
      - 8000:8000
    working_dir: /project
    command: python manage.py runserver
    depends_on:
      - requirements

  requirements:
    image: python:3.7-alpine
    volumes:
      - pip37:/usr/local/lib/python3.7/site-packages
      - .:/project
    working_dir: /project
    command: pip install -r requirements.txt

volumes:
  pip37:
    external: true

PS: I created a named volume for the pip modules so I can preserve them across different projects. You can create one yourself by running:

docker volume create mypipivolume

Comments

[Robert Christopher]

nice solution. upon installing all requirements, the server never seems to start. it appears no logs are being dumped to stdout.

[Dan Lowe]

Notice that the script changes to the /home/app directory - perhaps that is not where your manage.py is located? Your Dockerfile didn't show the WORKDIR, so I am not sure what the image is assuming. Maybe you simply need to cd somewhere else before running the exec.

[Robert Christopher]

WORKDIR is set to /home/app. Confirmed it by running 'ls' in the entrypoint script. Appears to be in the correct location.

[Dan Lowe]

@RobertChristopher Oops! I forgot an important bit... you need to set ENTRYPOINT in the Dockerfile. See updated answer.

[Robert Christopher]

Am I supposed to specify entrypoint in the compose file? thanks.

[Dan Lowe]

You can specify entrypoint and/or command in the compose file, those will override whatever the image itself has. The image always has an entrypoint whether you specify it or not; it defaults to /bin/sh -c if you don't tell it what to use. Using the CMD and ENTRYPOINT in Dockerfile builds it into the image, and in that case you don't need either in docker-compose.yml unless you want to override the image.

[Robert Christopher]

Still encountering issue in which python manage.py runserver returns nothing to stdout. Server doesn't appear to boot either. Any suggestions?

[Robert Christopher]

Attached (docker attach) to the running container and sent ctrl + c keys. Something on the process seems to be blocking.

[Robert Christopher]

Let us continue this discussion in chat.

[Dan Lowe]

@RobertChristopher Updated the answer to include one detail that emerged in chat (exec).