Docker how to make python 3.8 as default

Question

I'm trying to update an existing Dockerfile to switch from python3.5 to python3.8, previously it was creating a symlink for python3.5 and pip3 like this:

RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN ln -s /usr/bin/python3 /usr/bin/python

I've updated the Dockerfile to install python3.8 from deadsnakes:ppa

apt-get install python3-pip python3.8-dev python3.8-distutils python3.8-venv

if I remove python3-pip, it complains about gcc

C compiler or Python headers are not installed on this system. Try to run: sudo apt-get install gcc python3-dev

with these installations in place I'm trying to update existing symlink creation something like this:

RUN ln -s /usr/bin/pip3 /usr/local/lib/python3.8/dist-packages/pip
RUN ln -s /usr/bin/pip /usr/local/lib/python3.8/dist-packages/pip
RUN ln -s /usr/bin/python3.8 /usr/bin/python3

it fails, saying

ln: failed to create symbolic link '/usr/bin/python3': File exists

which I assume fails because python3 points to python3.6. if I try: RUN ln -s /usr/bin/python3.8 /usr/bin/python it doesn't complain about symlink and image gets build successfully, but fails while installing requirements later (we use Makefile targets to install dependencies inside the container using pip and pip-sync):

ERROR: Cannot uninstall 'python-apt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

which I assume because python-apt gets installed as part of the default python3.6 installation and python3.8 pip can't uninstall it.

PS: my Dockerfile image is based on Ubunut 18.04 which comes with python3.6 as default.

How can I properly switch Dockerfile / image from python3.5 to python3.8? so I can later use pip directly and it points to python3.8's pip

Solution 1

Replacing the system python in this way is usually not a good idea (as it can break operating-system-level programs which depend on those executables) -- I go over that a little bit in this video I made "why not global pip / virtualenv?"

A better way is to create a prefix and put that on the PATH earlier (this allows system executables to continue to work, but bare python / python3 / etc. will use your other executable)

in the case of deadsnakes which it seems like you're using, something like this should work:

FROM ubuntu:bionic

RUN : \
    && apt-get update \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        software-properties-common \
    && add-apt-repository -y ppa:deadsnakes \
    && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        python3.8-venv \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* \
    && :

RUN python3.8 -m venv /venv
ENV PATH=/venv/bin:$PATH

the ENV line is the key here, that puts the virtualenv on the beginning of the path

$ docker build -t test . 
...
$ docker run --rm -ti test bash -c 'which python && python --version && which pip && pip --version'
/venv/bin/python
Python 3.8.5
/venv/bin/pip
pip 20.1.1 from /venv/lib/python3.8/site-packages/pip (python 3.8)

---

disclaimer: I'm the maintainer of deadsnakes

Solution 2

Why not just build a new image from ubuntu:18.04 with the desired config you need? Like this:

FROM ubuntu:18.04
RUN apt update && apt install software-properties-common -y
RUN add-apt-repository ppa:deadsnakes/ppa && install python3.8 -y
RUN ln -s /usr/bin/pip3 /usr/bin/pip && \
    ln -s /usr/bin/python3.8 /usr/bin/python

Comments

[Aarif]

I'm actually doing all these steps, I've skipped those lines where I'm adding deadsnakes:ppa to keep the question short, let me try this

[Aarif]

it builds the image successfully but as I said, there's a step involved later in which we use Makefile targets to install dependencies inside the container using pip and pip-sync that fails on Cannot uninstall 'python-apt'.

[Aarif]

thank you @Anthony for contribution, initially, I was looking for a solution that didn't involve virtualenv but installed python3.8 wanted to tinker with packages installed at the system level that it didn't like ... long story short incorporating virtualenv was the best option available .. again thanks!

[Maxime Chéramy]

Please don't do a separate RUN for the apt update, this creates a new layer that will be stored in the cache which could lead in some situations to packages not found when you try to install them later.