python - Activate and switch Anaconda environment in Dockerfile during build

Question

I've been trying for hours and can't figure out how to activate and switch anaconda environments in a Dockerfile during the build process

Here's the initial code:

FROM nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04

# Set user
ENV SETUSER myuser

RUN useradd -m $SETUSER
USER $SETUSER
WORKDIR /home/$SETUSER

# Install Anaconda
RUN wget https://repo.anaconda.com/archive/Anaconda3-2019.03-Linux-x86_64.sh
RUN bash Anaconda3-2019.03-Linux-x86_64.sh -b
RUN rm Anaconda3-2019.03-Linux-x86_64.sh

# Set path to conda
ENV CONDA_ENV_NAME mynewenv
RUN /home/$SETUSER/anaconda3/condabin/conda create -q --name $CONDA_ENV_NAME python=3.6 && 
    /home/$SETUSER/anaconda3/condabin/conda clean --yes --all
RUN /home/$SETUSER/anaconda3/condabin/conda activate base #Just for testing anaconda environment

At first, anaconda in Docker will complain that the shell is not setup properly, so after the conda create command I added:

RUN /home/$SETUSER/anaconda3/condabin/conda init bash
RUN /bin/bash -c "source /home/$SETUSER/.bashrc"
RUN /home/$SETUSER/anaconda3/condabin/conda activate base

Running the 3 commands after building the docker image works (i.e. running interactively after calling docker run container-name), but for some reason it does not work when building the container. I figured out that the $PATH variable was not being updated during the build, so comparing my $PATH when building and after building.

ENV PATH /home/$SETUSER/anaconda3/envs/$CONDA_ENV_NAME/bin:$PATH
ENV PATH /home/$SETUSER/anaconda3/condabin:$PATH
ENV PATH /home/$SETUSER/anaconda3/bin:$PATH
RUN conda init bash
RUN /bin/bash -c "source /home/$SETUSER/.bashrc"
RUN conda activate base

Now, the Docker $PATH when building and the $PATH when modified interactively when running the container after-building is the same, but I'm still getting the shell not properly setup error.

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run $ conda init Currently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell See 'conda init --help' for more information and options. IMPORTANT: You may need to close and restart your shell after running 'conda init'.

Why is this not working???

I've seen that there may be workaround using a miniconda docker template, but I cannot use that. How do I create and switch anaconda environment during the Docker building process? Thanks!

Answer 1

You've got way too many RUN commands in your Dockerfile. It's not just that each RUN creates a new layer in the image. It's also that each RUN command starts a new shell, and conda activate applies only to the current shell.

You should combine logical groups of actions into a single RUN command. Use && to combine commands, and to break lines for readability:

RUN conda activate <myenv> 
 && conda install <whatever> 
 && ...

Keep in mind: at the end of that RUN command, the shell will be gone. So if you want to do something else to that conda environment afterwards, you've got to run conda activate again, or else use -n <myenv> to put something into an environment without activating it first.

When you start a container from the image, you will also have to call conda activate inside the container.