3. Docker#
Some short notes on using docker to create reproducible dev environments.
3.1. What is a Docker Image?#
A docker image can be thought of as a snapshot of a read-only virtual machine that contains everything needed to create a runnable virtual machine.
Docker images come in all shapes and sizes and can be downloaded from Docker
hub which is a huge online repository for Docker images. For example, the
Docker image for Ubuntu can be downloaded using
docker pull <image-name>.
$ docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
e96e057aae67: Pull complete
Digest: sha256:4b1d0c4a2d2aaf63b37111f34eb9fa89fa1bf53dd6e4ca954d47caebca4005c2
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
The images you have downloaded can be listed using docker images.
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest a8780b506fa4 8 days ago 77.8MB
Again, Docker images are read only. We use docker run to create a
Docker container from the images which is essentially a runnable virtual
machine.
3.2. Docker Python Dev Environment#
On Docker Hub there are a number of python images on Docker hub. The
python:3.10-slim-buster includes a minimal linux virtual machine with
python already installed. In this section, we will discuss how to use this
image to create a simple development environment.
3.2.1. TLDR#
To create an interactive shell from the python:3.10-slim-buster image,
use the following command.
$ docker run -itd 3.10-slim-buster
38e12b993c037e2447cd63d4d7e5cb64d9c4c03e0df37041136627daab0db222
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
38e12b993c03 python:3.10-slim-buster "python3" 33 seconds ago Up 31 seconds condescending_galois
$ docker exec -it 38e12 /bin/bash
root@38e12b993c03:/# python
Python 3.10.8 (main, Oct 25 2022, 05:40:26) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Above, we created a docker container with ID 38e12. We then list the
running containers. Then we execute bash inside the container which gives us an
interactive shell from which we can run python.
This gives us a basic python environment inside a docker container.
In the next few sections, we explore the run command and its flags more deeply.
3.2.2. Explore Docker Run#
Docker run lets you run a command in a new container [DRUN].
$ docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Docker run first buts the container from an image and then executes a command. For example, we can write hello world to the console after the container has been built.
$ docker run python:3.10-slim-buster echo Hello, World!
Hello, World!
We can use docker ps to list all the running containers and
docker ps -a to list all containers including those that have been
stopped.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e6289b05d40d python:3.10-slim-buster "echo Hello, World!" 4 minutes ago Exited (0) 4 minutes ago quirky_varahamihira
The container exited as soon as the command echo Hello, World was
executed. We can execute any shell command here, in-fact we can start a bash
shell.
$ docker run python:3.10-slim-buster /bin/bash
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7677b3ae48c8 python:3.10-slim-buster "/bin/bash" 6 seconds ago Exited (0) 4 seconds ago competent_jennings
However, we see that the shell exits instantly. We need a way to keep the container running such that we can give input to the shell. We can do this using interactive mode.
3.2.3. Exlore Interactive Mode#
Interactive mode keeps the standard input (STDIN) of a container open
and allows us to execute commands in the container. Containers can be run in
interactive mode by using the -i or --interactive flags.
$ docker run -i python:3.10-slim-buster /bin/bash
ls
bin
boot
dev
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
Running the command above puts us in a bash shell withing the container that we
can interactive with. For example, above we executed the ls command
which lists the files and directories of the current directory.
This shell may not look familiar, to get a more familiar shell, we have to
allocate a TTY to the container using the -t flag.
$ docker run -it python:3.10-slim-buster /bin/bash
docker run -it python:3.10-slim-buster /bin/bash
root@c6b8389f4237:/# ls
bin dev home lib64 mnt proc run srv tmp var
boot etc lib media opt root sbin sys usr
root@c6b8389f4237:/# exit
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c6b8389f4237 python:3.10-slim-buster "/bin/bash" About a minute ago Exited (0) 55 seconds ago fervent_merkle
The -t flag gives a more familiar shell. Note however that when we exit
the container, we see the container also stopped. To prevent this, we run the
container in detached mode.
3.2.4. Exlore Detached Mode#
By default, docker run runs a container in foreground mode. This means
that when the containers command has been executed, the container is stopped.
We can run the container in the background by using the -d flag.
$ docker run -it python:3.10-slim-buster
85ad08af58b567f35ca367b67f5ab09ba7d454e0506c9952c8cc06902f3ed496
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
85ad08af58b5 python:3.10-slim-buster "python3" 2 minutes ago Up 2 minutes determined_chatelet
Here, the docker run command returns the ID of the container and from
docker ps, we see the container is running. Note, we have not included
the /bin/bash command here.
So, we have a container that is running in the background but we have not been
put into a shell in the container. To create a shell in the container, we use
the docker exec command.
$ docker exec -it 85ad /bin/bash
root@85ad08af58b5:/# echo Hello, World!
Hello, World!
root@85ad08af58b5:/# python
Python 3.10.8 (main, Oct 25 2022, 05:40:26) [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
We give docker exec flags -it which, like before, runs the
command interactively with a pseudo TTY. We then give the ID of the container
we want the command to be executed in and finally the command
/bin/bash itself. This starts an interactive bash shell that we can use
for development.
When we exit the shell, the container will still be running in the background.
To stop the container, use the docker stop command.
3.3. Volumes#
Suppose you are are starting a project and you want to do your development inside a docker container. You could create a container like in the previous section and do everything inside the container. In particular you could store the files for your project inside the docker container. However, getting the files from the container onto the host machine can be tricky.
Alternatively, you could attach your project files to the container using a volume. Think of this as sharing files and data between the host machine and the container. This means any changes made to the volume files inside the container will be reflected in the files on the host machine. Also, if the container is stopped or deleted, the files still exist on the host machine so no data will be lost.
Suppose we have a simple project on our system with one file main.py.
$ pwd
/home/user/my_project
$ ls
main.py
The file main.py is a simple script that prints some text.
# main.py
print("This is main.py")
Let us create a python docker container and attach the my_project
directory to the container. This is done with the -v flag.
$ docker run -itd -v /home/user/my_project:/my_project python:3.10-slim-buster
1f181219691a3837f1830db590e88d8f1644be251b5915687db523096dff93fc
The argument passed to -v contains two paths. The first path before the
colon is the path to the directory (or file) on the host machine that we want
to attach to the container. The second path after the colon is the path to the
location where we want the directory (or file) to be stored in the docker
container.
Now, if we execute bash in the container, we should be able to navigate to our
main.py file.
$ docker exec -it 1f18 /bin/bash
root@1f181219691a:/# cd /my_project
root@1f181219691a:/my_project# ls
main.py
root@1f181219691a:/my_project# python main.py
This is main.py
root@1f181219691a:/my_project# touch created_in_container.py
root@1f181219691a:/my_project# ls
created_in_container.py main.py
root@1f181219691a:/my_project# exit
$ docker stop 1f18
1f18
$ docker rm 1f18
1f18
Above, we navigate to the /my_project directory and run the
main.py script. We also create a new file called
created_in_container.py from within the container and exit the
container. Finally we stop and delete the container.
Since we attached the my_project directory to the container using
volumes, the created_in_container.py file will exist on the host
machine.
$ cd /home/user/my_project
$ ls
created_in_container.py main.py
Despite having stopped and deleted the container used to create
created_in_container.py, the file still exists on the host machine. One
can use volumes to attached many files to a container such that changes made
inside the container are reflected on the host machine.