How I Switched from Docker-Compose to Pure Ansible

Humble Beginnings

Let me tell you a story. It’s 2014 and I had read so many articles about Docker (as the project was called then), how awesome it is and how it makes the lives of developers so much easier. Being one, I decided to try it out. Back in the day, I was working on some django applications. Those apps were really simple: just a webserver and a database. So I went straight ahead to docker-compose. I read in the docs that I should create a docker-compose.yml file and then just docker-compose up. An error message here and there but I was able to navigate the containers to success with no big issues. And that was it. One command to run my application. I was sold on containers.

I Need to Tell Everyone

I was so excited that I started talking about Docker and docker-compose to everyone, everywhere. In the office breakroom, to my dad, at a meetup, to a crowd of 50 at a local conference. It wasn’t completely easy, since some people argued or did not understand fully. But I definitely made some converts. We even made a workshop series with my friends Peter Schiffer and Josef Karasek.

Things are Not Perfect

After some time, especially as my application got bigger and more complex (read: more services), I started running into issues which had odd solutions I did not like. I realized that Dockerfiles and docker-compose is not the solution I wanted to stick to. Back then I wasn’t familiar with Ansible. Luckily my friend Peter was. He started working on a talk about managing containers with Ansible. I realized that I could have full functionality of docker-compose, but also the full power of Ansible.

Enter: Ansible

The longer I used docker-compose, the more I realized it wasn’t meeting all of my needs. I needed a more powerful tool with full templating, more modules, easier setup and well-defined abstractions to better meet my needs. Ansible was that solution for me.

Show Me!

Let’s have a look at some common tasks and how we would do them with docker-compose and then the same thing but with Ansible.

Database Initialization

The problem: You need to initialize a database before a django development web server starts, or else it crashes. Take into account that it may take seconds for a database to come up.

The docker-compose community suggested the creation of a new container which performs the initialization — meaning, write it in shell. Unfortunately, docker-compose does not support dependencies between services — wait for a service to become ready, then start the dependant one. Therefore, the solution is to initialize the database from one container and do sleep 10 && ./manage.py runserver from the one which holds the web service. Not an ideal fix in my opinion. 

Database Initialization with Ansible

Let’s say we need to start a database in a container, wait for it to come up and populate it with desired schema:

---
- hosts: localhost
  tasks:
  - name: Run database container
    docker_container:
      name: 'database'
      image: 'docker.io/centos/postgresql-96-centos7'
      state: 'started'
      env:
        POSTGRESQL_USER: 'user'
        POSTGRESQL_PASSWORD: 'password'
        POSTGRESQL_DATABASE: 'my_db'
    register: db_cont_metadata
  - name: Wait for postgres to accept connections
    wait_for:
      host: '{{ db_cont_metadata["ansible_facts"]["docker_container"]["NetworkSettings"]["IPAddress"] }}'
      port: 5432
      state: drained
      connect_timeout: 1
      timeout: 30
    register: postgres_running
    until: postgres_running is success
    retries: 10
- name: Populate database
  command: ./manage.py migrate

That wasn’t so hard. We created a container with postgresql running inside and waited for the database to accept connections then populated it afterwards.

Does My Application Work?

The problem: After I deploy my application, how do I verify that it works? Docker introduced health checks to solve this problem. Is this a good solution? Some have raised an interesting issue: with health checks, you need to have all the tooling present inside the container. The proposed solution is to check healthiness remotely.

The following is a brief overview of how health checks work. You can define them in a dockerfile:

FROM registry.fedoraproject.org/fedora:28 
HEALTHCHECK CMD ls / 
CMD sleep infinity 

and then when you start the container:

$ docker build --tag=image-with-healthcheck . 
$ docker run -d --name cont image-with-healthcheck 
c8bbc7cbb23ee7209f17b6d934ee8b6c152c75ac6177985cbaa0dd8e4925444a

you can see the healthiness status:

CONTAINER ID    IMAGE ...                STATUS  ...               NAMES                            
c8bbc7cbb23e    image-with-healthcheck   Up 4 minutes (healthy)    cont

You can also define health checks inside docker-compose.yml.

Does it Work? Checking with Ansible

If our application is a web service, we can use uri Ansible module. The main difference here is that we would check the healthiness from the host which invokes the playbook, not from within the container. Bear in mind this is just a sanity check after you deploy, and proper monitoring solutions should be done nevertheless. Docker health checks can run in intervals and therefore can check how your app is doing on a regular basis. However, you still need some mechanism for reporting.

docker-compose.yml Templating

The problem: With moderately complex docker-compose.yml, there may be values which are repeated in multiple places. It’s can be frustrating to change a value in multiple places, e.g., environment variables and mount configuration.

At some point, variable substitution was added. The way it works is it picks up the variables from your present environment, and you can’t define variables inside docker-compose.yml. Let’s take a look.

This is my compose file:

version: "3"
services:
  x:
    image: fedora:28
    environment:
      y: b
    command: echo $y

Let’s run it:

$ docker-compose up
WARNING: The y variable is not set. Defaulting to a blank string.
Creating network "asd_default" with the default driver
Creating asd_x_1 ... done
Attaching to asd_x_1
x_1  |
asd_x_1 exited with code 0

It says $y is not defined even though it’s present in the environment section.
You can see there is no output:

$ docker-compose logs                                                            
WARNING: The y variable is not set. Defaulting to a blank string.
Attaching to asd_x_1
x_1  |

Let’s define y in our shell:

$ export y=c                                                       
$ docker-compose up
Recreating asd_x_1 ... done
Attaching to asd_x_1
x_1  | c
asd_x_1 exited with code 0

And suddenly we have output. Why not have a variables section in a docker-compose.yml file?

Templating with Ansible

Ansible is built on top of jinja. Jinja is a powerful templating system. It has a wide range of features, i.e., variables, conditionals, loops, filters and more. These features are accessible to you when writing Ansible Playbooks.

Caching

The problem: cache invalidation doesn’t always work as one would want - typically commands depend on time and may result in a different output even if they are the same.

A very nice example here is:
RUN git clone a-repo

This command depends on content of the repository. Caching the output of the command might be dangerous because it could install an older version of your application with bugs, which may be already fixed. Or your application may not even start.

The solution? Invalidate cache as needed by introducing noop commands:
RUN git clone a-repo || echo 1

Caching with Ansible

One can structure a playbook like this:

1. Check if content is present and is in desired state
2. If yes, continue
3. If not, perform tasks to get it to such state

A typical example would be to clone a repository and make sure HEAD points to the desired commit. Another use-case would be to check whether a set of packages is already installed: if yes, just continue; if not, install them.

This is a way you can tailor caching mechanism to fit your needs.

Dockerfile Best Practices

The problem: one should master dockerfiles in order to be more efficient.

Dockerfiles are easy to start but it can take time to master them. The caching mechanism is a good example. You also want to know which instructions should come first in order to save time. Image layering is another topic which should be understood well in order to not make mistakes (e.g., by adding ssh keys in a layer).

Once you master Dockerfiles, you can benefit from this knowledge only in docker or moby ecosystems. You are not able to use Dockerfiles for anything other than creating container images for docker engine, moby or containerd. On the other hand, if you invest into Ansible, you can benefit from this knowledge across your whole infrastructure: you can provision nodes, manage networks, set up services, deploy (not just containerized) applications, manage containers and more.

Conclusion

I believe Ansible is the more general-purpose tool. It’s able to handle a lot of use cases and the solutions are effective and portable — you can write a single Ansible Role and use it to provision a service without giving consideration to where it’s supposed to run: container, OpenShift service, VM or even bare metal. On the other hand the tools from docker ecosystem feel more compact and easy to start with to me. In the end, it’s up to you to pick the tooling which suits you best.