TripleO

OpenStack Containerization with Podman – Part 2 (SystemD)

In the first post, we demonstrated that we can now use Podman to deploy a containerized OpenStack TripleO Undercloud. Let’s see how we can operate the containers with SystemD.

Podman, by design, doesn’t have any daemon running to manage the containers lifecycle; while Docker runs dockerd-current and docker-containerd-current which take care of a bunch of things, such as restarting the containers when they are in failure (and configured to do it, with restart policies).

In OpenStack TripleO, we still want our containers to restart when they are configured to, so we thought about managing the containers with SystemD. I recently wrote a blog post about how Podman can be controlled by SystemD, and we finally implemented it in TripleO.

The way it works, as of today, is that any container managed by Podman with a restart policy in Paunch container configuration, will be managed by SystemD.

Let’s take the example of Glance API. This snippet is the configuration of the container at step 4:

As you can see, the Glance API container was configured to always try to restart (so Docker would do so). With Podman, we re-use this flag and we create (+ enable) a SystemD unit file:

How it works underneath:

  • Paunch will run podman run –conmon-pidfile=/var/run/glance_api.pid (…) to start the container, during the deployment steps.
  • If there is a restart policy, Paunch will create a SystemD unit file.
  • The SystemD service is named by the container name, so if you were used to the old services name before the containerization, you’ll have to refresh your mind. By choice, we decided to go with the container name to avoid confusion with the podman ps output.
  • Once the containers are deployed, they need to be stopped / started / restarted by SystemD. If you run Podman CLI to do it, SystemD will take over (see in the demo).

Note about PIDs:

If you configure the service to start the container with “podman start -a” then systemd will monitor that process for the service. The problem is that this leaves podman start processes around which have a bunch of threads and is attached to the STDOUT/STDIN. Rather than leaving this start process around, we use a forking type in systemd and specify a conmon pidfile for monitoring the container. This removes 500+ threads from the system at the scale of TripleO containers. (Credits to Alex Schultz for the finding).

Note about PIDs:

If you configure the service to start the container with “podman start -a” then systemd will monitor that process for the service. The problem is that this leaves podman start processes around which have a bunch of threads and is attached to the STDOUT/STDIN. Rather than leaving this start process around, we use a forking type in systemd and specify a conmon pidfile for monitoring the container. This removes 500+ threads from the system at the scale of TripleO containers. (Credits to Alex Schultz for the finding).

Stay in touch for the next post in the series of deploying TripleO and Podman!

Software Engineeer at Red Hat, Private Pilot, French guy hiding somewhere in Canada.