FetchIt: Life-cycling and configuration of containers using GitOps and Podman

by | Aug 9, 2022 | Containers, Hybrid Cloud

GitOps is a great solution for continuous delivery of Kubernetes applications, as it’s based on Git, a tool that many if not all developers are familiar with. Developers can manage deployed applications by storing their desired state in Git and reap the benefits, such as revision history, code reviews, and branching for their deployments.

GitOps tools such as ArgoCD and Red Hat Advanced Cluster Management (RHACM) have been focused on Kubernetes environments for the most part, but what about lighter weight environments where Kubernetes isn’t required?

Welcome to FetchIt! It’s a lightweight, simple, hands-off GitOps tool for deploying and managing containers through Podman. FetchIt supports various deployment methods, such as systemd, Ansible, and podman-kube. It consumes the same files you would use to deploy heavier weight Kubernetes clusters. In this post, I explain how to run FetchIt and how FetchIt works. I also describe the methods of deployment that FetchIt supports.

Note: Red Hat’s Emerging Technologies blog includes posts that discuss technologies that are under active development in upstream open source communities and at Red Hat. We believe in sharing early and often the things we’re working on, but we want to note that unless otherwise stated the technologies and how-tos shared here aren’t part of supported products, nor promised to be in the future.

FetchIt setup

FetchIt has a few requirements: a config file, Podman, and having the appropriate Podman socket running. Podman has a user socket and a root socket. To enable and run the user socket, use this command:

systemctl --user enable podman.socket --now

For the root user, use this command:

sudo systemctl enable podman.socket --now

How does FetchIt work?

FetchIt works by monitoring the remote Git repositories specified in the configuration file and polling them every so often to check for changes. If a change is detected, based on the methods assigned to the target repository, FetchIt will perform the required update to the deployment on your server(s). This is achieved by communicating with the Podman socket to create containers or pods, or by writing a systemd service file to your server(s) then running it.

Running FetchIt

There are two recommended ways of running FetchIt: as a systemd service or directly within a container. In either case, it will be running within a container.

Running FetchIt as a systemd service
You can run FetchIt as root, using the Podman root socket, or as a user using Podman’s user socket. Ensure the socket you intend to use is enabled. The systemd service runs the FetchIt container for the user, and all a user has to do is make sure they have written a config at ~/.fetchit/config.yaml.

Running FetchIt in a container
The other way, running it directly within a container, requires running that container with some options.

The command can be found below:

podman run -d --rm --name fetchit \
  -v fetchit-volume:/opt \
  -v $HOME/.fetchit:/opt/mount \
  -v /run/user/$(id -u)/podman/podman.sock:/run/podman/podman.sock \
  quay.io/fetchit/fetchit-amd:latest

Note: if running with SELinux in enforcing mode, add the podman flag --security-opt label=disable to the run command . This turns off label separation in the fetchit pod.

The important arguments in this command are the volume mounts; the rest are optional. This command mounts a volume with the same name specified in the config at /opt, which is where FetchIt clones the Git repositories it’s monitoring. This command also mounts the user’s config.yaml at /opt/mount. The final mount is mounting the host’s podman socket to the corresponding socket in the container. That’s it—you now have an instance of FetchIt running on your machine.

Configuring FetchIt

The YAML configuration file is the interface for users to configure FetchIt. It contains a list of target repositories, the options for the repository, and which methods to run for each repository. There is also a field for specifying the name of the Podman volume being mounted at /opt. There is a directory full of examples on the FetchIt GitHub, so let’s take a look at the full-suite.yaml, a config file used to test all methods in our CI:

targetConfigs:
- url: http://github.com/containers/fetchit
  raw:
  - name: raw-ex
    targetPath: examples/raw
    schedule: "*/1 * * * *"
    pullImage: false
  systemd:
  - name: sysd-ex
    targetPath: examples/systemd
    root: true
    enable: false
    schedule: "*/1 * * * *"
  ansible:
  - name: ans-ex
    targetPath: examples/ansible
    sshDirectory: /root/.ssh
    schedule: "*/1 * * * *"
  filetransfer:
  - name: ft-ex
    targetPath: examples/filetransfer
    destinationDirectory: /tmp/ft
    schedule: "*/1 * * * *"
  branch: main
prune:
  All: true
  Volumes: false
  schedule: "*/1 * * * *"

You start by configuring a list of targets. Each target has a URL field and their methods fields. The URL points to the remote repository you want to target. The methods contain fields defining each method you want to run on this repository, and their options. Each method has a name field, a target path field, and a schedule field. The target path field defines the directory containing the files describing what that method should deploy. The schedule field describes when the method should run, in cron format. Other fields on methods are usually specific to that method. I explain individual methods below.

Fetchit reliability
FetchIt attempts to run every method it can and will run other methods even if one fails. The goal is to have FetchIt keep going even if it hits some errors and notify the user that these errors are happening so the user can identify the cause. This is the level of reliability designed into FetchIt.

Here’s a recap before continuing:

  • FetchIt requires a user to have a config file, Podman, the appropriate Podman socket running, and SELinux in permissive mode.
  • To run FetchIt, you have two options: as a systemd service using one of the two service files from this repository, or directly in a container, as described above.
  • FetchIt’s configuration is organized into targets and methods to be run for each target.
  • Each method requires a schedule to specify at what intervals it should run.
  • Many methods require the targetPath field to specify where to find the files to be deployed using that method.

Methods

You can see that FetchIt is organized into targets that represent git repositories. Users can specify multiple targets, with each target running up to one of each unique method described below. Schedules are scoped to their respective methods, and the schedules of different methods on the same target are completely independent. A target can not have two instances of the same method running. If the method or methods copy files to the same directory on the machine, or create containers with the same name, there will be conflicts that must be resolved by the user.

Raw

The raw method is the simplest. It lets users create containers from a small spec. Here is an example of a raw method configuration:

raw:
  - name: raw-ex
    targetPath: examples/raw
    schedule: "*/1 * * * *"
    pullImage: false

The only unique option for this method is the pullImage option, which forces a pull of the image specified in the spec below on every run of the method. In the examples/raw directory specified in the targetPath option, you can find YAML files with the following format:

Image: "docker.io/mmumshad/simple-webapp-color:latest"
Name: "colors2"
Env:
  APP_COLOR: "pink"
  tree: "trunk"
Ports:
  - container_port: 8080
    host_port: 9080
    range: 0

This YAML specifies a simple container configuration with a container name, image, environment variables and a list of port mappings. On every run of the raw method, FetchIt will parse the files in the target directory, check whether those files’ images are on the machine, and pull them if they aren’t (or if the pullmage option is set to true). Then it will remove previous containers with the same name as the one specified in the file and create new containers according to the spec supplied in the files.

Systemd

The systemd method lets users specify systemd service files to deposit on the local machine in the proper systemd directory. There are many options for the systemd method configuration. The first is a simple one:

systemd:
  - name: sysd-ex
    targetPath: examples/systemd
    root: true
    enable: false
    schedule: "*/1 * * * *"

As you can see in the example above, there are three unique options for the systemd method. The first is the root option, which lets the user decide whether they want the service file to be deposited in the root systemd directory to run as a root service, or placed in the non-root systemd directory to run as a non-root service. At the moment, this must be set to true if a user wants FetchIt to enable the service automatically. If the root option is not set, users will have to enable the service themselves using a command such as systemctl –user enable.

The second option is enable, which, if set, enables the service for the user automatically after depositing the file in the appropriate location on the machine. The final option is restart, which restarts the service (after applying the changed service file) when systemd detects a change.

Filetransfer

The filetransfer method allows a user to deposit files from a remote repository into a specific location on a local machine. This method is used within the systemd method to get the service files into the right location. Here is an example configuration:

filetransfer:
  - name: ft-ex
    targetPath: examples/filetransfer
    destinationDirectory: /tmp/ft
    schedule: "*/1 * * * *"

For this configuration the unique option is the destinationDirectory option. This option names the directory to deposit the file.

Ansible

The Ansible method allows a user to run the Ansible playbooks found at the target path, within a container that FetchIt deploys to modify the host running FetchIt. Here is an example configuration for the Ansible method:

 ansible:
  - name: ans-ex
    targetPath: examples/ansible
    sshDirectory: /root/.ssh
    schedule: "*/1 * * * *"

The unique option here is sshDirectory. This option lets FetchIt know which directory to mount the required .ssh directory containing the SSH keys and authorized_keys file.

Kube

The kube method uses the podman-play-kube functionality to allow users to deploy a Kubernetes YAML specification defining config maps, PVCs, pods, and deployments to run via Podman. PVCs map to Podman volumes, and config maps are loaded into memory when a kube spec is applied. Here is an example kube method configuration:

kube:
  - name: kube-ex
    targetPath: examples/kube
    schedule: "*/1 * * * *"

There is no unique option for this method, but within the targetPath directory you will find Kubernetes specification files such as this one:

apiVersion: v1
kind: ConfigMap
metadata:
  name: env
data:
  APP_COLOR: blue
  tree: trunk
---
apiVersion: v1
kind: Pod
metadata:
  name: colors_pod
spec:
  containers:
  - name: colors-kubeplay
    image: docker.io/mmumshad/simple-webapp-color:latest
    ports:
    - containerPort: 8080
      hostPort: 7080
    envFrom:
    - configMapRef:
        name: env
        optional: false

FetchIt will run podman play kube on this file, and Podman will create a pod that has the container declared here with the port mapping declared in the file, consuming the config map declared at the beginning of the file. You can learn more about podman kube play in the Podman documentation.

Prune

The prune option is a special method that does not use any files and thus does not need a target, but serves as a way to remove unused containers and images. It runs podman system prune with the options set in the method configuration, according to the schedule provided in the configuration:

prune:
  All: true
  Volumes: false
  schedule: "*/1 * * * *"

The All and Volumes options in the configuration map to the podman system prune flags that share the same name. Learn more about podman system prune in the documentation.

Config reload

Finally, the configuration reload method is also a special method, like the clean and systemd auto-update. This method is a little complicated and deserves its own article, but I mention it here for completeness.

Conclusion

It’s important to mention again that all of these methods are used in a GitOps fashion. There are remote target repositories that FetchIt knows about from the configuration file it is watching for changes. For each change, FetchIt fetches the update to that file and runs what has to be run to update the state of the running containers so they match what is declared in the repository. Because of this, FetchIt ensures that each machine will make the smallest amount of changes to its deployments, to match the state declared within the remote repository..

The FetchIt GitHub repo has a very complete examples directory to explore if you’re interested in learning more about how to configure FetchIt. There is also a read the docs website with more information about the project.

If you are interested in contributing to the project, feel free to visit the repository and open issues or pull requests.

Author