It’s no secret that to do their jobs well, developers often need to use as many tools as they can get their hands on to build the best application they can. For them, the right tools for the right job may consist of this version of component X and that version of component Y. But for another tool, entirely different versions of the same components might be needed.
For coders, this is usually just a matter of grabbing the different version of software they need off the internet, installing it, and using it to their heart’s content. No problem, right? Perhaps not for the developer, but from a systems administrator’s point of view, such installations can create systems that are very difficult to manage, particularly on the server side, where having software in packages that are supported and auditable is very much the preferred option.
In the past, Linux distributions solved this problem by offering packaged software that has been tested and integrated, and regularly updated and supported. Each component has been included just once, in a single version, shared by all applications. That made it much easier to manage and worked well for a long time, until different applications started to require different versions of libraries, language runtimes, and other dependencies.
Because of that, some systems use an old version of an OS having the right libraries in it. In other cases, newer versions of libraries had to be carefully brought in to satisfy the application’s dependencies. One technology enabling that are Software Collections (SCLs), allowing a system to have an alternative version of a piece of software without affecting the system.
Finally, we have Linux containers — a way to bundle dependencies together with the app, making it possible to run it on almost any system having a container runtime.
But, according to Red Hat engineer Adam Samalik, none of these solutions completely addresses the problem of balancing the desires of developers for cutting-edge software and the needs of operators for stable software. Samalik outlined the challenges of this particular problem in a conversation after his recent presentation at Open Source Summit Europe. In his talk in Edinburgh, Samalik outlined the way Fedora will address this issue with modularity.
Balance has been a problem that many have tried to solve before. SCLs seem to have come close, but SCLs, while enabling parallel installation, “are very hard to produce and maintain. And you need to manipulate the install paths and run ‘scl enable’ to make them work. And it turned out that in the enterprise world, parallel installation is not a common use case. It’s usually one app for machine, VM, or container,” Samalik indicated.
Containers also seemed to be a strong approach, but while wrapping just what is needed for an application’s runtime and letting that run on a container layer makes things more portable and easier to deploy, the problem of what software is actually being gathered and run can still make an operator’s life difficult, since it may not conform to a distribution’s standard set of software.
Modularity, Samalik explained, is able to mitigate this problem, by (from a user’s standpoint) enabling multiple versions of a given application, library, framework, or language to be available to the same machine. Moreover, the available software originates from the distribution’s own package repositories, which means developers can build code that works with that distribution no matter what. At the same time, available software straight from the distro’s repositories simplifies management for administrators, Samalik added.
According to Samalik, modularity “provides multiple versions of language runtimes or applications, such as databases, available in a Linux distribution. So developers can choose and easily get the correct version they need, since there is now much bigger chance the version they want is packaged. And when the app they wrote is ready to go to production, ops can just install the same packages from the distro, which is much nicer for them to maintain.”
Under the Hood
So how will Fedora Modularity enable end users to make multiple versions of an application available on any given version of Fedora with modularity enabled?
Modularity provides “package groups on steroids” — using standard packaging format, but available in multiple versions (called streams) to choose from. And with their own lifecycle that can be potentially longer or shorter than the standard Linux distribution release. So you can choose the right version of the piece of software you need, regardless of the OS version you’re running. It’s as if their lifecycles have been separated.
First, Samalik described, “there are high thousands of packages in the Linux distro and shipping them in multiple versions would be very messy. So that’s why we group packages into logical units called modules–representing a language runtime and some of its libraries, or an application such as a database or a web server. That way we increase the granularity–instead of high thousands packages we can have tens or low hundreds of modules–that’s much easier to navigate! And we provide them in multiple versions, known as streams. So you can choose the version you want (just one) and install it on your system.”
The solution may feel similar to the concept of branches in Git — a version control system many developers are already familiar with.
With Git, repositories of files (which can be used to build code, host content for web sites, and store multiple versions of files for any reason) are held in a “master” branch. But, when changes need to be made, a new branch is created, where files can be modified as needed without any of those changes affecting the master branch. More interestingly in the context of modularity, is the behavior of a branch’s files from a user’s point of view.
Git allows you to store multiple versions of a code base in a single directory. How? Using the concept of branches. Branches hold different versions of the code in a virtual space, and the developer can switch between those in order to see it in the directory.
Suppose I have five files in the master branch:
File1
File2
File3
File4
File5
I need to make changes, so I will create a new branch with those five files within it. I make my changes, which include deleting one of the files and making changes to two others:
File1
File2New
File4
File5New
If I were to look at my file directory while I was working in the new branch, this list above would be what I would see. But (and this is the interesting part) if I were to check back into the master branch, the original list of five files would be displayed. What is seen (and accessible) in a git-maintained directory is entirely dependent on what branch I, the user, am in.
Samalik uses another analogy to help people visualize how modularity works. For him, it’s about imagining customizing a laptop.
“You can choose what screen you want, the hard drive, maybe the GPU, but you can always have just one of each in your laptop. But what’s good is that you get a standard laptop,” he explained. “An analogy for SCLs would be an external hard drive on a cable, an external monitor that better meets your requirements, an external GPU, etc.” Definitely usable, but might feel a bit clunky.
And you can apply the same analogy to containers: “Containers without modularity is like ordering a standard non-configurable laptop, going out to buy different components, and changing them yourself. And then repairing them yourself,” Samalik added. “Containers with modularity is like clicking ‘customize’ while ordering the laptop.”
The application of git-like version control to packages and containers is at the core of what makes Fedora modularity work. The benefits also can go beyond just balancing the needs of developers and operators.
Immutable Operating Systems
Modularity, when extended for containers, also satisfies the hygiene goal for an operator, since, as with packages, containerized applications can consume their dependencies in the container from the distribution’s registry, thus providing stability and known environments all the way up the stack. “Now containers can be built and used the right way,” Samalik said.
Because containers only need an operating system that has a container runtime layer to run applications in said containers, “the operating system can become immutable,” Samalik said.
This means that the entire operating system can be treated as a single object–an image–with no regard needed for the packages that comprise the operating system. At the operating system, everything just works to run the container layer, and the applications in containers no longer need to rely on what’s going in with variations within an operating system.
This is the idea behind Fedora CoreOS, which will be tuned as a server platform, and Fedora Silverblue, which will be designed to function as a workstation. Applications can be installed within containers and managed independently of the OS. That makes OS upgrades much less stressful because the possibility of breaking the app is significantly smaller. And even when something does happen, the OS can be reverted to a previous state because it’s treated as an immutable image, with a previous state being available as a backup.
“Modularity gives developers the choices they want and operators the quality they need,” Samalik added.
