Most modern operating systems (including iOS, Android, macOS, and ChromeOS) have been using image-based architectures for a while, and in recent years we’ve been making progress towards this on the desktop side as well. It’s a necessity if we want to provide the security, usability and reliability that people expect from their devices nowadays.
However, broadly speaking the adoption of image-based designs has been limited in the free desktop world so far because there are still a few major gaps. While Flatpak and Flathub have more or less solved the app distribution on these systems, the developer-facing story still looks much more incomplete.
Traditionally, people have been developing against the host OS they are running. Assuming you’re working on something like NetworkManager and need a new dependency, you’d just “apt install” it globally on your system, and then configure/compile against that. Similarly, if you need a command-line utility, compiler toolchain, or other build dependency, you’d also install them from the distribution repository.
In an image-based world this approach doesn’t work, and people have come up with various ways to address this.
RPM-Ostree and Package Overlays
Fedora Silverblue can be extended similar to package-based distros with rpm-ostree overlays, but the issue with this approach is that overlayed packages can completely break the system in unexpected ways or stop it from further updating. This is why the next iteration of Silverblue is based on “bootc”, and will be explicitly avoiding this paradigm. Same issue are present with other package overlay approaches as well.
Monolithic “Development” Overlay
The approach used by Android, iOS, Windows, et al is to have a single monolithic overlay with “system development tools”. Developers can install this overlay, and it provides all the utilities people developing the system itself need.
GNOME OS does something similar: There is a “developer” system extension that overlays the toolchain used for building the OS on top of the user-facing OS image. However, this is a finite list of utilities needed specifically to build the OS. As such, it can not cover the long tail of development and debugging tools developers in different areas need (e.g. kernel development).
Toolbox
Another approach is trying to replicate the same traditional package-based experience inside a container. Toolbox and distrobox are examples of this.
Unfortunately, with this approach you are still relying on the same old package infrastructure, while also being inside a more restrictive container environment that none of the tools expect to be run in. As such, you often run into limitations when developing system components and need to bypass the container layers in order to debug the system itself.
Homebrew
Homebrew provides an independent tool chain to develop against, but in order for homebrew binaries to be usable directly in the terminal, they have to be prioritized over system binaries. This means that the system can break if there is a mismatch between what the system expects and what homebrew provides. For example, you install QEMU, but it overrides the GLib your system uses for everything else.
There are other architectural issues with homebrew, but in my opinion this alone disqualifies it for system development.
Flatpak
Lastly, we have Flatpak. Like Toolbox it uses “containers”, so the same issues and limitations are also present here, but it’s even more restrictive because the assumption is that apps will use portals to access system resources such as directories and devices. Flatpak was designed specifically with desktop apps in mind, and its architecture and integration points are not well suited for command line apps.
Somewhat tangentially, there are apps that can’t be made to fully work with Flatpak as it is today, particularly debugging utilities and IDEs. While it’s technically possible (GNOME Builder and the Ptyxis terminal are proof), older applications were not designed with sandboxed resources in mind, or have not adapted to this model yet.
What Could We Do Instead?
While we would love for everything to be sandboxed and confined, that is sadly not yet possible. All the existing approaches that attempted to containerize tooling, run into the same conflicts between the desired functionality and the restrictions inherited by this approach. This is a topic we will discuss in more depth later on.
Additionally I believe there are two different use cases here. The build tooling/toolchain used by projects, and the developer utilities.
Project Build Setups
One aspect of this is the build setup used by individual project, which would ideally be something more deterministic, and containerized like Buildstream, flatpak-builder, bazel or even nix, instead of the old status quo “apt install -yqq gcc meson libone-devel libtwo-devel”. This is also a topic for another day though.
Developer Utilities
The other aspect, and what I want to focus on today, is that a lot of developer utilities that people rely on can’t realistically be run in a confined environment (such as a container) without a near complete rewrite.
We need a way to make things like strace, ripgrep, and qemu available. Shipping them in the host system is one option, but there is a real long tail issue here. You can’t (and don’t want to) provide every single utility any developer might need. Thus we need them to be shipped independently, and as such, not attached to the host OS.
Toolpak
If we were to take a fresh look at this issue and design something from scratch, what would we want it to look like? Over the past months we’ve had a number of discussions on these topics, and it feels like we’re finally converging on a solution would address most people’s concerns and needs.
These are the important properties we would want:
- Tools should be independent from the host OS , and they should not break the host OS if something goes wrong. They need to be able to access every resource in the system, much like today.
- A large catalog of existing tooling , like we find today in distribution repositories today.
- Tools work out of the box , and will befully functional, as it’s not feasible to modify all of them. Things like shelling out to other tools should work like it does now.
Here is what I imagine an implementation, which we will call Toolpak, would look like:
- Discoverable Disk Images (UAPI.3) as the image format. This will provide us state of the art security practices, like Verity. It will also give us a good base for allowing Reproducible builds, given that the build tooling permits.
- A mount namespace for the binaries. The /usr and /app split from Flatpak is a great idea and we should also steal it much like portable services now can use Base/Runtime images! /usr should be provided by the “Runtime Image” and shared among tools. And /app will be the main contents of the Tool image.
- Tools have unrestricted access to everything else. This should also satisfy the “no need to port” requirement (Some edge cases where the mount namespace will conflict, but they are minor).
- We should prepend the tools, to the PATH of the user (ie. our binaries will override the system ones), but it should avoid breaking the system as you can only override the system binaries. Said binary will then setup the mount namespace and execute the real binary from inside the image. This avoids messing with the linker and shared libraries. Unless you overriding the system GNU Tar with the BSD Tar (or other incompatible cases of the same binary), things should work fine. More research is needed to explore what other safeguards will be required.
- Tools can not depend on other tools . One of the issues with traditional distributions is dependency resolution and package management (this is subcategory of a heavilydiscussed topic, I also talked about it in myLAS talk from 2025 . We should avoid making individual packages or having dependencies between tools, to avoid all the unnecessary complexity that comes with it.
- Tools will bundle all their dependencies . This will ensure that they will always execute against the environment they were tested against. It additionally allows tools to bring their own versions of libraries that might be present in the system, without conflicts. We mentioned the /usr and /app split above.
- Great user experience to discover and install tools . There should be a flagship “app store” and ideally the tools should be packaged and distributed by the developers themselves, similar to Flathub. No more “download this static binary and chmod +x it”.
- Tools have complete and arbitrary access to the system , so it’s crucial that it not be simple to install random images. They will be have to be signed with a trusted key, and verity checked at runtime, and be thoroughly reviewed before appearing on the flagship “app store”, and all the other practices we should expect from distributing software securely.
- Apps that can be packaged using Flatpak should be rejected . Unless they are less functional, like IDEs on Flathub.
In order to be adopted, it will also have to come with tooling that will make it easy to build said Toolpaks. Here are some specific properties the build tooling should have:
- It should be easy to orchestrate builds of your tool and its dependencies.
- Great caching for all build artifacts in a Content Addressable Storage.
- Reproducible by default, with tooling to easily verify the output.
- Can be used for both local development and composing the final image.
- Handles all the licensing/SBOM/etc requirements.
- A Buildstream plugin or wrapper around it should satisfy all these needs.
Next Steps
While this topic has been discussed for years, there’s now concrete work towards a prototype as part of a Prototypefund project. We plan to share more on this in the coming weeks and are interested in feedback from the wider community. In the mean time, if you have any other feedback, find us in #gnome-os:gnome.org on Matrix, or leave a comment.