Debusine is a Free Software project developed by Freexian to manage scheduling and distribution of Debian-related tasks to a network of worker machines. It was started some time back, but its development pace has recently increased significantly thanks to funding from the Sovereign Tech Fund. You can read more about it in its documentation.
For more background, Enrico Zini and Carles Pina i Estany gave a talk on Debusine in November 2023 at the mini-DebConf in Cambridge.
We described the work from our first funded milestone in a post to debian-devel-announce in March.
We’ve recently finished work on our second funded milestone, culminating in releasing version 0.3.0 to unstable. Our focus on this milestone was on new building blocks to allow us to automatically orchestrate QA tasks in bulk. Full details are in our release history document. As usual, debusine.debian.net is up to date with our latest work.
Collections
In the previous milestone, debusine could store artifacts and run tasks against those artifacts. However, on its own this required the user to do a lot of manual work, because the only way to refer to an artifact was by its ID.
We now have the concept of a
collection,
which can store references to other artifacts (or indeed to other
collections) with some attached metadata. These are structured by category,
so for example a debian:suite collection contains references to source and
binary package artifacts with their names, versions, and architectures as
metadata. This allows us to look up artifacts using a simple query
language
instead of just by ID.
At the moment, the main visible effect of this is that our Getting started
with
debusine
tutorial no longer needs users of debusine.debian.net to create their own
build environments before being able to submit other work requests: they can
refer to existing environments using something like
debian/match:codename=trixie:variant=sbuild instead.
We also have a basic user interface allowing you to browse existing collections, accessible via the relevant workspace (such as the default System workspace).
Workflows
We’ve always known that individual tasks were just a starting point: real-world requirements often involve chaining many tasks together, as many Debian developers already do using the Salsa CI pipeline. debusine intends to approach a similar problem from a different angle, defining common workflows that can be applied at the scale of a whole distribution without being tightly coupled to where each package’s code is hosted.
In time we intend to define a way for users to specify their own workflows,
but rather than getting too bogged down in this we started by building a
couple of predefined
workflows
into debusine. The update_environments workflow is used to create
multiple build environments in bulk, while the sbuild workflow builds a
source package for all the architectures that it supports and for which
debusine has workers. (debusine.debian.net currently has amd64 and arm64
workers, supporting the amd64, arm64, armel, armhf, and i386 architectures
between them.)
Upcoming work will build on this by adding more workflows that chain tasks together in various ways, such as workflows that build a package and run QA tasks on the results, or a workflow that builds a package and uploads the result to an upload queue.
Next steps
Our next planned milestone involves expanding debusine’s capability as a build daemon. For that, we already know that there are a number of specific extra workflow steps we need to add, and we’ve reached out to some members of Debian’s buildd team to ask for feedback on what they consider necessary. We hope to be able to replace some of Freexian’s own build infrastructure with debusine in the near future.