Planet CCC - Blogs and more around CCC, CCC-Hamburg and Attraktor

test 2 @ entropia Licensed to the public under https://creativecommons.org/licenses/by/4.0/ about this event: https://c3voc.de

Video:entropia-2-eng-Test_2_hd.mp4

A fundamental aspect of an OS security policy is the ability to manage mounting. That includes filesystem mounts, remounts, bind-mounts, and property changes. Contrary to popular believe Linux exposes only barebones policy hooks. This is about to change. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/J9TM7X/

Video:asg2026-441-eng-Managing_the_new_mount_api_sd.mp4

In this talk we will look at how binfmt and bpf can be integrated. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/8SCGQC/

Video:asg2026-440-eng-binfmt_bpf_sd.mp4

Varlink is slowly getting adopted in the desktop space. This brings a few new challenges, one of them being observability: D-Bus allows us to capture the entire system and session bus and then drill down on specific interactions in the system. Varlink on the other hand is just a protocol spoken over AF_UNIX sockets and thus has no bus and no global view of the system. The talk explores how we can use eBPF and extended attributes on sockets to give us back the observability we're used to with D-Bus. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/FKWEC8/

Video:asg2026-437-eng-Varlink_observability_sd.mp4

In this talk I'd like to provide a vision for accessing systemd hosts via local IPC APIs and remote RPC APIs. With the Varlinkification of systemd going full steam ahead, I'd like to explain what I think should be next for systemd and how things fit together with our focus on APIs. We'll talk about metrics/reports, Varlink in general, the Varlink HTTP proxy specifically, about security models, and zero-agent remote access, secure system parameterization and more. We'll talk about design patterns to follow, both for systemd components, and for other OS components, outside of it. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/RZ3PUB/

Video:asg2026-433-eng-An_API_for_systemd_Machines_sd.mp4

`systemd-sysupdate` provides a lot of incredibly useful primitives for doing system updates, but it is still mostly designed around old-school `SHA256SUMS` distribution and static transfer definitions. The need to have more autonomous systems for updating systems and adding components became quite clear when we started developing such systems at Amutable. This approach also dove-tailed nicely into developing an autonomous provisioning system using the same underlying mechanism, [leading us to build Quarry](https://amutable.com/blog/distributing-images-quarry) (which has been [recently open sourced](https://github.com/amutable-systems/quarry)). This talk will contain a discussion of the architecture we came up with, the pain points with integration with `systemd-sysupdate`, as well as some of the caveats that such an architecture has with generic distributions. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/BBJYFH/

Video:asg2026-431-eng-Modernising_Software_Distribution_with_Quarry_sd.mp4

For years, kexec was the standard way to skip lengthy firmware and bootloader delays at reboot. However, in modern cloud environments, even a fast reboot causes severe service degradation. Guests evacuation in virtualized environments or rebuild of massive in-memory caches for large databases is costly and disruptive. Kernel HandOver (KHO) and Live Update Orchestrator (LUO) introduce a complete framework for seamless kernel and userspace state preservation across upgrades. KHO provides the infrastructure to preserve kernel data structures and arbitrary memory regions across kexec. Physically contiguous memory areas managed by KHO let the incoming kernel to bootstrap itself without touching the preserved memory. On top of KHO, LUO provides userspace APIs for coordination of the transition and allows applications to preserve file descriptors across kexec. With initial support for memory file descriptors (memfd), the ecosystem is rapidly expanding to cover subsystems like PCI, VFIO, KVM, guestmemfd, and iommufd. Moreover, the ecosystem now features native systemd integration, leveraging the standard fdstore interface to pass resources across the kernel reboot. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/J8GNND/

Video:asg2026-430-eng-Seamless_Upgrades_with_KHO_LUO_and_systemd_sd.mp4

In this talk we'll cover the developments in kmod over the past few years. We'll wiz past the documentation and contribution updates, outline some of the performance improvements and discuss the latest APIs introduced. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/TTFUPB/

Video:asg2026-425-eng-Kmod_-_teaching_old_dog_new_tricks_sd.mp4

BuildStream is a tool for building / integrating software stacks. In a way, it has a similar goal to bitbake / yocto and Android's repo, but takes a completely different approach. It can be used to take software from various sources, build it with various buildsystems in a reproducible sandbox, and cache results for speedy rebuilds. mkosi is a tool for easily building customized OS images. It is generally used with distros that have a package manager such as dnf or apt. There is however recent development to add support for using it with BuildStream. In this talk I give a brief overview of Buildstream, and freedesktop-sdk which is a BuildStream project that can be used as a base to build your own operating system. While BuildStream on its own can build the whole thing, there are some advantages to using mkosi which I'll go over in this talk. I'll also go over the challenges faced moving GNOME OS to use mkosi for generating the image. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/DUNFF9/

Video:asg2026-423-eng-Building_image-based_operating_system_images_with_BuildStream_and_mkosi_sd.mp4

Delivering a Linux OS via OCI images unifies infrastructure management but often introduces severe network and state-reconciliation bottlenecks. This talk dives into the mechanics of solving these challenges by applying OS updates directly from OCI images to btrfs snapshots. We will explore how to drastically optimize these atomic upgrades using zstd-chunked compression, enabling systems to execute file-level delta updates via HTTP range requests instead of pulling full layers. Finally, we will demonstrate how to leverage snapper for seamless 3-way merges, safely reconciling upstream OS changes with local user configurations. The transition to image-based, immutable Linux systems fundamentally changes OS management, but it also introduces complex network and storage challenges that are often abstracted away. This session explores the architecture and technical workflows required to provision and upgrade an OS using OCI images, advanced compression, and copy-on-write filesystems. The foundation of this approach relies on mapping OCI images directly to btrfs snapshots. OCI image layers are pulled, unpacked, and directly applied as immutable btrfs snapshots, forming the read-only root volume of the system while maintaining distinct writable areas for persistent state. To eliminate the network bottleneck of pulling full OCI layers for minor patches, we will detail efficient delta updates with zstd-chunked OCI images. The OS agent identifies and downloads only the specific files that have changed between the current running image and the target image. We will explain the mechanics of using HTTP range requests to extract these payloads and write them directly into the target btrfs snapshot on the fly, drastically reducing update times and bandwidth consumption without requiring full local extraction. Finally, an immutable OS still requires localized configuration (e.g., in /etc), creating a challenge when the base OS image is updated. We will explore how to solve this using a 3-way merge strategy. In particular we will see how snapper is utilized to navigate btrfs snapshot trees and isolate the precise differences between the original base image, the user's localized state, and the incoming update. Having a clear picture of the differences in multiple directions allows us to define an algorithmic approach to safely merging these three states. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/E7L8GD/

Video:asg2026-417-eng-Delivering_the_OS_via_OCI_images_Btrfs_snapshots_delta_updates_and_3-way_merges_sd.mp4

KDE's new image-based Linux distribution embraces modern technologies whenever possible. We'll explore all the tools that contribute, and how they are being utilized to form the final product. From building the disk image to eventually updating it on the user's system. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/VXGVJF/

Video:asg2026-416-eng-KDE_Linux_Under_the_Hood_sd.mp4

A decade ago, container networking was in its infancy, relying heavily on complex bridges and iptables rules. Today, the landscape has fundamentally shifted towards highly performant, secure, and observable dataplanes powered by eBPF. In this talk, a core contributor to Cilium reflects on a decade of building container networking solutions - from the early days of Weave Net to modern eBPF-based architectures. We will explore the hard-fought engineering lessons from migrating away from traditional kernel bottlenecks and discuss how Cilium evolved into the most widely used container networking plugin (CNI) and a foundational platform for advanced networking research. Finally, we will look to the horizon to discuss what the next decade of Linux system networking holds. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/R7BEEK/

Video:asg2026-408-eng-Ten_Years_Building_Container_Networking_Lessons_Learned_and_What_the_Future_Might_Look_Like_sd.mp4

When a VM is confidential (CVM), the host cannot access its memory, enabling data-in-use protection. Hardware vendors like Intel (TDX) and AMD (SEV-SNP) provide the underlying technology to achieve this. The use case becomes more complex when a persistent disk is added to a CVM: the disk must also be encrypted, but how can we safely manage its encryption keys without trusting the host? We propose a mechanism to encrypt and decrypt a disk using remote attestation: the passphrase is stored in a Key Broker Service (KBS) that only releases it when the identity of the VM is verified. During the first boot, systemd-repart encrypts the root disk using a random key that is generated and stored in KBS. On subsequent boots, systemd-cryptsetup requests the same key to decrypt the disk. We present the changes we are working on to support this use case in components like KubeVirt, Trustee, and systemd-repart. We present PRs for Trustee, systemd-repart, and KubeVirt, along with a new LUKS token plugin that integrates systemd-repart and systemd-cryptsetup with KBS for key retrieval. This use case extends beyond KubeVirt and applies to any environment where confidential workloads require encrypted persistent storage, including bare metal deployments. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/TKKLMZ/

Video:asg2026-405-eng-Disk_encryption_using_attestation_for_Confidential_VMs_sd.mp4

[IncusOS](https://linuxcontainers.org/incus-os) is an immutable OS based on Debian 13 and built using mkosi. It's designed to be as secure as possible, actively relying on both UEFI Secure Boot and a TPM 2.0 module to provide strong boot security as well as encryption at rest through LUKS+TPM. Its whole purpose is to provide an ideal environment for running [Incus](https://linuxcontainers.org/incus/) with a focus on running on bare metal with pretty complex local and remote storage as well as a variety of networking options. Unlike most other Linux systems, IncusOS doesn't provide any local or remote shell access to the system. Instead it's entirely API driven with its management API being available through the Incus API itself. That management API allows for system configuration, update management, installation of additional components (systemd system extensions), configuring a variety of system services, ... But it also needs to handle a variety of fallback and recovery/patching mechanisms for when something goes wrong. This talk will cover why we built IncusOS, how we built it, its current state and what we're working towards in the near future. [Incus](https://linuxcontainers.org/incus/) is a modern container and virtual machine manager with native support for both system and application containers as well as virtual machines. It's designed to behave just like a private cloud, but one that you can run on your laptop or Raspberry Pi as well as on clusters of hundreds or thousands of servers. It's got support for a wide variety of local and remote storage options as well as both software defined and traditional networking. It has built-in clustering allowing for hundreds of servers to act as one with a consistent API and distributed database providing high availability and automated recovery. Incus is widely available through packages in most Linux distributions with IncusOS providing another method of deployment particularly targeted to those who don't want to have to manage yet another Linux system or those who want to perform large scale deployments with known identical systems. IncusOS can be deployed and managed at scale through [Operations Center](https://github.com/futurfusion/operations-center), allowing for hundreds of clusters to be deployed, updated and tracked in one place. Incus, IncusOS, Operations Center and generally everything related to Incus is Open Source and released under the Apache 2.0 license. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/ER3SSR/

Video:asg2026-402-eng-Building_IncusOS_sd.mp4

Image-based Linux systems have different storage requirements from traditional mutable systems as they rely on signed, immutable disk images for software delivery and separate writable filesystems for data. Technologies like systemd-homed require that multiple writable filesystems share the same backing storage. This thin provisioning is not something that is well supported by Linux today as filesystems are not aware of it. Another area for improvement is that many disk images often contain similar data that could be deduplicated. We will present a storage system prototype designed for the usecases of image based systems and our plans for improving thin provisioning on Linux. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/WSNAQL/

Video:asg2026-401-eng-OSPool_and_thin_volumes_Storage_for_Image-based_systems_sd.mp4

Confidential VM (CVM) instance types are becoming ubiquitous, with all major cloud providers offering them as a key component for achieving “zero trust.” While complete zero trust is currently more of a marketing term than a technical reality, we can acknowledge that CVM technologies help reduce the Trusted Computing Base (TCB) when utilized properly. But do we actually know how to use them effectively? When transitioning from a world of traditional, mutable operating systems, users often expect to gain security benefits simply by moving their existing workloads to new CVM instance types, perhaps with some level of attestation. Unfortunately, in many cases, this looks more like a role play in a “security theater”. Over the past few years, we have investigated what security benefits can truly be unlocked when mutable RHEL or Fedora workloads are deployed on confidential VMs, whether in the public cloud or on-premises. While immutable OSes offer distinct advantages in such scenarios, we pose the question: can traditional (mutable) OSes also benefit? Over the last year, we have contributed several features to systemd (and other components) that we believe can improve the situation. In particular, we are focused on preventing certain attack scenarios against a VM which is normally deployed from a publicly available cloud (marketplace) image. We have further ideas, and in this talk, we will share our vision for how these pieces can fit together. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/VNXPQS/

Video:asg2026-399-eng-Running_mutable_OS_images_on_Confidential_VMs_is_there_any_hope_sd.mp4

Enterprise Linux deployments lack a standardized, native enrollment flow. Current solutions rely on distro‑specific first‑boot mechanisms (systemd-firstboot, cloud‑init, realmd, ignition, kickstart and more) or external configuration management layered post‑deployment, resulting in poor feature sets w.r.t. modern features enabled by TPM2 or UEFI setting management. This talk proposes to explore minimal, vendor‑neutral set of APIs and protocols for OS enrollment, *analogous to* Windows Autopilot but designed for the Linux userspace stack and specifically for the ParticleOS ecosystem. We will examine a concrete model based on device identity (TPM, SMBIOS information), secure attestation, policy discovery, and management‑hint delivery, decoupled from any particular configuration management system. There might be a live demonstration presenting a reference implementation on NixOS communicating with a lightweight enrollment service over HTTP(S) leveraging a bunch of Varlink APIs built across the past years and presented at ASG. The primary outcome is a community‑reviewed specification draft for enrollment APIs, intended for integration into systemd and distro installers. The talk is not a proposal for a new tool but rather invite the attendees to think about the problem space around corporate enrollment for Linux distributions and how do we build native solutions for distributions interested by this idea. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/3RTJNA/

Video:asg2026-397-eng-Native_OS_enrollment_for_the_Linux_userspace_A_proposal_for_open_APIs_and_protocols_sd.mp4

When a machine/server/node is serviced and experiences a downtime, that results in a service interruption, customers or users tend to get quite sad. This talk will explore two recent additions to systemd that aim to make these service interruptions as minimal as possible. systemd v260 and v261 added two features which aim to let service owners minimize downtime. Firstly, when services use dm-verity images, profiling data showed that the largest bottleneck by far was loading the new images, which used to happen after the running service had already stopped during a unit restart. Since v260, when very specific conditions are met, this bottleneck has been removed from the critical path. Secondly, a common issue causing slowdowns for services is having to serialize and deserialize large amount of state from slow disks after a reboot due to an OS update. Since v261, systemd supports the kernel's Live Update Orchestration, and uses it to persist the content of the FD Store across a kexec reboot. Services can use a memfd for their runtime state, hand it over the systemd when they are stopped/restarted, and they will get it back and can mmap it and use it straight away on restart. With LUO support, services can rely on this even across a kexec, removing the need to use serialize and deserialize state to disk. This talk will explore both features, explaining how they work and how they can be used. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/XFZKN8/

Video:asg2026-393-eng-systemd_reducing_impact_of_servicing_interruptions_on_restart_reboot_sd.mp4

In this talk, we'll walk through the architecture of Amutable's OS image build system. We'll dive into how we mirror rpms from Fedora and orchestrate building those rpms from source, as well as building OS images with buck2, an open source monorepo build system. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/YSR3PP/

Video:asg2026-392-eng-Hold_My_Beer_Im_Building_100_Images_at_Once_sd.mp4

This talk will give an update on make NoNewPrivs the default on a Linux distribution. Running systems with “NoNewPrivs” by default is already possible without issues for many use cases, but there are still some cases where it doesn't work. As followup to last year's presentation “Accessing shadow records via varlink” this session provides an up-to-date status report on the progress made and the issues that remain. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/CG9K7H/

Video:asg2026-387-eng-Making_NoNewPrivs_the_default_sd.mp4

`systemd-sysupdate` provides a lot of incredibly useful primitives for doing system updates, but it is still mostly designed around old-school `SHA256SUMS` distribution and static transfer definitions. The need to have more autonomous systems for updating systems and adding components became quite clear when we started developing such systems at Amutable. This approach also dove-tailed nicely into developing an autonomous provisioning system using the same underlying mechanism, [leading us to build Quarry](https://amutable.com/blog/distributing-images-quarry) (which has been [recently open sourced](https://github.com/amutable-systems/quarry)). This talk will contain a discussion of the architecture we came up with, the pain points with integration with `systemd-sysupdate`, as well as some of the caveats that such an architecture has with generic distributions. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/BBJYFH/

Video:asg2026-431-eng-Modernising_Software_Distribution_with_Quarry_hd.mp4

This talk will give an update on make NoNewPrivs the default on a Linux distribution. Running systems with “NoNewPrivs” by default is already possible without issues for many use cases, but there are still some cases where it doesn't work. As followup to last year's presentation “Accessing shadow records via varlink” this session provides an up-to-date status report on the progress made and the issues that remain. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2026/talk/CG9K7H/

Video:asg2026-387-eng-Making_NoNewPrivs_the_default_hd.mp4