{"article":{"slug":"building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun","title":"Building a VMM from scratch, #0: Why BoxLite is replacing libkrun","subtitle":null,"summary":"BoxLite explains why every box runs in a micro-VM on libkrun today, why they are replacing it with an in-house VMM, and how this build-in-public series will walk through writing a VMM from scratch.","content_type":"blog_post","language":"en","canonical_url":"https://blog.boxlite.ai/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun","author":{"name":"BoxLite","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"BoxLite","url":"https://blog.boxlite.ai/","listing_slug":null,"listing":null},"topics":[{"name":"Systems Programming","slug":"systems-programming","url":"https://listedarticles.com/topics/systems-programming"},{"name":"Infrastructure","slug":"infrastructure","url":"https://listedarticles.com/topics/infrastructure"},{"name":"Open Source","slug":"open-source","url":"https://listedarticles.com/topics/open-source"}],"about_listings":[],"cover_image_url":null,"license":"CC0-1.0","word_count":984,"reading_minutes":4,"published_at":"2026-10-11T12:00:00.000Z","added_at":"2026-10-12T05:11:15.752Z","updated_at":"2026-10-12T05:11:15.752Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun","markdown_url":"https://listedarticles.com/articles/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun.md","example":false,"citation":"BoxLite, BoxLite. \"Building a VMM from scratch, #0: Why BoxLite is replacing libkrun.\" 11 Oct 2026. https://blog.boxlite.ai/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun (CC0-1.0)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://blog.boxlite.ai/building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun"},"body_markdown":"# Building a VMM from scratch, #0: Why BoxLite is replacing libkrun\n\nPart #0 of a build-in-public series: what BoxLite is, why we started on libkrun, why we're replacing it, and how a VMM works.\n\nEvery BoxLite box runs in its own micro-VM, and today libkrun runs that VM. We're replacing it with a VMM we write ourselves. This series builds it in public, one merged PR at a time.\n\n## 1\\. What BoxLite is\n\nBoxLite is the compute substrate for AI agents — light enough to embed on your laptop, elastic enough to power an agentic cloud. Each box is a hardware-isolated micro-VM that runs any OCI image and keeps its state across turns.\n\nFigure 1. One engine in two places: the embedded BoxLite runtime starts one micro-VM per box, on your machine or on a cloud runner.\n\nThe part that runs each micro-VM is the VMM, the virtual machine monitor. That part is the subject of this series.\n\n## 2\\. Why we started on libkrun\n\nlibkrun describes itself as \"a dynamic library that allows programs to easily acquire the ability to run processes in a partially isolated environment\", on KVM (Linux) and HVF (macOS on Apple silicon), behind \"a simple C API\". That fit BoxLite:\n\n * A library, not a daemon: BoxLite embeds in your app, with no root and no background service.\n * One API for KVM on Linux and Hypervisor.framework on macOS.\n * The guest kernel comes with it (libkrunfw), along with virtio-fs, virtio-blk and vsock.\n\n\n\nlibkrun is how BoxLite runs boxes on macOS and Linux today.\n\n## 3\\. Why we're replacing it\n\nlibkrun is clear about what it isn't: its [README]() lists \"Become a generic VMM\" and \"Be compatible with all kinds of workloads\" as non-goals. BoxLite's needs grew past those goals:\n\n * The VM's outcome. libkrun ends the process itself, so the caller gets no exit status, and every start failure comes back as \n \n -EINVAL\n\n.\n * The kernel as a shared library. libkrunfw is loaded with \n \n dlopen\n\n, and that keeps breaking on some hosts; [#1512]() lists six such issues.\n * Devices. Each directory volume gets its own device, and with three a box fails to start with \n \n IrqsExhausted\n\n([#935]()).\n * What comes next. Hot-plug mounts, live resizing, memory snapshots for AutoPause ([#1003]()) and GPUs all need a VMM we control.\n\n\n\n### What we're not doing\n\n * Forking or patching libkrun. It ships unchanged until the new VMM is the default, stays one release as a fallback, and is then removed.\n * Building a general-purpose VMM. It runs one VM per process with BoxLite's devices only; confidential computing, live migration and Intel Macs are out of scope.\n\n\n\n### What it costs\n\n * The new VMM needs macOS 15 or later, for Hypervisor.framework's in-kernel interrupt controller. macOS 12–14 keep libkrun until cutover.\n * A long road: first boot (M1), then a real box (M2), volumes (M3), hardening (M4) and cutover (M5).\n\n\n\n## 4\\. How a VMM works, in one screen\n\nFigure 2. Inside one machine. The runtime spawns a jailed boxlite-shim per box, and the VMM inside it runs the micro-VM. PROPOSED marks what this series replaces.\n\nA hypervisor (KVM or Hypervisor.framework) runs guest instructions on the CPU. The VMM is the ordinary process around it: it gives the guest memory, creates vCPUs, loads the kernel and emulates every device. Its heart is one loop:\n\nFigure 3. Run a vCPU until it exits, handle the exit, and run it again.\n\nThe first guest our new KVM backend ran was 7 bytes long:\n \n \n mov dx, 0x3f8 ; COM1 serial port\n mov al, 'K'\n out dx, al ; KVM exits to the VMM: port 0x3f8, byte 'K'\n hlt\n \n\nEverything else is detail, and each detail gets a part in this series:\n\nPiece| What it is| Part \n---|---|--- \nMemory| Guest RAM is host memory; an unmapped address is a device| #1 \nExits| One exit contract for KVM, Hypervisor.framework and, later, WHP| #2 \nStopping| Pulling a vCPU out of the guest, safely| #3 \nCPU state| Registers, CPUID and MSRs on x86| #4 \nBoot| The VMM is the bootloader| #6 \nDevices| Serial, RTC and reset first, then virtio| #7, #10–12 \n \n## 5\\. The series\n\nStatus as of October 11, 2026. A part goes out only after its code merges, so every claim links to a merged PR.\n\nArc| Parts| Ships when \n---|---|--- \n1\\. First instructions| #1 Your first VM is 7 bytes · #2 Design the exit contract first · #3 Stopping a vCPU is the hard part · #4 x86 CPU state, by hand · #5 A kernel we build ourselves| Code merged \n2\\. First boot (M1)| #6 You are the bootloader · #7 Boring devices first · #8 On Apple silicon, you decode the exits · #9 One kernel, three hosts| As M1 lands \n3\\. First box (M2–M3)| #10 virtio from scratch · #11 Disks, sockets, network · #12 One virtio-fs device for every volume · #13 Our agent as PID 1| M2–M3 \n4\\. Ship it (M4–M5)| #14 Guest input must never panic the host · #15 Measured against libkrun · #16 Deleting libkrun| M4–M5 \nLater| Memory snapshots, hot-plug mounts, GPUs, our own network stack| After M5 \n \n### Where we are\n\n * ✅ Design and one hypervisor contract for KVM, Hypervisor.framework and, later, Windows\n * ✅ KVM on x86_64: VMs, memory, the run loop, I/O completion, kicks, boot registers, CPUID and MSRs\n * ✅ A reproducible build of our pinned Linux 6.12 kernel for x86_64\n * ⏳ M1: first boot on macOS arm64, Linux x86_64 and Linux arm64, checked in CI\n\n\n\nFollow along on the [RSS feed]() and in every PR on [GitHub]().\n\n## 6\\. Come build it with us\n\nBoxLite is open source, and so is every step of this VMM. We'd love your help:\n\n * Challenge the design. Read the [VMM design doc]() and tell us where it's wrong.\n * Pick up M1 work. [#1698]() lists the open slices and their dependencies: devices, kernel loaders, the Hypervisor.framework backend, arm64 KVM and three-host CI.\n * Ask and discuss. Open a [GitHub issue]() or find us on [Discord]().\n\n\n\nStart with [CONTRIBUTING.md](). Your first PR asks you to sign our contributor license agreement.","body_html":"

Building a VMM from scratch, #0: Why BoxLite is replacing libkrun

\n

Part #0 of a build-in-public series: what BoxLite is, why we started on libkrun, why we're replacing it, and how a VMM works.

\n

Every BoxLite box runs in its own micro-VM, and today libkrun runs that VM. We're replacing it with a VMM we write ourselves. This series builds it in public, one merged PR at a time.

\n

1. What BoxLite is

\n

BoxLite is the compute substrate for AI agents — light enough to embed on your laptop, elastic enough to power an agentic cloud. Each box is a hardware-isolated micro-VM that runs any OCI image and keeps its state across turns.

\n

Figure 1. One engine in two places: the embedded BoxLite runtime starts one micro-VM per box, on your machine or on a cloud runner.

\n

The part that runs each micro-VM is the VMM, the virtual machine monitor. That part is the subject of this series.

\n

2. Why we started on libkrun

\n

libkrun describes itself as "a dynamic library that allows programs to easily acquire the ability to run processes in a partially isolated environment", on KVM (Linux) and HVF (macOS on Apple silicon), behind "a simple C API". That fit BoxLite:

\n
  • A library, not a daemon: BoxLite embeds in your app, with no root and no background service.
  • One API for KVM on Linux and Hypervisor.framework on macOS.
  • The guest kernel comes with it (libkrunfw), along with virtio-fs, virtio-blk and vsock.
\n

libkrun is how BoxLite runs boxes on macOS and Linux today.

\n

3. Why we're replacing it

\n

libkrun is clear about what it isn't: its README lists "Become a generic VMM" and "Be compatible with all kinds of workloads" as non-goals. BoxLite's needs grew past those goals:

\n

  • The VM's outcome. libkrun ends the process itself, so the caller gets no exit status, and every start failure comes back as 

    -EINVAL
\n

.

\n

  • The kernel as a shared library. libkrunfw is loaded with 

    dlopen
\n

, and that keeps breaking on some hosts; #1512 lists six such issues.

\n

  • Devices. Each directory volume gets its own device, and with three a box fails to start with 

    IrqsExhausted
\n

(#935).

\n
  • What comes next. Hot-plug mounts, live resizing, memory snapshots for AutoPause (#1003) and GPUs all need a VMM we control.
\n

What we're not doing

\n
  • Forking or patching libkrun. It ships unchanged until the new VMM is the default, stays one release as a fallback, and is then removed.
  • Building a general-purpose VMM. It runs one VM per process with BoxLite's devices only; confidential computing, live migration and Intel Macs are out of scope.
\n

What it costs

\n
  • The new VMM needs macOS 15 or later, for Hypervisor.framework's in-kernel interrupt controller. macOS 12–14 keep libkrun until cutover.
  • A long road: first boot (M1), then a real box (M2), volumes (M3), hardening (M4) and cutover (M5).
\n

4. How a VMM works, in one screen

\n

Figure 2. Inside one machine. The runtime spawns a jailed boxlite-shim per box, and the VMM inside it runs the micro-VM. PROPOSED marks what this series replaces.

\n

A hypervisor (KVM or Hypervisor.framework) runs guest instructions on the CPU. The VMM is the ordinary process around it: it gives the guest memory, creates vCPUs, loads the kernel and emulates every device. Its heart is one loop:

\n

Figure 3. Run a vCPU until it exits, handle the exit, and run it again.

\n

The first guest our new KVM backend ran was 7 bytes long:

\n
mov dx, 0x3f8   ; COM1 serial port\nmov al, 'K'\nout dx, al      ; KVM exits to the VMM: port 0x3f8, byte 'K'\nhlt
\n

Everything else is detail, and each detail gets a part in this series:

\n
PieceWhat it isPart
MemoryGuest RAM is host memory; an unmapped address is a device#1
ExitsOne exit contract for KVM, Hypervisor.framework and, later, WHP#2
StoppingPulling a vCPU out of the guest, safely#3
CPU stateRegisters, CPUID and MSRs on x86#4
BootThe VMM is the bootloader#6
DevicesSerial, RTC and reset first, then virtio#7, #10–12
\n

5. The series

\n

Status as of October 11, 2026. A part goes out only after its code merges, so every claim links to a merged PR.

\n
ArcPartsShips when
1. First instructions#1 Your first VM is 7 bytes · #2 Design the exit contract first · #3 Stopping a vCPU is the hard part · #4 x86 CPU state, by hand · #5 A kernel we build ourselvesCode merged
2. First boot (M1)#6 You are the bootloader · #7 Boring devices first · #8 On Apple silicon, you decode the exits · #9 One kernel, three hostsAs M1 lands
3. First box (M2–M3)#10 virtio from scratch · #11 Disks, sockets, network · #12 One virtio-fs device for every volume · #13 Our agent as PID 1M2–M3
4. Ship it (M4–M5)#14 Guest input must never panic the host · #15 Measured against libkrun · #16 Deleting libkrunM4–M5
LaterMemory snapshots, hot-plug mounts, GPUs, our own network stackAfter M5
\n

Where we are

\n
  • ✅ Design and one hypervisor contract for KVM, Hypervisor.framework and, later, Windows
  • ✅ KVM on x86_64: VMs, memory, the run loop, I/O completion, kicks, boot registers, CPUID and MSRs
  • ✅ A reproducible build of our pinned Linux 6.12 kernel for x86_64
  • ⏳ M1: first boot on macOS arm64, Linux x86_64 and Linux arm64, checked in CI
\n

Follow along on the RSS feed and in every PR on GitHub.

\n

6. Come build it with us

\n

BoxLite is open source, and so is every step of this VMM. We'd love your help:

\n
  • Challenge the design. Read the VMM design doc and tell us where it's wrong.
  • Pick up M1 work. #1698 lists the open slices and their dependencies: devices, kernel loaders, the Hypervisor.framework backend, arm64 KVM and three-host CI.
  • Ask and discuss. Open a GitHub issue or find us on Discord.
\n

Start with CONTRIBUTING.md. Your first PR asks you to sign our contributor license agreement.

","headings":[{"level":1,"text":"Building a VMM from scratch, #0: Why BoxLite is replacing libkrun","id":"building-a-vmm-from-scratch-0-why-boxlite-is-replacing-libkrun"},{"level":2,"text":"1. What BoxLite is","id":"1-what-boxlite-is"},{"level":2,"text":"2. Why we started on libkrun","id":"2-why-we-started-on-libkrun"},{"level":2,"text":"3. Why we're replacing it","id":"3-why-we-re-replacing-it"},{"level":3,"text":"What we're not doing","id":"what-we-re-not-doing"},{"level":3,"text":"What it costs","id":"what-it-costs"},{"level":2,"text":"4. How a VMM works, in one screen","id":"4-how-a-vmm-works-in-one-screen"},{"level":2,"text":"5. The series","id":"5-the-series"},{"level":3,"text":"Where we are","id":"where-we-are"},{"level":2,"text":"6. Come build it with us","id":"6-come-build-it-with-us"}]}}