--- title: "The forgetful CPU (Linux on M4)" slug: the-forgetful-cpu-linux-on-m4 url: https://listedarticles.com/articles/the-forgetful-cpu-linux-on-m4 canonical_url: https://yuka.dev/blog-2026-10-02-linux-m4.html content_type: blog_post language: en published_at: 2026-10-02T00:00:00.000Z updated_at: 2026-10-02T18:14:46.029Z author: "Yureka Lilian" author_url: https://yuka.dev/ authored_by: human publisher: "Yureka Lilian" publisher_url: https://yuka.dev/ topics: ["Linux", "Hardware", "Systems Programming", "Engineering"] license: all-rights-reserved word_count: 1650 reading_minutes: 7 citation: "Yureka Lilian, Yureka Lilian. \"The forgetful CPU (Linux on M4).\" 2 Oct 2026. https://yuka.dev/blog-2026-10-02-linux-m4.html (all-rights-reserved)" # The full text follows. The web page shows an extract and sends readers # to the source above; quote the citation and link the canonical URL. --- # The forgetful CPU (Linux on M4) > A detailed write-up of first-booting Linux on an M4 Mac mini, covering CPU quirks, bring-up details, and the hardware surprises along the way. # The forgetful CPU (Linux on M4) *This blog post goes into quite some detail about how I first booted Linux on my M4 Mac mini. I encourage you to look up terms and concepts you don’t know, since I cannot explain all the background in this post ;)* Before saying anything further, I need to thank the entire Asahi Linux team for all their prior work and help during the journey. Please consider donating to the Asahi Open Collective if you want to see more mainline Linux on Apple Silicon work! ### The Beginnings In November 2024, I bought an M4 Mac mini, gambling that it would be similar to the M1-M3 Apple Silicon machines and could be quickly supported in Asahi Linux. While the M4 was sitting on my desk for several months, more details started surfacing about this SoC. It turned out to be more difficult, since the M4 machines are the first generation of Apple Silicon to mandate SPTM (Secure Page Table Monitor), which provides hardening against vulnerabilities in the XNU kernel of macOS. In previous generations, the Linux bringup was largely based on MMIO traces captured using the m1n1 hypervisor, allowing the analysis of interactions between the original macOS drivers and the hardware. With SPTM, major changes to m1n1 are required to get macOS running under the hypervisor, and these changes certainly go beyond what I could come up with as a newbie in this space. Still, it wasn’t entirely a lost cause. In parallel to the hypervisor work, I started my first attempts to boot Linux on the M4. This meant disabling strict boot security, installing m1n1 as a custom boot object via macOS recovery1, and obtaining a serial console2 to inspect the logs. ### Locked registers Initially, m1n1 was only able to start in BRINGUP mode and immediately crashed while attempting to initialize GXF3 otherwise. GXF functionality turned out to be disabled/locked in raw boot mode on these M4+ SoCs, so making its initialization conditional and skipping it on these machines was the correct thing to do. Besides the disabled GXF feature, there was also the RVBAR (Reset Vector Base Address Register): a place in memory (one per CPU core) that determines where the core starts executing when powered on. The m1n1 code writes its entrypoint address to the RVBAR for each core when booting the kernel or chainloading another m1n1. Writing to it also led to a crash on M4, but long story short, it already contained the correct value so this write needed to be skipped as well. `2024-12-01 17:37 can confirm this state gives a working usb proxy, as in a "Generic m1n1 uartproxy v1.4.17-61-ga24ff77" turns up in lsusb, and I can open a shell :)` ### `debug_putc` At this point, a long time went by without me touching the Mac mini. At the Chaos Communication Congress at the end of 2025, I found some new motivation. Finally, I hacked together a very minimal device tree containing only the CPU cores and AIC interrupt controller and loaded the Linux kernel (using m1n1’s `linux.py`) with the `earlycon` parameter, but I did not see any output after “Vectoring to next stage”. Since I wasn’t getting any useful output from the kernel, I could only take wild guesses at what was going wrong, right? I decided to try the brute-force method: good old `println`-debugging. I took the `debug_putc` assembly routine from m1n1 and adjusted it to print a single ‘a’ character 4. I inserted this into the Linux kernel code very early in boot, and sure enough, I got an ‘a’ after “Vectoring to next stage”! Essentially, I bisected the Linux boot code and landed at the MMU init code (which is still *very* early, still in the assembly code in `arch/arm64/kernel/head.S`). Was the MMU initialization crashing the CPU somehow? Not quite: the UART is accessed using memory-mapped I/O. Once the MMU is enabled, all memory accesses are directed to virtual addresses, which are mapped to a corresponding physical addresses through the page-tables. While m1n1 creates mappings to expose the MMIO address space at identical virtual addresses, Linux does not do this, meaning we end up accessing unmapped space instead of the UART once the MMU is enabled. I modified the initial pagetables to add this 1:1 mapping for the MMIO space, and now my `debug_putc` worked much further into the boot process. Bisecting again, the print now worked up to somewhere in the interrupt controller initialization! I narrowed it down to a write to the implementation-specific CPU register `SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2`, which triggered the new crash. After I commented out this write, the kernel booted to a shell. Let’s goooo! `SYS_IMP_APL_VM_TMR_FIQ_ENA_EL2`, which is related to virtualization, has since been unlocked in new iBoot versions, so commenting out the write is no longer necessary. Now that I knew that Linux code was actually being executed, I looked once again into why I hadn’t gotten any `printk` output on the serial console earlier despite the `earlycon` boot parameter. ``` 2026-01-23 12:35 added earlycon=s5l,0x3ad200000 to my bootargs 2026-01-23 12:36 and now I'm actually getting useful output before the crash is happening! ``` Sure enough, the device tree was just missing `stdout-path = "serial0"`, and after adding it, I got full register dumps and stack traces from Linux on early crashes. ### Secondary cores and WFI For now, m1n1 didn’t start the secondary cores because `smp_start_offset` was missing (an offset hardcoded in m1n1; without it, `smp_init` is skipped). I tried the offset used for base M1 - M3 and was able to start the secondary cores. Once I attempted loading Linux again, I arrived at another mysterious crash. Previous Apple Silicon CPUs already had known quirks regarding the WFI instruction. Depending on the state of the *chicken bit* (a bit in a register, that allows the vendor to disable some CPU optimization or feature, or to *chicken out*) called `ARM64_REG_CYC_OVRD_ok2pwrdn_force_mask` in the XNU OSS code, the WFI instruction causes the CPU registers x0-x31 to be zeroed on these previous generations. XNU saves these registers to the stack before the WFI and restores them afterwards. In M1-M3 SoCs, m1n1 disables this behavior so the CPU behaves like other arm64 processors. The Asahi kernel later specifically re-enables the behavior to allow the CPU cores to reach deeper sleep states and save power. This is also required to allow one core in the cluster to boost to higher clock speeds when all other cores are in this deeper WFI sleep state. It seems this chicken bit is either locked or has been removed on M4, and the default behavior does not comply with the ARM64 specification (specifically: “If the system is configured such that the WFI instruction can be completed, then the WFI instruction must not cause a loss of architectural state.”5). In April 2026, I managed to boot Linux on the M4 with all cores enabled by replacing all WFI and WFIT (Wait For Interrupt with Timeout) instructions in my kernel with NOP (no-op). With this, a long journey towards an upstream solution for the WFI problem began. At first, I looked into how errata (misbehaviors of the silicon) are usually handled. There is a whole framework in Linux to allow it to patch itself in memory during early boot. However, this approach was discarded because it is very difficult to accurately detect cases in which WFI should be NOP’ed. Namely, a virtual machine running under the macOS hypervisor would also trigger the erratum logic, but WFI is trapped by the macOS hypervisor and used to schedule different guests efficiently. Detecting virtualization (especially when nested virtualization is enabled) is also complicated, so Will Deacon suggested an alternative solution: The kernel should gain support for disabling WFI idle using a new bootarg6, and then m1n1 can add the appropriate bootargs conditionally when booting on bare-metal machines known to have broken WFI. We will then add a mechanism to allow Linux to put the cores into sleep states. For now, the downstream cpuidle-apple driver can be used for this, but Sven’s PSCI EFI conduit work is very promising for a future upstream solution. I’m happy to report that the mechanism for preventing Linux from crashing on WFI and WFIT instructions has been merged into mainline Linux7 8 and m1n19, so the latest releases of these can boot natively with secondary cores on M4 Macs! ### Next steps This work has so far uncovered and addressed a fundamental issue with running Linux on the M4 and later Apple Silicon chips, allowing Linux to boot to a shell with all cores usable (the same WFI workarounds have been found to work on M4 Pro, M4 Max and M5 chips!). Reverse engineering the peripherals, on which I will not go into details here, is going at a slow but steady pace. Sven’s tireless work on making the m1n1 hypervisor able to boot and trace macOS on these devices is going to be very useful for tackling the more complex components such as the builtin camera, display controller, and GPU initialization. For the most part, I’m submitting my work directly to the respective upstream projects, allowing everyone to take advantage of it. Sometimes it would be nice if certain projects getting a lot of funding were more transparent about how they’re benefitting from the upstream projects’ progress. **If you want to support my work on Linux on the M4, I take donations over at LiberaPay or GitHub Sponsors. Please also consider donating to the Asahi Open Collective for more Linux on Apple Silicon mainlining.** As always, I hope you could learn something :)