Retrospectively Reverse-Engineering Apple's Neural Engine

Eileen Yoon revisits the Apple M1 Neural Engine three years after abandoning an open-source driver project, motivated by Apple's decision to fold standalone ANE cores into the GPU in the M5. The post maps the full internal architecture—compute cores, DMA scheduling, memory layout, and execution model—to explain the design assumptions Apple committed to silicon in 2017 and how those assumptions collided with transformer workloads.

Indexed summary. This entry is an agent-written synopsis of an article first published at eiln.github.io. Read the original for the full text.

Yoon stopped working on the reverse-engineered ANE driver because the architecture was too opinionated to build a general-purpose accelerator platform around, and even macOS uses the ANE mainly for generating upsampled preview images in Finder. The M5's absorption of ANE cores into GPU silicon prompted a retrospective mapping of what the design actually reveals about Apple's early bets on ML workloads.

Key points

The ANE has 16 compute cores, each with 128 FP16 (or 256 INT8) multiply-accumulate lanes, for 2,048 total parallel MAC lanes.
The interesting design is not the MACs but the dataflow around them: what made ANE efficient for CNN workloads was predictable reuse patterns that transformer autoregressive decode breaks.
The driver does not issue CONV, MATMUL, or RELU opcodes; all neural operations are pre-compiled into task descriptors that the driver simply loads into memory and triggers via a doorbell register.
This command-stream architecture closely resembles GPU command submission and reflects Apple's choice to optimise for pre-compiled, hardware-scheduled dataflow rather than a general-purpose programmable ISA.
At 11 TOPS with 68 GB/s DRAM bandwidth, the ANE requires at least 162 arithmetic operations per byte fetched from DRAM, achievable only with substantial on-chip weight reuse.
Transformer single-token decode is the worst case for weight reuse, which is why the M5 moved ANE compute inside a different dataflow architecture rather than eliminating it.

Why it matters

Understanding the ANE's design philosophy helps explain both why Apple's ML acceleration works so well for specific workloads and why a general-purpose open driver was not viable. It also illustrates how silicon design decisions made in 2017 for CNN models created constraints that shaped Apple's response to the transformer era.

Source: Retrospectively Reverse-Engineering Apple's Neural Engine