Neural Engine

Getting 50 GB/s Back Out of the ANE

Eileen Yoon identifies an RTL performance bug in the Apple M3 Neural Engine where DRAM throughput collapses from 45–60 GB/s to 17–19 GB/s whenever total weight size is an exact multiple of 1 MiB. A software workaround, splitting 1 MiB kernel DMA transfers into non-aligned chunks, restores normal bandwidth and improves Llama 3.2 1B token throughput from 10 to 24 tokens per second.

Research · Apple Silicon · Neural Engine · Performance · Hardware

Retrospectively Reverse-Engineering Apple's Neural Engine

Eileen Yoon revisits the Apple M1 Neural Engine three years after abandoning an open-source driver project, motivated by Apple's decision to fold standalone ANE cores into the GPU in the M5. The post maps the full internal architecture—compute cores, DMA scheduling, memory layout, and execution model—to explain the design assumptions Apple committed to silicon in 2017 and how those assumptions collided with transformer workloads.

Blog post · Apple Silicon · Neural Engine · Reverse Engineering · Machine Learning