Getting 50 GB/S Back from the Apple Neural Engine

The article describes a significant Apple M3 Neural Engine performance erratum that can cut DRAM weight-streaming throughput from a nominal 45–60 GB/s to about 17–19 GB/s when model weight sizes hit 1 MiB-aligned boundaries, affecting a sizable share of deployed models. For CIOs and technology leaders, the strategic takeaway is that AI inference performance on edge/client devices can be unexpectedly constrained by low-level hardware quirks, creating material impacts on latency, user experience, and capacity planning even when software is correct. IT organizations should treat model sizing and hardware-specific benchmarking as a first-class operational concern, since avoiding these pathological dimensions more than doubled throughput in tested workloads.