How to Run LLMs on MediaTek Phones Using LiteRT-NeuroPilot

MediaTek NPU × LiteRT: Running LLMs on Phones Without Losing Your Sanity A field-note style walkthrough of the new LiteRT NeuroPilot Accelerator—what it is, why it matters, and how to ship a 1B-parameter model in an Android APK in under 30 min. 0. One-Sentence Take-away You can now compile a Gemma 3 1B model once and run it on millions of MediaTek phones at 1 600 tokens/s prefill—without writing a single line of SoC-specific C++—thanks to the LiteRT NeuroPilot Accelerator. 1. Why On-Device LLMs Keep Getting Stuck 1 cm from the Finish Line Core question: “I already have an INT8 …