It's Mobile World Congress week in Barcelona, which means my timeline has spent the last few days arguing about a keynote. There's always one. This year the theme is, predictably, AI on the device: every vendor with a stand has a slide claiming the model now runs locally, the assistant now anticipates your needs, and the whole thing is private because nothing leaves the handset. The demo reel is glossy. The opinions are loud. Half of them were posted before the speaker had finished.
I've stopped having strong feelings about keynotes themselves, because the keynote is marketing and I knew that going in. What I have feelings about is the gap between the stage and the spec sheet, and that gap is where the actual engineering lives.
Here's the thing the demos never show you. "Runs on device" is a real, genuinely good change when it's true. Inference that doesn't round-trip to a datacentre is faster, works on the train, and keeps your data off someone else's GPU. When a vendor ships that properly, I'll say so plainly, because it's the right direction. But the demo runs on a flagship with a top-tier NPU and a full battery in a room with perfect connectivity for the fallback path. The question I actually care about is what happens on the phone people can afford, eighteen months from now, with a battery that's been through six hundred charge cycles.
Nobody benchmarks the keynote against the thermal envelope. A model that runs beautifully for the ninety seconds of a stage demo is a different beast from one running continuously whilst the chassis is already warm from the screen, the radio, and a navigation app. On-device inference is power-hungry, and power becomes heat, and heat becomes throttling. The honest number isn't tokens per second in a cool room. It's tokens per second per watt after twenty minutes, and that number is never on a slide.
The privacy claim deserves the same scrutiny, and not because anyone's lying. "Nothing leaves the device" is usually true for the headline feature and quietly false for the long tail: the fallback to a bigger cloud model when the local one isn't confident, the telemetry, the model updates. None of that is sinister. It's just more complicated than the four words on the slide, and the complications are exactly the part worth reading the documentation for.
So I'll watch the keynote, same as everyone. I'll even enjoy it, because a good demo is a good demo and there's craft in pulling one off live. But I won't form an opinion from it, because the keynote is the trailer and the spec sheet is the film. The opinions worth having arrive in about three months, when someone with a power meter and a thermal camera tests the thing against the marketing, on a mid-range handset, doing the work continuously. Those posts get a fraction of the engagement and roughly all of the truth.
Until then, everyone has an opinion on the keynote, and mine is that it's a keynote. Ask me again when the review units ship.