On-device AI needs a feature resource ladder
In short: Apple's elastic on-device model shows why each AI feature needs a tested resource tier, hardware boundary and cloud fallback.
On 8 June 2026, Apple introduced AFM 3 Core Advanced: a 20-billion-parameter on-device model that activates 1 to 4 billion parameters for a request. The full model sits in flash storage while a prompt-level router loads a subset into active memory.
AFM 3 turns on-device AI into a product-routing problem as well as a model-compression problem. Every feature needs a feature resource ladder that binds its supported hardware, active parameter tier, measured quality envelope and cloud fallback. Review those rows separately and the user can receive a different product without anyone noticing which boundary moved.
Apple AFM 3 makes model size a runtime choice
Apple's architecture note says AFM 3 Core Advanced selects experts per prompt and can periodically revise the selection during generation. Apple calls the result inference-time elasticity: each use case gets a predetermined active-parameter budget instead of one model size for every task.
This is a different control problem from the previous dense on-device model. Apple's 2025 technical report described a roughly 3-billion-parameter model using 2-bit quantisation-aware training and cross-layer KV-cache sharing. Those optimisations helped one model fit. AFM 3 Core Advanced adds a choice about how much of a larger model each feature may load.
Apple also reports feature results at a named activation tier:
| Apple evaluation | Active parameters | AFM 3 Core Advanced | Production or previous baseline |
|---|---|---|---|
| General voice, 5-point MOS | 1B | 4.15 | 3.87 |
| Conversational voice, 5-point MOS | 1B | 4.24 | 3.82 |
| Dictation overall preference | 1B | 44.7% | 17.6% |
Apple's first-party feature evaluations; the dictation comparison also reported 37.7% ties.
The resource tier and feature result arrive together. “Runs on device” is too coarse once one feature can activate a different slice from another.
The feature resource ladder I would require
This is editorial judgement, not a Trigent or client deployment I ran. I would require one reviewed row per feature: task and model version; eligible device class; active parameters and memory budget; measured p95 latency, energy and task quality; then the fallback route with its trigger.
The row should fail closed when evidence is missing. A dictation quality score cannot approve an extraction feature. A one-billion-parameter result cannot silently approve the four-billion-parameter tier. A lab latency number cannot stand in for battery and thermal behaviour across the supported devices.
The fallback belongs in the same row because it changes the data path. Apple separates on-device work from harder requests that can use Private Cloud Compute. In June, Apple extended PCC to NVIDIA GPUs in Google Cloud while retaining requirements such as stateless computation, no privileged runtime access and verifiable transparency. Those controls do not make cloud execution the same boundary as local inference.
Where Apple's evidence holds and where it breaks
Apple's note supports the architecture, the 1-to-4-billion activation range and the reported feature comparisons. It does not publish an independent reproduction, a per-feature battery table, thermal results across devices or a third-party application study. The models were still in a beta period when Apple described the evaluations.
So I would use AFM 3 as evidence that elastic on-device inference is a real product architecture, not as a universal performance claim. The ladder must be filled with measurements from the exact feature, operating-system build and device set being released.
What's in it for you
- Review the resource tier and feature acceptance result together.
- Treat a cloud fallback as a data-path change with its own trigger and controls.
- Re-run the row when the model, operating system, supported hardware or fallback changes.
On-device is a routing promise; version the resource tier, evidence and fallback that make the promise true.
