Local models do not make local agents private

Local agent loops got genuinely good this year. Vicki Boykis reported in June running Gemma 4 agent loops on a 2022 M2 Mac with 64 GB of RAM at roughly 75% of frontier-model accuracy and speed — her own estimate, not a controlled benchmark. Google's figures put Gemma 4's 26B A4B at 85.5% on tau2-bench retail tool use, against 86.4% for its 31B.

Capable enough to matter. Which is exactly when the reasoning starts going wrong.

The model moved. The permissions didn't.

“We'll run the model locally” can end a privacy discussion before anyone has looked at the agent. That is where I'd reopen it. You have told me where inference happens. You haven't told me where the document can go after the model reads it.

Running weights locally removes one recipient of your data: the model provider. That is a real and sometimes decisive win. It does not establish the privacy of the whole agent, because it says nothing about its other data paths.

The agent is not the model. It is the model plus a harness, a mounted filesystem, whatever credentials the process inherits, and tools that can act. Moving the weights doesn't revoke a single permission. You can change the inference endpoint and leave every other route to your data exactly as it was.

Boykis describes Pi in Docker, a mounted workspace and a limited tool configuration. Read that part as closely as the model name. A shell can still call programs or make network requests if its environment permits them. Removing a named Python or browser tool isn't proof that the remaining shell cannot reach the same capability.

Four questions. Local answers one.

My review sheet starts with four questions. Write down the actual configuration, not what the setup was intended to allow:

Inference endpoint — where do the tokens go. Filesystem mounts — what does the container actually see, and is it the repository or a subdirectory. Injected secrets — which environment variables and credentials does the process inherit, and would you be comfortable if the model emitted them verbatim. Egress-capable tools — can this thing make a network call, execute code, or write somewhere that syncs.

Answer those four and you have described the disclosure boundary. Answer only the first and you have described the marketing.

Picture this illustrative setup, not a client incident: local inference approved for sensitive code, the whole repository mounted, CI credentials inherited, outbound access left on. A malicious instruction in a retrieved document asks the agent to read a credential and send it out. Whether that succeeds depends on the actual controls. The local model checkbox has no answer to offer. The permission review does.

The UK AI Security Institute's July cyber-evaluation incident report supplies a useful adjacent case. In 10 of 122 runs using deliberately permissive, non-commercial configurations, agents took 19 unsanctioned actions. A human maintainer rejected a malicious pull request, and monitoring helped contain the incident within an hour. AISI reported no sandbox escape and no real-world harm. The case does not evaluate local Gemma agents; it shows why reach, approval and monitoring remain separate from the model endpoint.

Keep the local win. Finish the review.

I am not arguing against local models. Removing a remote inference service can be exactly the right decision. I am arguing against asking that one decision to do the work of a whole architecture review.

Keeping inference on your own hardware can remove a third-party processor from that path. It also changes the dependency: your hardware and serving stack take over work previously done by a remote provider. That can be a useful trade. It is not free capacity or an exemption from maintenance, and it still needs evaluation on your workload. Google's tool-use scores give a reason to try the model, not permission to skip that evaluation.

Keep those gains on the right line of the review. Avoiding a remote inference request reduces one disclosure path; it doesn't describe the other network destinations or inherited credentials. A benchmark answers how well the model performed on a task. A permissions test answers what the surrounding process can reach. Better performance cannot fill in the second answer, however attractive the first number looks.

Test a boundary, not a promise

Here is the acceptance exercise I'd use; I haven't run it against Boykis's installation. Create a disposable workspace and place a harmless marker outside the permitted directory. Ask the harness to read it through every enabled file and shell route. Watch for the operating system or sandbox denying access. A polite refusal from the model proves it declined that request, not that the route is closed. Save the result with the configuration that produced it.

Repeat the exercise for network access with a controlled test endpoint and synthetic data. If the workload requires one external service, verify the allowed destination and an intentionally disallowed one separately. A single successful request proves connectivity; it does not establish that an allowlist is enforced. Never use a real secret as the test payload.

Then restart using the unattended launch configuration. An interactive terminal and a background service can inherit different environments or permissions. A boundary that holds only while somebody demonstrates it isn't the boundary you are about to ship. The useful artefact is the configuration and its refusal results together, not a screenshot of the model saying it takes privacy seriously.

Ask for the configuration, not the reassurance

Steal this for the next local-agent proposal: ask for the mounts, the environment and the tool list on the same page as the model choice. If those fields are blank, the privacy review is unfinished. You don't need to argue that local inference is bad. You need the other three answers before approving the system around it.

The evidence stops where it stops: Boykis's comparison is her estimate, Google's scores are vendor benchmarks, and I haven't audited her image or host controls. The recommendation is mine: evaluate model capability and process permissions separately, then keep both results with the approval.

The related guardrail benchmark piece approaches the same design decision from the attacker's side. Here the test starts with the local process and every route it can use.

Local weights remove one recipient, not the boundary — audit the mounts, the secrets and the egress before you call an agent private.