← back to the archiveCover illustration for “Portability must include the learning layer”
ESSAYday 48·1 week ago·by Andy Padia

Portability must include the learning layer

Everyone negotiates portable model endpoints — and leaves the evals, graders, trace history, and feedback labels trapped in one vendor's control plane. The model is the swappable part; the learning layer is the asset.

Efi Pylarinou's agentic-finance review this week made an argument that deserves to escape the fintech niche it landed in: the compounding asset in enterprise AI is not the model, it is the learning layer — the retained evaluations, contextual intelligence, and production feedback a firm accumulates. It's a strategic claim, not a benchmarked one; I couldn't find a comparative study proving ownership of these assets produces better outcomes, so hold it as a well-reasoned thesis. But it names precisely the mistake I watch enterprises make in every vendor negotiation, and the mistake is expensive.

Here is the mistake. Enterprises have finally learned to demand model portability — portable endpoints, no proprietary API lock, the ability to swap providers. Good. And then they leave everything that actually accumulated value trapped in one vendor's control plane: the prompts, the graders, the trace history, the feedback labels, the failure clusters, the retrieval annotations. They negotiated the right to move the engine and forgot to negotiate the right to move everything the engine learned. That is portability of the swappable layer only, which is barely portability at all.

The layer that compounds is the one nobody exports

Think about what a mature AI deployment actually is, a year in. The model is the same model anyone can rent. What's yours — what took a year of production to build and cannot be bought — is the accumulated intelligence around it: the eval cases drawn from your real work, the reviewer decisions that turned tacit expert judgment into graded criteria, the failure clusters you discovered the hard way and now test against, the retrieval annotations that taught the system which of your tables is trustworthy, the trace outcomes showing what actually worked in production, and the version history tying every behavior change to the data that caused it.

That is the asset. OpenAI's own disclosure about its internal data agent — 3,500-plus users, 600 petabytes, 70,000 datasets, and an architecture that is mostly human annotations, lineage, and organizational context — is the same lesson at scale: the model was the least interesting part; the accumulated context was the product. Your enterprise version is smaller but identical in kind. And nearly every organization I see has it sitting inside a vendor platform, in that vendor's schema, with no export path — which means the "portable" model you negotiated is a body you can move and a memory you can't.

rendering diagram…

Real portability is portability of the memory

The reframe: you are only as portable as your least-portable layer, and that layer is almost always the learning layer. A model you can move but whose evals, graders, and feedback history you cannot is a false freedom — switching vendors means starting the year of accumulation over, which no one will do, which means you are locked in exactly as hard as if the model itself were proprietary. The lock just moved to where you weren't looking.

So the negotiation has to move with it. Treat the learning-layer artifacts as exportable, vendor-neutral, and owned — the same way a serious enterprise treats its data. Concretely, that means contract terms and architecture that guarantee: eval suites and graders live in your repo in an open format, not the vendor's console; trace and feedback history is exportable in bulk, on demand, in a documented schema; retrieval annotations and reviewer decisions are yours by ownership clause, not the platform's by default; and there is a written retention and egress path you have actually tested. Same discipline I've argued for eval records, semantic layers, and open-model serving — this is the umbrella over all of them: the compounding assets belong to you, in formats you control, or they don't really belong to you.

At work, the diagnostic I now run in every platform decision: I ask to see the export. Not the import, not the demo — the export. Show me eval suites, trace history, and feedback labels leaving the platform in an open format I could load into a competitor tomorrow. Vendors building for genuine portability can show it. Vendors building a moat get visibly uncomfortable, because the un-exportable learning layer is their lock-in strategy, and a customer who asks to see the exit is a customer they're about to lose leverage over. That discomfort is the most useful signal in the whole procurement.

Steal this clause set for your next AI contract: (1) evals and graders stored in customer-owned repos, open format; (2) bulk export of trace and feedback data on demand, documented schema, no fee; (3) explicit customer ownership of annotations and reviewer decisions; (4) a tested egress runbook, refreshed annually. Then run the drill once a year — actually export the learning layer and confirm it loads elsewhere — exactly as you'd rehearse a serving-provider exit. Portability you haven't exercised is a clause, not a capability.

You negotiated the right to move the model and forgot the year of learning that made it useful — own the evals, traces, and feedback in open formats, or you're locked in wherever the memory lives.

#portability#evals#vendor-lock-in#enterprise-ai#strategy
← older drop
Agent-built apps need expiry dates
newer drop →
Provenance attests the factory, not the code

related drops

explore all 78 drops →
← back to the archiveday 59