Data agents need semantic infrastructure, not smarter models

OpenAI published the architecture behind its in-house data agent — the internal tool that answers analytics questions for more than 3,500 employees over roughly 600 petabytes and 70,000 datasets. If anyone on earth could brute-force "text to SQL" with raw model intelligence, it is the company that makes the models.

They didn't. Read the writeup and count what the architecture is actually made of: schema metadata, table lineage, historical queries, curated human annotations about what tables mean, and access control passed through from the underlying platform. The agent grounds itself in explicit context layers — starting with table usage and human-written domain notes — before it ever touches broader organizational context.

The model is in there somewhere. It is the least interesting part of the diagram.

The failure mode nobody benchmarks

Every data team that has piloted natural-language analytics knows the specific way it fails, and it is not syntax errors. The SQL comes back clean, runnable, and plausible. It just queried orders_v2_legacy instead of orders_current, because both exist and nothing told the model which one the finance team actually trusts. Or it computed "revenue" — a word your company defines one way in the billing warehouse and a subtly different way in the board deck — and produced a number that is correct by one definition and wrong in the meeting.

I have debugged exactly this at work: a client's pilot agent writing beautiful SQL against the wrong grain of a table, monthly figures pulled from a daily snapshot, everything plausible and everything off by design. No amount of model upgrade fixes that, because the failure isn't reasoning. The failure is that the knowledge of which table is real and what the metric means lived in two senior analysts' heads, and the agent was never given a way to read it.

That is the quiet message of OpenAI's disclosure. Their answer to "which table is real" is not a bigger model — it is human annotations, usage statistics, and lineage wired into context. The organisation wrote its tribal knowledge down and made it machine-readable. That is the product.

What the stack actually is

Strip the branding and OpenAI's data agent is a layered grounding system:

rendering diagram…

Four of those five layers existed as a category before agents. We used to call them a data catalog, a lineage graph, a query log, and row-level security. The agent interface is new; the infrastructure it depends on is the semantic layer your data team has been asking to fund since 2019.

My claim, stated plainly: natural-language analytics is mostly a semantic-layer and permissions problem presented through an agent interface. The companies that get magical-seeming data agents in the next two years will be the ones that did unglamorous metadata work in the last two. The model is becoming a commodity component inside somebody else's information architecture — which also means a "data agent" bought off the shelf inherits none of this, because the semantic layer is, by definition, yours.

Note the honest gap too: OpenAI published the architecture but no independent accuracy benchmark for the agent. Even the reference implementation is asking to be trusted on outcomes. If they had a number they loved, I suspect we would have seen it.

The permissions layer is not optional garnish

The detail I would underline twice: access control passes through to the agent. The agent can only read what the asking user could read. That single decision kills the most common enterprise data-agent disaster — the friendly chatbot that happily aggregates tables its user was never allowed to see, because the service account it runs on is God. If your data agent pilot runs on a privileged service account "just for the demo", you do not have a pilot. You have an incident with a delay timer.

Steal this sequencing

If a data-agent initiative is on your roadmap, reorder it. First: pick the fifty tables that answer 80% of real questions, and write one paragraph per table — what it is, what grain, what to never use it for. Second: wire lineage and the query log into retrieval. Third: pass through user permissions, not service-account permissions. Fourth — only fourth — pick the model, and pick it by eval on your own questions, not by leaderboard.

Run the pilot on the annotated fifty tables only, and publish the coverage number to the business — "the agent currently speaks for 50 tables, verified" sets expectations the way a chatbot demo never will. An agent that says "I don't have context for that table yet" is trustworthy infrastructure with a roadmap. An agent that answers everything is a liability with good manners.

The budget conversation gets easier too. Semantic-layer work used to be a hard internal sell because its beneficiaries were dashboards nobody loved. Now the same investment is the difference between a data agent that works and one that embarrasses you in a QBR — same metadata, suddenly fundable.

A data agent is your semantic layer wearing a conversational interface — fund the layer, and the agent gets smart for free.