Should you build or buy AI agents? A decision method
In short: Decide build versus buy by the cost of operating and exiting an accepted workflow, measured with one acceptance suite run against every option.
Google Cloud's AI-strategy piece of 10 October 2024 reports that only 35% of companies say they have an AI strategy at all, and recommends ranking use cases by business value, feasibility and actionability. Useful — and silent on the question that lands on a tech lead's desk a week later: do we buy an agent, configure a platform, or build our own?
My answer is a method rather than a verdict. Decide build versus buy by the cost of operating and exiting an accepted workflow — not by the demo. A demo is optimised to look finished; it cannot show who carries integration, evaluation, review and migration once the thing is live. Measure those with one acceptance suite run against every option and the decision mostly makes itself.
Why the demo is the wrong unit of comparison
A demo compares capability. A purchase commits you to an operating model. Google Cloud's own deployment guidance, last reviewed 19 November 2024, is explicit that a managed service does not remove application-level work: before release you still verify permissions, infrastructure and code dependencies, and you still run load tests against your own requirements. Buying moves the model and some of the plumbing; it does not move the responsibility for the workflow being correct inside your systems.
So compare at the workflow level — this ticket type, these tools, these permissions, this volume — with the same cases for every candidate. Anything else compares a polished demo against your imagination of a build.
Run one acceptance suite against every option
Write the acceptance cases before you talk to anyone: the task outcomes that must be true, the actions that must never happen twice, the refusals that must occur, and the volume you expect on a bad day. This is the same discipline as evaluating any LLM feature before production; here it doubles as the procurement script. Then run the identical suite against the purchased agent, the configured platform and a thin custom build, under the same load assumptions, and keep the transcripts and traces from each.
Two things usually surface. The vendor's accuracy number was measured on their cases, not yours. And the option that scored best on the demo is not always the one that lets you see why it failed — trace export and eval hooks are a dimension of their own, as vendor choice for coding agents already shows.
The decision matrix
| Dimension | Buy | Configure a platform | Build | Hybrid |
|---|---|---|---|---|
| Differentiation | low — same agent your rivals buy | medium | high | high where it matters |
| Data rights and retention | contract-bound | contract-bound | yours | split by component |
| Connectors and action permissions | vendor's scope | mostly covered | all yours | yours at the boundary |
| Eval and trace export | often partial | usually available | native | native at the boundary |
| Review effort per release | low until a vendor update | medium | high | medium |
| Operating ownership | vendor + you | shared | you | you |
| Exit cost | re-platform workflow | re-platform config | rewrite | smallest blast radius |
The rows are the questions to put to each option; the cells are typical, not universal — score your own candidates after the suite run.
The hybrid row is the one most teams overlook: buy the model and the generic tooling, own the boundary where your data, your permissions and your acceptance checks live. It usually scores worst on the demo and best on the fourth column.
Count the costs the quote leaves out
A quote prices subscription or inference. The workflow costs five more things: integration into the systems that hold the state, the review effort every time a vendor ships a model update you did not schedule, incident response when the agent does something wrong at 2 a.m., the evaluation suite you now have to maintain anyway, and the migration if you ever leave. Open-source shortlists need an operating-cost column for the same reason, and outcome pricing moves the argument about those costs without removing them.
Put explicit assumptions on each line — volume, review minutes per exception, update cadence — rather than invented vendor prices. A decision built on stated assumptions can be revisited; one built on a demo cannot.
The pilot contract I would write
I have not trialled vendors for this piece and no quote was inspected, so this is editorial judgment, labelled as such. The pilot I would insist on has three parts. Expected task outcomes, written as the acceptance cases above, with the pass rate and sample size agreed in advance. Failure handling: what the agent does when a tool is down, who is paged, and how a wrong action is reversed. And an exit rehearsal before go-live — export the traces, the configuration and the data, stand the workflow up on the fallback option, and time it. If the exit cannot be rehearsed, the exit cost is unknown, and an unknown exit cost is the strongest argument for the hybrid column.
Where this holds and where it breaks
It holds for workflows with real side effects — refunds, records, provisioning — where the cost of a wrong action is countable and the acceptance suite is mostly state assertion. It is weaker for advisory or drafting agents, where "accepted" is a judgment call and the suite needs calibrated human review. It breaks when the organisation cannot write the acceptance cases at all; then the workflow is not understood well enough to buy or build, and the honest next step is a two-week discovery, not a procurement.
FAQ
Is "buy" always faster?
Faster to a demo, usually. Faster to an accepted workflow inside your systems, only if the vendor's connectors and permissions already match yours — which the suite run tells you in a week.
When does building win?
When the workflow is differentiating, the data rights matter, or the exit cost of a vendor is unbounded. Even then, build the boundary and buy the model.
What is the hybrid option, concretely?
A purchased model and tooling behind an interface you own: your acceptance checks, your permission layer, your traces. The vendor can be swapped without rewriting the workflow.
What's in it for you
- A decision matrix with the eight questions that separate the options, including the hybrid most comparisons skip.
- One acceptance suite that doubles as the procurement script, so every candidate is judged on your cases.
- A pilot contract shape — outcomes, failure handling, exit rehearsal — that turns the exit cost from a guess into a number.
Buy capability if you must, but own the boundary where your data, permissions and acceptance checks live — that is where the cost of leaving is decided.
