Agent ROI is often feature activation in disguise

SaaStr's latest agent dispatch has a great collections story: three humans and twenty-plus agents in production, six figures behind on receivables during their conference, and their finance agent goes into bill.com, sets up dunning, and gets collections back on autopilot. The end-to-end flow around it is genuinely impressive — seconds after a signature, the agent reads the contract, flips the Salesforce opportunity, appends missing signer contacts, creates split invoices with correct terms. Real work, well integrated. First-party and unaudited, as ever, but I believe the shape.

Then read what the collections win actually was, because SaaStr is honest enough to tell you: the agent discovered that bill.com had built-in automated reminders the team had never turned on in eight years — reminders before the due date, escalation after, all behind one toggle. The companion episode repeats the pattern with "a setting we'd missed for 8 years." The agent's edge here was not reasoning. It was reading the manual exhaustively and flipping a switch a human could have flipped on day one.

I want to be clear this is a real win — money got collected, and "the tool you already pay for, finally configured" is legitimate value. But it is a specific kind of value, and mislabeling it is about to distort a lot of ROI math. A measurable share of reported agent ROI is feature activation in disguise: the agent switching on paid-for SaaS capabilities the humans never configured. When that gets booked as "AI ROI," you are pricing configuration debt as artificial intelligence — and paying a premium for a machine that read a settings page.

The distinction that matters is between two categories of win, because they have completely different economics. Activation wins are one-time and cheap: a capability that already existed, now switched on. Enormous first-time payoff, then it's done — the reminders don't get re-discovered next quarter. Reasoning wins are durable and genuinely priceable: the agent handling a novel case, exercising judgment no toggle encodes, doing something the software couldn't do at any setting. Both show up as "the agent saved us money." Only the second is a recurring reason to pay for an agent, and only the second scales with model quality.

There's a real strategic inversion hiding in here too, and it's the optimistic half. SaaS vendors have long enjoyed an "undiscovered feature" moat — you pay for a hundred capabilities and use twelve, and the unused eighty-eight are pure margin. An exhaustive machine reader collapses that gap: it activates what you already bought. That's great for buyers (you finally use what you pay for) and quietly threatening to vendors whose renewal economics assumed you'd never find the settings. But notice it also means much of the early "agent ROI" is a one-time harvest of accumulated configuration debt — a backlog that, once cleared, doesn't refill. Extrapolating year-one numbers that were mostly activation into a permanent run rate is how you overpay for year two.

At work, this is now the first cut I make in any agent-ROI review: for each claimed win, ask could a human have done this by changing a setting in software we already own? If yes, it's an activation win — bank it once, celebrate it, and do not put it in the recurring-value column that justifies the agent's price. If no — if it required reading a situation, weighing a tradeoff, handling a case no configuration covers — that's a reasoning win, and that's what you're actually buying an agent for. Most pilots I see have their columns mixed, and the activation wins are doing the heavy lifting in a number that's supposed to prove durable intelligence.

Steal this audit: split your agent's wins into two lists — "activated a feature we already paid for" and "did something no setting could." Sum them separately. The first list is a fantastic one-time consulting outcome and an indictment of your last five years of SaaS configuration. The second list is the only one that should set the agent's ongoing price. If the second list is thin, you didn't buy an AI. You bought a very thorough audit of your own software, which is worth something — just not what the invoice says.

Half of "agent ROI" is the agent finally reading the manual — separate switching-on wins from thinking wins, or you'll price a one-time cleanup as permanent intelligence.