Vendor-selection advice needs the same evidence check as vendors

I would run a vendor-selection memo through the same evidence check I apply to the vendors. Advice should not become exempt from scrutiny because it is written in a confident, helpful voice.

GAI Insights' Claude-versus-ChatGPT comparison uses $4 billion for OpenAI's first-quarter cash burn in its body and $3.7 billion in its FAQ. It also discusses an annualised revenue figure around another period while characterising first-quarter burn as more than half of that quarter's revenue.

Those statements warrant a source and period check. They do not, by themselves, prove fraud or even a numerical contradiction: $4 billion may be a rounded version of $3.7 billion, and a later annualised run rate is not the same thing as revenue actually earned in an earlier quarter.

My criticism is therefore about the unresolved bridge between figures and recommendation. A reader should not have to reconstruct which number was measured, estimated or rounded before deciding how much weight to give the conclusion.

Correcting the memo requires more than dividing by four

A tempting rebuttal is to divide a $40 billion annualised run rate by four and declare that quarterly revenue must have been $10 billion. That would repeat the period error rather than fix it.

A run rate extrapolates recent performance under some convention. Actual quarterly revenue accumulates over a defined period. For a fast-changing business, the two can differ substantially without either being internally impossible.

I would ask for the original financial source, the measurement period and the definition of cash burn. If those are unavailable, mark the comparison as insufficiently supported. Do not manufacture a precise correction from another incompletely defined figure.

Rounding deserves similar treatment. A memo can use a rounded figure for readability, but it should not let the rounded and precise-looking versions imply different levels of evidence. The source record should explain the relationship.

This is an argument for a cleaner advisory method, not a claim that the opposite vendor recommendation must be right.

Separate the observed customer pattern from the market

The comparison also draws on the author's workshop experience. That can be useful practitioner evidence. It remains a sample of organisations that hired that adviser, not automatically a representative account of all enterprise buyers.

For a hypothetical selection team, I would record the advice as one input and test the finalists against the organisation's own tasks, access requirements and operating constraints. A recommendation becomes more useful when the team can state the conditions under which it should hold.

The adviser’s commercial activities belong in the context too, but their existence does not establish that a conclusion was bought or knowingly distorted. Ask for relevant interests to be clear and evaluate the supporting evidence on its merits.

A small review can catch a great deal: trace the consequential figures, align the periods, distinguish observation from generalisation and identify what the recommendation assumes about the buyer. That work should happen before the memo is compressed into a board slide.

I want strong advice to survive scrutiny. The useful standard is that its conclusion remains defensible after the confident framing has been removed.

Audit the source, period and definition in a vendor-selection memo before using its numbers to choose a platform; critique needs that discipline too.