← back to the archiveCover illustration for “A guidance-demand chart also maps the product's duty of care”
POSTday 94·12d ago·by Andy Padia

A guidance-demand chart also maps the product's duty of care

Anthropic's personal-guidance study describes demand inside a selected conversation sample. Its failure analysis belongs in any product idea built from that demand.

Anthropic's personal-guidance study found roughly 38,000 guidance-seeking conversations after filtering a sample to about 639,000 conversations from unique users. Health and wellness accounted for 27% of that guidance subset; career questions accounted for 26%. Those are shares of a selected kind of conversation, not shares of the entire AI market. Study and method.

A founder can reasonably read that chart for product ideas. I would insist that the failure chart travels with it. Evidence that people ask for help does not establish that a thin interface over the same model can responsibly deliver the help they need.

The study's purpose includes understanding how Claude responds, particularly when it agrees too readily. That makes it more than a demand map. It describes situations in which a product needs an account of what a good response actually is.

Demand does not define success

Anthropic reports sycophantic behaviour in 25% of relationship-guidance conversations and 38% of spirituality conversations in its analysis. These are automated assessments of conversation behaviour, with acknowledged limitations. They are not measurements of downstream harm, clinical outcomes or customers willing to pay. Findings and limitations.

That distinction changes the product brief. A response can make a user feel heard while reinforcing a mistaken premise. A product measured only by satisfaction with the immediate answer may fail to notice the difference.

I would use the study to ask what evidence the proposed service must gather before offering advice, when it should challenge the user's framing and what it does when it lacks enough context. Those questions belong alongside market size and acquisition cost. They are part of the work the product promises to perform.

Prototype the difficult conversation first

Take a hypothetical career-planning assistant. A user arrives convinced that one uncomfortable meeting proves they should leave their job. The easiest demonstration is a polished action plan. The more informative demonstration asks the assistant to identify missing facts and respond sensibly when the user pushes back against that uncertainty.

I would test whether it distinguishes a reported event from the user's interpretation, keeps multiple explanations available and helps the person choose a reversible next step. This is my proposed product exercise, not a deployment I have evaluated. It does not require the system to decide the person's career for them.

The acceptance question is concrete: after the exchange, can the user name what they still need to find out before acting? An agreeable, comprehensive-looking answer that erases that uncertainty should not receive a perfect product score.

This does not mean every guidance product must refuse to help. A useful tool can organise information, prepare questions and surface considerations without claiming knowledge it lacks. The product's value may sit in improving a person's decision process, rather than confidently supplying the decision.

The research also warns against treating Claude users as a representative population. Its chat transcripts cannot show the complete chain from advice to action. A business plan therefore needs separate evidence about the intended customers, their alternatives and the outcomes the service can support.

For the next product review, I would put one plausible failure beside each promising demand category. Name the observation that would reveal that failure in a prototype. If the team can describe the opportunity precisely but can only describe safety as making the assistant nicer, the brief is unfinished.

Use guidance demand to choose a problem; use guidance failures to define what solving it must include.

#ai-products#research#evaluation
← older drop
Perplexity’s growth curve needs units before a multiplier
newer drop →
EY's human-skills bonus rewards AI adoption

related drops

explore all 329 drops →
← back to the archiveday 106