AI9 min read
Human behaviour in an AI world: designing products people actually trust
AI changes what products can do. It does not change how people decide whether to rely on something. Trust in AI products follows the same human rules it always has, with higher stakes.
By Strategy Practice, Brand, strategy & AI
Much of the conversation about AI products focuses on capability: what the model can do, how accurate it is, how fast. Far less attention goes to the human side. How do people decide whether to rely on a system they cannot see inside? When do they hand over judgement, and when do they take it back?
The answers are not new. People have always calibrated trust in tools, colleagues and institutions through experience. AI products have to work with that process, and the ones that fight it end up either ignored or dangerously over-relied on.
Trust should match capability
The goal of AI product design is not maximum trust. It is calibrated trust: people relying on the system where it is reliable and checking it where it is not. Both failure modes are costly. Under-trust wastes the investment, because people redo work the system could have done. Over-trust is worse, because people stop checking exactly where checking matters.
| Under-trust | Over-trust |
|---|---|
| Users ignore good suggestions | Users accept bad suggestions |
| Work is duplicated | Errors pass unnoticed |
| Adoption stalls | Accountability blurs |
| Visible and frustrating | Invisible until something goes wrong |
Explain in the user's terms
Explanations of AI behaviour often describe the model: confidence scores, token probabilities, technical caveats. Users need something different. They need to know what the output is based on, what the system might have missed, and when they should look more closely. 'Based on the three contracts you uploaded; the 2024 renewal was not included' is an explanation a person can act on.
Keep people in control where it matters
Automation works best when its scope matches the stakes. Low-risk, repetitive tasks can run on their own with a way to review later. Consequential decisions, about money, health, employment, legal standing, should be supported by the system and made by a person, with the interface making that person's role clear and easy.
Consistency builds trust faster than brilliance
People trust what they can predict. A system that is excellent most of the time and baffling occasionally is harder to trust than one that is good and consistent. Where variability is unavoidable, make it visible: show when the system is on familiar ground and when it is not.
People will trust a machine the way they trust a colleague: slowly, from evidence, and with a long memory for mistakes.
The products that earn lasting trust in AI will not be the ones with the most impressive demos. They will be the ones that are honest about their limits, easy to correct, and respectful of the person who carries the consequences.
Strategy Practice
Brand, strategy & AI
The strategy practice covers positioning, product strategy, UX diagnosis and AI opportunity work. It writes about the commercial reasoning behind design decisions, and the organisational conditions that make good products possible.
