Trust in AI Products: How to Design for Calibrated Trust

By Nick Babich

Key Takeaways

• Aim for calibrated trust: users should rely on an AI feature as much as its record on that task justifies, no more and no less.
• Stated trust is low, reliance is not: fewer than half of people say they trust AI, yet many accept its output without checking.
• Decide autonomy per action: rate each AI action by impact and reversibility, and give risky, hard-to-undo actions evidence and a human decision.
• Plan for the first visible error: acknowledge it, explain the cause, fix what it touched and show what changed.
• Measure behavior: track over-reliance and under-reliance next to a validated trust questionnaire.

Trust in AI products is a user’s willingness to rely on an AI feature’s output or action when they cannot fully check it themselves. It is an attitude, and it is not the same as trustworthiness (how reliable the system actually is) or reliance (what the user actually does). The design goal is calibrated trust: people rely on the AI exactly as much as its record on that specific task justifies. With too little trust, users override correct outputs or abandon the feature after one mistake. With too much, they approve wrong outputs without looking, which is known as automation bias.

This guide is for product managers, designers and founders who ship AI features: where trust in AI stands in late 2026, what drives it, a five-step method built around a stakes and reversibility matrix, the trust lifecycle, a worked example and measurement. It does not repeat the interface patterns for uncertainty, errors and human approval that we cover in a separate guide. If you plan to bring in outside help for this work, our ranking of AI UX design agencies, compared side by side is a good place to start.

What Is Trust in AI?

The idea is older than generative AI. In Trust in Automation: Designing for Appropriate Reliance (2004), John D. Lee and Katrina A. See argue that automation causes problems when people fail to rely on it appropriately. Trust that matches what the system can actually do is what designers now call calibration.

Three terms get mixed up in most product discussions, so it helps to separate them:

• Trust: the user’s attitude. It is what they would tell you in an interview or a survey.
• Trustworthiness: a property of the system, such as accuracy on the task, consistency, security and data handling.
• Reliance: behavior. Did the user accept, edit, verify or override the output?

Calibration is the relationship between the three. Under-trust means trust sits below trustworthiness: people redo work the AI did correctly or switch the feature off. Over-trust means trust sits above it: people rubber-stamp outputs and let errors through. Both are design failures, and a team that only asks how to “increase trust” usually fixes the first and makes the second worse. If a feature is wrong on roughly 1 task in 20, the right amount of trust is the amount that still makes users catch that 1.

It is also the first thing I check in the case studies of the UX teams we review for AI-powered products: does the work show what happened when the AI was wrong, or only the polished happy path?

Is Trust in AI Falling?

Short answer: stated trust has fallen while use has kept rising. These are the most recent sources I could verify as of November 2026:

• Global trust is below half: the KPMG and University of Melbourne global study surveyed over 48,000 people in 47 countries between November 2024 and January 2025. 66% use AI with some regularity, but only 46% are willing to trust it, and compared with a 17-country survey from before ChatGPT’s release in 2022, people have become less trusting and more worried.
• Concern is rising in the US: in a Pew Research Center survey of 5,023 US adults in June 2025, 50% said they are more concerned than excited about the increased use of AI in daily life, up from 37% in 2021.
• Generative AI advice is doubted: in an August 2026 McKinsey podcast on consumer research, Danielle Bozarth said trust in generative AI as a source of shopping advice sits below 40% on the firm’s net consumer trust index (trusters minus non-trusters), even though consumers keep using it.
• People verify before they act: an Idea Grove survey of 1,000 US adults in April 2026 found that only 2% would buy from an unfamiliar brand on an AI recommendation alone. Only 15% fully trust AI brand recommendations.

The same KPMG study points the other way too. Among employees, 66% say they rely on AI output without evaluating its accuracy, and 56% say they have made mistakes in their work because of AI. Low stated trust and careless reliance exist side by side, so the problem for a product team is miscalibration in both directions: skeptical in general, uncritical in the moment.

What Drives Trust and Distrust in AI Products

Trust in a specific AI product rests on a few user-side judgments. When I review an AI feature, I check these first:

• Track record on my task: how often it has been right on the kind of work this user does, not a benchmark score.
• Predictability: similar inputs give similar outputs, and behavior does not change without notice.
• Understood limits: the user knows what the feature cannot do before finding out the hard way.
• Control that fits the stakes: the user can stop, review or reverse an action in proportion to the damage it can do.
• Data handling: it is clear what the AI reads, keeps and learns from.
• Accountability: a named person or team owns the outcome when the AI is wrong.

Distrust comes from the mirror image: oversold accuracy, hidden errors, model updates that silently change results and data use users discover on their own. One silent failure costs more trust than ten visible, well-handled ones.

How to Design for Calibrated Trust: A Five-Step Method

Calibrated trust does not come from one screen or one disclaimer. It comes from five decisions that run through the whole feature, and the fifth one feeds back into the second.

Step 1: Set Accurate Expectations

Most trust problems start with a gap between what users expected and what they got. State the task the AI is built for, the inputs it handles well and the cases it handles badly, where the user starts the task, not in a help article.

Watch the claims outside the product too. A landing page that promises “99% accuracy” sets the bar the first visible error will be judged against. A modest claim that holds up builds more trust than a big one that breaks in week two.

Step 2: Match Safeguards and Autonomy to the Stakes

Trust is not one setting for the whole product. A single AI agent can draft a reply, edit records, email a customer and move money, and each of those actions deserves a different level of autonomy. I rate every action on two questions: how much damage it does if it is wrong (impact) and how easily it can be undone (reversibility).

The stakes and reversibility matrix turns those two answers into an autonomy level, a safeguard and the evidence the user needs:

Two rules make the matrix work. Rate actions, not features: “AI assistant” is not a quadrant, but “assistant sends the email” is. And when a team cannot agree on a rating, move the action toward more human control and revisit it once there is usage data.

The top-right quadrant often carries legal duties. If your product approves money, credit or consent, read our notes on designing approval and disclosure flows for regulated financial products first.

Step 3: Show the Evidence the Stakes Need

Explanations do not automatically help. In a CHI 2021 study by Gagan Bansal and colleagues, AI assistance improved the accuracy of human and AI teams, but explanations did not add to that gain. Instead they increased the chance that people accepted the AI’s recommendation whether it was right or wrong.

So I choose evidence by what it helps the user catch, scaled to the quadrant. A low-stakes draft needs almost nothing. A review-queue action needs a clear record of what changed. A confirm-first action needs a precise preview. A human-decides action needs the inputs the AI used and the alternative it rejected. If a reviewer cannot spot a wrong answer from the evidence shown, the evidence is decoration.

Step 4: Repair Trust After Errors

Every AI feature will make a visible mistake, and that moment sets long-term trust more than any onboarding flow. Plan the repair before launch, like an incident response:

• Acknowledge: say plainly what went wrong and how many items it touched. Do not bury it in a changelog.
• Explain: give the cause in the user’s terms, not the model’s.
• Fix: correct the affected records, or give the user a one-step way to do it.
• Show what changed: tell the user what the product now does differently, and lower its autonomy for that action until it re-earns it.

The same goes for changes nobody asked for. When you swap a model or retrain on new data, tell users which tasks may behave differently.

Step 5: Measure Reliance, Not Just Sentiment

Surveys tell you how users feel. Behavior tells you whether their trust is calibrated. Track how often users accept wrong outputs and override right ones, per action, and use those numbers to move actions between quadrants. Autonomy expands one action at a time after a task-specific track record, and contracts after errors.

The Trust Lifecycle: From First Use to Earned Autonomy

Trust changes over time, and the design job changes with it. I map AI features against five stages:

1. Before first use: users bring expectations from marketing and other AI tools. Disclose the AI, state its scope and name its weak spots.
2. First output: users check closely. Show evidence generously so they learn what good output looks like.
3. First visible error: trust drops. The four repair moves from step 4 exist for this moment.
4. Repair: if the product acknowledges, explains, fixes and shows what changed, trust recovers to a calibrated level. If it hides the error, users drift to under-trust or quietly leave.
5. Routine use and earned autonomy: checking fades and complacency becomes the risk. Keep evidence on high-stakes actions and widen autonomy only where the record supports it.

Worked Example: An Invoice-Matching Agent

Here is the method applied to a fictional product. Quarrowby Payables is an accounts payable tool with an AI agent that matches supplier invoices to purchase orders. The product and every number in this section are illustrative, not drawn from a real deployment.

• Rating the actions: matching an invoice under $2,000 is easy to undo before payment, but the agent has no record yet, so it starts in “act, then review” with every match queued for a person. Matches from $2,000 to $25,000 sit in the “confirm first” quadrant. Changing vendor bank details and paying an invoice are “human decides” by policy.
• Earning autonomy: after four weeks the team has 412 checked matches under $2,000, 409 of them correct (99.3%). The rule set before launch was 400 checks at 99% or better, so that one action becomes automatic.
• The first visible error: on November 3 the agent matches 3 invoices to the wrong purchase order because two vendors use overlapping PO numbers. A reviewer catches it the same day.
• Repair: a notice states the date, the count, the cause and the fix: the agent now checks the vendor ID as well as the PO number. The 3 entries are corrected, and auto-matching is paused until the fix ships.
• Measuring reliance: in the confirm-first tier this month, reviewers accepted 2 of 9 wrong suggestions and overrode 5 of 151 correct ones. The first number calls for sharper evidence on mid-size invoices.

The screen below shows those decisions in one interface. It has no single “trust score” for the agent, because trust is earned per action.

How to Measure Trust in AI Products

Measure trust two ways, because they disagree. Stated trust comes from validated questionnaires; reliance comes from product analytics. Don’t write your own trust survey from scratch. These scales have published development work behind them:

For reliance, I track these per action, not per product, and compare them before and after every model or design change:

Ground truth is the hard part. Sample a fixed share of each action for expert review every week, including the automatic ones, or over-reliance stays invisible until a customer finds it.

Governance Signals That Build Organizational Trust

Users and buyers also judge the organization behind the AI. Four signals matter most:

• AI disclosure: Article 50 of the EU Artificial Intelligence Act (Regulation 2024/1689) requires providers to make sure people are told when they are interacting with an AI system, unless that is obvious, and requires providers of generative AI systems to mark their output in a machine-readable way. Under the regulation as published, these duties apply from 2 August 2026; check the current text with counsel.
• A risk framework: the NIST AI Risk Management Framework, released on January 26, 2023 for voluntary use, helps teams build trustworthiness considerations into the design, development, use and evaluation of AI products. NIST says it is being revised, so name the version you follow.
• Change notices: a short note whenever a model, data source or autonomy level changes.
• A public AI policy: one page on which features use AI, what data they use, who is accountable and how to report a problem.

A Short Trust Audit Checklist

Use this short audit before launch and again each quarter. It complements, and does not replace, a review of the interface itself:

• Every AI action is rated by impact and reversibility, and its autonomy level matches the rating.
• The scope and known weak spots are stated where users start the task.
• Evidence for each high-stakes action lets a reviewer reject a wrong output quickly.
• A written repair plan exists for the first visible error, with an owner.
• Autonomy promotion rules (volume and accuracy) are written down before launch.
• Over-reliance and under-reliance rates are tracked per action, with weekly ground-truth sampling.
• Model and data changes trigger a notice to affected users.
• A named person is accountable for each human-decides action.

Frequently Asked Questions

What is trust in AI?

Trust in AI is a person’s willingness to rely on an AI system’s output or action when they cannot fully verify it. It is an attitude, separate from trustworthiness (how reliable the system is) and reliance (what the person does). Good AI products aim for calibrated trust, where trust matches trustworthiness.

Is trust in AI falling?

Yes, on most measures. The 2025 KPMG and University of Melbourne study found people less trusting and more worried than before ChatGPT, and Pew found US concern rising from 37% in 2021 to 50% in 2025, while use keeps growing.

Can explanations make users trust AI too much?

Yes. Research presented at CHI 2021 found that explanations raised acceptance of AI recommendations whether or not they were correct. Evidence should help users catch errors, not make the AI look convincing.

When should an AI feature act without approval?

Only for low-impact or easy-to-undo actions, and only after a track record you defined before launch, such as a minimum number of checked outputs at a set accuracy. Pull the permission back after a visible error until the cause is fixed.

Conclusion

Trust in AI products is not a number to maximize. Users trust AI less than they did a few years ago, and many still accept its output without checking. The design job is to close both gaps.

Rate each action by impact and reversibility, show evidence that helps users catch mistakes, plan the repair for the first visible error and let reliance data, not enthusiasm, decide when the AI gets more autonomy. That way trust is earned one action at a time.