The distinction that matters

In pension governance, the choice is not whether AI arrives. It is whether it arrives by design or by drift.

A capable AI placed inside a trustee board can make challenge cheaper, or it can make ratification smoother. The two outcomes look identical in the audit trail. They are not the same decision.

A new Knowa white paper, The Architecture of Real Decisions, is about that distinction and what it takes to build trustee technology where judgement is architected in, not engineered out.

See what our CEO, Amédée Levillain, is saying about it: Read his post on LinkedIn.

The calculator on every desk

The paper opens with a study in which researchers deliberately programmed an on-screen calculator to give undergraduate students wrong answers. Even when the calculator produced a patently absurd result (an age of 114 for someone born in 1945 and assessed in 1994), a large minority of users showed no suspicion. The students could have caught the error by hand in seconds. They did not, because the cost of overruling a confident machine was psychological, not practical.

That finding, the paper argues, is what trustee technology needs to take seriously. The capacity for independent verification does not disappear when the machine becomes more sophisticated and the stakes become consequential. The psychological cost of challenge compounds. Pension governance is a setting where the integrity of millions of retirement outcomes depends on whether decisions are genuinely made by the people legally responsible for them.

Fifty years of human-factors research

The paper draws on five decades of research in human-factors science: automation complacency, automation bias and the problem the literature calls the vanishing loop. When a system is reliable most of the time, the human supervising it stops genuinely supervising. When a confident machine output meets an uncertain human, the human predictably defers (even when the machine is wrong, and even when the human possesses information the machine does not).

These findings have reshaped cockpit design, surgical checklists, nuclear control rooms and ICU monitoring protocols. They have not yet meaningfully reshaped the design of the technology that sits in front of trustee boards.

The trustee board as a system under stress

The paper examines the operating conditions of a UK pension trustee board: quarterly meetings, materials that frequently exceed several hundred pages, arriving late in the week before the meeting. Investment consultants have done the analysis and framed the choices. The scheme actuary has set out the assumptions. The legal adviser has drafted the resolution. The board's job, as a matter of law, is to bring independent judgement. The operational reality exerts ratification pressure.

Into this system, AI is already arriving: not because anyone has chosen to deploy it, but because consumer AI is now ambient. ChatGPT in the browser, Copilot embedded in Office, generic LLMs one prompt away from any laptop in any meeting room. The paper describes conversations already happening in the market: a trustee pasting a consultant's recommendation into ChatGPT to ask whether it looks reasonable, an adviser running a draft manager paper through Copilot to tighten the language. None of it is governed. None of it was designed for the work.

Two futures

The paper describes two possible futures for trustee technology.

In the first (what it calls the smooth-ratification future), AI makes it easier for boards to confirm. The trustee arrives under-prepared, asks the platform whether a recommendation looks reasonable, receives a confident affirmation, and the board approves. The minutes record a thoughtful exchange. The audit trail is comprehensive. The actual cognitive content of the meeting has been hollowed out further than before.

In the second (the architected-judgement future), the platform is designed around a different question: not "how do we make it easier for the board to confirm?" but "how do we make it cheaper for the board to challenge?" The platform surfaces inconsistencies, flags unanswered questions from previous meetings, points to drift in fund characteristics that headline performance numbers disguise. It makes the uncomfortable second question (the one most trustees lack the time or context to formulate) almost free to ask.

Care as architecture

The paper introduces a taxonomy of friction in trustee decision-making. Wasted friction (locating documents, reconciling versions) should be removed without hesitation. Productive friction (the pause that surfaces an inconsistency, the flag that points to an unanswered question) should be preserved and engineered for. Cosmetic friction (the performance of deliberation that did not happen) is the most dangerous category. Constitutive friction (the cadence of meetings, papers and reviews that the law has built around trustee decisions) exists because the decisions matter.

A care-architected platform removes wasted friction aggressively, engineers productive friction deliberately, refuses to manufacture cosmetic friction even when asked, and respects constitutive friction even when faster alternatives exist.

What this means for Knowa

The paper sets out Knowa's product vision in three stages: Knowa Core (live today, providing a single source of truth for board work), Knowa Connected (currently in development, wiring compliance modules into the same underlying record), and Knowa on Demand (the horizon, where governance tasks trigger themselves). Knowa Q, the intelligence layer, runs across all three stages and is where the boundary between augmentation and advice is drawn most carefully.

Knowa Q answers questions about the record. It does not recommend. It will surface an inconsistency between two papers and not tell the board which is right. It will show that an action assigned three meetings ago has not been closed and not opine on whether closing it should change the present decision. Those are judgements, and judgements belong to the trustees and their advisers.

The full white paper is available to download on the Knowa white papers page.

Judgement, architected in. Not engineered out.