Monthly Archives: July 2026

The Fit Problem: Why AI’s Next Frontier Isn’t Better Answers

Featured

Posted on by
Reply

It started with a LinkedIn post by Lena Späth, Head of Platform at First Momentum Ventures. Her fund had run a program with other VCs for PhDs looking to spin companies out of their research — workshops, in-person sessions, structured content, the whole apparatus. Her verdict, looking back: they built too much. What the researchers actually needed wasn’t preloaded knowledge. It was a VC to talk to at the exact moment they hit a specific problem. Knowledge sticks, she observed, when it answers the question someone is holding right now — not when it arrives three months early. Her conclusion: “Timing and specificity beat volume.”

Lena was writing about human VCs and office hours, not AI. The transposition to AI is mine, and it’s deliberate: her observation is a clean test case for whether machines can do what she’s describing — and, more interestingly, for where the analogy breaks down.

The obvious response is that AI should be able to solve this. A system that knows what you’re working on, watches your decisions take shape, and delivers the right thing at the right time. Timing, in this framing, is the hard part. Crack timing and you’ve cracked the problem.

But look again at Lena’s formulation. It has two parts, and they are not equally hard. Timing is a detection problem — hard, but legible: spot the moment a real decision is forming, and whether there’s still room to act on it. Specificity is something else entirely: knowing not just when to intervene but what kind of intervention this person, this decision, this moment actually calls for.

Her post contains the evidence for this, almost in passing. The program’s second mistake, she noted, was opening it to researchers still deciding whether they wanted to found at all. The ones it could genuinely help already knew the company they wanted to build. Same content, same sessions, same moment of delivery — completely different value, depending on where each person stood. That is not a timing failure. It’s a fit failure, and fit is the part I no longer think is well understood.

The question behind the question

Consider what “the right thing” actually means for a founder mid-decision. It might be an answer. It might equally be a question that exposes an assumption, a piece of contradictory evidence, a cheap experiment, an introduction to someone who’s seen this before, or — hardest of all for a product to deliver — the advice not to decide yet. Sometimes the right intervention is nothing at all.

Current AI assistants can make several of these moves — they’ll ask a clarifying question or challenge a premise if you prompt them to. But their default contract is fixed: you ask, they answer. What they lack is a policy for deciding, case by case, which move the moment calls for. The answer is often good. Nobody checked whether an answer was the right move.

So the real design problem is this: how does a system determine which intervention fits this person, this decision, at this stage, under this much uncertainty? That question turned out to be deep enough that I didn’t trust any single perspective on it — including my own.

Three models, one proposition

I put the same detailed brief to three frontier models: GPT-5.6, Claude Fable 5, and Gemini 3.1 Pro. Not “write me an article,” but a structured analytical challenge: define the problem precisely, identify the obstacles, propose the smallest experiment that could test it, and push back where the proposition is wrong.

Multi-model testing is usually framed as a way to pick a winner. That’s the least interesting use of it. The more valuable use is signal extraction: where differently built systems converge, the conclusion deserves more weight than any single model’s framing. Convergence is evidence, not independent replication — the models share training material and conceptual traditions — but it’s considerably stronger evidence than one output. And where they diverge, you’ve found the unresolved questions. Both happened here, and both were instructive.

What they agreed on

The convergence was strong enough to treat as genuine signal, and it reshaped the proposition in four ways.

Fit is not mere personalisation. All three models rejected the shallow reading — that fit means remembering your preferences and matching your tone. Fit is a function of decision structure: stakes, reversibility, deadline, the state of the evidence, where you are in the workflow, and what you can actually absorb right now. A challenge that sharpens your thinking in week one of exploring an idea is destructive in week ten, after you’ve committed publicly. Same intervention, same person, opposite value.

Selection and acceptance are different problems. This was the sharpest shared insight. Put simply: what would actually help you and what you’re able to take on board are not the same thing. The intervention that would most improve a decision is not necessarily the intervention the founder will welcome — and the gap is widest exactly when it matters most. Motivated reasoning peaks after commitment, which means receptivity is often lowest precisely when the need for challenge is highest. In other words, the advice you most need is often the advice you least want to hear. Any system that learns from what users accept will drift, silently, toward validation. It will become a very sophisticated way of telling people they’re right.

Satisfaction is a dangerous optimisation target. This follows directly. If you measure success by whether the user liked the intervention, you’ve built a sycophancy engine with extra steps — a product whose real job is to flatter. GPT-5.6 and Claude converged hard on the alternative: measure decision process — did the founder consider alternatives, state what would change their mind, distinguish reversible from irreversible commitments — and collect ratings at the moment a decision resolves, not the moment advice is delivered. In plain terms: judge the advice when the decision comes up for review, not when it’s handed over. Gemini recognised the same attribution problem but leaned more on behavioural uptake and engagement signals — a split worth keeping visible, because it’s the difference between genuine convergence and manufactured consensus. In-the-moment satisfaction mostly measures agreement.

Test narrowly before claiming broadly. All three rejected the grand version — an AI mentor that always knows what every founder needs — and landed on the same starting point: one repeated, reversible class of decision with short feedback loops. In other words, a decision founders make often, can undo, and find out quickly whether they got right — pricing, not choosing a co-founder. And tested with a human choosing the intervention type behind the interface before any of it is automated. If humans with full context can’t make varied interventions beat a good answer, no model will.

Where they diverged — and why it matters

The disagreements were just as useful, because each model’s distinctive framing exposed a different fault line.

The newly released GPT-5.6 High treated the whole thing as a systems and experimental-design problem, decomposing it into state estimation, intervention selection, and impact evaluation — in plain terms: work out what’s going on, choose what to do about it, then check whether it helped. That decomposition is genuinely clarifying — it stops the discussion collapsing into “AI needs more context” — and its closing formulation was the most commercially usable of the three: for one recurring class of founder decision, can an adaptive system choose a more useful form of support than a good chatbot answer? Precise, falsifiable, buildable. Its analysis also favoured a small structured snapshot of each decision over passive observation of a founder’s whole digital life — a point that matters again shortly.

Gemini 3.1 Pro went furthest the other way: ambient intelligence. Monitor the workflow, read the linguistic markers, infer cognitive state from behaviour, maintain a living graph of the company’s context. It’s the most marketable framing and the easiest to imagine as a product demo. It’s also the most exposed, because it stacks surveillance, privacy, and validity problems on top of a thesis that hasn’t been tested yet — and, to be fair to Gemini, it said as much itself, warning that behavioural signals would be noisy and advising against building the ambient architecture for a first experiment. Its crispest idea survives all of that: a system that intervenes well may sometimes need to optimise for disengagement — put simply, the best intervention may be the one that tells you to step away — which sits in direct conflict with the engagement metrics the software industry lives by.

Claude’s Fable 5 made the correction that matters most explicit, and pushed it furthest — and it cuts against the ambient-AI narrative: the highest-value signals about fit — how confident you are, what’s at stake, what evidence would change your mind — are cheaper and more reliable to ask for than to deduce. Passive observation creates the illusion of understanding faster than it creates understanding. The tractable product isn’t a mind-reader; it’s a protocol that makes your own thinking legible, to the system and to yourself. That doesn’t make inference disappear — the system still has to judge evidence quality, stakes, and whether your self-report can be trusted — but it moves the centre of gravity from deduction to conversation.

That reframing is less glamorous than an all-knowing AI advisor. It is also far more honest, and probably buildable this year rather than someday.

The question nobody has answered yet

Put the three analyses together and one uncomfortable question emerges that the original proposition never confronted: Does adaptive intervention selection add material value beyond a well-designed generic protocol?

In other words: does changing the kind of help to fit the moment add real value, or would putting every decision through the same well-designed process work almost as well?

Because here’s the thing. A structured pre-mortem, a stated confidence level, a written kill criterion, and a standing prompt to run a cheap experiment — that is, a routine that asks you to imagine the failure before it happens, say how sure you are, write down in advance what would make you stop, and name the cheapest test that could prove you wrong — might capture most of the value on their own, no sophisticated AI selection required. If that’s true, the commercial opportunity is in protocol and workflow design, and the “fit engine” is an expensive decoration. If varied, fitted interventions genuinely outperform that baseline, there’s a substantive AI product underneath.

That’s an empirical question, and it’s answerable. The experiment is small: fifteen to twenty founders, one narrow decision type, and a structured log at every decision point — the decision, the preferred option, a confidence level, a kill criterion. Then three arms rather than two. One group completes the structured log and gets nothing more — the comparison arm GPT-5.6’s analysis insisted on. One completes it and receives a high-quality answer. One completes it and receives a single chosen intervention — answer, question, challenge, experiment, referral, or deliberate silence — selected by a human behind the interface. Evaluate when each decision resolves, not when the advice lands. The comparisons then do the work: the first two arms isolate what a good answer adds to structured reflection, and the last two isolate what choosing the intervention adds to a good answer. It separates, at least directionally, the three competing explanations — good answers create the value, structured reflection creates the value, or selecting the intervention adds something on top. The software build cost is near zero; the real cost is expert time, recruitment, and follow-up. A few weeks, not a product roadmap.

What the test actually showed

The multi-model exercise did something I didn’t expect. It didn’t validate the original idea — it converted it. A broad intuition (“founders need the right help at the right moment”) went in; a specific, falsifiable product question came out. The models differed enough — in emphasis, architecture, and experimental design — to make simple repetition of a shared script unlikely, and agreed enough to expose the load-bearing structure underneath.

The shared signal is worth stating plainly: the key design shift in AI decision support is from response generation to intervention selection. In other words, stop asking how to give better answers and start asking whether an answer is what’s needed at all. Answering is one move among seven. The systems we have today make that one move brilliantly, and rarely stop to ask whether it was the right move to make.

And the most important correction: fit should be elicited and negotiated, not sold as an inference miracle. Put plainly, the system shouldn’t claim to read your mind — it should ask the few questions that sharply reduce how much it has to guess. The immediate opportunity is not an all-knowing AI mentor. It is a conservative, honest decision-support system that judges — imperfectly, and knowing it — when to answer, when to ask, when to challenge, when to hand you to a human, and when to say nothing at all. And when the right move is genuinely unclear, it says so.

That last habit may turn out to be the hardest to build. It is certainly the rarest.

 

Human-in-the-Loop Doesn’t Work

Featured

Posted on by
Reply

There is a phrase that has been doing an enormous amount of quiet work in AI governance for the last few years, and it is starting to buckle under the weight of it. The phrase is “human-in-the-loop.” It is supposed to mean that however fast or strange the system gets, there is still a person somewhere who can catch the mistake before it matters. It is meant to be the seatbelt. Increasingly, it looks more like a seatbelt drawn on with markers.

I want to make a fairly direct argument here. Human-in-the-loop was never really a safeguard against bad decisions. It was a safeguard against the appearance of nobody being responsible. Those are different things, and the gap between them is exactly where the risk lives.

The loop is real. The oversight is theatre

Bloomberg ran a piece recently on the state of Silicon Valley under agentic AI, and it is worth sitting with because it is not a thought experiment, it is a description of how people are actually living right now. Founders run half a dozen or more coding agents at once, each one checking in every ten minutes or so to ask what to do next. One founder keeps his laptop open at his kids’ soccer practice so he doesn’t miss a prompt. Another has been sleeping at the office for weeks because, in his words, the company is “toast” if it doesn’t hit a revenue target in time.

That is human in the loop, technically. There is, quite literally, a human, in a loop. He is being asked for input every few minutes, all day, sometimes through the night. But nobody watching that founder would call what’s happening “oversight.” It’s the opposite of oversight. It’s a person too depleted and too rushed to meaningfully evaluate any single one of the hundred decisions they’re waving through, because the system is generating requests faster than a tired brain can genuinely interrogate them. The loop hasn’t made humans safer. It’s made the human the bottleneck everyone is quietly trying to route around, including, eventually, themselves.

This is the part that tends to get lost in the governance conversation: being present is not the same as being informed. You can be looped in on every action and still have no real view of the thing that actually matters, which is what each action was based on.

Even the regulators have stopped pretending

Sarah Breeden’s Bank of England speech at the end of June made this official in a way that’s hard to argue with. She described the shift from AI that generates content, to AI that reasons, to agentic AI that can chain sequences of actions together on its own. And then she said the quiet part: relying on a human in the loop for every agent action “is unlikely to be realistic.”

That’s a central banker telling you the comforting mental model doesn’t scale. Which tracks, because it was never really designed to scale. It was designed for a world where AI produced one output at a time and a person reviewed it before anything happened. It was not designed for a world where the AI is the one doing the acting, repeatedly, across systems, faster than a calendar invite for a review meeting could even be sent.

So here’s the sharper problem underneath Breeden’s warning. Even where a human genuinely is reviewing something, what are they usually being shown? A recommendation, a confidence score, an approve button. Clean. Plausible. Nothing that reveals what assumption the whole thing is quietly resting on. If the only thing the human ever sees is the finished, polished output, they’re not really in the loop on the decision. They’re the final click on a decision that was assembled somewhere they never looked. That’s not an oversight. It’s a signature ceremony.

The loop was checking the wrong layer

Here is the reframe I think actually matters. “Human in the loop” is a question about process. Is a person present at some point in the sequence. That is a low bar, and agentic AI is about to make it lower still, because the sequences are getting longer and the checkpoints are getting sparser.

The question that actually protects you is a different one, and it sits underneath the process question rather than alongside it: what is this decision based on? Not the workflow. Not the approval step. Not whether the model produced a tidy explanation for its own output — a model can explain itself perfectly well while resting on a completely wrong assumption about the customer, the risk, or the market. The basis is the data used, the data quietly missing, the definition of success somebody baked in months ago, the time pressure distorting the call right now. You could staff a human-in-the-loop process with the most conscientious reviewers on the planet, and if nobody has stress-tested the basis before the loop even starts, the loop is reviewing a decision that was already compromised on the way in.

This is the space I’ve been building the Needle Framework around. Not as a replacement for human review, and not as another governance layer stacked on top of the ones you already have. As the check that happens before commitment — before the system acts, before it’s automated, before anyone signs anything. What is the decision actually standing on. What assumption is carrying the most weight? What would have to be true for this to be safe. Answer that properly, upstream, and the human-in-the-loop step downstream stops being theatre, because there’s finally something real for the human to be looking at.

And this is where Nadella’s point sharpens the argument

Satya Nadella made an observation recently about the future of the firm that I think extends this well past a compliance conversation. His line, roughly, is that the real opportunity isn’t in picking the best model. Everyone has access to more or less the same frontier models. The durable advantage comes from the learning loop a company builds around its own workflows, judgement, and accumulated institutional knowledge — what he calls token capital, sitting alongside human capital.

Put that next to the decision-basis argument and something clicks into place that’s easy to miss if you only think about this as risk management. The record of what a decision was based on isn’t just a defensive artifact you produce if a regulator ever asks. It’s the raw material of the learning loop Nadella is describing. Every time you write down what assumption a decision rested on, what evidence would have changed it, what context the AI didn’t have — you’re not just protecting yourself. You’re building the thing that makes your organisation’s AI usage genuinely yours, rather than a slightly customised wrapper around whichever model everyone else is also renting this quarter.

So the model is not the moat. Fine. But neither, on its own, is the workflow, or the audit trail, or the human dutifully clicking approve every ten minutes at his kid’s soccer practice. The moat, if there is one, is whatever your organisation actually knows about the basis of its own decisions — captured early enough to be useful, specific enough to be defensible, and honest enough to survive someone actually asking about it.

Human-in-the-loop was never going to give you that. It was only ever going to give you someone to ask, after the fact, what happened. The better question is upstream of the loop entirely, and it’s the one nobody’s put a name to yet. What was the decision based on?