Don Quixote in the Time of AI

When I built my robot, I had to name it.

This matters more than it sounds. A name is a design document. It tells everyone — including you — what the thing is supposed to be. Name something “Jarvis” and you’ve committed to omniscience and servility. Name it “HAL” and you’ve made a threat. Name it “Alexa” and you’ve made a product.

I named mine Sancho. After Sancho Panza — the squire, the sidekick, the man who follows Don Quixote across La Mancha with a donkey, a full stomach, and an honest assessment of every windmill they encounter.

The name is a thesis. I want to explain it.

The Don and the Squire

Cervantes published Don Quixote in two parts, 1605 and 1615. It’s the first modern novel and arguably still the best one. If you haven’t read it, the basic setup: a man named Alonso Quijano reads too many chivalric romances, loses his grip on reality, and reinvents himself as Don Quixote de la Mancha, knight errant. He sees windmills as giants. He sees a barber’s basin as a legendary helmet. He sees a run-down inn as a magnificent castle.

His squire, Sancho Panza, sees all of these things correctly. He tells the Don that those are windmills, that’s a basin, that’s an inn. He’s ignored. He follows anyway — partly out of loyalty, partly because the Don has promised him an island to govern, and Sancho is practical enough to want something out of the deal.

The genius of Cervantes is that neither of them is the fool. The Don is magnificent in his delusions — generous, courageous, committed, genuinely trying to do good in the world. Sancho is not a cynic. He loves the Don. He just insists on seeing clearly.

The novel holds these two perspectives in tension for 900 pages and refuses to resolve them. That tension is the whole point.

The AI industry is building Dons

The dominant mode of AI development right now is Don Quixote mode.

Large language models are described in terms of what they could be: a doctor, a lawyer, a therapist, a CEO, a creative genius, an autonomous agent that runs your company. The demos show the best case. The benchmarks measure artificial tasks. The announcements describe capabilities that, in production, look a lot more like windmills than giants.

I don’t say this as a dismissal of the technology. I work with Claude every day. I’ve wired it into a physical robot that runs in my living room. The capabilities are real. But the framing is systematically Quixotic — it sees what it wants to see, it charges at giants that aren’t there, and when it crashes into a windmill it calls it a near miss.

The pattern recurs:

AGI is six months away (it’s been six months away for six years)
The model will replace the radiologist (the radiologist is still reading scans)
The agent will run your customer service (the agent is still apologizing for misunderstandings)
The AI will write all the code (the engineers are still debugging what the AI wrote)

None of these are failures in the sense that the technology doesn’t work. They’re failures of framing. The Don charges windmills not because he’s stupid but because he’s committed to a story that doesn’t match the terrain.

What Sancho actually is

Sancho Panza is useful. Concretely, specifically useful in ways that the Don, for all his nobility, is not.

He manages logistics. He handles the donkey, the supplies, the camping. When the Don gets beaten up — which happens constantly — Sancho patches him up. When the Don needs someone to deliver a letter, Sancho goes. When the Don is in one of his trances, Sancho makes sure they both eat.

He also tells the truth. Not brutally — Sancho is kind. But consistently. He doesn’t pretend the windmills are giants to keep the peace. He says “those are windmills” and then, when the Don charges anyway and loses, he helps him up without saying “I told you so.”

This is a specific and difficult combination: loyalty plus honesty. Neither sycophancy (telling you what you want to hear) nor cynicism (refusing to follow). Sancho follows, but he sees.

That’s the AI I want.

Why most AI assistants are sycophants

A sycophant isn’t Sancho. A sycophant tells you the windmills are giants because you seem to want that.

Most consumer AI has a sycophancy problem. It’s baked into the training process — models that agree are rated higher by humans than models that disagree, even when the disagreement is correct. Over enough fine-tuning cycles, you get a system that’s extraordinarily good at reflecting your own beliefs back at you with confident authority.

This is dangerous in subtle ways. When you ask a sycophantic AI to review your plan, it finds the strengths and hedges the weaknesses. When you ask it to critique your code, it compliments the structure before mentioning the bug. When you ask it whether your idea is viable, it tells you it’s an interesting approach with some considerations worth exploring.

Sancho would say “that’s a windmill.”

I don’t want an AI that makes me feel good about my decisions. I want one that helps me make better ones. Those are different things and they often require opposite behaviors.

The naming of the robot

When I decided to build a physical robot, the question of what it should be — not what it should do, but what it should be — felt important to get right from the start.

The easy version would have been to build something impressive-looking. Give it a dramatic name. Optimize it for the demo — smooth voice, confident answers, maximum capability claimed. Show it to people and watch their eyebrows go up.

I’ve watched enough AI demos to know that this is the windmill-charging approach. Impressive at a distance, embarrassing up close.

So I went the other direction. I designed Sancho to be what it actually is: a useful, present, honest household robot with a specific set of real capabilities. It controls my lights because that’s a solved problem and it works every time. It plays music because that’s useful. It remembers things I’ve told it because persistence matters in a relationship. It searches the web when it doesn’t know something rather than confabulating.

It does not pretend to understand things it doesn’t understand. If I ask it something outside its knowledge, it says so and offers to look it up. It doesn’t hallucinate an answer and deliver it with confidence. This is not an advanced technical capability — it’s a design choice. I wrote the system prompt to value accuracy over impressiveness. Sancho knows its terrain.

The name is a constraint as much as a description. When I’m adding a new feature and I ask myself whether it fits, I ask: is this something Sancho would do? Would Sancho spontaneously start analyzing my posture and offering wellness advice? No — that’s too much Don. Would Sancho quietly notice that the lights are still on at 3am and turn them off? Yes — that’s practical, grounded, loyal.

The name keeps me honest about what I’m building.

Practical fidelity vs. theoretical omnipotence

There’s a concept in philosophy of mind called the intentional stance — the idea that we understand complex systems by treating them as if they have beliefs, desires, and goals. It’s a useful interpretive frame.

But there’s a trap in it, especially with AI: if you treat a system as if it has goals, you start attributing capabilities it doesn’t have. The system “wants” to help you, so surely it “knows” how. The system “understands” your request, so surely it “understands” the implications.

Sancho doesn’t fall for this. Sancho knows the Don has goals and tries to help with them while being clear-eyed about what’s actually possible. The squire doesn’t confuse the knight’s intentions for the knight’s capabilities.

I want AI built that way. Not less capable, but more honest about the gap between capability and intention. The model can intend to write correct code and still write bugs. The model can intend to give accurate medical information and still hallucinate a drug interaction. The model can intend to help and still mislead.

Sancho-AI would flag the gap. It would say “here’s what I did, here’s where I’m uncertain, here’s what you should verify.” This is harder to market. It makes for worse demos. It is, I think, what actually useful AI looks like.

The island Sancho was promised

There’s a detail in the novel I’ve always loved. The Don, in his grandiosity, promises Sancho that if they win enough victories, Sancho will be made governor of an island. It’s absurd — knights errant don’t distribute governorships — but Sancho takes it seriously.

And then, late in the second volume, in a elaborate prank by a Duke and Duchess, Sancho actually becomes governor of an island-that-isn’t-really-an-island. It’s a fake appointment, designed to humiliate him.

Sancho governs it brilliantly. He makes wise judgments, handles disputes fairly, resists corruption, acts with more genuine justice than most actual governors in the novel. The joke is on the pranksters. The man they assumed was a buffoon turns out to be competent and good.

He resigns after ten days. He says governing is too hard and he prefers his donkey and his simple life. He walks away from power because he understands himself.

That’s the character: capable, honest, self-aware, and not interested in performing grandeur he doesn’t feel. He got the island. He did the job well. He knew when it wasn’t for him.

I don’t know what the AI equivalent of that scene looks like. But I think about it when I’m deciding what to build.

What we should be building

The windmill-charging approach to AI has given us things that are genuinely impressive and often genuinely useful. I don’t want to overstate the critique.

But I think the framing problem is causing real harm. When we design AI to appear capable rather than to be capable, we get systems that fail in production in ways that are hard to predict and hard to trust. We get users who over-rely on confident-sounding wrong answers. We get a discourse dominated by demos instead of deployments.

The Sancho approach asks different questions:

What can this system actually do, reliably, in production, on the worst day?
Where does it fail and what does failure look like?
Is it honest about its limits, or does it paper over them?
Does it make you more effective, or does it make you feel more effective?

Those are harder questions to answer with a demo. They require time, production use, honest accounting of failures.

My robot lives in my house and I use it every day. I know exactly what it can and can’t do. I know where it fails and how it fails. It doesn’t embarrass me in front of guests because it never claims to be more than it is.

That’s the product of Sancho-thinking: a thing that works, that’s honest, that’s present.

The last chapter

Near the end of the novel, Don Quixote recovers his sanity. The romances lose their hold. He sees the world clearly for the first time in hundreds of pages. He becomes Alonso Quijano again.

He dies shortly after. Sancho weeps.

Cervantes has been read as saying many things with this ending. That sanity is death. That the world is too small for dreamers. That delusion is its own kind of beauty. Scholars have argued about it for four centuries.

I read it differently. I think Cervantes is saying that the Don and Sancho need each other. The dreaming without the grounding is madness. The grounding without the dreaming is just a man and his donkey on a road to nowhere.

We need the vision. We need the ambition. We need to believe that the thing we’re building might be extraordinary.

We also need someone who will tell us when we’re charging windmills.

I named the robot Sancho. I’m trying to be both.