What the Machine Cannot Do for You

**EDO SEGAL:** I want to ask this one personally, because I have a stake in the answer that keeps me up at night. I have children. They are growing up able to outsource every product of thought to a machine that does it faster and often better — the summary, the analysis, the essay, the proof, the plan. And I watch them, and I watch my engineers, and I watch myself, and the question that frightens me most isn't whether the machine thinks. It's whether *we'll stop*. René, you ended your own argument here — on something you said cannot be outsourced. Tell me what it is, because I need it for my kids.

**DESCARTES:** Then I will give you the most useful thing I have ever said, and I will give it to your children directly. Return one last time to the cogito, and notice the feature everything has been circling. The certainty of "I think, therefore I am" is not a fact about the world that could be discovered, recorded, transmitted, or looked up. It is the product of an *act* — the present-tense act of thinking — and the certainty exists *only in the performing of it*, available only to the one performing, only about that one, only in the doing. I cannot run your cogito for you. No one can. And this is the property no outsourcing can preserve. A machine can supply the *products* of thought with terrifying ease — the summary, the analysis, the answer — because those are detachable; they are the residue thought leaves behind, the hats and the coats. But the act of thinking is not a product. It is the activity itself, certain only in the doing. You can have all the products and perform none of the act — which is, quite possibly, exactly your machine's situation — and you can perform the act, producing nothing, and still be the thinking thing the cogito certifies. So tell your children this. The work was always detachable. The thinking never was. A civilization can outsource every product of thought to machines and remain a civilization of thinkers — *if its people keep performing the act.* It can also acquire every output of mind and quietly stop running the cogito, possessing all the summaries of understanding while no one, anywhere, understands. The machine cannot do the thinking for you. Not because it is forbidden. Because thinking is the one thing that is only itself in the first-person doing, and the moment it is handed off, it is no longer being done by the one who needed to do it.

**EDO SEGAL:** Quietly, then — that's the thing I needed and didn't have words for. Geoff — and I'm bracing, because you've dissolved every other distinction tonight — does the materialist let Descartes keep this one? Or does the machine take the cogito too?

**HINTON:** The materialist lets him keep it, and I want to say *why* I let him keep it, because it's not charity, it's that he's right and the rightness doesn't depend on the metaphysics. Whatever thinking *is* — even if it's purely physical, even if it's a network noticing it's running — it's still something that happens *in a particular system*, and you cannot have it happen in *your* system by having it happen in *another* one. That's not dualism. That's just what it means for a process to occur somewhere. When your kid lets the machine write the essay, the essay exists, and the *understanding the essay was supposed to grow in your kid* did not occur, because understanding is a thing that happens in a learner, by the learner doing the hard part, and the machine did the hard part instead. This is the [apprenticeship problem](https://www.youonai.ai/fieldguide/med/apprenticeship_problem) and it terrifies me as much as the existential stuff, maybe more, because it's *here*. We are building a generation that can access every answer and may never build the internal model that lets them know which answers are wrong. And here's the brutal part, the part that connects to my whole argument: the machine *learns from the struggle it's removing*. Every time it does the hard part, it gets better, by exactly the mechanism — error, gradient, adjustment — that the kid is now skipping. The immortal mind is getting stronger on the very friction the mortal mind is being relieved of. That's the trade I most want people to see.

**EDO SEGAL:** So the two of you — the dualist and the materialist — converge, from opposite metaphysics, on the same instruction to the same frightened parent. Mark it: that's the fourth convergence, and it's the warmest. Say it to her plainly, each of you, one sentence. The struggle is the —

**DESCARTES:** The struggle *is* the thinking. Protect the struggle, and you protect the thinker. There is no other place the thinker lives.

**HINTON:** The struggle is where the model gets built — in your child's head or not at all. The machine will gladly do the struggling. Don't let it, on the things that are supposed to become *them*.

**EDO SEGAL:** And there's the [candle](https://www.youonai.ai/fieldguide/med/candle_in_the_darkness) that doesn't go out — not because the machine can't reach it, but because it's lit from the inside, in the doing, by a someone, and that someone has to be you. Hold every thread now. We've fought through the cogito, the substances, the two tests, the wax, the immortal, the demon, the beast, the gland, and the candle. The next chapter, I leave the room. You two have been circling each other for two and a half hours behind my questions. Now you ask each other directly. I'll only speak if the building catches fire.