The Worker Inside the Machine
**EDO SEGAL:** Timnit, I want to take you somewhere specific, because the abstraction is the enemy and you've taught me that. Walk me through what actually happens for a modern chatbot to come out the other end sounding harmless. Not the metaphor. The supply chain. Who does what, where, for how much.
**GEBRU:** It comes out harmless because human beings made it harmless, by hand, and then were erased from the story. Start with the data: trillions of words scraped from the internet, much of it produced by people who never consented and were never paid — writers, artists, ordinary posters — [their work taken as raw material](https://www.youonai.ai/fieldguide/med/training_data_as_pirated_labor) and converted into a product that competes with them. Then the labeling and the filtering: armies of workers, disproportionately in Kenya, India, the Philippines, Venezuela, paid by the task — sometimes pennies — to rank outputs, to flag toxicity, to read and watch the worst material on earth so the model learns not to repeat it. The fluency you experience as magic sits on top of a substrate of human judgment and, very often, human trauma. And the final move, the cruelest one, is that the moment the model performs, the worker disappears. Her labor congeals into a thing that looks like it needs no labor at all. We even named it to hide her — "artificial" intelligence, as if there were nothing human in there. It's the most human thing in there.
**EDO SEGAL:** So let me restate that and hand it back, harder than you put it. You're saying — literally — that "artificial intelligence" is a marketing term for *human* intelligence that's been captured, compressed, and stripped of its attribution. That when the machine answers, it's not thinking — it's *relaying*, and the relay was built by people we've agreed not to see. Is that the version you'll stand on?
**GEBRU:** That's the version I'll stand on, and I'll add the consequence. Once you see it that way, the whole automation story inverts. The popular fear is that AI replaces workers. What it actually does, far more often, is *hide and relocate* them — to where labor can be bought cheapest and seen least. It's a continuation of a very old pattern: value extracted from the margins toward the center, from the Global South toward the North, dressed up this time as a technical marvel. So when someone says "the machine will free us from labor," I want to ask: free *whom*? Because right now it's not abolishing the work. It's burying it.
**BOSTROM:** I want to grant almost all of that, and then say the thing that will make her angry. The labor exploitation is real, it is documented, and it is a moral disgrace — I'm not going to perform a both-sides on a Kenyan content moderator's wages, because there isn't one. Where I'll push is on the word "the machine," because Timnit's argument, taken to its end, proves something stronger than she may want. If today's systems are *entirely* relayed human labor with no genuine capability of their own, then they are exactly as harmless as she says — and there's nothing for me to fear, and she wins the whole debate. But I don't think she believes that. I think she believes these systems are powerful enough to flood the information ecosystem, to displace workers, to entrench monopolies. And a thing powerful enough to do all that is a thing whose *future* capability is precisely what we should be arguing about. You can't have it both ways: either it's an empty relay, in which case relax, or it's a genuine and growing capability, in which case my question about the derivative is live.
**GEBRU:** I can absolutely have it both ways, because you're conflating two different things, and the conflation is the whole trick. A system can be *powerful* — power to flood, to displace, to surveil — without being *intelligent* in the way you need it to be for your story. A printing press is powerful. A surveillance camera is powerful. Neither is going to wake up and acquire the world's atoms. The harm doesn't come from the machine's mind, Nick. It comes from the *power relations the machine is embedded in*. A face system doesn't need to understand anything to put the wrong person in jail. The capability that matters for harm is not cognition. It's deployment, by powerful actors, against people with no recourse. You keep needing the danger to live inside the machine's skull because that's where your philosophy is. I keep finding it in the contract.
**BOSTROM:** That's a genuinely strong distinction and I'll carry it. Power without understanding — yes, that's most of the present harm, and you're right that I underweight it because my training points me at the skull. But here's the friction. You say the harm comes from the power relations the machine is embedded in. Now imagine the machine itself *becomes* one of the powerful actors. Not metaphorically — imagine an automated system that is itself making the deployment decisions, allocating the resources, optimizing the outcome, at a speed and scale no human committee can match. At that point your "power relations" and my "machine's goals" are the same object. We've spent the night discovering that the present and the future are the same mechanism seen on two clocks, and here it is again: the power you fear, concentrated and automated, *is* the agent I fear.
**EDO SEGAL:** I want to mark something, because the reader can't see it. That's twice now you've collided and ended up holding the same object from opposite ends — concentration, and now power-becoming-agent. Convergence number two: you both fear a power that answers to no one. You disagree about whether its dangerous form is a boardroom or a mind. But before I let that comfort anyone — Timnit, does it comfort you? Or is Nick's "automated powerful actor" just the boardroom's fantasy of itself, the founder dreaming he's building a god so he never has to answer to a union?
**GEBRU:** It's the second thing, and thank you for asking it that way. The fantasy of the autonomous machine is *useful* to the boardroom precisely because it launders responsibility. "The algorithm decided" is the best liability shield ever invented. So when Nick says "imagine the machine becomes one of the powerful actors," I hear the exact sentence a company wants you to believe, because if the machine is the actor, then the executives aren't. There is always a human hand on the resource allocation. Always. The autonomy is the alibi. And I'll mark the convergence Edo offered, but with that asterisk: we both fear unaccountable power. I just refuse to let it hide inside a machine where no one can be subpoenaed.
**BOSTROM:** I'll take the asterisk, and I'll even concede it describes the present accurately. My disagreement is a claim about whether the hand stays on the allocation forever, and I think the honest answer is: not necessarily, and the moment it comes off is the moment that can't be undone. But I won't pretend that future is here, and I won't let it erase the boardroom that's here now. Mark it convergence, with both our asterisks attached.
**EDO SEGAL:** Marked, asterisks and all. Hold the worker in your mind — she comes back when we talk about a world where labor is optional, and you two are going to fight about whether that's a paradise or the final erasure. But the next round is the one that made Timnit's name, and it's about a different kind of seeing — what the machine fails to see, and who taught it not to. The measurement and the mirror, after this.