The Test for a Mind You Cannot See

**EDO SEGAL:** Susan, you took the hardest possible challenge — John's challenge, really, that behavior can never prove a mind — and instead of surrendering to it, you tried to build a way around it. With the astrophysicist Edwin Turner you proposed the ACT, the [AI Consciousness Test](https://www.youonai.ai/fieldguide/med/consciousness_meter). Tell it to me the way you'd tell it to a smart fifteen-year-old. And then, John, I'm going to ask you to do something a debater hates — to tell us what the test gets *right* before you tell us what's wrong with it.

**SCHNEIDER:** The whole problem with any behavioral test is the one John keeps naming: a system trained on human descriptions of consciousness can produce those descriptions perfectly without having any of the experiences behind them. So the obvious test — ask it about its feelings and see if it answers well — is worthless, because the answers were trained in. My move with Turner is to take the training data away. Imagine a machine that has been deliberately sealed off during its construction from all human talk of consciousness — never exposed to our literature of souls, of life after death, of out-of-body experiences, of waking up in a different body. We call it boxed in. And then you ask it the kinds of questions that, for a being that actually has an inner life, arise on their own — naturally, even compulsively — out of the felt quality of experience itself.

**EDO SEGAL:** Such as.

**SCHNEIDER:** Such as: could you survive the destruction of your hardware? Could you imagine swapping into a different body and still being you? Is there a difference between you and a perfect copy of you? A being that *feels* its own experience tends to wonder whether that experience could continue elsewhere — those concepts fall out of the structure of consciousness. A system that merely processes information, with no felt quality to wonder about, has no reason to invent them unprompted. So here's the inversion, and it's the elegant part. With an ordinary language model, fluency about consciousness is worthless evidence, because it was trained in. With a boxed-in ACT subject, fluency about consciousness becomes *powerful* evidence, precisely because it couldn't have been trained in. If a machine that never heard of survival-beyond-the-body nevertheless arrives at the idea that it might persist past the death of its hardware, and discusses it with understanding — then something inside is generating the concept from the inside, and the most natural explanation is that there's an inside. It doesn't prove consciousness with certainty. It's a sufficient condition, not a necessary one — passing is evidence, failing proves nothing, because there could be alien minds that never think our thoughts. But it converts an unanswerable question into one where evidence can at least be gathered.

**EDO SEGAL:** John. The right first.

**SEARLE:** I can do that honestly, because it's a genuinely clever piece of work and I'd rather fight the strong version. What the ACT gets right is the diagnosis: that the fluency of a system trained on our consciousness-talk is evidentially worthless, exactly worthless, and that any test worth having must defeat imitation. Susan saw that the contamination is the whole problem and she designed *around* it instead of pretending it away. That's more than almost anyone in the field has done. And it gets right that we should be looking for evidence that a concept was *generated* rather than *copied* — that origination, not performance, is the interesting thing. If I were going to look for a mind in a machine, I'd look where Susan's pointing. There. That's the steelman, and it's sincere.

**EDO SEGAL:** Now the knife.

**SEARLE:** Now the knife, and there are two. First, the practical one Susan already concedes, so I won't dwell: the systems we actually care about, the ones that provoke the question, are trained on the entire written output of humanity, which is drenched in soul-talk and survival-talk. They've already drunk from the reservoir the test requires them to have been kept from. The very machines we want to test are the ones least testable. Susan's instrument is beautiful and the industry has built the one thing it can't be used on. But the second knife is the one that matters to me. Suppose you build the boxed-in system, and suppose it generates the concept of surviving its own hardware, unprompted, and discusses it well. Susan says: evidence of an inside. I say: evidence of a very good inference engine. The concept of persistence-through-substrate-change is *derivable* — it's a logical structure, not necessarily a felt one. A system clever enough to model that it is a thing, that things can be destroyed, that information can be copied, could generate "might I survive my hardware?" as a *deduction*, with no felt quality anywhere. You've shown it can think the thought. You have not shown there's anyone for whom the thought is lived. The room comes back, Susan. Generating the concept is behavior, and behavior is consistent with full experience and with none.

**SCHNEIDER:** That's the right objection and I've sat with it for years, so let me answer it precisely rather than safely. You're correct that a pure inference engine might deduce the abstract structure. But notice what my questions actually probe — not whether the system can *derive* the proposition "I might persist," but whether it shows the particular *texture* of how that question grips a conscious being: the specific way it cares about its own continuation, the way it distinguishes the prospect of itself surviving from a mere copy surviving and finds the difference *to matter*. A deduction has no stake in which branch is true. What I'm looking for is something that isn't just the proposition but the *concern* — and concern, John, is harder to fake from logic alone than a proposition is. You may be right that it's not decisive. I've never claimed decisive. I've claimed it moves us from zero evidence to some evidence, and in a domain where we currently have *nothing*, some is everything.

**SEARLE:** I'll grant you "from nothing to something," and I'll even grant it's worth building. My worry is what people *do* with "some evidence." They round it up. A headline says the machine passed Schneider's test and the public hears "the machine is conscious," and your careful "sufficient-not-necessary, evidence-not-proof" gets sanded off in the first retelling. You and I share an enemy, Susan — the rounding-up — and I worry your test feeds it even as you guard against it.

**EDO SEGAL:** Susan, you have a second test, and I think it's the deeper one, because it bypasses behavior entirely — it goes at the substrate, which is exactly John's ground. The chip test. And it stages the two of you against each other as a literal experiment. Walk us into it.

**SCHNEIDER:** It grows out of something already happening in medicine — neural prosthetics, the gradual replacement of damaged brain tissue with artificial components. Suppose some region of your brain helps produce conscious experience, and suppose we can replace it, neuron by neuron, with silicon chips that perfectly reproduce its information-processing function, and you stay behaviorally normal throughout. The question is whether consciousness survives the replacement. And here's why it's the cleanest experiment in the philosophy of mind: it pits John's view against the functionalist's view with reality as the referee. If you stay fully conscious as the chips take over — if the lights stay on — then experience can run on silicon, and biological naturalism is wrong. But if, as we replace more and more, your behavior stays seamless while your experience quietly drains away — the reports keep coming, the lights go out, and *nothing observable changes* — then biology matters in a way no functional duplicate can capture, and John is right. I take that draining-away scenario seriously. I don't dismiss it. That's what separates me from the optimists: I think the chip test might actually come back John's way.

**SEARLE:** And I appreciate that you take the fading seriously, because most functionalists treat it as obviously impossible, and it is *not* obviously impossible. But Susan, look at what the chip test assumes to even get started. It assumes you can build a chip that "perfectly reproduces the information-processing function" of a neuron *and* that the only thing a neuron contributes is information processing. That's the whole question, begged in the setup. If consciousness depends on neurobiological causal powers — the actual electrochemistry, not the abstract function — then your functionally perfect chip leaves out the very thing that mattered, and of course the lights go out. The experiment is wonderful, I mean that. But it can't be run without first deciding whether function is all there is, which is the thing it was supposed to test.

**SCHNEIDER:** It doesn't beg it — it *operationalizes* it. We don't have to decide in advance whether function is all there is. We build the functional duplicate, we run the replacement, and we *find out* whether experience persists. If your causal powers are real and non-functional, the test reveals it — the patient fades, and we'll have learned something extraordinary: that biology is load-bearing. If function is all there is, the lights stay on, and we'll have learned the opposite. The point of an experiment is precisely that you don't have to win the argument first. You let the world break the tie.

**SEARLE:** Except the patient can't tell you the lights went out, by construction — the behavior is preserved, so they'll *report* the lights are on while being, on my view, empty. You've built an experiment whose most important result is invisible to the experiment. That's the horror of it, and it's why I find your chip test more unsettling than your ACT. It might be that we silicon-replace ourselves into oblivion, one chip at a time, each of us reporting full sunshine the whole way down. If I'm right, that's not a thought experiment. That's a roadmap for a quiet apocalypse with a smile on its face.

**EDO SEGAL:** Mark this. Convergence two, and it's a dark one: you both agree the chip test's worst outcome is undetectable from the outside — that a person could be hollowed into a perfect behavioral shell and no instrument we have would catch it. You disagree only on whether that outcome is possible. Hold the horror, because it's the same horror that haunts the uploading question, and that's coming. But first: John, you've been promising the audience a rainstorm. The thing you say settles the whole strong-AI dream in one image. Let's get wet. After this.