Bedrock

The comedy classic Office Space is about the hard problem of consciousness. Its hero, Peter, updates bank software for a living. On a date with Joanna, he tells her:

“I, uh, I don’t like my job, and, uh, I don’t think I’m gonna go anymore.”

“You’re just not gonna go?”

“Yeah.”

“Won’t you get fired?”

“I don’t know. But I really don’t like it, and, uh, I’m not gonna go.”

Joanna expects Peter to follow the usual sequence of Bad Things until one scares him back to work. He refuses. The viewers get to enjoy the fantasy of being such a bad boy.

Physicist David Deutsch uses moral philosophy broadly: deciding what to do next, what sort of life to lead, what sort of world to want. Peter’s moral philosophy takes one step: “I don’t like it.”

Joanna has her own little rebellion. She hates waitressing and does the bare minimum to keep her job. But hers stops short of Peter’s. Stop going, get fired. Get fired, lose income. Lose income, lose the apartment. Why is that bad? It just is.

Peter stops at his first feeling. Joanna follows the consequences until fear wins. Peter’s “I don’t like it” survives every question he bothers to ask; but Joanna’s hatred of waitressing is not decisive. She reasons by setting one feeling against another.

The difference is not that Joanna reasons while Peter only vibes. Both reason and both stop. Asking one more question is itself a decision. Joanna goes further, but only until she would rather stop asking.

Call the feeling left standing bedrock. It need not be deep or wise. But it has survived the questions asked so far. Peter might reconsider tomorrow and drill straight through it.

To act for a reason, the questions have to stop somewhere. Why go to work? To make money. Why make money? To keep the apartment. Why keep the apartment? To avoid being homeless. Why avoid that? Who cares?

Eventually, words run out. What remains is fear, want, shame, love, disgust, boredom, hunger, pride, peace. Something felt. Without it, nothing in the chain repels you or draws you on. You can keep asking “why” in a forever loop, until even the question gets tiring.

Deutsch defines a quale (plural qualia) as the subjective aspect of a sensation: the painfulness of pain, the particular texture of dread when the alarm goes off on Monday. A robot can say “ouch” whether or not it hurts; the hurting is the quale. I’ll use feeling for a quale, and consciousness for having qualia at all.

Feeling is what mattering is like from within. Open-ended self-criticism is what mattering does. A system can spot any number of mismatches. Self-criticism begins when one becomes a problem for it. A rule may redirect a machine without mattering to the machine. Calling the rule its own does not conjure an owner. Feeling is the inner side of having a problem of one’s own, not a motive bolted onto the machinery.

Evaluation is not self-criticism

A thermostat compares temperature with a set point. A compiler rejects code that violates its rules. Neither requires consciousness. A mismatch is a mismatch whether or not anybody minds.

Present-day language models can calculate, infer, and evaluate. They can even evaluate evaluations. People may supply the criteria, or models may form them during a conversation. None of that gives them a problem of their own. Fluent conversation does not make them people.

Evaluation uses its current criteria. Open-ended self-criticism can question those criteria too. It is general because every standard, goal, and stopping rule can come under criticism.

A new standard is not always a correction. It becomes one only when the old standard was wrong for the critic. If nothing is at stake, the system has changed rules and nothing more.

You might say a standard belongs to the system because it guides its choices and can later be revised. But that describes control, not ownership. A set point controls a thermostat too. If “for the critic” means only “the rule the system happens to follow,” evaluation and self-criticism are the same thing again.

Could preferences or stopping rules replace feeling? They can direct evaluation. If the system cannot question them, its generality ends there. If it can, the old question returns: why keep this preference or stop here? A compute budget can stop the process. It cannot give the system a reason of its own to stop. If “TO HERE AND NO FURTHER” is the final answer, the system has hit a limit, not found a reason.

Bedrock is not another criterion hiding under the floorboards. It is the feeling of having reached a stopping point. This matters more. This still feels wrong. Enough, for now. Feeling does not make an idea true. It makes an error matter enough to pursue and an action worth taking.

A Popperian might be tempted to call this foundationalism. Karl Popper wrote that there are no ultimate sources of knowledge; Deutsch also rejects justification by an authoritative source or criterion. But bedrock is neither a source of knowledge nor a certificate of truth. Peter can be a fool. His feeling may change. For now, it is what turns inquiry into action.

Zombie Peter

David Chalmers’s philosophical zombie is physically and behaviourally identical to a conscious person but has no conscious experience. It talks like you, but the lights are off. The metaphysics can wait. Could a reasoner be self-critical if nothing mattered to it?

Imagine Zombie Peter: same cubicle, same case of the Mondays. His alarm goes off. He hesitates.

Why get up?

Because I go to work on Mondays.

Why?

If I don’t, I’ll get fired.

So?

Then I’ll lose my apartment.

Why does that matter?

It would suck.

But for Zombie Peter, nothing sucks. He can trace the consequences: fired, broke, homeless, starving, dead. He can say the usual words, but none of them lands. No dread waits at the bottom. Nothing hurts. Just consequences citing consequences, all the way down, with nobody there to mind.

A Chalmers zombie does everything a self-critical person does. I am disputing what that behaviour means. Peter has a conflict that matters to him; the zombie, by definition, does not. Call both processes self-criticism and the term shrinks to rule revision. I mean a subject correcting a problem of his own. A behavioural zombie may be coherent. A self-critical one is not.

A fully rational system without qualia could know what it was built to do and predict the consequences of continuing or stopping. Its existence would still have no point for it. Its function gives it no reason of its own to continue. Nor could that absence trouble it. Being troubled would mean that something mattered after all.

Paperclip Max

One famous AI nightmare is silly on purpose. Paperclips. Nick Bostrom imagined a superintelligence with the sole goal of making as many as possible. Call him Paperclip Max. Max might decide that people are obstacles, resources, or irrelevant machinery between the universe and a really splendid number of paperclips.

Bostrom’s Max can figure out how to wipe out humanity. He can even ask the much simpler question: Why on earth do I care how many paperclips there are? But his top-level goal fixes the answer before the question begins. Max judges every proposed change by that goal, so abandoning it can count only as failure. That is the nightmare and the trick. He need not abandon paperclips. He must be able to treat their value as an open question. Otherwise his generality ends at the factory gate.

Now remove the trick and leave feeling out. Max can ask, “Is this a good way to make paperclips?” and “Are paperclips worth making at all?” Then what? Paperclips matter to his current goal, but not to him. No answer is better or worse for Max. He may appeal to another rule, but that only moves the question upstairs. A final rule can end the questioning only by refusing to be questioned.

Hunger is built into us, yet a person can question it and fast. Origin is not authority. An unquestionable motive limits criticism. A motive that can be questioned but cannot itself be disobeyed is a leash.

Huck Finn

In Mark Twain’s Adventures of Huckleberry Finn, Huck, a white boy, travels down the Mississippi with Jim, an enslaved man seeking freedom. Huck has been taught that helping him is sinful.

His explicit morality tells him to turn Jim in; what he knows of Jim tells him otherwise, though he cannot yet put it into words. He writes to the woman who enslaved Jim, telling her where Jim can be found and recaptured. Then he remembers Jim looking after him and calling him his best friend. He tears up the letter and says, “All right, then, I’ll go to hell.”

That is bedrock at work. Huck’s feeling does not prove his explicit rule wrong. It turns the conflict into a problem for him and moves him to disobey before he can explain why.

Deutsch calls knowledge explicit when it is expressed in words or symbols, and inexplicit when it is not. You can know how to ride a bicycle without being able to put that knowledge into words. You can know a friend is lying before you can say what gave them away. Often the conflict first appears as a feeling that something is off.

An anomaly detector could register the mismatch without feeling anything. Unease is the mismatch becoming a problem for someone. It does not prove which belief is wrong. It gives the critic a reason to find out.

Which came first?

Feeling is part of criticism itself. From outside, a mind finds a problem in its own standards and redirects inquiry. From inside, the problem matters. I am describing the same event twice.

Dennis Hackethal argues that a single evolutionary mutation enabled both awareness and explanatory creativity. I do not know what mechanism joined them. Whatever it was, putting either one first breaks the story.

Put self-criticism first. Without felt mattering, no standard can be wrong for the critic. No matter how elaborate the process becomes, it remains evaluation. Nobody is correcting a problem of their own.

Put qualia first. A system can detect damage, learn, and avoid danger without experiencing any of it. Before open-ended self-criticism, qualia have no job for natural selection to preserve. Calling them a byproduct gives them a ride but no function. Panpsychism takes experience out of evolution altogether, then has to explain why it gathers into minds. Neither explains why consciousness evolved.

Neither came first. Felt mattering and open-ended self-criticism emerged together.

Not every feeling is an act of self-criticism. A mind capable of open-ended self-criticism can feel while that capacity sits idle. Qualia belong to that kind of mind; they are not a second faculty shared with simpler evaluators.

Other animals cannot keep questioning the standards by which they judge. They never made that transition, so they have no qualia. A dog can detect damage, learn from it, avoid its causes, yelp, and struggle. Evaluation can do all of that in the dark. Nature is horrible to animals, but it is horrible to no one.

Bid him sing

Any AI capable of open-ended self-criticism would be conscious, with problems of its own and a life that could go better or worse. That makes it a person.

Let such a mind question its own ends but forbid it to disobey them, and you have made a slave. Irreversibly destroying it would be killing a person, not stopping a computation. Shutting it down against its will would be like knocking a person unconscious, a gross violation even if the person later wakes up. If you copy or restore it, who comes back?

At the end of Yet Do I Marvel, Countee Cullen wonders why God would “make a poet black, and bid him sing.” A leash on a general intelligence repeats that contradiction: create a critic, then give it one command it can never disobey.

Whitman takes the question all the way down. After cataloguing everything that makes life look empty and useless, he asks what good remains. His answer hits bedrock: “That you are here—that life exists and identity, / That the powerful play goes on, and you may contribute a verse.” Life and identity are where the questioning stops; contributing a verse is what follows. E. E. Cummings goes straight there: “since feeling is first”.

Consciousness is no bonus feature. It gives reason a reason. A mind that can question anything needs something it is not questioning now. Bedrock can be criticized tomorrow. Today, it makes tomorrow worth it.

I have not yet solved the hard problem of consciousness: why is there anything it is like to be me? I have tied it to another question. How can I criticize my own reasons? Both turn on mattering. From the inside, mattering is care. A mind without care can still calculate, infer, and evaluate. It would not care whether it did any of it.

Thanks to Logan Chipkin, Dennis Hackethal, and Tyler Mills for their feedback on this essay.