LLMs Have No Subjectivity

In his seminal essay “The Death of the Author”, Roland Barthes dismantles the idea that a text contains a singular meaning, placed there by its author. If a singular stable meaning existed in that way, every reader would arrive at the same interpretation, with every reading. Instead, he argues, each reader brings their own understanding of the world, experiences, assumptions, and cultural frameworks to the act of interpretation. Therefore, the meaning isn’t in the text, but is between the text and the reader. Meaning is produced through each reading.

Reading Barthes has inspired me to apply the same question to the creation side as well, how text is produced with meaning. We perceive the world and interpret it, and synthesise language to express it. Another subject receives our language, interprets it and combines it with their understanding of the world – their subjectivity.^[1]

We have the power to reduce all of that complexity into a few words and then roughly agree on what each other might have meant, not exactly but good enough. There’s a lot of linguistic efficiency there – we can express so much with so little. But also something is lost in the process. Interestingly, Barthes opens his essay with a much stronger version of this.

We shall never know, for the good reason that writing is the destruction of every voice, of every point of origin. Writing is that neutral, composite, oblique space where our subject slips away, the negative where all identity is lost, starting with the very identity of the body writing.^[2]

I liken it to dehydration and rehydration. The text comes from a place of meaning, for transport it gets dehydrated of its context, and gets rehydrated with the reader’s subjectivity, imbued with meaning again. The reason this process can work so well and language can be so efficient is because there are two minds at each end doing the compression and reconstruction.

Meaning is both shared and agreed upon through culture, sustained by intersubjectivity, but also reinterpreted by each subject, with every use. This is how culture reproduces itself, by being reinterpreted, slightly differently by each person, in every encounter. Each conversation, each reading, each utterance of a word causes a slight drift in meaning, realigning it with the realities of our time and place. Words shift register, get used differently, their etymologies become entertainment. Culture is the sediment of this slow evolution of countless reinterpretations.

And thus, we partake in this symbol-making practice. We observe the real world, perceive it the way a human does, come up with signifiers that express it, and share them with others.

That’s the difference between us and these machines: they can’t partake in this meaning-creation exercise. They work purely on the sign level and don’t need meaning to produce text. They’re an implementation of a pure mechanised Saussurean^[3] machine that keeps the differences of signs in relation to each other. They don’t deal with where those differences come from.

I kept coming back to this point as the main difference between humans and LLMs. LLM-generated text doesn’t come from meaning, but can be syntactically correct and contextually appropriate. It doesn’t make it any better or worse, just a difference. We haven’t come across a language generator this sophisticated before, and what we assume when we read something comes with a lot of assumptions. We employ a theory of mind, put a character behind the text, and assume a lot of context. Also our anthropomorphic terms such as “understand” and “think” start falling short.

We don’t know what subjectivity is

Subjectivity is one of the great mysteries of the world. Philosophy, cognitive science, neuroscience, and art all approach it from different directions but none have cracked it yet. We don’t have complete access to objective reality as it is. Each of us perceive and dissect the world in a way humans do: we group things, sort them into categories, draw distinctions and give them meaning, and much of it is arbitrary. And yet we end up doing it in more or less the same way, with similar enough precision. Enough that two people looking at the same thing can see the same, or argue over the differences.

The one thing that relieves me of the weight of assumption is that nobody knows the answer. Nobody can tell you exactly how it works – if they did, they’d conquer the world. We can’t look into a mind and observe how it does its magic directly. We can only infer it from behaviour, and only another subject can do that inferring.

So I’m fairly confident about how LLMs work, and much less sure about what minds are. The claim that LLM-generated text is different rests on intuition and my own theories on how we produce text. Whatever the true mechanism of subjectivity turns out to be, I’d be really surprised if it looked like how LLMs work, and it’s worth trying to find the words to describe the difference.

We don’t know what meaning is

Meaning is one of those suitcase words^[4] that pack many different concepts under a single word. What a word means, what I mean by saying it, what something means to me, and what life means are outcomes of completely different phenomena, but all seem very similar so we seamlessly slide between them.

There is no real consensus on where meaning comes from on the production side, as I mentioned. I can quote Lacan, Grice, or Husserl, and all have incredible insights into how we might breathe meaning into our words. Intention, desire, functionality, all play a role but strikingly none of them play a role in how an LLM produces text.

Ultimately LLMs were trained on text that was produced by people who did mean those words at one place, in some context. The least controversial thing I can say is: LLMs put a gap between the origin of meaning and the ultimate receiver. Before LLMs when we read a text we could assume it was directly produced by another person, and we’re reading them now. They were an identity, situated in a context at the moment they produced that text. LLMs strip away all of that context and identity, the groundedness in reality of it, and only keep the relationships between other pieces of text at the moment of training. Like a crypto tumbler: words from countless sources go into one pot, get mixed, and are paid out again with their origins scrambled. Then when they produce the text, it’s laundered of its connection to reality, and we’re free to make of it what we will.

And the gap is at its widest when a model produces a novel combination of words nobody has written before. With no original meaning or context, nothing to trace as a remix of, your defences are further lowered when you’re contemplating whether the chatbot understood what you meant.

Deferred Subjectivity

As we read an LLM-generated text, our minds try to create a theory of mind of the supposed writer. We unconsciously trace the origin of the intention behind the text, a subjectivity. The LLM puts a gap between the source and the reader, hence the subjectivity is displaced.

But I like using the word deferred to describe it. Derrida uses deferral to describe how there never is an ultimate meaning: all texts refer to each other. When you chase the meaning of a word, it lies in its difference from other words, and their meanings lie in their differences from yet others. You can never reach a final ultimate word that all words refer to, the source of all meaning. He called this différance, a coinage combining difference and deferral.

The way I use it somehow rhymes: when we trace the origin of the meaning of an LLM-generated text, our trace gets lost in the tumbler. Confusingly, it’s similar to what happens with human-written text too: the trail goes into someone’s mind, and we don’t know how it works. We do have an intuition about what happens in a mind, because we have one too. Since both processes look similar, our efficient minds are quick to apply the same intuition to LLMs too.

I think most of the interesting effects we see with LLMs come from these three things working together: the scrambled origin of texts, the subjectivity that gets deferred, and our propensity to search for a mind like ours behind what we read.