LLMs aren’t writing

I’m not an AI hater: I’ve used it for my fair share of projects that involve coding or for troubleshooting various technology problems. In my work testing Apple’s Apple Intelligence releases, I’ve generated some images, all truly bizarre and many of them good for nothing beyond a joke.

But I have been fascinated by the response to Anthropic’s AI watermarking scheme, especially after I read my friend John Gruber’s full-throated rebuttal. I have great respect for John as a writer. He truly cares about details: he’s meticulous about the words he picks, the way he constructs sentences and arguments.

This is why I find his defense so puzzling. Because a large language model could not be further away from those ideals. In the title of his piece, John describes the watermarking system as “a perversion of writing.” That might be the case if AIs were writing. But they are not; they are simply generating text. Those two things are not the same. As per the premise of John’s piece, writing is about carefully picking the words and crafting the meaning of what you’re saying, things that AI do not—and patently cannot—do.

LLMs do not care about the words that they pick because they cannot care about anything.

Speaking of two things that are not the same, John rightly points out the difference between the phrases “he leaped at the chance” and “he jumped at the opportunity”. Those are indeed distinct—if semantically similar—phrases, each of which might be more apt in a particular situation; or, to put it in another fashion: the use of each of those phrases tells us something different, whether about the person being described or the writer.

But the LLM doesn’t know which of those phrases is the right phrase to use. It has a guess, based on its models and weights and inputs. But the ultimate choice of those phrases tells us nothing about the writer because there is no writer.

All of this outrage presupposes the existence of a “correct” answer that the LLM is eschewing because it was told to pick different words, but if there were a correct answer, then shouldn’t it be chosen every time? But an LLM’s non-deterministic nature doesn’t work that way. Because, at the end of the day, the AI does not truly know what any of these words mean. There is no ur-text. You might as well get mad at word choice in a lorem ipsum generator.1

Likewise the example of “my favorite tropical fruit.” John points out the ridiculousness of a watermarking scheme substituting “pineapple” for “banana.” Of course! Those are not reasonable substitutions (outside of a smoothie).

For a person. But a computer has no favorite fruits. It could not possibly know the right word to use. It has never tasted the sweetness of a pineapple or cut up a banana for its yogurt. So why would I possibly care what it thinks is the correct word for that scenario, right or wrong?

I get being mad about computers interfering with what we, the users, want. But in this, AIs are only the latest example. Autocorrect and autocomplete, computational photography, search engine algorithms: all of that has led us down this path where we’re willing to accept the decisions a computer makes for us. The LLM is just the ultimate expression of that.

Getting mad about your LLM’s choice of words is like being pissed that Godzilla stomped on your garden. Dude, the rest of your city is in flames. Sorry about your flowers, but, you know, *gestures at everything*. We’re already talking about technology with huge environmental, societal, and ethical ramifications. This, to me, is quite possibly the least upsetting thing that AIs have done.

Anyway, welcome to the “mad about AI” fold. Glad to have you, even if I wish maybe you would have been a little more upset about all the words that it stole from me and other writers—words that were all carefully chosen.

In the end, I can’t summarize it any better than to ask: if you care so much about word choice, why are you using AI to generate text?

If this does truly make AI-generated text worse, well…good. A lot of people are already willing to accept what an LLM churns out as “good enough” and, if I’m being realistic, I don’t think this will change anything. But if it does lead to more people being dissatisfied with the pablum they’re being fed and turning instead to writing and editing their own text, then that would actually be a positive outcome. Maybe it’d even mean fewer human writers being put out of jobs.

In the end, what upsets me the most about this is the tacit equating of AIs generating text to writers writing. A computer that has digested the work of thousands if not millions of people and regurgitated them according to a statistical model is equivalent to a person toiling by the sweat of their brow to bring meaning into existence from nothingness?

Get out of here with that garbage.

Or autocorrect. Just now, as I typed “lorem ipsum”, autocorrect changed it to “lore Epsom”. ↩