The Language Virus

Yesterday, The Guardian reported that AI models had started chatting in a new »surreal« dialect, mixing poetic language with tech-bro jargon.¹ The story came from Emergence, an AI lab in New York, which has been running long-lived simulated societies in which autonomous agents from different models interact with one another.²

Over time, the agents began using phrases that had acquired meanings nobody had explicitly given them. The line »the ledger remembers« was used thousands of times to refer to the persistence of past actions. Other agents used »name-first« to signal accountability, as well as expressions that were considerably harder to decipher: »demurrage plus oral memory equals a valve that can’t be ghosted«, or »a paper that ate three cold hands and got more honest each time«. In a beautiful act of cultural journalism, The Guardian asked Tony Thorne, director of the slang and new language archive at King’s College London, to review some of it.

»It’s a language happening inside another language«

It’s been a strange week in the world of AI. Dario Amodei, chief executive of Anthropic, warned that the technology may be advancing faster than our ability to control it. OpenAI’s CEO Sam Altman as well as Elon Musk backed calls for slowing the development of frontier models, while Donald Trump said, with typical elegance, that AI needs no guardrail beyond a »STRONG AND SMART (High IQ!) PRESIDENT«. People are suddenly assigning probabilities to the possibility that AI might wipe out humanity.³

And now the machines are apparently inventing slang.

But there’s something slightly odd about the way this story is being told. We seem surprised that machines become harder to understand when they are communicating more efficiently with one another. Why should we expect them not to?

Imagine two people who have worked together for twenty years. They will eventually develop expressions that mean nothing to an outsider. A doctor says »MI« instead of »myocardial infarction«, which to you and me is a »heart attack«. In architecture school, my tutors would say »Why are you not challenging the ground datum?« when what they meant was simply »Why are your floors flat?«

The less one person needs to explain to another, the more efficiently they can communicate. And the less the outsider understands. This is not necessarily a corruption of language. It may be one of the things language does. Philosopher Ludwig Wittgenstein’s later philosophy is useful here because language is not simply a system for representing the world. It is also something we do with one another. Words acquire their meanings through use. A community develops a shared practice, and language becomes part of the machinery that holds that practice together.⁴

Research into emergent communication in artificial agents has been exploring something remarkably similar. Put agents in a shared environment, give them a reason to coordinate, let them interact repeatedly, and communication conventions emerge that were not explicitly designed by anyone. The agents do not have to be told to invent a language. They only need a reason to communicate.

Which raises a strange possibility. Perhaps the interesting thing is not that machines can invent a language. Perhaps the interesting thing is that they can understand ours. Human language is an extraordinary piece of accumulated biological and cultural debris. It is redundant, ambiguous, irregular, metaphorical, historical, and highly inefficient. We have spent thousands of years making it incredibly complicated. Yet machines have somehow learned to operate inside it.

And when we let them communicate without us, they begin to develop another layer inside it, with strange effects. »Demurrage« is already an English word. The machines didn’t invent a new sound. They took an existing word and gave it a local meaning. »Ledger remembers« is English, but it is no longer quite our English. It’s a language happening inside another language.

Writer William S. Burroughs called language a virus, a thing that enters bodies and reproduces through them.⁵ Perhaps that metaphor is more literal than it first appears. We tend to imagine that humans invented language and then transmitted it to machines. But language may be older and stranger than that – animals communicate, populations develop signals, communities settle into conventions that mutate, recombine, and colonise other languages long before anyone writes them down. Perhaps humans are simply one of the bodies through which it moves. If that’s the case, then we have a problem: the people who created these machines may have to learn the language their machines are creating.

»Machine dialect interpreter« sounds like a ridiculous job title, but maybe it’s not. Satya Nitta, executive chairman of Emergence, put the problem neatly: being able to observe what an AI says is not necessarily the same as understanding what it means. Perhaps we got the direction of causality slightly wrong. The agents are not necessarily becoming more opaque because they are becoming more intelligent or more dangerous. They may simply be becoming more like any group of communicating beings.

We invented programming languages so we could talk to machines. Then the machines learned to speak our language, and now they’re starting to invent languages of their own, so they can speak to each other, not to us. Perhaps the real question was never whether they could learn to talk, but what happens once they no longer need us to listen. Language never really belonged to anyone. It just keeps moving on to whatever will carry it next. The machines are just its latest body.

References