The Book-Eating Toy

Secondhand booksellers in Britain and Ireland have begun receiving strange bulk orders: hundreds of unrelated books, bought together by mysterious customers. Some suspect AI companies are behind them, acquiring obscure books to scan for training data. Anthropic has already spent millions buying physical books for this purpose.¹

»The book was made to be read. Then a new machine arrived that could use it for something its author could never have imagined«

Those books were made to be read. But if the suspicions are correct, they are now being turned into raw material for something else.

The twentieth century digitised much of the physical world into information. But plenty of information never made it into the digital world, or made it there incompletely. The twenty-first century is now reaching back into the physical world to recover some of it, because new machines have found other uses for information earlier systems had no reason to collect.

Anthropic's reported internal project, Project Panama, allegedly involved buying books, removing their bindings, scanning the pages, and discarding the physical copies.² Those books were not given a second life. Their text was extracted, transformed into data, and consumed by a machine that produces something else.

The transformer – the mechanism behind today's large language models – did something similar on a vastly larger scale. The internet was built for communication, publishing, commerce, and information exchange. It subsequently became training material for machines that learn statistical relationships between words and other forms of data.³ The output of one technological system became the input of another.

This has happened before, of course. Petroleum became plastics, solvents, and Vaseline. Recorded sound became raw material for musique concrète, the Mellotron, sampling, and hip hop. Newspapers became material for artists’ collages and fish’n’chips wrappings. A spring mechanism intended to stabilise sensitive ship equipment became the Slinky toy.⁴ Something built for one purpose became material for something that didn’t exist at the time of its inception.

A successful technology eventually becomes part of its environment. We stop noticing it as technology and start treating it as infrastructure.⁵ Then something else arrives and discovers possibilities in it that its makers never intended, predicted, or needed.

Perhaps we need a word for this. I propose probolism: the process by which the output of one system is thrown forward into another, where it becomes an input and changes what that second system can do. Catabolism breaks something down. Metabolism turns what enters the system into what the system can become. Probolism throws what a system produces into the future.⁶ It is what happens when an output becomes an input: when something made for one system becomes material for another.

Nobody writing a book in 1975 knew that its text might one day become training material for a language model. Nobody recording a song in 1972 knew that a fragment of it might become the rhythmic foundation of a hip hop banger fifty years later. The future system creates a new use for the old system's output.

This may change what it means to produce information. We normally make things because we know what they are for. A company commissions a report because somebody needs it. A photographer takes a photograph because somebody wants that photograph. A scientist records a measurement because it answers a question. But a new production system can create demand for information that nobody previously had a reason to collect.

Our AI models have already consumed enormous quantities of the readily available web. The interesting frontier may move elsewhere: into the gaps between the things we thought worth recording. An obscure agricultural treatise. A manual on breeding working donkeys. A history of the local watermills of a valley in northern Spain. A forgotten technical journal tracing the development of railway signalling. A book documenting the dialect of a village whose population is now 47 people. Each describes a tiny corner of the world. Together, they describe the texture between the lines of the Internet.

These are the sorts of books that describe worlds most people have no reason to enter. The techniques of a particular trade. The history of a particular valley. The accumulated knowledge of an industry that disappeared decades ago. The habits of an animal breeder, a boatbuilder, a surveyor, a millwright. The Internet has excellent coverage of what millions of people wanted to say. It has much poorer coverage of what ten thousand people knew, what a thousand people practised, what a hundred people remembered, and what one person bothered to write down.

For a human reader, most of this is marginal. We want meaning, narrative, argument, experience. A language model can also extract statistical relationships between words, concepts, and contexts. An obscure book may therefore be valuable precisely because almost nobody has read it: it contains unusual relationships, another patch of territory, another way the world has been described.

Perhaps the next generation of writers will produce books that nobody particularly needs to read, but that fill in the spaces between what today's machines already know. That would invert ordinary product development. We usually identify a need, then make something to satisfy it. A new production system can work in the opposite direction: »make the record first; discover the use later«.

And the process need not stop with artificial intelligence. Every technological system produces outputs. Those outputs enter the world and become part of the environment for whatever comes next. A photograph becomes pixels. Pixels become training data. Training data becomes a model. A model becomes an input to another system.

This is less like a chain of inventions than an ecology: systems altering the conditions in which other systems can emerge. The book does not disappear when something new learns to use it. It acquires a new position in the technological environment.

The book was made to be read. Then a new machine arrived that could use it for something its author could never have imagined. The value of an invention is partly the invention it makes possible next.

References