The Quizzing Problem

Yesterday, Nature Methods introduced CauTrigger, a computational method researchers have built to identify the factors that may drive a biological system from one state to another – a cell on its way to becoming a different kind of cell, say, or a gut microbiome tipping into a different balance.¹

»Going from everywhere you could possibly look to a handful of places actually worth testing is most of the hard part«

The software sorts through everything that changes when a biological system changes, and tries to flag which of those changes are in the driver’s seat, and which are just going along for the ride. The researchers tested it on data that tracks embryonic stem cells as they differentiate into a different cell type, and on gut-microbiome data that compares people on the autism spectrum to those that aren’t, trying to figure out if specific gut bacteria drive biological differences between the two groups.

Let’s say you start quizzing your child over dinner, and the grades improve that same term. Did the quizzing help? Or did you only start quizzing because you’d already noticed your child getting more interested in school – in which case the quizzing never had anything to do with it, and the child would have gotten the better grades either way? This is one of the oldest problems in science, dressed up yesterday in stem cells and gut bacteria.

CauTrigger tries to get at the causes without physically intervening, by arranging biological signals into layers that separate upstream triggers from their downstream effects. It does this through a mathematical framework called do-calculus, built by the computer scientist Judea Pearl.² His insight is that there’s a third option between merely noticing a correlation and actually running an experiment. If you state your hypothesis about which parts of a system might affect others, you can sometimes work out what an intervention would do without having to perform it. Causal reasoning doesn’t have to happen in the laboratory – it can happen at a desk, as long as you’ve got the structure right to begin with.

But getting that structure right is exactly the catch. Before CauTrigger can calculate anything, a researcher has to tell it which signals are even allowed to affect which others. The software doesn’t discover that shape from the data – it just uses the data to judge how strongly to believe in the shape it was already given. The philosopher Nancy Cartwright built a career on exactly this catch, and summed it up in a sentence sharp enough to be a slogan: no causes in, no causes out.³ Her point is that something only becomes a cause under the right background conditions – the same gene might drive a change in one setting and do nothing in another – and no amount of clever math can tell you, from inside the model, whether those conditions actually hold. You still have to go out into the world the model is supposed to describe and check your results.

That’s not a case against CauTrigger – the team behind it is upfront that it narrows the field rather than closes the case. The researchers didn’t switch a gene on or off to watch the predicted change occur. But in a system with more relationships than anyone could test one at a time, going from everywhere you could possibly look to a handful of places actually worth testing is most of the hard part. What’s left – running the actual experiment to confirm a cause – is the easier problem to have.

The same caution applies well beyond biology. If a policy passes and crime falls the same year, or a company reorganises and productivity rises the same quarter, or a new habit catches on and the people who adopt it suddenly report feeling better, we might assume that the intervention caused the change. But the honest next question to ask is the one CauTrigger’s authors put to their software: what else changed at the same time, what did the explanation have to assume, and what would it take to tell this story apart from other possible accounts?

Pearl supplies the language for asking what would happen if you changed something. Cartwright supplies a reminder: the answer is only as good as the model’s picture of the world it describes. Neither lets the laboratory, or the dinner table, off the hook. A cause isn’t just the explanation that fits the evidence you happen to have. It’s a claim about what would happen under circumstances that haven’t occurred yet. Tools like CauTrigger let us calculate, rank, and test such claims with unprecedented precision. The world still has to answer back. But now we know which questions are worth asking it.

References