Facebook
Twitter
Pocket
Reddit
Email
On stage in New York a couple years ago, noted neuroscientist Christof Koch handed a very nice bottle of Madeira wine to philosopher David Chalmers. Chalmers had won a quarter-century-long bet about consciousness—or at least our understanding of it.
The philosopher had challenged the neuroscientist in 1998—with a crate of fine wine on the line—that in 25 years, science would still not have located the seat of consciousness in the brain. The philosopher was right. But not without an extraordinary—and revealing—effort on the part of consciousness researchers and theorists.
Backing up that concession were the results of a long and thorough “adversarial collaboration” that compared two leading theories about consciousness, testing each with rigorous experimental data. Now we finally learn more about the details of this work in a new paper in the journal Nature.
Nicknamed COGITATE, the collaboration pitted “global neuronal workspace theory” (GNWT)—an idea advocated by cognitive neuroscientist Stanislas Dehaene, which associates consciousness with the broadcast of information throughout large swathes of the brain—against “integrated information theory” (IIT)—the idea from neuroscientist Giulio Tononi, which identifies consciousness with the intrinsic cause-and-effect power of brain networks.
The philosopher was right. But not without extraordinary effort from consciousness researchers.
The adversarial collaboration involved the architects of both theories sitting down together, along with other researchers who would lead and execute the project (hats off to them), to decide on experiments that could potentially distinguish between the theories—ideally supporting one and challenging the other. Deciding on the theory-based predictions, and on experiments good enough to test them, was never going to be easy. In consciousness research, it is especially hard since—as philosopher Tim Bayne and I noted—theories often make different assumptions, and attempt to explain different things even if, on the face of it, they are all theories of “consciousness.”
The predictions settled on by the COGITATE consortium were admirably clear. They had to do with the neural activity that accompanies conscious perception, with a focus on where and when in the brain information about visual experience could be decoded from, using the latest machine-learning methods. These predictions were to be tested in an experiment in which participants simply viewed images of different kinds of objects, for different lengths of time, while their brain activity was recorded.
Oversimplifying horribly, GNWT predicted a major involvement of regions in frontal parts of the brain—the “workspace areas”—with strong brain responses when the image (the stimulus) appeared and disappeared. IIT, on the other hand, predicted sustained activity throughout the duration of each stimulus in a different region—the so-called posterior “hot zone” toward the back of the brain.
Importantly, although these predictions were derived from their respective theories, they do not test the core claims of either theory. Which is a pity, but a necessary cost of making the theories comparable at all, in experiments like these. What the predictions could do, however, was test ideas about the theories’ biological implementations in the brain.
Already, and even before any data was collected, it was clear that neither theory would be roundly refuted, or celebrated as the one true theory. This surely came as no surprise to philosophers of science. The Duhem-Quine thesis (named for Pierre Duhem and William Van Orman Quine) noted decades ago that theories are almost impossible to falsify in single experiments, since one can always find reasons why things didn’t work as expected. The in-vogue philosopher Imre Lakatos had similar ideas, arguing that theories can be judged on their ability to generate testable predictions with explanatory power, even if their core commitments remain difficult to experimentally assess.
COGITATE was a large, complex study, with many findings. To cut a long story brutally short: Neither theory won. And neither theory lost. As the Nature paper puts it, the results “align with some predictions of IIT and GNWT, while substantially challenging key tenets of both theories.”
So what have we learned from the massive effort? Some people might be tempted to dismiss the study as a waste of time (and money), given that both theories remain in play. But this would be a mistake. Here are three take-aways that stood out to me.
First, the most interesting experimental results—to me—were the ones that went against each theory. Contrary to the predictions of IIT, the researchers did not find evidence for sustained neuronal synchronisation (in the so-called gamma range) in the all-important posterior hot zone. And contrary to the predictions of GNWT, there were no sharply defined “offset” responses in workspace areas when conscious perception ceased.
These results put some pressure on theorists to explain what happened. Perhaps the prediction did not follow the core elements of the theory closely enough, perhaps the data weren’t the right data, or perhaps the theory needs revising. Either way, we often learn more when predictions are disconfirmed compared to when they are confirmed.
Second, science done like this moves the needle on science itself. One of the more remarkable aspects of the COGITATE study was its commitment to the principles of “open science.” The experimental predictions were pre-registered—written-up and published in advance—so that they couldn’t be tweaked with the benefit of hindsight. The data collection was done in theory-neutral labs, using multiple brain imaging methods, and with independent replications. This data-set—which is freely available—is therefore unusually robust and reliable, and will no doubt spark many fascinating follow-up analyses. It’s worth emphasising that many of these virtues—in particular pre-registration and transparency—are not limited to these large collaborations. All scientists can get in on this game, and we should.
The most interesting experimental results were the ones that went against each theory.
Third, minds change slowly, and sometimes not at all. A movie adaptation of COGITATE might finish with one protagonist, after some deep soul-searching, conceding defeat and reflecting on how they could have been so wrong. (Not far from how this played out in real life with Koch’s relinquishing of the wine to Chalmers, although with a different kind of reflection!) But—recalling the Duhem-Quine thesis—it’s very unlikely that the main proponents of a given theory will change their minds, whatever the outcome. And perhaps this is not a bad thing. Science thrives on a diversity of ideas and the systematisation of doubt. Daniel Kahneman—the late Nobel Prize-winning economist and advisor to the COGITATE project—once said that adversarial collaborations can be valuable even without the adversaries changing their minds. How? Because the rest of us might change our views, and in this process the community as a whole can move ahead.
There’s much more of value in the COGITATE results, far too much to summarize here. As someone marginally involved in another adversarial collaboration (the INTREPID project, contrasting IIT with two other theories—“active inference” and “neuro-representationalism”—watch out for results soon!), I am grateful to the COGITATE project leaders—neuroscientists Lucia Melloni, Liad Mudrik, and Michael Pitts—for setting a strong example of how to conduct a project of this nature and scale. The value of this example may well be its most enduring collaboration.
But after the initial COGITATE results were announced in 2023 at that Association for the Scientific Study of Consciousness’ meeting in New York, the project became embroiled (at least in public) in a fiery debate about whether IIT counted as a scientific theory at all. As neuroscientist Àlex Gómez-Marín and I wrote recently, we think the charge of pseudoscience against IIT was unwarranted, but we also found fault with the defence offered by IIT. This intellectual conflict, too, illuminates the value of putting these ambitious theories through their experimental paces.
Adversarial collaborations, as lengthy and challenging as they are, offer a different way of doing things. They are adversarial, but they do not have to be combative. They are certainly not the only way science can advance, but they bring many unique benefits. As Kahneman put it, adversarial collaborations are an alternative to “angry science,” and that has to be a good thing. 
Lead image: Աննա Ավագիան / Wikimedia Commons
