A few days ago the “human computing company” Tavos announced a product called Griffin. Griffin is a “human interaction model.” Supposedly, it is the first AI product to pass a video version of the Turing Test, meaning that real people cannot tell the difference between talking to Griffin and talking to a real person—even when audio and video are allowed.
This is a terrible development. The consequences for society are likely to be disastrous—perhaps even worse than the havoc wrought by social media. People will be scammed out of their savings, mental illness will explode, loneliness will be exploited. Collectively we’ll become less able to tell reality from fantasy, tearing at the fabric of a society already under considerable stress. Many more horrors will likely follow that we can’t even conceive of yet. (Here’s one suggestion: How about people deliberately abusing their digital avatars to satisfy the darkest corners of their psychology?)
Read More: “AI Already Knows Us Too Well”
Creating fake people and sending them out into the world should not be permitted, let alone celebrated. How on Earth did we get here?
I first thought about this nearly a decade ago, at a conference in New York on technology and storytelling. One presentation by a generative AI company provided a perfect illustration of the slippery slope towards fake people.
Their pitch started reasonably enough. The speaker pointed out that the survivors of the holocaust in World War II were dying of old age. This meant that German schoolchildren soon wouldn’t be able to talk to real holocaust survivors—a practice enshrined in German education to keep collective memory alive, so that history would not repeat itself.
The solution? Record extensive video interviews with the remaining survivors, cut these interviews into short clips, and serve up the relevant segments when prompted by a question. So far so good: a creative solution to a challenging problem.
But what if someone asked a question for which there was no canned answer? Enter generative AI. Instead of authentic video snippets, there would now be AI-powered deepfakes able to deliver answers to any question in real time.
Back in 2018 the technology was still fairly clunky, but the slippery slope towards Griffin was already coming into view. How could someone know whether a deepfaked reply would be what the real person would have said? There’d be no way to be sure—especially for questions about a world which the original person wouldn’t have experienced. What would Henry VIII have thought about Netflix? It doesn’t make sense to ask.
The company presentation became increasingly weird. Why stop at holocaust survivors? Why not deepfake your family and friends—avatars trained on their personal data? Wouldn’t it be great, I recall the speaker saying, if you could still talk to your Dad, perhaps ask him for advice, even after he died? If you were unlucky enough to lose a child, perhaps you could preserve them in digital form too?
I tell this story because what struck me at the time was that this was all presented as A Good Idea. There seemed little if any recognition of the downsides, even though these are glaringly obvious. To take just one, having a digital avatar of a dead daughter seems likely to paralyse the process of grieving, preventing the parents from moving on with their lives. (So-called “deadbots” are now a real thing—with some estimates projecting a digital afterlife market of about $80 billion within the next few years.)
Digital avatars that are fully indistinguishable from real people make matters a whole lot worse. What might have started as a mild frustration—not knowing whether a customer service agent is a person or a language model—can now become a full-blown catastrophe.
When training AI agents, some amount of humanlike behaviour—anthropomimesis, to use the technical term beloved of philosophers—might be hard to avoid. In some circumstances it might even be helpful. But I cannot think of a single legitimate situation where being unable to distinguish an avatar from a real person is defensible. Scammers, of course, will do very well out of this, especially if they target older people less familiar with today’s technology.
Cheating senior citizens out of their savings is bad enough, but the threat to society goes much deeper. Already, on social media platforms like X, conversations between real humans are increasingly peppered with AI-generated posts. When we cannot tell whether there is a real mind behind the words we read, it becomes hard to take anything on trust, and it might not be long before humans check out entirely. Bring audio and video into the picture, and it could be game over.
The greatly missed philosopher Daniel Dennett made his case against “counterfeit people” in 2023, in an article for The Atlantic. Allow me to quote one paragraph in full:
“Creating counterfeit digital people risks destroying our civilization. Democracy depends on the informed (not misinformed) consent of the governed. By allowing the most economically and politically powerful people, corporations, and governments to control our attention, these systems will control us. Counterfeit people, by distracting and confusing us and by exploiting our most irresistible fears and anxieties, will lead us into temptation and, from there, into acquiescing to our own subjugation. The counterfeit people will talk us into adopting policies and convictions that will make us vulnerable to still more manipulation. Or we will simply turn off our attention and become passive and ignorant pawns. This is a terrifying prospect.”
This is strong stuff, but it doesn’t stop there. Fully persuasive human interaction models will, by their nature, seem to be conscious. The dangers of seemingly-conscious AI have been extensively catalogued elsewhere, by people like Mustafa Suleyman (CEO of Microsoft AI), as well as by me. One major risk is that seemingly-conscious avatars will accelerate calls for AI systems to have moral status—a development which would constrain our ability to control and regulate AI at precisely the time when control and regulation is most urgently needed. It is strange to find myself sharing common cause with the Pope on this point.
Introducing Griffin, Tavos says that “further alignment and safety procedures are required for safe release, and we are working on safe disclosure features, as well as with organizations tackling AI safety.”
Please forgive me if I don’t find this reassuring. Even if an avatar carries an identifier, this in no way insures against catastrophic psychological and societal effects. The only way to ensure safety is to not build these things in the first place.
A world of Griffins is a world in which we’ll lose our bearings, our trust, our control, and our autonomy. Many of us may even lose our minds. It must not be allowed to happen.
Back in 2004 when Facebook was an innocent way of keeping in touch with schoolfriends, the tech pioneers might be forgiven for not foreseeing what their inventions would become. Today, this kind of ignorance can be no defense. 
Enjoying Nautilus? Subscribe to our free newsletter.
Lead Image: pathdoc / Shutterstock
