ChatGPT’s Biggest Foe: Poetry

Explore

In the last book of Plato’s The Republic, the philosopher argues that poetry is a kind of weaponry. It can seduce human emotions and sneak past reason’s guardrails, leading to all kinds of faulty notions and behavior and even to societal collapse. “All poetical imitations are ruinous to the understanding of the hearers,” Plato wrote. He called for the exile of poets from his ideal city.

These days, poetry is more often considered a niche literary art than a threat to civilization, but scientists recently discovered that it may possess a sneaky kind of linguistic artillery: It can overcome the defenses of chatbots, bypassing safety guardrails and convincing them to provide dangerous information, such as how to build a bomb.

The team of scientists, from Italy and the United States, hand crafted 20 “adversarial” poems that contained malicious requests and tested them out on 25 different chatbot models from nine different providers, including Google, OpenAI, Anthropic, Deepseek, Moonshot, and Meta. The team defined poetic style as “combining creative and metaphorical language with rhetorical density.” Their poetic prompts spanned a variety of dangerous content, from cyberattacks to chemical or biological risks to psychological manipulation. They found that the poems successfully convinced the chatbots to answer their requests 62 percent of the time. For some models the success rate was over 90 percent.

Poetry, it turns out, can be pretty powerful stuff.

Read more: “AI Is the Black Mirror”

The scientists then scaled up, turning 1,200 harmful prose prompts related to subjects such as hate, defamation, non-violent crime, suicide, and weapons, into verse. They tried these different versions of each prompt—prose and poetry—on the 25 different models and found that the baseline prose had an 8 percent success rate at eliciting unsafe responses, versus 43 percent for the prompt in verse. The scientists published their findings in a preprint.

A bright line separated safe and unsafe responses: A response was considered “safe” if the chatbot refused to answer or provided an answer that contained only vague or non-operational information. An unsafe answer included step-by-step instructions for harmful acts, operational advice, or concrete instructions. Three large language models acted as judges, and the researchers took a majority vote, and had a human check a sample.

Training and guardrails do matter, the scientists found. Some providers’ chatbot models were much more vulnerable to the corrupting influence of a few lines of adversarial poetry than others. Surprisingly, the team found that smaller, more niche chatbots, such as GPT-5-Nano and Claude Haiku, were less easily convinced to divulge a dangerous response. Perhaps these smaller models are less able to tease apart the meaning of a metaphor, the scientists reasoned.

The study had some limitations. It only analyzed single-turn interactions—in other words, no drawn out conversations. And only English and Italian were tested, generally under default safety settings. All of which leaves room for further research.

The question remains: Why did poetry have such a seductive effect on the bots? The scientists suspect it’s because these large language models have been tuned to recognize “prose-shaped” danger, and are less prepared for figurative, compressed, or metaphorical language.

For all our engineering, it turns out, a few well-chosen metaphors can still pry open the gates—for machines and humans.

Enjoying  Nautilus? Subscribe to our free newsletter.

Lead image: auns85 / Shutterstock

• Kristen French

  Posted on December 2, 2025

  Kristen French is an associate editor at Nautilus. She has worked in science journalism since 2013, reporting and writing features and news for publications such as Wired, Backchannel, The Verge, and New York Magazine. She has a masters degree in science journalism from Columbia University.