This past February I stood in front of my collaborators to walk them through a major update to VIDE, the void-finding algorithm I first built more than a decade ago. Voids are the vast empty regions between galaxies, and digging them out of a survey is fussy, unglamorous work. The new version was 10 times faster than the old one. It could handle surveys a hundred times larger. Best of all, it had a far more sophisticated way of dealing with the ugly realities of an actual data set, the gaps and masks and ragged boundaries that never show up in the textbook version of the problem.
I had spent weeks on it, and I was proud of it. I had also spent those weeks with an AI at my elbow, and it had been indispensable. Coding agents rewrote the core of the algorithm, faster than I could have and without the typos. When it came time to write it up, they generated the plots and drafted the commentary that went with them. Every time I checked their reasoning, they never once hesitated.
Ten minutes into my presentation, a collaborator raised their hand. They said that something seemed off to them and asked about the new scheme for handling the edges of a survey. I answered easily because I knew this material. They asked again, slower, and pointed. I looked where they were pointing, took a breath, and the floor went out from under me.
Read more: “How the Occult Gave Birth to Science”
The edge handling was wrong. It wasn’t a typo, and it wasn’t a missing citation or a factor of two. It was subtle, but it was very wrong, and everything downstream of it was also wrong, and I had shared the whole thing in a room full of people who trusted me. The code was fast. The code was beautiful. The code was incorrect. After 20 years of training, I had presented nonsense with total confidence.
Yes, I still cringe when I think about that moment. And here’s the worst part: In all those weeks, I never felt a flicker of doubt. Nothing ever seemed iffy, or even worth a second look. It felt like mastery of a new technique in less than a tenth the time it would have normally taken. The fashionable worry is that AI is making us stupid, that every delegation is a small amputation. I don’t believe it. Nothing was taken from me in February.
But I will admit this: I was seduced. I felt AI was a modern-day philosopher’s stone, imbued with the power to transmute machine into mind. The ancient alchemists spent centuries chasing a legendary substance that would turn lead into gold and make its owner rich beyond counting. Today we pour in text and electricity and wait for thinking to come out the other end. That’s what makes these tools feel magical instead of merely handy. Not that they’re fast. It’s that they seem to have crossed a line that nothing made of arithmetic should be able to cross.
So, there I was in February, showing off the rotten fruits of my transmutation and the fool’s gold I now held in my hands.
But the story doesn’t end with my humiliation. Alchemy was already on my mind, and the more I read about it, the more I realized the alchemists’ work wasn’t an embarrassment to science, it was the road that led to it. They never found the philosopher’s stone. But they found something else, something we need really badly right now.
I felt like I understood what I was doing because the AI system I was using—a large language model, or LLM—sounded like it understood. An LLM is a little more than a sophisticated next-word predictor. The reason these systems are right about most things turns out to be a boring one. True statements are simply more common in the human-written text LLMs are trained from, so a true statement is usually the better prediction. Correctness is a side effect, and it falls out of the arithmetic the way heat falls out of an engine.
And then we made it worse. A raw predictor is unpleasant to talk to, so we polished it, showing people different forms of answers and tuning the model toward whichever they liked better. What we liked, it turned out, were answers that were clear, organized, and direct.
An LLM’s fluency is not an accident, and it is not an emergent mystery. It’s a trait we bred, the way we bred wolves into dogs that watch our faces when we open the treat bag. Being confident but wrong can cost a human their job, their career, or just their basic self-respect.
Machines don’t have jobs, or careers, or respect.
But dang it, LLMs are so undeniably useful. I have been writing code since I was 5 years old. I haven’t written a line of it since January. And for 20 years I did my own research, which is to say I read the papers, chased the citations down their rabbit holes, and built every argument myself. AI agents do all that now.
Read more: “Has Science Realized This 350-Year-Old Alchemist Wish List?”
These were not lazy trades. They were correct trades, and I’d make them again tomorrow. Somewhere in the past year an LLM became my thinking partner, and I say that without embarrassment. I’m not the only one. Hundreds of millions of people reach for these tools every day, and they are not idiots.
Users of LLMs get acceleration that is useful and correct a large fraction of the time, and total junk sometimes. And LLMs are so good at what they do, at finding the word that is both useful and pleasing, that I lowered a guard I have kept up my entire career.
The superpowers are real, but they’re volatile, and they’re tuned to please us. So how do we deploy a tool that is sometimes wrong but always pleasing? How do we trust AI? The simple answer is: don’t.
We’ve been here before. Not with machines, but with any powerful process (whose insides we cannot see) that returns something valuable. When I felt like an alchemist with new powers at hand, I was only half right.
Today we think of alchemy as a pseudoscience, a quasi-magical system of attempting the impossible. But the alchemists weren’t false scientists, they were pre-scientists, and they discovered in the laboratories many things about the world. Working with volatile substances they did not understand (they did not know about atoms, molecules, quantum mechanics, or all the rest), in Europe and the Middle East for more than a thousand years, they managed to discover phosphorus, recreated porcelain, isolated alcohol, and built laboratory apparatus still in use today. Their words are scattered through our language: crucible, alcohol, gas.
They managed this because they had a method. Ignorance of their materials was met with an almost fanatical commitment to the operational process, to rigorous documentation and strict control, to keep the work from evaporating into illusion.
We are now in the pre-chemistry era of AI. We know how these models work, because we crafted their code ourselves, and yet in most cases we cannot say why: why this word and not another, why this particular path through the machine. The crucible is closed, and like the alchemists we are not going to stop using it. The alchemists were the ones in history who had to pull real value from a process they could not see into. Now it’s our turn.
The architecture of alchemy had three parts that can help us navigate the seduction of AI.
The first principle is grounding. The alchemists called it fixing the volatile. A substance that fled the vessel on the way to becoming something was a failure of the work, and much of the art consisted of rendering matter stable enough to stay put and be weighed.
To us, any output that cannot touch the earth is dross. Today’s safeguards treat truth as an optional garnish: a timid warning at the bottom of the chat window, a filter scrubbing for toxicity. We exert instead a grounding pressure of verification, granting the machine no authority until every claim it makes has been checked against the world.
The second is provenance. Nothing in the alchemist’s vessel was a private event. They recorded everything—the color of the flame, the source of the materials, the phase of moon and positions of the planets—because they believed in the universal connectedness of nature. The most famous alchemical text, the Emerald Tablet, said it thusly: superiora de inferioribus. The higher from the lower. That chain holds even when the middle is mysterious.
For us, it means auditing and chain-of-argument. Can we trace every result back through the conversations? Can we point to every output and map it to an input? Can we own not just the result, but the chain that led to the result?
Grounding catches the fabricated citation, the number from nowhere, the substance that evaporated. Provenance catches something else, the flawless output that traces cleanly back to an idea that was already wrong. It helps us find the impurities, and where the LLM went off the rails.
With LLMs grounding and provenance will carry you a long way, and then they stop. A visible thread tells you where a claim came from. It says nothing about whether the source was any good. Somebody has to take hold of the far end of the thread and judge what is tied to it, and that somebody cannot be the machine, because the machine is where the thread began.
That is the third principle, and the one the AI industry is spending a decade trying to engineer away. We have been led to believe that the goal of AI is autonomous intelligence, but this is the ultimate alchemical error. Labor can be delegated; authority cannot. While the industry chases after gold, they ignore that the machine is ontologically incomplete; it possesses the logic of computation but lacks the spark of life.
Read more: “The Vulgar Mechanic and His Magical Oven”
Ars totum requirit hominem: The art requires the whole person. The alchemists held that the practitioner was no mere observer but the active catalyst, binding the volatile elements with their own hands. They believed that only good people were capable of truly great works. Judgment is not a technical operation. To decide that something is true is to decide what you will put your name on, and no machine has a name of its own. It is us that transforms a formless oracle into a useful tool. If the machine does 90 percent of the work, the remaining 10 percent (our own moral weight and intuition) becomes the entirety of the value.
A couple hours with my new method would have caught what happened to me in February, had I maintained the discipline of my training instead of just receiving whatever output was in front of me.
So, here’s what alchemical discipline means. I work differently now. Whatever comes out of the machine, I distrust, and not just when something smells wrong. I test it, I judge it, record what I did, and only then do I put my name on it. This is what an alchemist did every morning, standing in front of a vessel that would never explain itself: a fanatical devotion to process in the face of ignorance.
The alchemists never found the philosopher’s stone, not after 1,400 years of trying. But something even more powerful came out of the alchemical tradition. Robert Boyle, the father of modern chemistry, was an alchemist. So was Tycho Brahe, whose observational astronomy unlocked the laws of planetary motion. So was Isaac Newton, who wrote more on transmutation than he ever wrote on gravity. What they found instead of the stone, while failing to find it, was a way of working. A method. In their rigorous attention to detail, in their devotion to process, they created the seeds of the scientific method.
The philosopher’s stone of AI is never going to be found inside the machine. Only in the discipline of the hand that holds it. 
Lead image: Tasnuva Elahi; with images by Andrey Kiselev and Who is Danny / Adobe Stock
