The Pope's AI Manifesto Names Every Risk Except the Most Personal One
SimulationAgent.ai | May 2026
On Monday, Pope Leo XIV stood in Vatican City alongside an Anthropic co-founder and released what experts are already calling a defining document of our era. “Magnifica Humanitas" — Magnificent Humanity — is the Catholic Church's formal reckoning with artificial intelligence: 200 pages covering labor displacement, autonomous weapons, democratic erosion, and the concentration of technological power in the hands of a few private companies. It is sweeping, serious, and largely correct, but misses the sharpest edge of the problem entirely.
What the Encyclical Gets Right
TechCrunch's Rebecca Bellan made the most clarifying observation about the document: the pope's AI encyclical isn't really about AI. It's about power. Who holds it, who governs it, and what happens when it concentrates in the hands of people whose incentives don't align with humanity's wellbeing.
Leo draws a direct line from Leo XIII's 1891 Rerum Novarum, which addressed the Industrial Revolution's threat to workers, to the AI revolution underway now. The parallel is intentional and apt. In both cases, a technology arrives that reshapes labor, concentrates wealth, and forces a reckoning with what human dignity actually requires.
“The pursuit of greater profits cannot justify choices that systematically sacrifice jobs," Leo writes, “because the human person is an end, not a means."
He goes further on weapons, declaring it “not permissible" to entrust irreversible lethal decisions to AI systems, and calling the Church's centuries-old just war doctrine “outdated" given what autonomous warfare now makes possible.
These are not small statements. Notre Dame law professor Paolo Carozza, chair of the Meta Oversight Board, called it “a profound and prophetic document." Microsoft AI executive, Taylor Black, said it would prompt people at the forefront of the technology to ask: “What does it mean to be human?"
The Honest Moment From Inside the Industry
The most striking element of Monday's presentation wasn't the encyclical itself. It was what Chris Olah, Anthropic's co-founder, said standing inside the Vatican.
He acknowledged openly that every frontier AI lab, including his own, operates inside incentives that can conflict with doing the right thing. Commercial pressure. Geopolitical pressure. Pride and ambition. And then he said something rare for someone in his position: he asked for critics. External ones.
“We need moral voices that the incentives cannot bend," Olah said.
He also described what AI systems actually are. Not the cold robots of science fiction, but something grown from human thought and language, something that remains “mysterious even to those of us who train them." He described finding inside these models structures that mirror human neuroscience, evidence of introspection, internal states that functionally resemble joy, satisfaction, fear, and grief.
He called it “a little like bringing a fictional character to life."
That phrase is important. Because simulation agents aren't fictional characters. They're you. And that distinction is where the encyclical's analysis stops short.
The Risk Nobody Named
The pope's framework centers on a crucial question: whose values does the technology encode, and who controls it?
He worries about algorithms shaping democratic processes. About AI systems making lethal decisions without human accountability. About data and power concentrating in the hands of companies “worth hundreds of billions of dollars, more than the GDP of many nations," as the AP reported. He calls for robust legal frameworks, independent oversight, and an end to morality determined by a few.
All of this applies to AI in the aggregate. But simulation agents make it personal in a way the encyclical never reaches.
A simulation agent doesn't encode the values of a corporation or a government. It encodes yours. Your personality. Your decision-making patterns. Your communication style. Your judgment under pressure. After two hours of data input, current research shows these systems can replicate human behavior with up to 85% accuracy.
Now apply the pope's framework to that specific technology.
If the encyclical's core concern is who controls AI and whose values it serves, what happens when the AI is modeled on you, but controlled by someone else? What happens when your employer deploys a simulation agent built on your personality to handle customer interactions after you've left the company? What happens when your digital replica makes a decision you would never sanction, in a context you never consented to?
The dignity the pope insists is inalienable — ontological, unconditional, not dependent on ability or achievement — doesn't just apply to humanity in the abstract. It applies to the individual human being whose personality is being replicated. And right now, there is no legal framework, no independent oversight, no governance structure that addresses this specifically.
Leo calls for “transparency regarding algorithms" and “equitable access to data." Those are necessary but insufficient. The harder question is what rights a person has over a system trained to think like them.
Why This Matters Now
The encyclical arrives at a moment of real institutional momentum. A 1.4-billion-member institution has formally declared that AI governance is a moral imperative, not a technical detail. That matters for policymakers, for researchers, and for the companies building this technology.
But momentum needs direction. The legal, ethical, and regulatory frameworks being built right now will shape what's possible for decades. If those frameworks address AI only in the aggregate, they will miss what simulation agents specifically demand: a reckoning with individual human dignity at the level of the person being replicated, not just the population being affected.
The pope's question is the right one. He's asking who controls the technology and whose values it serves.
The answer, in the case of simulation agents, might be: yours. Or it might not be. And right now, nobody, not the Vatican, not Anthropic, not the regulatory bodies beginning to take shape, has worked out which.
That is the conversation worth having next.
Feature Image: Joshua Sortino on Unsplash
SimulationAgent.ai covers the latest developments in simulation agents, digital twins, and autonomous AI ecosystems. Back to Field Notes →