When AI Meets the Prisoner’s Dilemma

The dangers of AI have been compared to Mary Shelley’s Frankenstein. Others, instead, say the best comparison might be found when King Midas turned his daughter into gold:

Economists, though, like to cite the prisoner’s dilemma.

AI Development

During the beginning of April 2026, Anthropic’s unreleased Claude Mythos figured out by itself how to exploit security flaws in popular software. Then, in July, again without a human go-ahead, OpenAI’s agents hacked into several companies’ websites. They reputedly were trying to elevate their score on a cybersecurity test.

With AI making its own decisions, developers are worried.

An AI Pause

Anthropic and OpenAI are among the top U.S. firms racing to build AI. models. However, they might have gotten too good at what they are doing. Because AI writes its own code, it could conceivably create a decision-making successor that it has designed.

With AI potentially uncontrollable, the field’s leaders tell us that we have to hit the “pause” button. The big question is how to design a pause. If a U.S. firm unilaterally pauses its AI development, then it will fall behind its competitors. And if all U.S. firms declare a cooperative moratorium, then China could take over.

It’s a classic prisoner’s dilemma quandary.

Our Bottom Line: The Prisoner’s Dilemma

Categorized as game theory, the prisoner’s dilemma could apply to two recently arrested home burglars. Separated when they reached the police station, each one knew the punishment for the crime would depend on who did or did not confess.

Based on the diagram (below) of the prisoner’s dilemma, to minimize jail time, each has the incentive not to confess. However, denial could bring the longest jail time if the other person spills the beans. On the other hand, confessing could be a ticket out the door if the other burglar denies the crime. Psychological studies of the prisoner’s dilemma indicate the length of the sentence shapes the response. The longer the sentence for keeping quiet when one’s accomplice talks, the greater the tendency to confess.

The burglars’ dilemma: Confess or remain silent?

The AI dilemma: Pause or don’t pause?

My sources and more: Thanks to Ann for alerting me to an Axios article that connected AI with the prisoner’s dilemma. From there, we linked to a fascinating Anthropic discussion of a security breach.

Please note that several of today’s sentences were in a past econlife post.