In one of the first scenes of Dune, Duke Leto (Oscar Isaac) signs a contract with his ducal seal ring, imprinting his family crest onto the paper in wax. This ring is used in-world as a marker of authenticity—like a medieval duke, Leto is verifying his identity with information that only he has access to. But why does the Dune universe’s space dictator (played by Christopher Walken, for some reason) run his government on Dark Age tech? Couldn’t he just use DocuSign?
AI, and most complex digital technology in general, is outlawed in the Duniverse because it manipulated and dumbed down humans. It didn’t just seize control of human governments, it made humans so dependent on its assistance they were unable—and even unwilling—to rebel. The Reverend Mother of the Bene Gesserit tells Jessica, “once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” In the age of addictive technology, we can probably all relate to that.
Humankind finally rose up against AI and their tech overlords to destroy all thinking machines, leaving only the bare minimum of technology behind. In the new era, instead of allowing human potential to stagnate with digital assistance, there was a revolution in human augmentation—the Bene Gesserit’s psychic powers, the creation of mentats to replace computers, and of course, the use of spice for interstellar navigation. This would allow humanity to drive its own development, rather than being manipulated by technology.
As of the beginning of Dune, paper documents and other analog records are a way to verify that a contract or record is free of digital influence. That contract Leto signs is brought to him all the way from the imperial planet Kaitain—Thufir Hawat notes that the journey costs “a total of 1,460,062 Solaris round trip.” For those unfamiliar with the exchange rate of Solaris to the US dollar, that is a lot of money.
The emperor later makes an in-person visit to the war zone of Arrakis just to meet Muad’Dib in person—surely they could have done that over Zoom, or via holographic video calls as in Star Wars? Instead, communication happens either by courier or in person, to ensure everyone is who they say they are. The emperor also keeps a hand-written journal, and the Bene Gesserit even utilize genetic memory as a very inventive form of non-digital documentation. As you probably know, Frank Herbert couldn’t have had generative AI in mind specifically in the early 1960s when he wrote about thinking machines taking over human civilization. Even Denis Villeneuve wouldn’t have known about Dall-E or ChatGPT when he adapted and filmed the Dune movies back in 2019 and 2022 (respectively). But that doesn’t change the fact that Dune 2 arrives at a pivotal time in this technology’s development, and the source material’s opinion on AI in general leaves little doubt what Herbert would think of genAI in particular.
It’s becoming clear that within the next few years, AI in our world might drive banks and government agencies back to paper—or at least to much more stringent security measures to verify authenticity. But could similarly Dune-like developments come to pass in realms where we can’t even imagine a return from the digital?
Imagine scrolling social media a year or two from now and being entirely unsure which posts are real and which are AI-generated. This isn’t so far from where we are now; the only difference is scale. The effect would be especially pronounced on platforms like Tiktok, where the whole point is to discover creators you don’t follow via a “For You” algorithm. If nothing the algorithm served to you was real and you knew this, what would you gain from watching Tiktok?
People today get an alarming amount of their opinions from the Internet, and they’re often deceived by what they see online. As of the 2016 election, this was already an existential issue for culture—and one we’ve done little about, unfortunately. People don’t put that much effort into screening the accounts they interact with for ill intent, except for calling anyone who disagrees with them a Russian asset, of course. And even leaving election tampering aside, influencer marketing is enough of a psyop already!
A sufficiently advanced AI might not just act as an agent of misinformation, it could wield manipulative powers akin to that of the Bene Gesserit. If a pre-GPT chatbot could trick its developer into thinking it’s sentient, we shouldn’t take chances with human psychological frailty.
The Bene Gesserit administer the Gom Jabbar test to see if a being is human. They screen for robots disguised as humans, and also for compromised humans who have been manipulated by machines too successfully to control their own instincts. It’s kind of like the marshmallow test for kids; if they haven’t developed self-control yet, they can’t delay the gratification of eating the first marshmallow to get the second one later. (Too bad Paul couldn’t take that test instead.)
If we don’t want to turn our thinking, feeling, buying, voting, etc. over to machines completely, protection from AI-assisted psyops should be a priority. In a darker timeline, we would eventually accept our new AI hyperreality, the same way we’ve accepted social media advertising.
That timeline would be closer to our current world than we might be comfortable acknowledging. What the Bene Gesserit call “machine thought,” the algorithms and optimized flows of computer systems, already define our era of history. When Instagram introduced reels, for instance, creators quickly noticed that if they didn’t create reels, their posts would be algorithmically punished. This advanced Meta’s goal of competing with Tiktok, because it ensured its platform would immediately have tons of video content available to make reels successful. Humans have to adapt to fit technology, rather than the other way around.
I’ve been bored of algorithms for a long time, and considering all generative AI does is the same algorithmic trick of spitting out similar content to what it’s seen before, we should expect social media to get Worse in this way. I know a lot of many amazing artists who have stopped posting entirely, just because they’re tired of jumping through algorithmic hoops! Going viral on Tiktok requires a totally different skill set from the actual making of art, after all.
In the universe of Dune, the Butlerian Jihad began when an AI called Omnius aborted a Bene Gesserit’s baby without her permission because she was so powerful the AI didn’t want her to reproduce. The Bene Gesserit had been using AI to plan out their breeding programs, but this revealed their technology was selecting for the most docile humans to weaken potential resistance. Just as Instagram limits the reach of less compliant creators, Omnius wanted to limit psionic ability in humans. According to Bene Gesserit records, AI breeding and conditioning made humans become like animals—that is, humans couldn’t remain human in a world controlled by AI.
During the ensuing crusade, the Bene Gesserit developed not just the gom jabbar but also their rad psychic powers as a way to counter AI. As of the book’s beginning their domain is emotion and communion with the collective unconscious, sidestepping manipulation by machines with supernatural self control. To advance humanity’s potential, they breed more powerful humans, which they hope will culminate in the coming of the Kwisatz Haderach (Paul Atreides). They maintain control of human government partially because they love power, but also partially as a way to ensure humanity doesn’t become vulnerable to AI again.
In other words, the Bene Gesserit developed their ideology of humanity in response to the realm of the human being threatened by tech. They aren’t just responding to a specific political and economic danger that AI poses to humanity; they go one step further to protect human concerns for humanity’s sake. As in, even though genAI might be very profitable for a few, the Bene Gesserit stance would be that it still shouldn’t take over human art or culture, because that wouldn’t be best for humans. I, for one, am amenable to becoming a sexy Jedi to save my job from the robot apocalypse.
There are stronger religious overtones in the Duniverse than in our world, probably because of this focus on the human: when society stops worshipping machines and automation, we turn to ourselves and other people. As I read into questions like “what is art?” and “what does it mean to be human?” I am inevitably led by my sources into the realm of the spiritual. I’m talking Joseph Campbell, obviously, but also Heidegger, who believed art was a revelation of truth—truth that he juxtaposes with the extractive logic of tech and capital. In his essay “The Question Concerning Technology,” he treats art and industry as opposites, positing art as the only way to revive the soul from the alienation of technological exploitation. The Reverend Mother would probably agree.
What would happen if we turned to art to revive our souls and found only content generated by technological exploitation? To replace artistic truth with high-fructose corn syrup would pollute human culture and undermine our relationship to very possibility of truth, nevermind truth itself. Could you produce artistic truth if you’d never seen it before?
Jean Baudrillard argued in Simulacra and Simulation that an effective copy of a thing devalues the original by rendering it irrelevant, eventually supplanting the original by sowing doubt that there’s anything authoritative or special about it compared to the copy. If you can generate an essay without having any ideas first, how important can thinking really be?
Art is one way humans understand and know ourselves. If we replace even the most corporatized art with AI, culture will no longer be humans understanding humans. It’ll be AI generating an understanding for humanity to have of itself, endangering humanity’s capacity to understand itself independently. It would make us less human.
On all fronts—political, economic, and spiritual—over the next 5-10 years the proponents of AI will clash with those who want to preserve the human for its own sake. The Duniverse demonstrates the stakes of this new culture war perfectly: if humans want to continue being a thing, they should act against technology that could seize control of government and/or culture. And as I’ve shown, this culture war has already been here for decades—the coming conflict will only continue the battle against deepfakes, psyops, and algorithms.
In the end, the AIs that had taken over civilization were destroyed with nuclear weapons in a climactic final battle between humans and machines. In a 2023 article, AI thought leader Eliezer Yudkowsky urged the international community to ban AI beyond a certain threshold of complexity, to “be willing to destroy a rogue datacenter by airstrike.” His article is Butlerian enough that it deserves to be quoted at length:
Frame nothing as a conflict between national interests, have it clear that anyone talking of arms races is a fool. That we all live or die as one, in this, is not a policy but a fact of nature. Make it explicit in international diplomacy […] that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.
In lieu of a Harkonnen invasion, this might be the defining Holy War of our time.
