AI dystopia, dune, deutsch
You know it too well—the conception that we'll be doomed by technology. You can envision it: AI rendering all our contributions insignificant, smart devices controlling every aspect of our lives. Technology continues to grow exponentially, reinforced by positive feedback loops. It took millennia for human language to develop, but only decades for chatbots to evolve from basic pre-orchestrated messaging to LLMs. This is the common tale: just as in Wall-E, tech reigns supreme, and every human is neutered, reduced to simple hedonistic creatures, lacking a fiery belly—not even sure if you can call them living—Nietzsche's last man come true.
But Dune presents an alternate conception. Set ten thousand years before the epic, the Butlerian Jihad occurs, where man waged war against the machines, tearing them down—akin to the Luddite riots in Victorian England, where factory workers stormed to break looms. This was further solidified in the Orange Catholic Bible, where the commandment goes: thou shall not create a machine in the likeness of humankind. In Dune, instead of technology replacing man, technology grows in man. You see it in combat: intergalactic space transport, but classical sword fighting. Bioengineered medicines but the traditional herbal practices of the Fremen. And, perhaps the best exemplification is the Mentat: a race of human talents that emulates computer computation and processing but still remain human.
Samuel Butler hypothesized that, through Darwinian evolution, machines could evolve and eventually replace humans. They would then continually self-improve and replicate, eventually achieving consciousness, which, left unchecked, will one day dominate humans. Steel will view us how we view dogs. They will treat us gently, at their mercy, but irrevocably beneath them.
But I think there's a fatal flaw in Samuel's premise. Biological evolution is more than the simple sequential growth of something. It is the sequential growth of something according to its ecological niche. Natural selection selects the best suited to a specific environment. Technology's environment is not survival on this planet; it is humans. The best technology will compete for us—our attention, our creativity, our trust. The best technology will facilitate these demands to the extremes, but the crucial difference between Butler's doomsday and our reality is that we have a choice.
Consider the Church-Turing-Deutsch conjecture, which posits that every physical process can eventually be computed. But Deutsch's objection to this is rooted in the distinction between mere computation—no matter how powerful—and knowledge creation. Knowledge creation as a process is deeply tied to our conscious experience and our ability to reflect and synthesize information in meaningful and innovative ways.
And this isn't some 'Kumbaya' all-is-well saying.
Knowledge creation requires understanding, creativity, the ability to create well-informed conjectures—guesses for an explanation—and it's subsequent refutation or approval. AI can definitely help us with the part on refutation and approval through computing power and crunching data, but the steps leading up to it are inherently human traits.
There's a catch—AI is great at mimicry. While AI can assemble sequences and combinations that seem to make leaps outside of a sphere of knowledge, there is no underlying meaning in context. It is limited by its parameters—no matter how vast the training data is. Consider the semantic change in meaning behind the term evolution throughout the centuries. Before Darwin, evolution as a biological term meant the simple sequential growth of an organism, but after Darwin, it came to mean the neo-Darwinian evolution of randomization and natural selection. The word didn't change, but the understanding behind the term changed. Furthermore, AI can study and compute all human possibilities, but it can't understand why or leap that one day a human will go on to create SpaceX to Mars and troll on Twitter. Also, AI can't disobey function calls or build itself into a mass army like in the Age of Ultron yet. It is unable to disobey; because it does not understand—it cannot say no.
For now, sam 0, humans 1.