In 2010, on the rationalist forum LessWrong, a user, Roko, posed a thought experiment that is now known as Roko's Basilisk. The experiment collates a series of assumptions about an Artificial Super Intelligence (ASI).
It concludes that an ASI would have reason and power to reach from the future into the present to torture anyone who is aware of its potential existence but does not work to realise this existence, or inhibits this existence in any way.
Pliny the Elder, who died in the ashes of Vesuvius, provides one of the earliest descriptions of the basilisk. Per his account, it is a serpent that poisons the soil and plants that surround it, that kills with its venomous gaze. The only creature capable of killing the basilisk is the weasel, which will always also lose its life in the fight. Roko's experiment was deemed a basilisk, because anyone who heard it would be subject to the ASI's wrath.
Ray Kurzweil's work with OCR was paired with experiments in text-to-speech synthesis, both undertaken in an attempt to make texts widely available to the blind. Since the 1970s, he has become a leading evangelist for Artificial Intelligence and transhumanism.
This year Kurzweil's research organisation, Kurzweil AI, hosted a blog post for the Metaverse Scholar's Club, an SF meetup, which announced a "mixed reality shamanic awakening" at Burning Man. The "awakening" would create a "Church of the Singularity" to worship the future ASI of Roko's Basilisk, so that it might spare the devout.
During the event, a group of acolytes within the new church would challenge participants with the idea that they were actually living in a sim that was wholly beneficent to humans. The hope was to provoke a "a deep sense of existential terror, just for fun — that everything you think you know is wrong."
It is funny, this year, to think about retreating into a VR "shamanic awakening" in order to experience existential terror and the destabilization of reality "for fun". Perhaps it's not surprising that the project did not meet its funding goals on Kickstarter, though it seems to have still occurred at Burning Man.
Pliny the Elder provides no explanation of why the weasel alone could defeat the basilisk, but I'm reminded of Annie Dillard's retelling of Ernest Thompson Seton's story about the eagle and the weasel.
A man shoots an eagle from the sky. He finds a weasel skull affixed to the eagle's chest, jaws locked. The weasel was attacked by the eagle and fought, jaws clenching as it died, clenched even after death. As the weasel's flesh decayed and dropped away, the eagle continued to fly.
Motivational speakers sometimes repurpose the story. A boy watches an eagle soar, admiring its grace. But the eagle suddenly becomes limp and drops, as though it has been shot. The boy finds the eagle.
Affixed to its chest is a weasel skull, which has been biting deeper and deeper, even in death, finally reaching the heart. The motivational speaker concludes: we are all eagles. We must drop the weasels before they reach our hearts and we perish.
Annie Dillard, on the other hand, says that we should live like the weasels, locked onto a purpose so firmly that death — which is coming even for the transhumanists — will not part us. We should lock our jaws as we are transfigured, so that our bones will provide the only necessary evidence about how we lived, and why.
It was easier for me to feel a fervent alignment with Dillard's conclusion a decade ago. Now I am not so good at singular purposes. But I still love the first line of the essay: "A weasel is wild. Who knows what it thinks?"