AI can trick you, warns book that hid AI’s help writing it
Andrea Colamedici invented a philosopher, presented him as an author and produced a book, secretly generated with the help of artificial intelligence, about manipulating reality in the digital age. People were deceived. Accusations of dishonesty, bad ethics and even illegality flew. But the man behind it, Colamedici, insists it was not a hoax; rather, he described it as a “philosophical experiment,” saying that it helps to show how AI will “slowly but inevitably destroy our capacity to think.”
Colamedici is an Italian publisher who – along with two AI tools – generated “Hypnocracy: Trump, Musk, and the Architecture of Reality,” a buzzy text ostensibly written by Jianwei Xun, the non-existent philosopher. In Dec, Colamedici’s press printed 70 copies of an Italian edition that he supposedly translated. Still, the book quickly gained outsize attention, being covered by media outlets in Germany, Spain, Italy and France, and being cited by tech luminaries.
“Hypnocracy” describes how powerful people use technology to shape perception with “hypnotic narratives,” putting the public in a kind of collective trance that may be exacerbated by relying on AI.
The book, Colamedici said, was meant to show the dangers of “cognitive apathy” that could develop if thinking were delegated to machines and if people don’t cultivate their discernment. “I tried to create a performance, an experience that is not just the book.” Colamedici teaches what he calls “the art of prompting,” or how to ask AI smart questions and give it actionable instructions, at the European Institute of Design in Rome. He tries to teach users how to discern fact from fabrication.
The book is an extension of this effort, he said. The AI tools he used helped him to refine the ideas, while clues (real and invented) about the fake author, intentionally suggested potential problems to prompt readers to ask questions, he said. The first chapter discusses fake authorship, for example, and the book contains obscure references to Italian culture unlikely to come from a young philosopher from Hong Kong, which eventually helped to lead one reviewer to the true author operating as a translator. Sabina Minardi, an editor at Italian outlet L’Espresso, picked up on the clues, exposing Jianwei Xun as a fake in early April. Colamedici then updated the fake author’s bio page, spoke to publications, including some deceived by his work. New editions and excerpts come with postscripts about the truth.
But some who first embraced the book now question whether Colamedici has acted unethically or broken a EU law on AI. French news outlet Le Figaro wrote about “L’affaire Jianwei Xun,” explaining the “problem” with its earlier interview of the Hong Kong philosopher was that “he doesn’t exist.” Spanish newspaper El Pais retracted a report, replacing it with a note that said “the book failed to acknowledge AI’s involvement in the creation of the text, a violation of the new European AI Act.” Colamedici is disappointed some initial champions have decried the experiment. He said he plans to have Jianwei Xun – describing it as a collective of humans and artificial intelligence – teach a course about AI next fall. NYT
