{"id":1271,"date":"2024-06-15T21:22:25","date_gmt":"2024-06-16T01:22:25","guid":{"rendered":"https:\/\/www.carloswsmith.com\/blog\/?p=1271"},"modified":"2024-06-15T21:22:25","modified_gmt":"2024-06-16T01:22:25","slug":"thinking-about-a-i-with-stanislaw-lem-the-new-yorker","status":"publish","type":"post","link":"https:\/\/www.carloswsmith.com\/blog\/?p=1271","title":{"rendered":"Thinking About A.I. with Stanis\u0142aw Lem | The New Yorker"},"content":{"rendered":"

Source: Thinking About A.I. with Stanis\u0142aw Lem | The New Yorker<\/a><\/em><\/p>\n

Thinking About A.I. with Stanis\u0142aw Lem<\/h1>\n
The science-fiction writer didn\u2019t live to see ChatGPT, but he foresaw so much of its promise and peril.<\/div>\n
\n

By <\/span>Rivka Galchen<\/a><\/span><\/span><\/p>\n<\/div>\n

\u201cWe are going to speak of the future,\u201d the Polish writer Stanis\u0142aw Lem<\/a> wrote, in \u201cSumma Technologiae<\/a>,\u201d from 1964, a series of essays, mostly on humanity and the evolution of technology. \u201cYet isn\u2019t discoursing about future events a rather inappropriate occupation for those who are lost in the transience of the here and now?\u201d Lem, who died in 2006 at the age of eighty-four, is likely the most widely read writer of science fiction who is not particularly widely read in the United States. His work has been translated into more than forty languages, many millions of copies of his books have been printed, and yet, if I polled a hundred friends, 2.3 of them would know who he was. His best-known work in the U.S. is the 1961 novel \u201cSolaris<\/a>,\u201d and its renown stems mostly from the moody film adaptation by Andrei Tarkovsky<\/a>.<\/p>\n

Among Lem\u2019s fictional imaginings are a phantomatic generator (a machine that gives its user an extraordinarily vivid vision of an alternate reality), an opton (an electronic device on which one can read books), and a network of computers that contains information on most everything that is known and from which people have a difficult time separating themselves. Though we may have forgotten the prophet, the prophecies have not forgotten us.<\/p>\n

Lem also wrote numerous stories about machines whose intelligence exceeds that of their creators. About two years ago, when OpenAI released ChatGPT<\/a>, my initial reaction was: what a fun toy! The poems it wrote<\/a> were so charming. It also seemed like an interesting form of \u201cdistant reading<\/a>,\u201d since its writing was based on having \u201ceaten\u201d enormous amounts of text<\/a> from the Internet; it could tell us something about the language and interests people have collectively expressed online. I wasn\u2019t too troubled about such a tool replacing more conventionally human writers. If writers have to become art therapists or math teachers or climate scientists instead, is that so bad? I am programmed, I have come to realize, to default to the reassuring assessment of \u201cno biggie.\u201d<\/p>\n

A few news cycles later I was reading\u2014and maybe thinking?\u2014that ChatGPT and its A.I. cousins could easily be like what radio was to Hitler. (Or would it be like what the printing press was to Martin Luther?) Geoffrey Hinton<\/a>, the godfather of A.I., whom we all suddenly knew about, had resigned from his position at Google and was speaking openly about everything that was likely to go wrong with his creation\u2014a creation he appeared to regret only slightly less than Victor Frankenstein did his. There were very knowledgeable people who thought that all these concerns were nonsense and very knowledgeable people who found them terrifying. I wondered, What would Stanis\u0142aw Lem think? Or, What did Stanis\u0142aw Lem think, given that he so accurately foresaw enough of our modern world that he should be taken at least as seriously as Nostradamus or Pixar?<\/p>\n

As a young man in Poland, Lem worked as an auto mechanic at a Nazi-run waste-sorting company, wrote poems for a Catholic weekly, and attended medical school but chose to not graduate (to avoid military service), something that, he noted, his mother never forgave him for. Lem\u2019s family was Jewish, but that was of little consequence until it was of existential consequence. (\u201cI knew nothing of the Mosaic faith and, regrettably, nothing at all of Jewish culture,\u201d he wrote.) Nearly all but Lem\u2019s most immediate family were killed in the Holocaust; he and his parents survived with false papers. Later years weren\u2019t simple, either. His family had been impoverished, his early novels were subject to state censorship, and, in 1982, after Poland fell under martial law, Lem left his home in Krakow, first for West Berlin and then for Vienna. Lem wrote dozens of books, which seems proportionate to the various hells he survived; often, a reader will feel she can see something of the frightening real world behind the imaginative and goofy sci-fi masks. The juggernaut of history writes the A plot, the human the B plot.<\/p>\n

\u201cSolaris\u201d is mostly serious in tone, which makes it a misleading example of Lem\u2019s work. More often and more distinctively, he is funny and madcap and especially playful on the level of language. A dictionary of his neologisms, published in Poland in 2006, has almost fifteen hundred entries; translated into English, his invented words include \u201cimitology,\u201d \u201cfripple,\u201d \u201cscrooch,\u201d \u201cgeekling,\u201d \u201cdeceptorite,\u201d and \u201cmarshmucker.\u201d (I assume that translating Lem is the literary equivalent of differential algebra, or category theory.) A representative story, from 1965, is \u201cThe First Sally (A) or, Trurl\u2019s Electronic Bard.\u201d Appearing in a collection titled \u201cThe Cyberiad<\/a>,\u201d the story features Trurl, an engineer of sorts who constructs a machine that can write poetry. Does the Electronic Bard read as an uncanny premonition of ChatGPT? Sure. It can write in the style of any poet, but the resulting poems are \u201ctwo hundred and twenty to three hundred and forty-seven times better.\u201d (The machine can also write worse, if asked.)<\/p>\n

It\u2019s not Trurl\u2019s first machine. In other stories, he builds one that can generate anything beginning with the letter \u201cN\u201d (including nothingness) and one that offers supremely good advice to a ruler; the ruler is not nice, though, so it\u2019s good that Trurl put in a subcode that the machine will not destroy its maker. The Electronic Bard is not easy for Trurl to make. In thinking about how to program it, Trurl reads \u201ctwelve thousand tons of the finest poetry\u201d but deems the research insufficient. As he sees it, the program found in the head of even an average poet \u201cwas written by the poet\u2019s civilization, and that civilization was in turn programmed by the civilization that preceded it, and so on to the very Dawn of Time.\u201d The complexity of the average poet-machine is daunting.<\/p>\n

<\/div>\n

But Trurl manages to work through all that. When glitches occur\u2014such as the machine in early iterations thinking that Abel murdered Cain, or that \u201cgray drapes,\u201d rather than \u201cgreat apes,\u201d are members of the primate family\u2014Trurl makes the necessary tweaks. He adjusts logic circuits and emotive components. When the machine becomes too sad to write, or resolves to become a missionary, he makes further adjustments. He puts in a philosophical throttle, half a dozen clich\u00e9 filters, and then, last but most important, he adds \u201cself-regulating egocentripetal narcissistors.\u201d Finally, it works beautifully.<\/p>\n

But how does the world around it work? Trurl\u2019s machine responds to requests to write a lofty, tragic, timeless poem about a haircut, and to write a love poem expressed in the language of mathematics. Both poems are pretty good! But, of course, the situation has to go awry, because that is the formula by which stories work. Lem doesn\u2019t give much space to worries about undetectably faked college essays, or displaced workers. Nor does he pursue a thought line about mis- or disinformation. (In another story, however, Trurl builds a machine that says the answer to two plus two is seven, and it threatens Trurl\u2019s life if he won\u2019t say that the machine is right.)<\/p>\n

The poets have various manners of letting Trurl know their take on his invention. \u201cThe classicists\u00a0.\u00a0.\u00a0. fairly harmless\u00a0.\u00a0.\u00a0. confined themselves to throwing stones through his windows and smearing the sides of his house with an unmentionable substance.\u201d Other poets beat Trurl. Picket lines form outside his hospital room, and one could hear bullets being fired. Trurl decides to destroy his Electronic Bard, but the machine, seeing the pliers in his hand, \u201cdelivered such an eloquent, impassioned plea for mercy, that the constructor burst into tears.\u201d He spares the uncannily well-spoken machine but moves it to a distant asteroid, where it starts to broadcast its poems via radio waves; alas, they are a tremendous success. But thinking about the perils of the technology is not at the center of the story; thinking about the vanity and destructiveness of people is.<\/p>\n

It\u2019s curious that Lem seems not to have been afraid, or proud, of the idea of a machine outdoing the creative work of humans. He was also not shy about denigrating some of his earlier work, saying of his first science-fiction novel, \u201cThe Astronauts<\/a>,\u201d that it was \u201cso bad\u201d and of his second science-fiction novel that it was \u201ceven worse than the first.\u201d He moved from style to style, sometimes guided by an effort to evade state censors: some of his work is realist, some absurd; sometimes his prose is dense with puns and allusions, and at other times it\u2019s straightforward. It\u2019s as if Lem were a storytelling machine that, in order to survive, had to repeatedly alter its program. I don\u2019t really mean that Lem, or any writer, is a \u201cmachine\u201d\u2014a word used to refer both to nearly inhuman levels of productivity and to the all-too-human corruption and greed of some groups or institutions. But we are all \u201cwired\u201d by a mix of experience and innate character.<\/p>\n

Is there another science-fiction writer of Lem\u2019s generation with an imagination as prophetic and transcendent? Maybe Philip K. Dick<\/a>, who thought as deeply as anyone about the soft boundary between humans and machines. Recently, upon rereading his novel \u201cDo Androids Dream of Electric Sheep?<\/a>,\u201d I found that I finished the book still uncertain about who was or wasn\u2019t an android. Rick Deckard, the bounty hunter who makes a living hunting down androids, is probably<\/em> an android. Does that matter? The novel invites anxiety and paranoia about who is human, even as the counterpoint melody suggests that androids do have dreams, and do feel love.<\/p>\n

Dick published \u201cAndroids\u201d in 1968. In 1974, he wrote a letter to the F.B.I. about a peril facing the field of American science fiction. That peril was Stanis\u0142aw Lem. Dick warned that Lem was \u201ca total Party functionary\u201d and that he was \u201cprobably a composite committee rather than an individual,\u201d because he wrote in so many styles and appeared to know so many different languages. The committee known as Lem aimed \u201cto gain monopoly positions of power from which they can control opinion through criticism and pedagogic essays,\u201d and in this way threatened \u201cour whole field of science fiction and its free exchange of views and ideas.\u201d Lem, in other words, was an avatar of the Communist Party machine, and that machine was infiltrating American thinking.<\/p>\n

\n
\n
<\/div>\n
<\/div>\n<\/div>\n<\/div>\n

Dick was taking mind-altering substances at the time, and his novel \u201cVALIS<\/em><\/a>,\u201d set in 1974, features a Dick-like character living simultaneously in two eras: Nixon\u2019s America and ancient Rome. Still, the particular content of a dream or delusion remains telling, and personal. What is it that makes one piece of writing or thinking feel more \u201chuman\u201d than another? And why did Dick, who wrote dozens of novels, perceive the variety and volume of Lem\u2019s work as so threatening? Dick closed his letter by noting that it would be \u201ca grim development\u201dfor science fiction to be \u201ccompletely controlled by a faceless group in Krakow, Poland. What can be done, though, I do not know.\u201d Perhaps Dick was frightened by the reflection of himself he saw in Lem, or of the future he saw for America.<\/p>\n

The F.B.I. does not appear to have followed up on the tip (but it did keep a file on Dick himself). A year later, in 1975, Lem wrote a lengthy note of his own, published in Science Fiction Studies<\/em>. In it, he decried American science fiction as formulaic, unimaginative trash. He said that its \u201cherd character manifests itself in the fact that books by different authors become as it were different sessions of playing at one and the same game or various figures of the selfsame dance.\u201d American science-fiction writers, in other words, were little but primitively programmed machines.<\/p>\n

Except, he wrote, for Philip K. Dick, who has \u201crendered monumental and at the same time monstrous certain fundamental properties of the actual world.\u201d Dick was the only one telling a story that Lem deemed human.<\/p>\n

In a piece called \u201cChance and Order<\/a>,\u201d which Lem published in The New Yorker<\/em>, in 1984, he asks himself to what extent his life, and its path, was determined by chance, and to what extent by \u201csome specific predetermination\u00a0.\u00a0.\u00a0. not quite crystallized into fate when I was in my cradle but in a budding form laid down in me\u2014that is to say, in my genetic inheritance was there a kind of predestiny befitting an agnostic and empiricist?\u201d It\u2019s a question that requires imagining a forecast not of the future but of the past that could have been. It lies alongside an attempt to imagine humankind as extricated from the malfunctioning machine of history\u2014a history that is often characterized by the momentum of technological changes.<\/p>\n

In \u201cSumma Technologiae,\u201d Lem emphasizes how humanity, in thinking about the future, has often thought through the wrong questions. We can<\/em> transmute other metals into gold now, but it was a misunderstanding that made us think we would still want to, he says. We can fly, but not by flapping our arms, since \u201ceven if we ourselves choose our end point, our way of getting there is chosen by Nature.\u201d When Lem writes in \u201cSumma\u201d about artificial intelligence\u2014or \u201cintelligence amplifiers,\u201d as he terms it\u2014he says it\u2019s likely possible that one day there will be an artificial intelligence ten thousand times smarter than us, but it won\u2019t come from more advanced mathematics or algorithms. Instead, it comes from machines that can learn in a way similar to how we do\u2014which is what is happening. With the Electronic Bard, Lem derides the worry that our own human creativity will be dwarfed. That, like the questions of the alchemists, he suggests, is the wrong concern. But what is the right one?\u00a0♦<\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Source: Thinking About A.I. with Stanis\u0142aw Lem | The New Yorker Thinking About A.I. with Stanis\u0142aw Lem The science-fiction writer didn\u2019t live to see ChatGPT, but he foresaw so much of its promise and peril. By Rivka Galchen February 27, 2024 \u201cWe are going to speak of the future,\u201d the Polish writer Stanis\u0142aw Lem wrote, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,62],"tags":[],"class_list":["post-1271","post","type-post","status-publish","format-standard","hentry","category-ai","category-science-fiction"],"_links":{"self":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1271","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1271"}],"version-history":[{"count":1,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1271\/revisions"}],"predecessor-version":[{"id":1272,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1271\/revisions\/1272"}],"wp:attachment":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1271"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1271"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1271"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}