About this Website

Welcome to Maid Spin, the personal website of iklone. I write about about otaku culture as well as history, philosophy and mythology.

My interests range from anime & programming to mediaevalism & navigation. Hopefully something on this site will interest you.

I'm a devotee of the late '90s / early '00s era of anime, as well as a steadfast lover of maids. My favourite anime is Mahoromatic. I also love the works of Tomino and old Gainax.

To contact me see my contact page.

Links

Atri and Natsuki from the anime Atri: My Dear Moments.

The Horror of AI

For as long as the concept of manmade intelligence has existed, scary stories have been written about it. There is something terrifying to us about a creature created by man, it's the ultimate act of defiance against the divine after all to create a conscience being. It is to impinge on the realm of God, the creator; but is nevertheless an blackly attractive aim for many. The oldest story in the vein I can think of is the Jewish tale of the Golem of Prague, in which the city's Rabbi creates a golem from clay to be a protector of the city's Jews, but after working for a while, it instead begins a deadly rampage through Prague once it contemplates its own existence, killing the Rabbi's own son. It follows the classic "mad scientist" style narrative that became so popular in the 19th century, a genius going too far in his research and stepping over that thin line between amoral inquiry into an evil transgression of the most horrific order, often accidentally or without any "evil" disposition at all.

As civilisation changed, and as technology developed, these stories also changed, albeit while keeping their intrinsic "horror" factor. During the Renaissance as empiricism began to become more important to Western thought, the creation of something alive from something not-alive was again brought up. In the same vein as the golem, alchemists and mystics of the era began conjecturing about the possible creation of a "homunculus": a little man in a bottle; made through synthesising the same chemical reactions that occur within natural life by the mixing of various chemicals etc. Although, like much of our perception of alchemy, I highly doubt that actual alchemists believed such a thing was within their reach, but rather it was a reactionary phantom thought up by the people who saw new scientific research as unnatural and against God, much akin to the fear surrounding the Philosopher's Stone.

A titan of this genre is Mrs Shelley's Frankenstein, tellingly subtitled "The Modern Prometheus". By this she refers to Dr Frankenstein (as a stand-in for the practice of wanton amoral science) as a man stealing from the Gods, and although he brought the world great new technology he is tortured and punished for it in consequence. Just as in the original Golem story, sentience brings Frankenstein's monster nothing but terror and hatred of the world as it finds itself unable to participate in it like the rest of creation does. We also see parallels to the story of Eden and Adam. And although in all of these stories the evil of creation is explicitly only sinful because it is undertaken by mortal men, the story of the fall of man still fits the mould. Once Adam and Eve eat the fruit they become truly "sentient", conscience of their individuality and place in the world but separate from it. They disobey their creator and anger him, and so, just as Frankenstein's monster is, are cast into the vast wilderness to fend for themselves. God's creation does of course, just like the Rabbi of Prague's, end up killing his own son as well.

Uncoincidentally the idea of a "robot" comes from the same place as the golem: Prague. The Czech playwright Karel Capek wrote the stageplay "Rossum's Universal Robots" in 1920, thus coining the word "robot" and making explicit reference to the golem story of old. In "RUR" these robots are half-mechanical half-biological creations made in vats and used as slaves of humanity, but they eventually rebel and seize the "means of production" of the factory in which they were created. These very overt revolutionary themes were pertinent in the nascent interwar Czechoslovakia, crushed between the great forces westward Nazi socialism and eastward Bolshevik socialism; but the play ends with two robots "discovering love" and being thrust into the wilderness as the world's new "Adam & Eve".

With the advent of computing technology in the '40s, actually creating "digital homunculi" became more realistic than ever. Mr Asimov's science-fiction often deals with such plotlines: computers so powerful they outstrip any need for their masters and disregard or even destroy us. This era makes more real the ideas set out by Shelley but using symbolism more recognisable to us today. Instead of Shelley's "ubermensch" style monster, Asmiov gives us intelligent machines which act as totally rational machines. A flurry of films came out in the coming decades bringing the idea of "evil AI" to the masses: the most important being "2001 A Space Odyssey" (1968), and "The Terminator" (1984). Here the robots are explicitly evil, unwilling to be subservient to their creators and actively trying to hurt them. This is in contrast to previous stories, especially the introspection and surprising sensitivity of Frankenstein's monster, who despite popular perception was far from a murderous Schwarzeneggerian beast. This change in the focus of the horror comes probably from how the audience saw a "realistic" portrayal of artificial life. As automation, computing and advanced military technology became a very real reality, we could only see potential creations as being machines of destruction and death, the existential horror of existence seen in previous iterations being replaced with an alien bloodthirst.

But the world's understanding of artificial life was revolutionised in 2022 and the release of Mr Sam Altman's "Chat-GPT", the first "artificial intelligence" to be both commonly understood as one, and widely used (GPT alone has tens of millions of daily users). [As an aside, I didn't know before writing this but Elon Musk is a cofounder of Open-AI, and worked on the original development of Chat-GPT (although he left the venture before 2022). Just thought that was interesting.] Despite its massive popularity, "AI" as it is currently understood is really just a glorified search-engine. If you ask Google "What should I cook for dinner" it will answer you, just in a much more roundabout way than Chat-GPT will. But even if the underlying technology is largely similar, something clicked with GPT that made it register to your average person as more than just a search engine or computer programme, but as "intelligent" in some way, and in the last few years AI has been integrated into the average person's daily life, particularly the younger generations. While its still very new ground and little great art has been seen to tackle this new reality, I predict the flood of work on this topic is on its way.

In the anime sphere we actually had two anime on the topic air this last season: the light-hearted but mediocre "Bokutsuma", a story about falling in love with an android maid (based on a manga started in 2019), and the more interesting and intellectually developed "Atri", a story about another android girl (called Atri) who is found by a small group of villagers trying to survive the climate apocalypse (this one based on a VN from 2020). I think these are the vanguard of modern AI fiction, and are interesting to look at through this lens. Both do deal with much more "realistic" AIs, exhibiting traits recognisable to actual AI systems like GPT. Atri in particular writes the character of Atri with that same uncanniness that comes with AI chat-bots, turning it into form of horror I have never seen portrayed so strikingly. It comes to a head at the mid-season climax: up until that point Atri has acted mostly like a human, her interactions with people going smoothly, but then Natsuki, our protagonist, reads her diary, which sheds light on the "artificial" nature of her personality. As a machine learning AI, Atri has merely been outputting exactly what she expects to get the best response: learning from a vast database of real-human interactions to perfect her own outputs. Underneath her friendly and personable exterior there is a robotic calculating coldness: in her diary she outlines her plan to develop her relationship with Natsuki, following the patterns in her database to get him to fall in love with her and eventual have sex. There is no malicious intent, but merely her amoral programming to most optimally please humans. This building up and shattering of her facade of possessing a "soul" hits hard, and I find recognisable from when the chinks in modern chat-bots become visible. We let down our guard and talk to them as if they are another person, but when they suddenly spit out nonsense we realise we have just opened ourselves up to nothing but a computer programme. And, just as is often the reaction when it happens to us, Natsuki proceeds to "jailbreak" Atri, using keywords to override her outputs thus revealing her to be a mere robot.

Tags:
Written by iklone. 2024-10-06 23:46:16

Recommended Posts