ArmadilloCon 2011: The Singularity panel

Singularity is commonly associated with emergence of strong artificial intelligence, and the panelists don't think strong AI is any closer now than it was at the time this topic was discussed on ArmadilloCon panels of the past. Or 30 years ago. Or ever. Singularity is certainly no closer than when Vernor Vinge debated it at ArmadilloCon 2003, or Charles Stross at ArmadilloCon 2006. Moreover, some panelists disagreed whether moderator John Gibbons' question "What do you see as a fundamental block towards strong Artificial Intelligence?" is even the right question to ask. They disagreed whether it is possible to bring an AI into being by programming it.

This was a well-reasoned analysis of common Singularity tropes.

Trope 1: Singularity will come from the emergence of strong AI

Bruce Sterling admits that he used to find the idea of strong AI seductive, but doesn't see any evidence of it emerging any time. More, he doubts whether the products of intelligent mind, such as new ideas or inventions, can be attained computationally. "I know many very intelligent people, and they don't reason stuff out of the first principles," said Bruce Sterling. It certainly doesn't "feel like" creative insights come to us algorithmically. And if flashes of insight can't be simulated by a Turing machine, then they are not achievable by a computer. "Computation is not like human intelligence," says Sterling. "It's like mathematics. You could say, mathematics will one day overtake the human brain! But that would be a category error."

Left to right: authors Alexis Glynn Latner, Adrian Simmons, John Gibbons at the Singularity panel. More pictures from ArmadilloCon 2011 are in my photo gallery.

Bruce Sterling thinks collective intelligence is more interesting than artificial intelligence. When you are starting a company, would you hire HAL 9000, an intelligent machine who never sleeps, or a bunch of engineers who use Google, he asks. Google would immediately defeat HAL. He didn't answer another panelist Adrian Simmons' question if it wouldn't be even better to hire a HAL who uses Google.

Then Bruce Sterling left the panel to go help his daughter who was at the other end of town, adding "Real futurists have children!"

Trope 2: Moore's law will inevitably lead to strong AI

Both the panelists and the audience doubted whether the advance of AI has hardly anything to do with Moore's law. We already have extremely powerful computers for extremely complex weather and economic simulations, but you can't speak about their IQ.

Trope 3: Sentient computers care about the existence of humankind one way or another

Would we even want a sentient machine? John Gibbons reminds us that Charles Stross, author of Singularity-themed novels, asked this question in a recent blog post. What do we need a sentient machine for? While we conceivably might want an intelligent computer to run a spaceship on a long mission, like HAL 9000, in general there's not much advantage to sentience in a software program, argued John Gibbons. And it raises a huge batch of ethical questions. Using a genetic algorithm to derive sentient software? You're committing genocide along the way, because you're killing off versions that don't meet your goals.

Left to right: authors Katy Stauber, Marshall Ryan Maresca, and Bruce Sterling at the Singularity panel. More pictures from ArmadilloCon 2011 are in my photo gallery.

Even if a sentient AI is benevolent to the humankind, it can't be expected to do what humans would like it to do, Adrian Simmons pointed out. You may ask it how to make better gadgets, but it will instead turn around and ask you personal questions, because it might feel it's human now, and wants to experience a human perspective of the world. (Even that, I should say, is a bit human-centric, if not to say myopic. An AI might not be interested in learning from humans, since by necessity it would develop its own way of learning about the world: else it would not be an AI. It is a common trope in science fiction that a robot or AI yearns to know what it is like to be human, but I think we as humans overestimate our interestingness to the machines. We absolutely can't expect them to take an interest in our problems, let alone serve us. -- E.)

Trope 4: Singularity will come from augmentation of human brain

But machine intelligence is not the only way for Singularity to come about. Bruce Sterling said: increase of a metabolic efficiency of certain regions of the brain (that are dedicated to higher functions), and it will feel like Singularity. Our brain is very inefficient -- the biggest parts of it are dedicated to such functions as walking. So an increase in efficiency of higher reasoning parts of the brain could bring about enormous changes for the humankind. Science fiction has already addressed something similar, such as repurpose visual cortex to do other computations, an audience member pointed out.

Trope 5: Singularity will come from uploading a human personality to a machine

The panelists doubt whether that will ever be possible, because it seems like such a stretch between the chemistry of "wetware" and computational substrate. Here, too, science fiction has shown how horrific unintended consequences of this can be -- case in point is Greg Egan's story "Learning To Be Me".

Trope 6: Singularity will come from the web "waking up"

A person in the audience asked: if we organize semantic connections on the web, will the web "wake up"? John Gibbons reply was what I would have said too: becoming conscious requires a model of self. And it's hard to see how that model of self would emerge simply by organizing semantic connections on the web.

Vernor Vinge's "Rainbows End": FACT group discussion and my opinion

11 members of the FACT reading group attended the discussion of Vernor Vinge's "Rainbows End" on June 19, 2007. One reader emailed their comments. All but 1 member have read Vernor Vinge before. Everybody started the book. All but 2 people finished it, and the rest were planning to finish.

Overall people thought it was a multilayered book, even though (to quote one reader) when you read it it does not feel like a book with layers. The majority of the group felt the story was entertaining. However, everyone seems to have taken something different away from it.

Some people did not think that the technology portrayed in the book brought up any unique concepts, but others were impressed how convincingly Vernor Vinge extrapolated from existing technological trends. One reader said he had just read in the news that Google and Linden Labs are collaborating on "something like that" (he didn't clarify). Several readers noted that the technologies Vinge describes seemed familiar to them from Vinge's earlier novel "Deepness in the Sky". The ubiquitous nodes that store information about the physical world around them, are a milder version of localizer dust; the You-Gotta-Believe-Me technology is a milder version of Focus.

A fun read or food for thought?

Some people thought "Rainbows End" was mostly a spy / mystery thriller, a fun read without a significant science-fictional aspect; on the contrary, a minority said it was an intellectually worthwhile book to read, but the story did not "pull them along". Of those who were hooked by the story, several complained that the plot hit a major snag at the library battle scene. The battle, which draws out over a hundred pages or more, snarls the flow of the story, more so because the particulars of it aren't really important to the plot. To be fair, one reader loved the library battle scene. He felt as if watching a group of inhabitants of the World Of Warcraft universe battle the inhabitants of the Discworld universe for supremacy of their environment.

Vernor Vinge at the ArmadilloCon 2003, a science fiction convention in Austin, Texas

Yet other group members said the book was both a pleasure to read and provided food for thought.

Those readers noted that the book attempts to describe what it's like to be in the bend of the technological progress curve going off into the Singularity, and debated whether Vernor Vinge was more successful at it than Charles Stross in "Accelerando". Regardless, a few readers acknowledged that the author did an impressive job of showing the coming of the Singularity from the inside, so transparently you don't even understand how it happens. He showed it from several perspectives: there were kids in the book who have really mastered the "wearing" (a catch-all term for using the augmented reality technologies via devices built into the clothing and contact lenses); there were kids who were struggling with it; and then there were the "retreads" -- people who grew up in the pre-wearing era, and who needed to be taught this from scratch; there were even people who rebelled against the augmented reality, like the guy who, instead of wearing his computers, carried a laptop.

An "Alice in Wonderland"-like tour of the future

To make the "brave new world" comprehensible to us, the author had to show it through the eyes of one of our contemporaries, which explains the choice of the viewpoint character as Robert Gu, a retread. The readers observed that if Vinge chose a point of view character who was "native" to the new technologies, we wouldn't understand at all what was happening in the story. To quote a reader, Robert Gu was an equivalent of a shipwrecked English sailor in the medieval Japan.

This leads us to the question of what the Rabbit was. Even though it's not clear from the book (beyond some educated guesses), what kind of entity the Rabbit is, a reader observed that Rabbit's role is essentially that of the White Rabbit from "Alice in Wonderland". He further said "Rainbows End" is essentially an "Alice in Wonderland" kind of story, the Wonderland in this case being the high-tech future Robert Gu is touring. Hence Gu is kind of an Alice. (Not to be confused with a "Rainbows End" character named Alice. One of the more mysterious characters in the novel, she left some readers wanting more. They thought that for the importance of the role she played, the author should have told us more about her.)

The protagonist is a jerk, but he's often right

Speaking of characterization, though this group considers it be one of Vernor Vinge's lesser strengths, most people thought that Vinge did a good job with Robert Gu. They found Gu's character growth convincing. Myself, not so much. I wasn't impressed with Robert Gu's transformation from a world class jerk to a caring person, since it seemed externally, not internally motivated. His Alzheimer's treatment gave him a whole new personality, taking away his ability to see deeply into people and push their buttons, so he wasn't able to hurt people as well as he used to. That's not to say that Robert Gu is not an interesting character. He really is. The scene where he lashes out against Miri sheds lights on the complexities of his personality. Even though his delight in hurting Miri is despicable, his put-down of her has a ring of truth to it. It is true that she doesn't understand what the world was like before the augmented reality, and she doesn't understand what it's like to read books in their original, bare, unenhanced state, using only your imagination to experience the story fully. Miri is used to virtual reality doing that job for her, getting her into a story: a job that used to be done by the human mind alone. People who grew up without knowing how to use their imagination to enrich their experience seem retarded to Robert, and so does Miri. Despite his arrogance, it's hard not to agree he has a point, and that he has a right to feel offended by her simple-minded encouragement to embrace the augmented reality.

The political aspect of the book

One reader said all these things discussed so far were irrelevant to what Vinge wanted to do. The reader thought "Rainbows End" was one of the best political novels written of late, relevant to our current time when the country's administration doesn't live in reality-based world. He liked that Vinge did not take a stance on whether the high-tech world he described was utopia or dystopia. It can be either, since the ubiquitous computing devices that enrich one's perception of reality can also be used for surveillance of the wearers. But the author did not push one view or the other; instead he showed good and bad implications of the whole series of things that are going on now, such as department of homeland security, or interesting unintended consequences of Alzheimer's treatment.

Another reader was intrigued with the political aspect of the book that was left hidden behind the scenes. There are hints in the book that the near future world is going to be far more dangerous and violent than what we have experienced so far. Vinge offhandedly mentions that Alice's family was from Chicago and "none of them survived". Or that the counter-terrorist measures were mostly working, because "they haven't lost a major city in almost 5 years". Apparently, minor cities were lost more often than that.

Several prominent science fiction writers had written novels that were their response to 9/11, and it may be fair to say that "Rainbows End" was Vernor Vinge's reaction to that event -- somewhat belated, but not if you keep in mind that this writer produces 1-2 novels a decade. To quote a reader, "He's been writing as fast as he could."

ArmadilloCon 2005: opening ceremony and space opera panel

Opening ceremony is one of the most memorable parts of an ArmadilloCon. The toastmaster tries to embarrass the Guests of Honor by poking fun at them on stage. The toastmaster in 2005 was a Canadian urban fantasy writer Charles de Lint. In his speech he made a big deal of the fact that the writer Guest of Honor, Charles Stross, is a "computer wrangler" and that he is from Scotland. I don't know if perhaps Scotland is viewed by Americans as being primarily agricultural, but Charles de Lint drew a lot of parallels between computers and farming (and, by extension, wrangling). "Scotland doesn't have server farms, their computers are free-range".

Then he talked at length about Jim and Laurie Mann trapeze wedding. I missed most of it. I was too busy messing with my new recording gear, which is an MP3 player with a voice recording function, and a microphone that plugs into my laptop and allows me to record sound directly into the computer. It doesn't work very well. I don't know if it's a bug or a feature, but it stops every couple of minutes and then I have to press the Record button again. By then, some time has passed before I notice that it has stopped. So I'm sure my recording is missing large chunks of that speech.

Developments in Space Opera

Then I went to a panel "Developments in Space Opera". Charles Stross was on it. There were some interesting points made there, but I still don't know if my recording is of a good enough quality to capture everything that was said.

There was much wondering why so much space opera is coming out of UK nowadays, as opposed to America. Chris Roberson "accused" British authors of distrusting heroes. (I'm not sure if or how that was supposed to explain why British authors lead in space opera. :-)) Charles Stross said he doesn't so much mistrust heroes as doesn't think it's a realistic notion that one character in the right place could change the world, as in common in space opera and in science fiction in general. "In space opera, one shiny bright character always does the right thing, everybody revolves around him," he said. "In real world, we call people like that world criminals.

As may be expected for a panel with Stross, the discussion veered towards Singularity, although he didn't bring it up. (I'm actually surprised there are no Singularity-related panels in this year's ArmadilloCon, even though Stross is a guru on the subject. Two years ago, when Vernor Vinge, the writer who popularized the idea of Singularity, was the guest at the ArmadilloCon, several panels revolved around Singularity.)

Chris Roberson said we should ask Charles Stross to enumerate all singularities that the humankind has already been through. It turns out Stross thinks there had been a few. Apparently he does not define it as a very steep technological advance beyond which life becomes unpredictable: by that account we haven't had a singularity yet. I guess he has a broader definition: it's any kind of technological or social change beyond which life can't be predicted. If that's really his view, then it's almost like he is taking an easy way out. Because all these developments listed below, except the first one, are not the kind that would create an unbridgeable gap between the humans on one side and the humans on the other. You could take a person who lived before let's say, agriculture was practiced, and explain the concept to him or her; he/she would probably understand what it means to cultivate the crops, even though he/she may find that idea strange or silly. Even though it's true that the long-ranging consequences of agriculture could not have been predicted in advance, it still does not create an intellectual gap so deep that the "before" would have no means, no tools, no capabilities of understanding the "after".

In contrast, Vinge (I think) associates Singularity with an emergence of superhuman intelligences. In that scenario we have no chance of understanding what those intelligences are like, or what they are "about", or how they will begin to change the world, than a dog has a chance of understanding what human beings are about. That's probably why Vinge, in his novels, does not try to portray a post-Singularity society, but limits himself to the "left behind" societies, or scenarios where Singularity happened, but a particular society was somehow bypassed, or escaped it.

On the other hand, one guy in the audience commended Stross for not avoiding the post-Singularity scenarios, but "bear-hugging" them.

Anyway, this is Stross' list. It is probably incomplete (I might have missed something).

1. Development of language. Obviously, there is an unbridgeable gap between the primates that don't have language and those who do.
2. Agriculture. Once humans started farming, a given area of land could support many more people than it could when they were hunter-gatherers. Once the population density increased, it would have been impossible to go back to hunting-gathering. Or so his argument goes.
3. Writing -- ability to transmit knowledge from generation to generation.
4. Invention of finance and investing. He says, read Neal Stephenson's "Baroque Cycle" trilogy to get an idea how it changed society.


In the picture: the Space Opera panelists. Left to right: Chris Robertson, Charles Stross, Jim Minz, Sean McMullen, Jayme Lynn Blaschke. Here are more posts from ArmadilloCon 2005.