I keep thinking about those Singular computers, the ones Kurzweil thinks are working on getting smarter than us. They’ll be passing Turing Tests, convincing humans they’re human, maybe even enslaving us, but also doing all kinds of incredible things. According to Kurzweil:
My view is that the likely outcome is that on the one hand, from the perspective of biological humanity, these superhuman intelligences will appear to be their transcendent servants, satisfying their needs and desires. On the other hand, fulfilling the wishes of a revered biological legacy will occupy only a trivial portion of the intellectual power that the Singularity will bring.
Of course they’d have to conduct the Turing tests like an orchestra audition, where Watson would be behind a screen. (I’ve always wondered about those orchestra auditions, by the way – have you ever wondered how so many male flute players end up in orchestras?)
And what are they going to do with all that leftover intellectual power? Will they enslave us? Will they take us over, destroy our society?
Society, as John Tierney wrote in yesterday’s Times, depends on people behaving as if free will exists (there’s something kind of screwy about this logic: if you don’t believe in free will, you’re free to act with impunity?). But, accepting this for now, what does it take to behave as if free will exists? You have to believe in free will – in other words, you have to be conscious. And since no one knows how to create consciousness (well, except for the way you create babies), these machines aren’t going to be conscious (unless they’re so so smart they can create it in themselves). No free will, then. Our society is safe.
But what if they’re so smart they convince us free will is just a delusion?