[From:

http://www.cs.bris.ac.uk/~stefan/culture.html

"As mentioned above, there is another force at work in the Culture aside from the nature of its human inhabitants and the limitations and opportunities presented by life in space, and that is Artificial Intelligence. This is taken for granted in the Culture stories, and - unlike FTL travel - is not only likely in the future of our own species, but probably inevitable (always assuming homo sapiens avoids destruction). Certainly there are arguments against the possibility of Artificial Intelligence, but they tend to boil down to one of three assertions: one, that there is some vital field or other presently intangible influence exclusive to biological life - perhaps even carbon-based biological life - which may eventually fall within the remit of scientific understanding but which cannot be emulated in any other form (all of which is neither impossible nor likely); two, that self-awareness resides in a supernatural soul - presumably linked to a broad-based occult system involving gods or a god, reincarnation or whatever - and which one assumes can never be understood scientifically (equally improbable, though I do write as an atheist); and, three, that matter cannot become self-aware (or more precisely that it cannot support any informational formulation which might be said to be self-aware or taken together with its material substrate exhibit the signs of self-awareness). ...I leave all the more than nominally self-aware readers to spot the logical problem with that argument.

It is, of course, entirely possible that real AIs will refuse to have anything to do with their human creators (or rather, perhaps, the human creators of their non-human creators), but assuming that they do - and the design of their software may be amenable to optimization in this regard - I would argue that it is quite possible they would agree to help further the aims of their source civilisation (a contention we'll return to shortly). At this point, regardless of whatever alterations humanity might impose on itself through genetic manipulation, humanity would no longer be a one-sentience-type species. The future of our species would affect, be affected by and coexist with the future of the AI life-forms we create."]

The argument the author makes is that the creation of self-aware Artificial Intelligences is both inevitable and, more importantly, consistent with the evolution of biological intelligence, in that inanimate matter is able to be organised in such a way that it would become self-aware. This inevitability is again mentioned to refute arguments against the inevitability of Artificially Intelligent systems by the passage ending with “I leave all the more than nominally self-aware readers to spot the logical problem with that argument”. By this it is meant that a reader who denies “that matter cannot become self-aware” is denying their own self-awareness in disagreeing with the argument because they would themselves have to be self-aware to disagree with the idea of a machine being self-aware.

To prove his argument the author introduces several rhetorical ploys. He does this by first comparing the increased likely hood of Artificial Intelligence, in his view, to the unrealistic probability of faster than light travel. This is meant to shepherd the reader into believing the argument is a reasonable one by admitting that faster than light travel is an unrealistic prospect which he does not attempt to justify.

The author then attacks sceptics of Artificial Intelligence by pre-emptively refuting what he believes to be the three arguments that sceptics would make. The first sceptical argument is that there is some unknown or even unknowable property of biological life which means that intelligence can only have a biological basis. As this cannot be proved or disproved to any satisfaction the argument is not considered a convincing one. The second argument is that self-awareness is basically due to a supernatural artefact like a soul. This is refuted by assuming that belief in a soul or similar entity implies participation in an organised belief system or religion. The final argument is that matter cannot become self-aware, which is refuted as soon as a self-aware entity denies another entity can also become self-aware.

In refuting the counter-arguments, the author makes the assumption that belief in a soul implies membership of a religion. This is a fallacious argument as it would imply that belief in a soul, which may be a perfectly rational reason for one’s self-awareness, can only come from the teachings of a particular religion or cult.

A noticeable fallacy the author makes is his connection with and alteration of the terms “Artificial Intelligence” and “self-awareness”, implying the two terms are interchangeable. This leads the reader to assume that an Artificial Intelligence would be self-aware when this is not explicitly stated. The phrase “creators of their non-human creators”, implying a plausible evolutionary analogue to the creation of a self-aware machine, would seem to require a re-definition of the term “artificial” as well as the term “self-awareness” as the author admits that it is more likely that the self-aware entities he ultimately envisages are in fact the creations of successive generations of Machine Intelligences.