Jury still out on whether Eugene Goostman passed the Turing test
Following recent reports that a computer program named Eugene Goostman which imitates a Ukrainian teenager has won an artificial intelligence competition at the Royal Society in London, Professor Colin Pattinson assesses the potential advancement in artificial intelligence.
It was reported yesterday that a computer had “passed the Turing test”, and this has been hailed as a major breakthrough in Artificial Intelligence (AI). Since the announcement, there has been a great deal of scepticism about the nature of this “breakthrough”, although we should acknowledge the technological work behind it.
Artificial Intelligence (AI) has long been an objective for computer science: the idea that a computer can possess the same kind of “intelligence” as humans - being able to think, react and make the kind of decisions humans do is a staple of science fiction, and – were it to be possible – would have a major impact on the way humans see themselves.
In 1950, Alan Turing, often described as “the founder of computer science” devised his test for AI. The basic idea of the test is that a human “judge” sits in front of a screen and keyboard, and if more than 1 in 3 such judges believe they are communicating with another human, when in fact it is a computer program, then that program must have some form of intelligence. Last weekend it was reported that just such an event had taken place, and that over 33% of a panel of judges did believe that they were communicating with Eugene Goostman, a 13 year old Ukrainian, when in fact it was a computer program.
Although it is the case that the Turing test has been “passed” in some form, we are still a long way from AI. It is more accurate to say that a very good computer program has been developed which is able to simulate a 13 year old who speaks English as a second language, to the satisfaction of a third of the people who interacted with it. This is no mean achievement, but it is not AI.
The “1 in 3 judges for five minutes” criterion was actually Turing’s prediction of the likely rate of progress by the end of the 20th century: In this he was not far off. A more realistic definition of AI is that a machine should be consistently indistinguishable from a human over an extended period of time (as proposed by Stevan Harnand). We would also hope that “intelligence” means more than being able to exchange text messages.
It is to be expected that as computers become more powerful, and are able to process more data more quickly, that they will be able to show more traits of “intelligence”: the most obvious example is being able to recognise more complicated patterns more quickly and with more accuracy. These are the more likely advances which will be made, and will have an effect on us all more quickly than AI.
Colin Pattinson is Professor of Mobile and Converging Technologies and former Dean of School of Computing and Creative Technologies at Leeds Beckett; his research focuses on sustainable 'green' IT and computer network management.