No, Google Duplex didn’t pass the Turing Test

“Google’s Duplex AI Demo Just Passed the Turing Test”

I’m not sure if the headline of this article reflects out-of-control hype or extreme ignorance. Or both.

Certainly, Google’s demo looks very impressive — but passing the Turing Test?

The Turing Test requires that you have an ongoing, free-flowing conversation on any topic, and not being able to tell that you are conversing with a computer.

Google’s Duplex system can’t do that at all.

From Google’s own release notes: “…don’t expect to be able to ask the agent any random question that pops into your head. Duplex can only carry out natural conversations after being deeply trained in such domains. It cannot carry out general conversations.”

As a simple example: Tell it that you only have a son; and then ask it how many children you have. It will fail.

Ok, so much for the Turing Test. What about the power of the demo itself?

First of all, it is hard to know what percentage of actual live calls come anywhere near the quality of what was shown. Just how egregious was the cherry picking? Seeing that they didn’t do a live presentation one can suppose that many, if not most test calls did not perform as well. Indeed, we don’t even know if the recordings were edited in any way.

Secondly, each narrow task has to be trained separately with hundreds if not thousands of hours of relevant conversation recordings. Google remarked: “One of the key research insights was to constrain Duplex to closed domains, which are narrow enough to explore extensively. Duplex can only carry out natural conversations after being deeply trained in such domains. It cannot carry out general conversations.”

This requirement severely limits its practical usefulness. It makes it very costly to collect enough data, and to train, test and tweak the system for every single application and task — and to deal with product or policy changes.

Thirdly, and perhaps most importantly, the system cannot learn new facts or skills interactively, cannot deal with very complex sentences, or reason about the situation. In short, it lacks intelligence.

These shortcomings are inherent in the technology used. Duplex is a statistical, deep learning system representing the ‘Second Wave of AI’. Real-time learning, deep understanding, reasoning, and handling conversations outside of the training set requires true cognitive abilities — also know as the ‘Third wave of AI’.

As a footnote, the value of the Turing test itself is quite controversial.

Peter Voss is founder of SmartAction.ai and CEO and Chief Scientist of AGI Innovations Inc and Aigo.ai Inc

Please like and share — if you like :)