Did Google Fake Its Duplex AI Demo?

A few weeks ago, at Google I/O, company CEO Sundar Pichai demoed an incredible advance in AI and voice technology. The demo involved a new AI assistant making phone calls to two businesses — a hairdresser and a restaurant — to make an appointment and book a reservation, respectively. You can see the original demo below, but for those of you who don’t have access to video, the calls are impressive because the AI responds (and sounds) far more human than your typical voice assistant.

It pauses. It says “Umm,” in the same way a human might, while considering options. When the reservation-taker at the restaurant confuses the AI’s request, the AI doesn’t start jibbering nonsense. It doesn’t even flail. Both demos are impressive, but the restaurant demo is the one, I personally think, that made people sit up and blink. But there are also oddities in the recordings that I noticed, even at the time. Now Axios has written a bit more on the event, including some questions that Google, apparently, doesn’t want to answer. First, here’s the video:

Note the oddities:

Axios called over two dozen hair salons, including some in Mountain View, and every business establishment promptly gave its name when contacted. This is simply standard procedure. It’s extremely rare to contact someone at a place of business and not be given either the name of the business, the name of the speaker, or both. Even something as simple as “This is Lisa, how may I help you?” tells the person on the other end of the phone that they’ve reached a company as opposed to a personal line. Yet Pichai specifically states: What you’re going to hear is the Google assistant actually calling a real salon to schedule an appointment for you. Let’s listen.

Google has clammed up on the topic. It refuses to say if the calls were edited, even just to remove the business names and identifying information. It refuses to disclose anything about the demo at all.

Does it Matter?

There are two ways to look at this situation. On the one hand, the call technology was impressive as hell. The AI did respond with believable pauses. It didn’t error out when presented with flustered communication. Assuming that the AI was actually generating those responses itself, that’s still an impressive achievement in voice replication (and we’ve covered Google’s work in this area before).

But Sundar Pichai didn’t pitch this demo as a genuinely impressive advance in voice technology. He was demoing (or claiming to demo) a situation in which an AI assistant could respond to unexpected conversation prompts, confusion, and real-world scenarios. And it’s entirely possible that the reason Google partially or completely staged the demo (and therefore lied in its framing of the presentation) is because it can’t guarantee that the person on the other end of the phone will have an accent that its AI can understand. It may not be able to guarantee that the conversation won’t take a turn that its AI can handle. And it may not be able to promise that its AI will understand the information provided depending on how that information is verbally ordered. This type of “fuzzy” processing is something human brains are very, very good at, and it’s an area where AI has generally struggled.

But here’s one thing we do know. When companies pull off major breakthroughs, especially in an area like artificial assistant technology, they typically can’t wait to show them off in as many scenarios as possible. When Apple launched Siri or Microsoft launched Cortana, they showed off their respective capabilities at great length. To this day, Microsoft pushes regular Cortana news about the features and capabilities of the platform.

All we have on Google Duplex, in contrast, is one canned AI demo shown at a Google event, under conditions that raise real questions about whether or not the event was staged. Google’s refusal to answer questions about its demo doesn’t leave it looking good in this scenario. If Google actually staged this event, then we withdraw our earlier remarks. The company didn’t pass the Turing Test at all. It just demonstrated a theoretical scenario that might one day lead to such an achievement.

Continue reading

Google’s Duplex AI Demo Just Passed the Turing Test

Google Duplex isn't just an impressive bit of technology — it's arguably evidence that Google has passed the Turing test, at least in this specific instance.

Google Duplex AI Could Be Headed to Call Centers

The next customer service rep you speak to could be a machine. Welcome to your dystopian nightmare.

Google Duplex AI Still Needs a Lot of Help From Humans

We met Google Duplex at the I/O conference in 2018 when Google showed off a suspiciously realistic AI that could call and make appointments for you. The service has now rolled out to users of Google Assistant in most US states, but you may not be getting the full AI experience.