IBM Built an AI Capable of Holding Its Own Against Humans in a Debate

IBM Built an AI Capable of Holding Its Own Against Humans in a Debate

Over the past few years, AI has gone from a niche topic to an exploding field. AI can improve audio and video quality, animate still images of long-dead people, and identify you from an analprint. One thing it hasn’t been able to do? Argue effectively within the context of a formal debate.

To overcome this problem, IBM created Project Debater, an AI development program focused on exactly what it sounds like. Many AI projects, especially those focusing on gaming, have a clear winner and a loser based on the evaluation of numerical criteria, such as pieces captured, lives lost, or the ratio between kills and deaths. Effectively debating a human requires a vastly different skill set.

A recent paper in Nature describes the results of a 2019 test between Project Debater and globally recognized debate champion Harish Natarajan. The AI and individual debated whether preschool should be subsidized. Each side was given 15 minutes for prep time without additional internet access, which Project Debater used to sort through its own internal database of content. Both sides gave a four-minute speech, followed by a two-minute closing statement.

IBM Built an AI Capable of Holding Its Own Against Humans in a Debate

Ultimately, Natarajan was judged to have won the debate, but Project Debater held its own, forming logical statements and arguments over the course of the discussion.

The researchers that developed Project Debater can’t compare it with other systems of its type. There aren’t any. Instead, they used PD to generate a single opening speech and compared it against various other methods.

In the graph below, “Summit” is a multi-document summary system, Speech-GPT2 is a “finely tuned language model,” and Arg-GPT2 was generated using concatenating arguments. Arg-Search refers to speeches extracted using ArgumenText. Arg-Human1 and Arg-Human2 refer to a hybrid approach that tested Project Debater’s argument mining module alongside human authorship and verification. Finally, speeches were included from human experts.

IBM Built an AI Capable of Holding Its Own Against Humans in a Debate

The graph above shows the baseline score where a score of 5 indicates “Strongly Agree” and a score of 1 means “Strongly Disagree.” Readers were asked to answer the following question: “This speech is a good opening speech for this topic.” This graph is not a full test of Project Debater’s capabilities — it only evaluates opening speeches — but it demonstrates that the system is capable of producing coherent arguments. IBM has a website for the project with links to the whitepaper, podcast, and the 2019 debate on preschool subsidies if you’d like to see more of how the system performed in action.

The question of who wins a debate will always be subjective, and humans still clearly outperform IBM’s Project Debater. For now, we’re still a long way from Data — but we’ve come a long way from Eliza, too.

Continue reading

How to Build a Face Mask Detector With a Jetson Nano 2GB and AlwaysAI
How to Build a Face Mask Detector With a Jetson Nano 2GB and AlwaysAI

Nvidia continues to make AI at the edge more affordable and easier to deploy. So instead of simply running through the benchmarks to review the new Jetson Nano 2GB, I decided to tackle the DIY project of building my own face mask detector.

This Is a Bad Time to Build a High-End Gaming PC
This Is a Bad Time to Build a High-End Gaming PC

We're not going to say it's the worst time to build a high-end gaming PC, but if you'll need to get lucky with some orders if you want to pull it off this year.

Why We Can’t Build Our Way Out of the Semiconductor Shortage
Why We Can’t Build Our Way Out of the Semiconductor Shortage

Eight months on, the semiconductor shortage seems likely to stretch into 2022. Increasing chip production is a slow and difficult business.

How I Spent $700 Building a ‘Life Aquatic’ Mechanical Keyboard
How I Spent $700 Building a ‘Life Aquatic’ Mechanical Keyboard

I recently waited about a year and a half for my Life Aquatic project to come together, and the total cost was around $700. I know what you're thinking: "How? Why? Are you crazy?" Allow me to explain.