IBM Built an AI Capable of Holding Its Own Against Humans in a Debate

Over the past few years, AI has gone from a niche topic to an exploding field. AI can improve audio and video quality, animate still images of long-dead people, and identify you from an analprint. One thing it hasn’t been able to do? Argue effectively within the context of a formal debate.

To overcome this problem, IBM created Project Debater, an AI development program focused on exactly what it sounds like. Many AI projects, especially those focusing on gaming, have a clear winner and a loser based on the evaluation of numerical criteria, such as pieces captured, lives lost, or the ratio between kills and deaths. Effectively debating a human requires a vastly different skill set.

A recent paper in Nature describes the results of a 2019 test between Project Debater and globally recognized debate champion Harish Natarajan. The AI and individual debated whether preschool should be subsidized. Each side was given 15 minutes for prep time without additional internet access, which Project Debater used to sort through its own internal database of content. Both sides gave a four-minute speech, followed by a two-minute closing statement.

Ultimately, Natarajan was judged to have won the debate, but Project Debater held its own, forming logical statements and arguments over the course of the discussion.

The researchers that developed Project Debater can’t compare it with other systems of its type. There aren’t any. Instead, they used PD to generate a single opening speech and compared it against various other methods.

In the graph below, “Summit” is a multi-document summary system, Speech-GPT2 is a “finely tuned language model,” and Arg-GPT2 was generated using concatenating arguments. Arg-Search refers to speeches extracted using ArgumenText. Arg-Human1 and Arg-Human2 refer to a hybrid approach that tested Project Debater’s argument mining module alongside human authorship and verification. Finally, speeches were included from human experts.

The graph above shows the baseline score where a score of 5 indicates “Strongly Agree” and a score of 1 means “Strongly Disagree.” Readers were asked to answer the following question: “This speech is a good opening speech for this topic.” This graph is not a full test of Project Debater’s capabilities — it only evaluates opening speeches — but it demonstrates that the system is capable of producing coherent arguments. IBM has a website for the project with links to the whitepaper, podcast, and the 2019 debate on preschool subsidies if you’d like to see more of how the system performed in action.

The question of who wins a debate will always be subjective, and humans still clearly outperform IBM’s Project Debater. For now, we’re still a long way from Data — but we’ve come a long way from Eliza, too.

Continue reading

Microsoft’s DirectStorage Will Support PCIe 3.0, All DX12-Capable Hardware

Microsoft has shared some details on how its DirectStorage standard will allow PCs to support the same kind of texture streaming available on Xbox Series S|X and the PlayStation 5.

Some TPM 2.0-Capable Systems Are Still Ineligible for Windows 11

Microsoft's Windows 11 unveil has quickly devolved into a confused mess regarding which CPUs will and won't be supported, but new information from both the company and end-users has shed a bit of light.

OWC Announces PCIe SSD Capable of 26GB/s Data Transfer and 64TB Capacity

If your rinky dink NVME M.2 PCIe SSD isn’t cutting the mustard with its laughable 7GB/s transfer rates and middling capacity, OWC has a solution that is sure to clear up any storage and capacity bottlenecks you might be dealing with. Its new ACCELSIOR 8M2 PCIe SSD “solution” holds up to eight M.2 SSDs, and when…

Software Utility and BIOS Reportedly Unlock Full Mining Capabilities on Nvidia GPUs

A crafty software developer has created a workaround for Nvidia's Low Hash Rate GPUs to restore them to full mining power.