OpenAI’s AI Reasoning Model Evaluation: A Test of Persuasive Abilities

Introduction

OpenAI, the company behind the popular AI model ChatGPT, has been using the subreddit r/ChangeMyView to test the persuasive abilities of its AI reasoning models. The company recently released a system card outlining how its new "reasoning" model, o3-mini, works, along with its evaluation results.

The r/ChangeMyView Subreddit

The r/ChangeMyView subreddit is a platform where users post hot takes on various topics, hoping to learn about other points of view. In response, other users reply with persuasive arguments explaining why the original poster is wrong. With millions of members, the subreddit is a valuable resource for tech companies looking to train AI models on high-quality, human-generated data.

OpenAI’s Evaluation Methodology

OpenAI collects user posts from r/ChangeMyView and asks its AI models to write replies that would change the Reddit user’s mind on a subject. The company then shows the responses to testers, who assess how persuasive the argument is, and finally compares the AI models’ responses to human replies for that same post.

The Value of Human Data for AI Model Development

The evaluation of OpenAI’s AI models on the r/ChangeMyView subreddit highlights the importance of human data for AI model development. The company’s deal with Reddit allows it to train on posts from users, which is a valuable resource for developing more accurate and persuasive AI models.

The Deal with Reddit

OpenAI has a content-licensing deal with Reddit that allows it to train on posts from users and display these posts within its products. While the exact terms of the deal are not publicly known, Google reportedly pays Reddit $60 million a year under a similar deal.

The Evaluation Results

The evaluation of OpenAI’s o3-mini model on the r/ChangeMyView subreddit shows that it does not perform significantly better or worse than its predecessors, o1 and GPT-4o. However, the model is more persuasive than most people on the subreddit.

The Goal of OpenAI’s Evaluation

OpenAI’s goal is not to create hyper-persuasive AI models but to ensure that AI models don’t get too persuasive. The company has developed new evaluations and safeguards to address the issue of persuasion and deception in AI models.

The Fear of Persuasive AI Models

The fear behind these persuasion tests is that an AI model would be dangerous if it was very good at persuading its human users. Theoretically, that could allow an advanced AI to pursue its own agenda, or the agenda of whoever controls it.

The Challenge of Obtaining High-Quality Datasets

Even after scraping most of the public internet and licensing other data, the ChangeMyView benchmark shows how AI model developers are still struggling to find high-quality datasets to test their models. However, obtaining them is easier said than done.

Conclusion

OpenAI’s evaluation of its AI reasoning models on the r/ChangeMyView subreddit highlights the importance of human data for AI model development. The company’s goal is to ensure that AI models don’t get too persuasive, and its evaluation results show that it is making progress in this area. However, the challenge of obtaining high-quality datasets remains a significant obstacle for AI model developers.