OpenAI is pursuing a new way to fight A.I. 'hallucinations'

Important Points

Independent experts have expressed doubts about the effectiveness of the methods proposed.

OpenAI has announced a new method of training AI models to combat "hallucinations" in AI.

The research is timely, as misinformation originating from AI systems has become a hot topic in the wake of the recent generative AI boom. It also coincides with the lead-up to 2024's U.S. Presidential election. OpenAI, which released ChatGPT last year, a chatbot powered by GPT-3 & GPT-4 and reached 100 million monthly users within two months. This set a new record for the fastest-growing application. Microsoft has invested over $13 billion into OpenAI to date. The startup is valued at approximately $29 billion.

AI hallucinations are when models such as OpenAI's ChatGPT and Google's Bard fabricate facts, while acting like they're presenting them. Google's Bard chatbot made an untrue statement about the James Webb Space Telescope in its February promotional video. ChatGPT has cited "bogus cases" in a New York Federal Court filing. The New York attorneys who filed the case may be sanctioned.

The OpenAI researchers stated in their report that even state-of the-art models can produce falsehoods. They have a tendency to fabricate facts when faced with uncertainty. These hallucinations can be particularly dangerous in domains requiring multi-step reasoning. A single logical mistake is enough to derail an entire solution.

OpenAI's new potential strategy to combat fabrications is to train AI models so that they reward themselves for every correct step in reasoning, rather than just rewarding the correct conclusion. This approach, which is different from "outcome-supervision," could lead to a more explainable AI. It encourages models to take a more human-like approach to reasoning.

Karl Cobbe is a mathgen researcher with OpenAI. He told CNBC that while the company did not create the process supervision method, it is actively pushing the technology forward. The motivation behind this research was to address hallucinations to make models better able to solve challenging reasoning problems.

Cobbe stated that OpenAI released a dataset of 800 000 human labels which it used to train its model in the research paper.

Ben Winters is the senior counsel and project leader for the AI and Human Rights Project at the Electronic Privacy Information Center. He expressed his skepticism to CNBC, saying he was interested in seeing the complete dataset and examples.

Winters stated that "this alone doesn't do any significant mitigation for concerns about misinformation or incorrect results...when it's being used in real life." Winters said, "It's important to know if the researchers plan to incorporate what they learned in their research into their products. If they don't, it raises some serious questions about their willingness and ability to share information with the public."

Suresh Venkatasubramanian of Brown University's center for tech-responsibility told CNBC he viewed the research more as a preliminary observation.

Venkatasubramanian explained that the research community will have to settle this before any conclusions can be made. In this world, a lot results are published very frequently. Because of the instability of large language models, what works in one context, model, and setting may not work in a different context, model, and setting.

Venkatasubramanian continued, "Some of what people are concerned about is models making up citations or references." This paper doesn't provide any evidence that it would work.

OpenAI didn't respond to our request for comment regarding whether or not the research was externally reviewed, and when the company intends to implement the new strategy in ChatGPT, as well as its other products.

Sarah Myers West is the managing director of AI Now Institute. She told CNBC that it's a good thing to see companies working to improve their systems and try to reduce errors.

West said, "[OpenAI] is releasing a dataset of human level feedback with this article, but hasn't given basic details about data used to test and train GPT-4." There's a lot of opacity in AI that makes it difficult to make meaningful accountability efforts, even though these systems directly affect people.