Are bad incentives to blame for AI hallucinations?

Ay New research paper From the open, asks why big language models like GPT -5 and ChatGP are still chatbots like Hallucinate and whether something can be done to reduce these hallucinations.

In Post a blog to summarize paperOpenly defines hallucinations as “praiseful but false statements generated by models of languages” and acknowledge that despite the improvement, hallucinations are “a basic challenge for all large language models” – a one that will never be completely eliminated.

To illustrate the matter, researchers say that they asked “a widely used chattabot” about Adam Touman Kalai’s PhD title. Research articles, they got three separate answers, they are all wrong. (One of the authors of Kalai paper)) They asked about his birthday and got three separate dates. Once again, they were all wrong.

How can a chatbot be so wrong – and it seems so confident in its wrongdoing? Researchers have suggested that a partially a pretending process that focuses on power to properly predict the next word, without the true or false labels attached to the training statements: “The model should only look at the positive example of the fluent language and the overall distribution needs to be estimated.”

“Spells and brackets follow continuous patterns, so the errors disappear with the scale there,” they wrote. “However, short-frequency information cannot be predicted from a single patterns like a pet’s birthday, and therefore leads to hallucinations.”

The proposed solution of the paper, however, focuses on the initial pretending process and how the larger language models are evaluated. It argues that the current evaluation models do not cause hallucinations themselves, but they “determine the wrong enthusiasm.”

Researchers compare these evaluations with multiple choice tests of random assumptions, because “you can become lucky and be right,” leaving the answer blank “guarantee a zero.”

“Similarly, when the models are simply graded on the basis of accuracy, they are exactly the percentage of the questions that are okay, they are encouraged to guess ‘I don’t know,’.” They say.

Suggested solutions, then, like testing (like sat) which contains “negative [scoring] In order to blank questions to discourage blind assumptions for the wrong answer or partial credit. “Similarly, Openi says that model evaluations are” more confident defects than punishing your uncertainty and partial credit for appropriate expression of uncertainty. “

And researchers argued that it was not enough to introduce “a few new uncertainty-conscious tests on the side”. Instead, “widely used, accuracy -based avals need to be updated so that their scoring is discouraged.”

Researchers say, “If the main scoreboards reward the lucky estimate, the models will learn to guess,” researchers say.