.png)
November 22, 2024
Large language models (LLMs) have revolutionized the world of artificial intelligence. Their ability to generate human-like text opens up unprecedented possibilities in a wide variety of fields. At the same time, LLMs struggle with a persistent problem: hallucinations. These refer to the tendency of LLMs to generate information that sounds plausible but is factually incorrect.
Hallucinations in LLMs are a complex phenomenon and can take various forms. They range from generating content that contradicts the given input, to violations of contextual coherence, to statements that contradict established facts. Hallucinations are particularly problematic in areas where accuracy and reliability are crucial, such as healthcare, finance, or law.
Research is striving to understand the mechanisms behind these hallucinations. A recent study examines the relationship between a model's knowledge awareness and its susceptibility to hallucinations. The central question is: "Does" the model know whether it has sufficient knowledge about a particular entity (person, place, object, etc.) to make correct statements about it?
The study suggests that entity recognition plays a key role in the emergence of hallucinations. Simply put: if the model can recognize an entity and link it to existing knowledge, the probability of a hallucination decreases. If the model cannot classify the entity, the risk of it inventing facts increases.
Using sparse autoencoders, a method for interpreting neural networks, the researchers were able to demonstrate that certain areas in the LLM's neural network are responsible for entity recognition. These areas seem to represent a kind of "self-knowledge" of the model by indicating whether the model has sufficient information about an entity.
The researchers were also able to establish a causal relationship between entity recognition and the occurrence of hallucinations. By specifically manipulating the areas in the neural network responsible for entity recognition, they were able to make the model refuse to answer questions about known entities or hallucinate about unknown entities.
Interestingly, these manipulations also had an effect on chat models trained through Reinforcement Learning with Human Feedback (RLHF). This suggests that the model refined through RLHF training continues to rely on the original mechanisms of entity recognition.
The study also provides initial insights into the mechanistic processes underlying hallucinations. According to the study, the areas responsible for entity recognition seem to disrupt the attention of certain neuronal components that are normally responsible for transmitting entity attributes to the final output. This could explain why the model tends to generate false information when uncertain about an entity.
The findings of this study contribute to a better understanding of the complex mechanisms that cause hallucinations in LLMs. The identification of the areas in the neural network responsible for entity recognition opens up new possibilities for developing strategies to mitigate hallucinations. Future research could focus on specifically training these areas or developing new mechanisms that improve the knowledge awareness of LLMs.
Bibliographie: https://openreview.net/forum?id=WCRQFlji2q https://openreview.net/pdf/416a1f938d5d9909169f7c553aef7a4747137d8e.pdf https://paperreading.club/page?id=267503 https://huggingface.co/papers/2408.07852 https://arxiv.org/pdf/2401.01313 https://arxiv.org/html/2311.05232v2 https://aclanthology.org/2024.lrec-tutorials.12.pdf https://medium.com/codex/harnessing-knowledge-graphs-to-mitigate-hallucinations-in-large-language-models-d6fa6c7db07e https://www.researchgate.net/publication/382296850_A_review_of_methods_for_alleviating_hallucination_issues_in_large_language_models/fulltext/6696a7c402e9686cd1078e9a/A-review-of-methods-for-alleviating-hallucination-issues-in-large-language-models.pdf https://2024.aclweb.org/program/finding_papers/