Large Language and Vision Models: Surprising Properties Unveiled

Large Language and Vision Models (LLVMs) have recently garnered significant attention and development efforts due to their remarkable generalization performance across a wide range of tasks requiring both perceptual and cognitive abilities. A key factor in their success is their straightforward architecture, which comprises an image encoder, a projector, and a large language model (LLM).

Despite their achievements in challenging reasoning tasks, their performance on fundamental perception-related tasks (e.g., MMVP) is surprisingly low. This discrepancy raises the question of how LLVMs actually perceive images and leverage the benefits of the image encoder. To shed light on this, researchers have systematically investigated this question with respect to various aspects: permutation invariance, robustness, mathematical reasoning, alignment preservation, and meaning. They analyzed the most popular LLVM families (e.g., LLaVA) using 10 benchmark datasets.

The investigation revealed several intriguing properties of current LLVMs:

Global Image Processing

LLVMs process the image globally internally, even when the order of visual patch sequences is randomly permuted. For instance, in LLaVA 1.5, the average performance drop across 10 benchmarks is 0.19 (< 1%), suggesting that LLVMs exhibit permutation-invariant properties.

Mathematical Capabilities Despite Lack of Detailed Perception

LLVMs are capable of tackling tasks when presented with synthetic versions of the MathVista dataset, with only a minor decrease in performance (1.8% for LLaVA 1.5). Additionally, it was observed that in certain scenarios, LLVMs can still solve problems even without access to the complete image, including detailed numerical and diagram elements.

Loss of Perceptual Ability Due to Overfitting

After alignment and visual instruction tuning, LLVMs are no longer able to retain their original perceptual capabilities, leading to a drop of up to 20% in image classification tasks (e.g., CIFAR-100). This phenomenon is referred to as catastrophic forgetting. Furthermore, they struggle to grasp common-world concepts within the representation space, as evidenced by the platonic representation hypothesis.

Importance of Lower Layers for Visual Processing

Analysis of model behavior reveals that LLVMs tend to focus on the central region of the image. Moreover, the lower layers in LLVM architectures are crucial for better generalization. In these layers (i.e., the bottom 20% of LLM layers), the model primarily processes visual information, whereas the higher layers concentrate on text interpretation.

Future Research and Development of LLVMs

The research findings highlight the importance of developing better LLVMs and creating more sophisticated benchmark datasets. Specifically, designing more interactive and complex benchmarks is necessary to minimize selection bias and enhance real-world applicability. Furthermore, preserving cross-modal alignment is critical when developing new LLMs.

The insights gained can assist other ML researchers and engineers in establishing a new paradigm for LLMs and further pushing the boundaries of artificial intelligence.

Bibliography

http://arxiv.org/abs/2410.04751
https://arxiv.org/html/2410.04751v1
https://www.aimodels.fyi/papers/arxiv/intriguing-properties-large-language-vision-models
https://openreview.net/pdf/328dddf22b858986bc8405c791736a9e2e5c5e68.pdf
https://openreview.net/forum?id=P5D2gfi4Gg
https://github.com/Muzammal-Naseer/IPViT
https://openaccess.thecvf.com/content/CVPR2024W/MMFM/papers/Schiappa_Probing_Conceptual_Understanding_of_Large_Visual-Language_Models_CVPRW_2024_paper.pdf
https://proceedings.neurips.cc/paper/2021/file/c404a5adbf90e09631678b13b05d9d7a-Paper.pdf
https://www.reddit.com/r/MachineLearning/comments/16ij18f/d_the_ml_papers_that_rocked_our_world_20202023/
https://openaccess.thecvf.com/content/CVPR2024/papers/Guo_RegionGPT_Towards_Region_Understanding_Vision_Language_Model_CVPR_2024_paper.pdf