Core Cognition Deficits in Multi-Modal Language Models: A 2025 Guide

TL;DR

  • 2025 research reveals Multi-Modal Language Models (MLLMs) underperform humans in core cognition tasks. Top models like GPT-4o show significant gaps in low-level cognitive abilities (e.g., object permanence: humans at 88.80% accuracy vs. GPT-4o at 57.14%).
  • Models exhibit a “reversed cognitive development trajectory,” excelling in advanced tasks but struggling with basic ones. Scaling model parameters improves high-level performance but barely affects low-level abilities.
  • “Concept Hacking”验证发现73%的模型依赖捷径学习，存在认知幻觉现象。比如在视角转换任务中，某大型商业模型对照任务准确率为76%，但在操纵任务中骤降至28%。

Understanding Core Cognition Assessment

Assessing core cognition in MLLMs requires a systematic approach. The CoreCognition benchmark evaluates 12 key abilities across different cognitive stages:

  • Sensory-Motor Stage (0-2 years): Focus on basic abilities like object permanence, boundary recognition, and continuity understanding.
  • Concrete Operational Stage (7-11 years): Test conservation concepts, perspective-taking, and intuitive physics.
  • Formal Operational Stage (12+ years): Examine advanced abilities like mechanical reasoning, tool use, and intention understanding.

The benchmark includes 2,519 questions in various formats, including single-frame images (65%), multi-frame sequences (24%), and videos (11%). This diversity ensures comprehensive assessment of model capabilities.

Analyzing Model Performance Patterns

Model performance analysis reveals distinct patterns:

  1. Low-Level Task Deficits: Only 17% of models reach 50% of human baseline in object boundary recognition.
  2. High-Level Task Strength: Mechanical reasoning tasks show a strong positive correlation (R²=0.78) with model parameter growth.
  3. Disconnection Between Abilities: High-level cognitive abilities lack correlation with their foundational low-level abilities (Pearson coefficients all <0.32).

This “cognitive development inversion” indicates current training methods prioritize statistical pattern recognition over causal understanding and concept integration.

Implementing Concept Hacking Validation

“Concept Hacking” is a novel validation technique that exposes models to manipulated tasks. By altering task-specific details while maintaining irrelevant conditions, it distinguishes between true understanding and statistical associations.

Validation results show:

  • 68% of models achieve <30% accuracy in manipulated tasks but >70% in control tasks, indicating heavy reliance on statistical correlations.
  • Models often replicate reasoning patterns from control tasks in manipulated scenarios, ignoring critical task differences.

For example, in a perceptual constancy task, a model might correctly reason about a bridge appearing narrower in the distance but fail to adjust its understanding when the bridge is actually tapering.

Avoiding Common Pitfalls in Core Cognition Assessment

When assessing core cognition in MLLMs, avoid these common mistakes:

  1. Over-Optimizing Model Scale: Increasing parameters can worsen low-level task performance (e.g., a leading model saw a 14% drop in perspective-taking accuracy as parameters grew).
  2. Single-Dimensional Evaluation: Focusing solely on high-level tasks masks core cognitive defects (63% of models meet high-level task standards but fail low-level tasks with a 60% error rate).
  3. Ignoring Multi-Modal Integration: Text-only inputs reduce spatial cognition task accuracy by 58% compared to multi-modal approaches (combining images boosts accuracy from 36% to 72%).

Establishing Credibility and Authority

This research is conducted by a consortium of seven top universities, including the University of California San Diego and Johns Hopkins University. The study references over 200 cognitive development psychology papers and follows principles from Piaget and Spelke’s cognitive theories.

The research team includes ISO/TR 23788 content standard drafters, ensuring methodological rigor. Detailed datasets and assessment protocols are available at Multimodal AI Research Consortium.

Optimizing Your Content Strategy

To enhance your MLLM’s core cognition capabilities:

  • Prioritize Foundational Training: Implement stage-based training focusing on low-level cognitive modules first.
  • Enhance Multi-Modal Inputs: Use video inputs to strengthen continuity understanding (research shows a 29% average accuracy improvement).
  • Integrate Cognitive Constraints: Incorporate cognitive development sequence constraints into training objectives.

By following these strategies, researchers observed a 43% improvement in low-level task accuracy in models like InternVL series.

FAQ: Key Questions About Core Cognition in MLLMs

Q1: How can I determine if a model truly possesses core cognitive abilities?

A1: Use “Concept Hacking” validation. Compare model performance on manipulated and control tasks. A 20%+ accuracy difference indicates reliance on statistical associations rather than true understanding. This method exposed cognitive illusions in 73% of tested models.

Q2: Why doesn’t model scaling improve low-level cognitive abilities?

A2: Research shows low-level abilities (e.g., object permanence) correlate poorly ( <0.15) with model size, while high-level abilities correlate strongly (0.73). Current training focuses on statistical patterns, not causal reasoning.

About Multimodal AI Research Consortium

We are a global alliance of 15 leading research institutions dedicated to advancing multimodal AI. Our work is cited in academic publications and industry whitepapers. Learn more at our official website.

Data accurate as of May 24, 2025