Nested Learning: A New Machine Learning Paradigm for Continual Learning

The past decade has witnessed remarkable advancements in the field of machine learning (ML), driven primarily by powerful neural network architectures and the algorithms used to train them. Yet, despite the impressive capabilities of large language models (LLMs), several fundamental challenges persist—particularly in the realm of continual learning. This critical capability refers to a model’s ability to actively acquire new knowledge and skills over time without forgetting what it has already learned.

Why Is Continual Learning So Important for AI?

When it comes to continual learning and self-improvement, the human brain remains the gold standard. It adapts through neuroplasticity—the extraordinary capacity to restructure itself in response to new experiences, memories, and learning. Without this ability, humans would be confined to immediate contexts (a condition similar to anterograde amnesia).

Current LLMs face a similar limitation: their knowledge is restricted either to the immediate context within their input window or the static information they acquire during pre-training.

A straightforward approach to addressing this—continually updating a model’s parameters with new data—often leads to “catastrophic forgetting.” This phenomenon occurs when learning new tasks comes at the cost of diminished proficiency in previously mastered ones. Researchers have traditionally tackled catastrophic forgetting through architectural adjustments or improved optimization rules. However, for far too long, we have treated a model’s architecture (its network structure) and its optimization algorithm (its training rules) as two separate entities. This division has prevented the development of truly unified, efficient learning systems.

In 2025, at the Neural Information Processing Systems (NeurIPS 2025) conference, a paper titled “Nested Learning: The Illusion of Deep Learning Architectures” introduced the “Nested Learning” paradigm, offering a fresh solution to this challenge. By breaking down the barrier between architecture and optimization, Nested Learning reimagines a single ML model as a system of interconnected, multi-level learning problems—all optimized simultaneously.

What Is the Nested Learning Paradigm?

Nested Learning reveals a key insight: complex ML models are not monolithic processes but rather collections of coherent, interconnected optimization problems that either nest within one another or operate in parallel. Each of these internal problems possesses its own “context flow”—a unique set of information from which it seeks to learn.

This perspective suggests that existing deep learning methods work essentially by “compressing” these internal context flows. More importantly, Nested Learning uncovers a new dimension for model design, enabling the creation of learning components with greater computational depth.

To illustrate this with a real-world analogy: consider learning to cook. You must remember ingredient pairings (long-term knowledge), adjust seasoning based on real-time heat levels (immediate information), and refine techniques through repeated trial and error (mid-term experience). These different levels of learning occur simultaneously and influence one another—mirroring the “multi-level optimization problems” at the core of Nested Learning.

Understanding Nested Learning Through Associative Memory

Associative memory, a concept from psychology, refers to the ability to map and recall one piece of information using another (e.g., remembering a name when you see a face). Nested Learning leverages the lens of associative memory to reinterpret core mechanisms of existing deep learning:

• The training process itself—specifically backpropagation—can be modeled as a form of associative memory. The model learns to map a given data point to its local error value, which serves as a measure of how “surprising” or unexpected that data point is. The more a data point contradicts the model’s existing knowledge, the higher the error.
• As demonstrated in previous research (e.g., the Miras paper), key architectural components—such as the attention mechanism in Transformers—can also be formally defined as simple associative memory modules. Their role is to learn the mappings between tokens in a sequence.

By defining an update frequency (i.e., how often each component’s weights are adjusted), we can order these interconnected optimization problems into “levels.” This ordered set forms the heart of the Nested Learning paradigm.

How Does Nested Learning Improve Existing Technologies?

The Nested Learning framework provides principled guidelines for enhancing existing algorithms and architectures, with two key applications:

1. Deep Optimizers

Nested Learning treats optimizers (e.g., momentum-based optimizers) as associative memory modules, allowing us to apply principles from associative memory to their design.

Many standard optimizers rely on simple dot-product similarity—a measure of how alike two vectors are, calculated by summing the products of their corresponding components. However, this update method fails to account for relationships between different data samples.

Nested Learning proposes redefining the optimizer’s underlying objective to use more standard loss metrics, such as L2 regression loss. This common loss function in regression tasks quantifies error by summing the squares of the differences between predicted and true values. By making this shift, we can derive new formulations for core concepts like momentum, rendering optimizers more resilient to imperfect or noisy data.

In simple terms, just as humans remember not only individual facts but also the connections between them, deep optimizers enable models to consider relationships between data points when updating parameters. This reduces learning biases caused by incomplete or noisy data.

2. Continuum Memory Systems (CMS)

In a standard Transformer, the sequence model acts as short-term memory, holding immediate context, while feedforward neural networks serve as long-term memory, storing pre-trained knowledge.

Nested Learning expands this concept into a “Continuum Memory System (CMS),” where memory is viewed as a spectrum of modules—each updating at a distinct, specific frequency. This design creates a richer, more effective memory system tailored for continual learning.

For example:

• Some modules update every 10 data points (handling short-term fluctuations),
• Others update only after 1,000 data points (preserving long-term patterns),
• Additional modules operate at intermediate frequencies (capturing mid-term trends).

This multi-scale update mechanism allows models to store and utilize information across different timeframes—much like the human brain.

Hope: A Self-Modifying Architecture Powered by Nested Learning

As a proof of concept, researchers designed “Hope”—a self-modifying architecture with continuum memory—based on the Titans architecture.

Titans architectures are long-term memory modules that prioritize memories based on their “surprisingness” (i.e., how much a piece of information deviates from the model’s existing knowledge). Despite their strong memory management capabilities, Titans architectures have only two parameter update levels, limiting them to first-order in-context learning.

Hope, by contrast, is a self-modifying recurrent architecture that leverages unbounded levels of in-context learning. Augmented with CMS blocks, it can scale to larger context windows. Essentially, Hope optimizes its own memory through a self-referential process, creating an architecture with infinite, looped learning levels.

Experiments and Results for Hope

Researchers conducted experiments to evaluate the effectiveness of deep optimizers and Hope’s performance across four key tasks: language modeling, long-context reasoning, continual learning, and knowledge incorporation (full results are available in the original paper). The findings validate the power of Nested Learning, continuum memory systems, and self-modifying Titans architectures:

• On a diverse set of commonly used public language modeling and common-sense reasoning tasks, Hope achieved lower perplexity (a metric for evaluating language model performance—lower values indicate better predictive accuracy) and higher overall accuracy compared to modern recurrent models and standard Transformers.
• In long-context Needle-In-Haystack (NIAH) downstream tasks, Hope demonstrated superior memory management. This proves that CMS offers a more efficient and effective approach to handling extended sequences of information.

The Significance and Future of Nested Learning

The Nested Learning paradigm represents a significant step forward in our understanding of deep learning. By treating architecture and optimization as a unified, coherent system of nested optimization problems, it unlocks a new dimension for model design—one where multiple levels can be stacked to build more powerful systems.

成果 like Hope demonstrate that a principled approach to unifying architecture and optimization can yield more expressive, capable, and efficient learning algorithms.

This paradigm provides a robust foundation for closing the gap between the limited, forgetful nature of current LLMs and the remarkable continual learning abilities of the human brain. Moving forward, the research community is encouraged to explore this new dimension and collaborate on building the next generation of self-improving AI systems.

Frequently Asked Questions (FAQ) About Nested Learning

What is the difference between Nested Learning and traditional deep learning?

Traditional deep learning treats model architecture (network structure) and optimization algorithms (training rules) as separate entities. In contrast, Nested Learning recognizes that these are fundamentally the same concept—just different “levels” of optimization, each with its own context flow and update frequency. This unification creates a more integrated, efficient system.

Why is catastrophic forgetting a major problem for continual learning?

Catastrophic forgetting occurs when a model loses proficiency in old tasks as it learns new ones, preventing it from accumulating knowledge over time. For example, a model trained first to recognize cats and then to recognize dogs might struggle to identify cats after learning the second task. This is a critical barrier for AI systems that need to adapt and learn continuously—and Nested Learning addresses it by enabling multi-level optimization that preserves old knowledge while integrating new information.

Does the Continuum Memory System (CMS) resemble human memory?

Yes. Human memory operates across multiple scales: short-term memory (e.g., remembering a recent conversation), mid-term memory (e.g., recalling details from a meeting yesterday), and long-term memory (e.g., remembering childhood experiences). Each scale has its own “update” and “forgetting” frequency. CMS mirrors this by using modules that update at different rates, simulating the multi-scale nature of human memory.

Why can the Hope architecture handle longer contexts?

Hope incorporates Continuum Memory System (CMS) blocks, which stratify memory modules by update frequency. Different levels of modules process information of varying context lengths. Additionally, Hope’s self-modifying capability allows it to dynamically adjust its memory management strategies, making it more adaptable to ultra-long sequences than traditional models.

What other fields can Nested Learning be applied to?

Based on current research, Nested Learning has shown promise in language modeling, common-sense reasoning, and long-context memory tasks. In the future, it could benefit any field requiring continual learning or multi-scale information processing, such as robotics control, personalized recommendations, and decision-making in dynamic environments.