ContextPilot Hands-on: Tencent’s Open-Source Solution for Long-Context AI Agents

Long-context agents have a dirty secret: bigger windows don’t always mean better results. Run a multi-hop retrieval or cross-document QA for dozens of turns, and you’ll watch the working context bloat into a chaotic mess. The model starts forgetting what matters while holding onto irrelevant noise.

Tencent just open-sourced ContextPilot to tackle this head-on. Instead of passively cramming more tokens into a fixed window, this framework teaches agents to actively manage their own context—deciding what to remember, what to forget, and when to reorganize.

The code is available on GitHub, and the paper is up on arXiv.

Why larger context windows don’t solve the real problem

Pushing context windows to 128K or even 1M tokens seems like the obvious fix. But in practice, long-horizon agentic tasks—like iterative retrieval, multi-step reasoning, and deep search—suffer from information clutter, not just capacity limits.

Every new turn adds more data while the agent tries to preserve earlier reasoning threads. With a massive window, you dump everything in and hope the model figures it out. With a smaller one, you’re forced to drop potentially useful content.

The typical toolset for managing this mess is thin: search, delete, and summarize. That works for simple scenarios but falls apart in long-running tasks.

ContextPilot takes a different route. It augments the agent with a full toolkit for context organization and uses reinforcement learning (RL) to teach the model how to use these tools effectively.

The toolkit: moving beyond search and delete

ContextPilot extends the standard toolset with three core capabilities.

Planning: The agent drafts a high-level outline before execution, breaking tasks into ordered steps. This plan stays in the context as a reference throughout the workflow.

Long-term memory: Critical information gets written into structured memory. Unlike working memory, this long-term store is queryable—the model can retrieve specific entries on demand.

Soft context offloading: Instead of permanently deleting content that’s temporarily irrelevant, the agent archives it to an external “drawer.” The data can be reloaded when needed, so nothing is truly lost.

These additions turn the context from a linear text stream into a dynamic workspace. The agent can decide what stays on the desktop, what goes into long-term archives, and what lands on the to-do list.

RL training: focusing on critical actions and per-step credit

Equipping the toolkit is only half the battle. Teaching the agent how to use it is where ContextPilot’s training design comes in.

1. Deep exploration only on critical editing actions

During training, the system evaluates which context-editing operations have the highest impact on the final outcome. Deleting a seemingly irrelevant detail that later supports reasoning—or storing a key fact at the right moment—matters more than routine operations.

ContextPilot lets the model explore multiple choices for these high-impact actions while skipping redundant exploration for mundane edits.

2. Per-action credit assignment

Standard RL assigns a single reward to the entire trajectory. If the task succeeds, everything gets credit; if it fails, everything gets blamed. But a long task may involve dozens of context edits, with only two or three making the real difference.

ContextPilot traces backward from the final outcome to evaluate each intermediate edit’s contribution to all subsequent branch trajectories. Useful edits get positive rewards; harmful ones get penalized. This fine-grained approach helps the model understand exactly which actions work and which don’t.

3. Identifying key nodes with context change and entropy variation

How does the system know which actions deserve deep exploration? It tracks two signals: the magnitude of context change and the variation in the model’s output entropy.

If an edit significantly alters the subsequent reasoning path—or shifts the model from uncertainty to certainty—that node gets flagged for deeper training.

This makes intuitive sense: the points where the context changes the most are usually where the task stands or falls.

Performance: smaller window, better results

Tencent trained Qwen3-14B with ContextPilot. The results are worth looking at closely.

With a 32K context window, the model outperformed the base Qwen3-14B running on its native 128K window. Cutting the window to a quarter of its original size actually improved long-context task performance. At the same time, the actual token consumption dropped significantly.

The evaluation covers four benchmarks: the longbook_choice_eng split of InfBench, NovelQA, LongMemEval, and BrowseComp+. These span long-document QA and deep-search scenarios. Detailed numbers are in the paper.

Setting up the inference environment

ContextPilot splits into inference and training components. Both are open-sourced.

Install the inference dependencies first:

bash infer/scripts/setup_environment.sh
source infer/.venv/bin/activate

The searchEngine tool depends on Elasticsearch. The setup script installs a local Elasticsearch distribution. To run the service separately:

bash infer/scripts/start_elasticsearch.sh

Running evaluation

The evaluation suite runs all four benchmarks in one go:

bash infer/scripts/run_full_pipeline.sh /path/to/checkpoint my-run

Results are written to infer/results/, including predictions, full trajectories, and scores.

Individual benchmarks can be executed separately:

bash infer/scripts/eval_infbench.sh /path/to/checkpoint my-run
bash infer/scripts/eval_novelqa.sh /path/to/checkpoint my-run
bash infer/scripts/eval_longmemeval.sh /path/to/checkpoint my-run
bash infer/scripts/eval_browsecomp_plus.sh /path/to/checkpoint my-run

Training your own model

The RL implementation builds on verl and includes context-aware partial rollout and per-action credit assignment. Training scripts live in the train/ directory.

For Qwen3-8B:

cd train

TRAIN_FILE=/path/to/train.parquet \
VAL_FILE=/path/to/validation.parquet \
MODEL_PATH=/path/to/qwen3-8b \
GPUS_PER_NODE=8 \
bash sh/run_qwen3-8b_longbenchv2.sh

For Qwen3-14B, use sh/run_qwen3-14b_longbenchv2.sh.

Important configuration notes

Git LFS: LongMemEval and BrowseComp+ data files are managed with Git LFS. After cloning, run:

git lfs install && git lfs pull

NovelQA access: NovelQA answer annotations cannot be redistributed. You need to request full access from the NovelQA dataset page on Hugging Face.

Judge endpoint: LongMemEval and BrowseComp+ scoring uses an OpenAI-compatible judge service. Store the endpoint configuration in a JSON file and set:

export JUDGE_OPENAI_FILE=/path/to/judge-endpoint.json

Check infer/README.md for the exact configuration format.

Quick setup checklist

  • [ ] Clone the repository: git clone https://github.com/Tencent/ContextPilot.git
  • [ ] Install Git LFS and pull data files: git lfs install && git lfs pull
  • [ ] Run the inference setup script: bash infer/scripts/setup_environment.sh
  • [ ] Activate the virtual environment: source infer/.venv/bin/activate
  • [ ] (Optional) Start Elasticsearch: bash infer/scripts/start_elasticsearch.sh
  • [ ] Configure the judge endpoint and set JUDGE_OPENAI_FILE
  • [ ] (For NovelQA) Request dataset access from Hugging Face
  • [ ] Run the full evaluation pipeline or individual task scripts
  • [ ] (For training) Navigate to train/, set data paths, and run the training script

FAQ

Q: How is ContextPilot different from just using a larger context window?

A: Larger windows increase capacity passively. ContextPilot teaches the model to actively organize and manage what’s already there. The result: a 32K window with ContextPilot outperforms the same base model with a native 128K window.

Q: What framework does the training component rely on?

A: Training is built on verl, which handles the RL infrastructure.

Q: Is Elasticsearch mandatory?

A: Only for the searchEngine tool. The setup script installs a local distribution, or you can start it separately with start_elasticsearch.sh.

Q: Do I need to configure the judge service?

A: Only for LongMemEval and BrowseComp+ evaluation. InfBench and NovelQA don’t require it.

Q: How do I get NovelQA data?

A: Request full access from the NovelQA dataset page on Hugging Face. The annotations aren’t redistributable.

Q: Which base models are supported?

A: The provided training scripts target Qwen3-8B and Qwen3-14B.

Q: Where are the evaluation results stored?

A: Everything goes to infer/results/, with separate files for predictions, trajectories, and scores.