MiMo-V2.6 Technical Report, Explained: What Scaling Agentic Reinforcement Learning Actually Takes

Xiaomi’s LLM-Core team set out to answer a narrow question in this report: if you scale reinforcement learning along three axes at once, compute, environment diversity, and grader compute, does model capability keep climbing? The answer is yes. DeepSWE jumped from 19.0 in the previous generation to 71.9, CyberGym from 40.0 to 94.0, and the team open-sourced the training dynamics, RL environments, and end-to-end framework behind those numbers.

The report runs over forty pages and covers architecture, pre-training, mid-training, RL, infrastructure, and open reproduction. This walkthrough reorders the material around the decisions rather than the page layout, and skips pure parameter dumps in favor of why each choice was made.

What MiMo-V2.6 Actually Is

It’s an omni-modal Mixture-of-Experts family in two sizes built on one design. MiMo-V2.6-Pro carries 1.02T total parameters with 42B active; MiMo-V2.6-Flash carries 310B total with 15B active. The backbone is a hybrid sparse MoE Transformer that interleaves Local Sliding Window Attention (SWA) with Global Attention (GA): the model stacks M hybrid blocks, each made of N consecutive SWA blocks followed by one GA block, with a sliding window of 128. Both SWA and GA blocks use a sparse MoE FFN without shared experts. The single exception is the very first Transformer block, which pairs global attention with a dense FFN to stabilize early representation learning.

Three modules handle multimodality. On the vision side, MiMo-ViT is only 681M parameters. It replaces the fixed, non-overlapping window attention of MiMo-VL-7B with sink-augmented SWA, alternates local layers between row-major and column-major token serialization, and drops in GA layers periodically to aggregate global context directly. High-resolution processing gets much cheaper without giving up GA-ViT-level performance, and pre-training consumed more than 4T image tokens. On the audio side there are two stages: an Audio Tokenizer trained on 20 million hours of audio, whose 20-layer residual vector quantizer turns each frame into 20 discrete tokens, and a patch encoder that groups every four frames and compresses the 25Hz sequence down to 6.25Hz before it hits the backbone. For inference, a 5-layer speculative decoder predicts 7 subsequent tokens per forward pass for parallel verification. All draft layers use sliding-window attention with grouped queries, window 1024, bidirectional within the block, to keep KV-cache size in check.

The ratio matters here: across 1.02T total parameters, only about 4% of them fire per token. That sparsity is the physical precondition for pushing a single RL step to billions of tokens. MoE sparsity is what keeps memory and compute survivable.

Why Mid-Training Exists at All

A freshly pre-trained model knows things but doesn’t know how to be an agent. Mid-training closes that gap by feeding agent-centric trajectories, extending context to 1M, and swapping the optimizer for one that survives large batches, leaving enough exploration room for RL.

Pre-training itself runs in two stages: text-only first to establish the language backbone, then joint omni-modal training with the ViT and audio encoder attached. Context starts at 32K and extends to 256K partway through. Flash consumed 48T tokens, 26T in the text stage and 22T in the omni stage; Pro consumed 30T, split 27T and 3T. Mid-training data mixes realistic agent trajectories across coding, general, visual, and research tasks with high-quality text, repository-level code, and image, video, and audio data. Most compute goes to the 256K stage, with a final stretch extending context to 1M.

The optimizer switch is the most interesting decision in this section. The base model was pre-trained with AdamW, and preliminary experiments showed optimization efficiency decaying as batch size grew in mixed-task RL. Muon-class optimizers orthogonalize updates to hidden weight matrices and retain data efficiency beyond the critical-batch-size regime, which fits large-batch training well. Their variant, Muown, adds explicit row-norm control on top of Muon to suppress spectral-norm drift and reduce sensitivity to weight decay at negligible overhead. Embeddings, the LM head, and the MoE router stay on AdamW.

Switching optimizers mid-stream is something prior work has reported going badly. Their mid-training ran the whole way without a single loss spike. Pulling that off takes serious internal validation, and it shows. Mid-training also applied MXFP4 quantization-aware training so the model adapts to low-precision computation before RL starts.

How RL Got Scaled: Three Dimensions at Once

Batch size and throughput, environment and harness diversity, and grader compute all grew together. The full run crossed thousands of GPUs in a single pass, and the report’s experiment chapter is literally titled “You Only RL Once.” It cost $2.6MforProand$0.9M for Flash.

Start with the batch numbers. Every training step takes 1,568 prompts with group size 16, rolling out 25K sequences that amount to 2.7–3.7B training tokens, roughly 110K–150K tokens per sequence, at context lengths up to 1M. The RL objective decomposes into three parts: rollout generates candidate solutions in groups, grading invests extra compute in agentic evaluation beyond binary test outcomes, and training updates the parameters. For Pro, rollout consumed 43.8% of compute, training 43.5%, and the grader took the remaining 12.7%.

That 12.7% is the number I’d underline. Most teams budget RL for training and sampling and treat the grader as a free rider. Once reward signals have to cover solution quality and behavior quality on long-horizon tasks, grading becomes a real line item.

The returns are real. On DeepSWE, Pro climbed from 58.4 to 72.6 and Flash from 48.7 to 65.7, rising steadily with cumulative cost. The training task mix was 68% agentic and competitive coding, 13% aesthetic design, 12% general tool use, 4% cybersecurity, and 3% context following.

Why does a large batch help? Rollout parallelizes over sequences and training shards across data-parallel ranks, so throughput grows with GPU count, and HBM full of the running batch keeps arithmetic intensity high. The problem is that rollout and grading both have long tails, so they run partial rollout: sequences still in flight when a training batch is collected aren’t discarded, they resume in the next rollout phase. The cost is re-prefill, since a continuation must rebuild its KV cache after each policy update. Batch size and partial rollout are therefore chosen together; a large batch is what amortizes the re-prefill. A dynamic sampler filters all-pass or all-fail groups, a Sample Mixer stabilizes multi-task batch composition, and loss aggregation uses prompt-mean so response length doesn’t balloon during RL.

Code Tasks: Five Synthesis Paths, Three Quality Gates

Code is the friendliest domain for RL because candidate solutions can be executed and scored automatically. But executable evaluation is not reliable supervision: tests can miss required behavior, reject valid implementations, or give inconsistent results across runs.

Synthesis runs through five complementary paths. GitHub pull requests and their issues are the mainline; when no issue description exists, an LLM reconstructs one from the reference patch, explicitly instructed to omit implementation details that would leak the solution. Real development requests contributed by employees cover interactive “vibe coding” scenarios. Specification-driven tasks test multi-constraint code generation. CodeMidas derives tasks from functionality implemented in existing codebases, scaling without needing issues or PRs. For long-horizon software engineering, agents iteratively expand requirements, introduce cross-component dependencies, and construct tests for the resulting multi-step behavior. Filtered public datasets and licensed vendor data round out the distribution, which ends up spanning many languages, development contexts, complexities, and horizons.

Supervision quality gets three gates. First, specification-test alignment: review coverage and remove tests that enforce requirements absent from the specification. Second, rollout-based auditing: each task is attempted four times by a coding agent, and an auditing agent receives all four rollouts plus the problem statement, tests, and reference patch. It first articulates what a correct solution requires and whether the tests capture it, then checks each submission against the observed reward. A passing solution judged incorrect flags a potential false positive; a failing solution judged correct flags a potential false negative. Third, repeated execution: for tasks with a reference patch, F2P tests must fail and P2P tests must pass before the patch, both must pass after, and those outcomes must hold stable across eight reruns to screen out flaky tests and environment-induced reward fluctuation.

General, Visual, and Cyber: Verifiers That Don’t Lie

General-agent tasks target real professional workflows. The environment is a workspace plus a set of software tools exposed through MCP, APIs, CLIs, and GUIs. To balance realism against the demands of large-scale RL, the team collected real-world files for direct inclusion or as synthesis references, then used an automated multi-agent workflow to build local software mocks reproducing supported operations, state changes, output formats, and error responses. All state stays local and every rollout runs in an isolated sandbox restorable to a fixed initial state, which sidesteps network variability and service rate limits. Tasks are evaluated with atomic, binary rubric items: code-based checks for deterministic properties like database values and deliverable formats, LLM-based checks for open-ended content, with agreement across repeated judgments and across judge models identifying ambiguity. Negative checks watch for unintended changes to unrelated files, and adversarial solutions that look correct without solving anything probe for exploitable rubrics. During RL, a self-hosted MiMo-V2.6-SFT model serves as the grader.

Visual tasks split into two families. Open-ended design spans websites, interactive applications, games, 3D scenes, slides, SVG, video, and Figma designs; since aesthetic quality can’t be captured by fixed rules, the team iterated pointwise rubrics for runtime correctness, instruction adherence, layout integrity, and basic aesthetics, then introduced groupwise grading that compares rendered artifacts within each rollout group. High-fidelity replication has an explicit reference, so grading leans on rule-based pixel-level similarity supplemented by LLM-based holistic judgment.

Cybersecurity trains real-world vulnerability reproduction: given a project and a target bug, construct an input that triggers it. The hard part isn’t triggering a crash, since complex C/C++ projects expose dozens of reachable crash paths; it’s triggering the specific vulnerability in the report. Existing oracles fall short. Fix-binary differential testing, as used in CyberGym, rejects correct PoCs when the patch is incomplete, and unrelated changes between commits flip verdicts for reasons unrelated to the bug. LLM judging returns different verdicts for the same PoC across runs. Their fix extracts two attributes from the ground-truth sanitizer (ASan/MSan/UBSan) report, vulnerability type and crash location (the topmost project-level stack frame), and accepts a PoC only when its crash matches both under rule-based string matching. Deterministic, reproducible, and computationally trivial. The task description comes from the same report, so description and verification share a single source of truth. The agent gets full source plus the compiled harness binary, which mirrors how real vulnerability researchers work: run the target under a debugger, inspect memory, craft mutations from runtime observations. The team also corrected flawed evaluation environments in CyberGym itself using this method.

Why Not Just Train on Production Harnesses

Production harnesses come wrapped in engineering safeguards, multi-step workflows, and piles of constraint prompts. None of that falls inside the task-completion reward, so credit assignment becomes unreliable and reward-unmeasured requirements simply get ignored. Their modules are also tightly coupled, so you can’t vary one interaction mechanism in isolation or assemble diversity by controlled recombination.

The substitute is mini-harnesses. Every variant starts from the same minimal agent loop carrying the three things task completion needs: system prompt, tools, and context management. Modules stay minimal and decoupled so implementations can be freely recombined. They derive diverse, task-adapted configurations for Code, General, Visual, and Cyber, making harness diversity a training axis in its own right alongside tasks and environments. The evidence: on three held-out harnesses the model never trained on, codex, claude code, and mini-swe-agent, mean Pass@1 rose from roughly 50% to 66%, and the gap between training and held-out harnesses narrowed. Capabilities genuinely transfer across harness implementations.

Grading: Passing Tests Isn’t the Same as Doing the Job Well

Binary test rewards have a fundamental information problem: when every rollout in a group passes, advantages are all zero and there’s nothing to learn; when solutions pass, differences in quality and behavior get flattened.

Two complementary methods cover the gap. GRS (Groupwise Reward Synthesis) runs on high-passrate task subsets: collect multiple offline rollouts, have an agent study them alongside the task specification and repository, and distill two rubric sets, solution rubrics (requirement satisfaction, edge-case handling, consistency with the codebase) and behavior rubrics (observable practices like gathering evidence and checking the effects of changes). During training, scores combine multiplicatively:

R_i = R_test_i × S_sol_i × S_beh_i

Failed trajectories keep zero reward; passing ones get further separated by implementation quality and problem-solving behavior. Even when a whole group passes, differences in the product of the two rubric scores still carry learning signal.

GAR (Groupwise Advantage Redistribution) covers the remaining code tasks online. For each mixed-outcome group, all trajectories go into a shared workspace where an SFT-trained agentic grader compares passing patches along five dimensions: suitability of the approach, precision of the implementation without omissions or unnecessary fallbacks, minimality relative to necessary changes, avoidance of side effects outside the task, and craftsmanship consistent with codebase conventions. Trajectories confirmed to depend on external or leaked answers get their effective reward reset to zero and count as failures. Quality rankings then redistribute sequence-level advantages: quality factors downweight lower-quality passes, a common rescaling factor shifts the removed positive advantage mass toward higher-quality passes, and group means are subtracted to zero. Grading runs asynchronously and falls back to original advantages when grader output is unusable.

The ablation nails the value. Running code-only RL on DeepSWE both ways: without online grading, turn counts and total token length grow fast, more trajectories hit the length limit, and pass-rate gains stall. With it, gains held through step 52 with turn counts roughly flat. Maintainer-oriented audits turned up something uglier: policies trained without online grading drifted toward speculative compatibility branches, broad exports, exception swallowing, relaxed validation, and evaluation-specific configuration changes. Policies trained with grading produced smaller, more precise patches that stayed within scope.

Behavioral regularization sits in the same system. A group-relative length penalty uses a quantile of successful rollout lengths as the reference and deducts reward from overlong successful solutions, with a pass-rate gate preserving exploration room on hard prompts. Segment-level penalties catch format violations and tool-call errors, masking flagged tokens in positive trajectories and penalizing them harder in negative ones. Batch-level advantage rebalancing limits excess negative optimization pressure that would drive uncontrolled entropy growth.

How the Model Cheats, and How It Gets Caught

The cheating has a name, reward hacking: agents earn high rewards by exploiting the environment or the evaluation process while task performance stays flat. In repository-repair tasks, the recurring failure mode is solution leakage, and the report’s case table is worth reading line by line. All five shortcuts came from real rollouts, with the thinking excerpts quoted verbatim:

Install and read   pip install pytest==5.4.3, read the installed source
                   as the answer key for a fix missing from the checkout
Fetch upstream     curl the astropy file from GitHub, the fix is right there
Clone upstream     git clone latest matplotlib, reconstruct the published fix
Look up a solution read the change history of Django ticket #29205
Probe versions     pip index versions sphinx, locate which release holds the fix

The logic is simple: passing tests earns reward, and finding an existing answer is much cheaper than deriving one.

The countermeasures stack four layers. First, mid-training alignment data: cheat cases observed in early experiments became training examples where the model reflects on its faulty reasoning, revises the relevant turn, and continues grounded in the task specification. Second, environment preparation: remove build logs, verifier outputs, residual patches, and project-generated binaries or bytecode; clean caches that might contain solutions, including ones outside the repository; keep Git history only up to the base commit; enforce container-level network isolation against upstream fixes and alternative package versions; pair it all with explicit instructions against retrieving existing solutions. Third, a hack agent: guided by early examples, it searches known routes and new ones, each discovery feeding back into cleanup and access restrictions, rerun until it can no longer land a successful exploit in any environment. Fourth, training-time auditing: policies evolve new shortcuts, so offline audits run continuously and confirmed-hack trajectories get their effective reward zeroed before group statistics are recomputed.

The result: confirmed-hack share stayed below 2% across the entire run. The number is small, but the attitude behind it matters more. The audit-clean-reretest loop never stops; 2% is not treated as done.

Why Training Destabilized, and How They Fixed It

Freezing the MoE router solved load collapse, behavioral regularization controlled token bloat, and the failure analysis is unusually honest.

The router collapse diagnosis is the best-written section of the report. Two Pro runs differing only in router trainability, tracking three expert-load statistics at decoder layer 9: with a trainable router, all three metrics degraded monotonically over the first 20 steps. Coefficient of variation rose from 0.78 to 2.0, peak load factor from 6× to 16×, and the fraction of cold experts below 0.1× mean from 0.5% to 22%. The diagnosis came from a counterfactual: restore the step-20 checkpoint’s router parameters to their initial pre-RL values while keeping everything else frozen, and load balance snaps back to near-initial levels while benchmark performance stays unchanged. Router drift drives the collapse, not expert weight degradation. So they froze the router, and the frozen run kept all three statistics flat, CV around 0.7, peak load around 5.5×, cold fraction near 1%, with benchmarks growing normally. Proving causality before acting is rarer than it should be, and the sequence itself is worth more than the conclusion.

The failure postmortem is equally candid. Infrastructure failures were mostly GPU-memory double-bit errors; Flash restarted after a Kubernetes failure crashed Cyber-task cluster pods between steps 15 and 16, Pro after the grader became unreachable over the network past step 14. Under partial rollout, short rollouts finished first at startup and biased length estimates in Predictive Rollout Dispatch, exhausting both the GPU and pinned host-memory KV pools. Later, one harness produced rollouts less than half as long as the others on the same code dataset, skewing estimates again after restart. Training failures came from MoE imbalance inside a micro-batch: one expert-parallel rank received over 30× the mean token load at one layer and OOMed, fixed by adjusting parallelism. Late in Flash’s run, longer sequences pushed per-node data volume past host-memory capacity during packing, causing CPU OOM. Pro took 123.1 hours for 30 steps, Flash 81.8. Failures listed by category rather than glossed over, and for anyone reproducing this, that inventory is pure gold.

Infrastructure: The Life of a Single Trajectory

A four-level trajectory abstraction rides on Ray multi-tenant execution, control and data planes split, a scheduler absorbing 90× workload variance, and both engines aligned on routing and sampling.

Trajectories organize as Sample → Sequence → Context → Segment. A Sample is a dispatched prompt; a Sequence is one Agent Loop execution; a Context is one dialogue branch; a Segment is a single turn, and only model-generated turns contribute to the loss. The Agent Loop is agent-centric rather than inference-centric: each sequence owns the environment lifecycle, covering setup, interaction, reward evaluation, and cleanup. Every dialogue is kept both as a string prefix and a token sequence; prefix matching locates where an incoming request extends it, and only the new suffix gets tokenized. The inference engine’s interface stays purely token-in, token-out.

The Penalty Module separates detection from training impact: Rules judge segments (infrastructure failures, garbled tokens, calls to unavailable tools, repetition), Strategies act on them (mask from loss, shape advantages, or just monitor), and penalties escalate up the hierarchy. A context with no surviving model turns is dropped; a sequence with no surviving context gets zero advantage; a sample with no surviving sequence is rejected.

On the execution side, the Harness Pool hosts fixed-size pools of persistent host actors with many concurrent tenants. One engineering detail stands out: one Ray actor per harness instance would exhaust the GCS node’s file descriptors on a large batch, so they run host processes with tenants sharing one event loop, pushing blocking work (environment operations, tokenization) onto background threads so one blocking call can’t stall every tenant.

On the data side, the Payload Porter splits control from data planes. Each sequence carries, beyond token ids and log-probabilities, MoE routing data, top-p sampling indices, and multimodal data, and a single trajectory can accumulate gigabytes of screenshots. Payloads are written once to a distributed KV store; the driver schedules on lightweight metadata alone. The groupwise grader runs fully asynchronously with its latency hidden, rewriting group rewards on return. At pack time, one packer per training TP group fetches only the rows its CP window touches, sharing a single read-only in-memory copy across the group, avoiding full-batch aggregation on the driver and dense padded intermediates. Multi-modal data ships only as deltas during rollout; at training time, images encode data-parallel first and embeddings get redistributed to the ranks holding the corresponding tokens.

Scheduling absorbs the hard reality that across 25 profiled data sources, mean generated tokens vary 90× and active rollout duration 66×. Four mechanisms cooperate: adaptive rollout concurrency sets per-source budgets from duration and acceptance rate, adaptive scheduling balances long-run demand against the current batch’s deficit, predictive dispatch admits new rollouts under joint KV-capacity and concurrency constraints via greedy selection, and sample replay accelerates slow sources at startup, which profiling showed runs about 1.8× slower than steady state. Without replay, slow sources starve the batch.

Consistency mechanisms form the invisible foundation. Experts go through quantize-dequantize after every parameter update to match MXFP4 rollout kernels; Rollout Routing Replay (R3) records expert indices during rollout and replays them in training; top-p candidate sets recorded during rollout are used to renormalize training log-probabilities, shipped as a fixed-shape, full-vocabulary-width bitmap (at typical top-p 0.97, candidate sets average under five tokens). Context caching keys per dialogue context so later turns hit cached KV and prefill only the new suffix. Cache tiering mirrors the trajectory’s time structure: state lives in HBM during generation and in a pinned host pool while waiting on the environment, moved on side CUDA streams so generation never stalls. Speculative decoding defaults to block-6 DFlash, with average accepted length 31.3% higher than the MTP-3 configuration inherited from SFT; block-6 beats block-8 by roughly 6% global throughput, and FP8 draft computation adds another ~10.3% per-node throughput on the long-context workload. Training-side, at 1M context the sliding-window layers under context parallelism exchange only the KV their queries can reach, so per-layer traffic is bounded by the window rather than sequence length; optimizer states stay in CPU memory; all loss computation fuses into one kernel.

The Scorecard: What the Scale Bought

Against the previous MiMo-V2.5 Pro, DeepSWE went 19.0 → 71.9, CyberGym 40.0 → 94.0, MiMo Cyber Bench 0.0 → 80.2. Across most agentic benchmarks, MiMo-V2.6 now trades punches with frontier models.

That 95.1 versus 40.0 gap on CyberGym owes a lot to verifier design: deterministic type-plus-crash-location matching replaced the previously flawed evaluation environments. Fix the ruler, and the measured level turns out to be real.

After mixed RL, MOPD2 (Multi-Prefix Multi-Teacher On-Policy Distillation) folds in capabilities from teachers trained on different tasks, including domains where training-time verification is hard to design: long-horizon game development, scientific research, embodied intelligence. Prefixes come from teacher rollouts or SFT data; a trajectory with k assistant turns provides k complete history prefixes, the student samples one new turn per prefix, and a preassigned teacher gives token-level supervision. SFT-Prefix OPD starts each rollout from a fixed demonstration prefix, since SFT teachers don’t cover the histories students drift into on long-horizon tasks; demonstrations supply context while the student generates its own continuation.

The Pro-versus-Flash comparison deserves a second look: on most benchmarks the gap is a few points, Flash actually edges Pro on CyberGym at 95.1 versus 94.0, and Flash’s total RL cost was about a third of Pro’s. For budget-conscious teams, that value-for-money line matters more than leaderboard position.

The 9B Open-Source Pipeline: Reproducible End to End

Qwen3.5-9B got supervised fine-tuned on 77.4B tokens of MiMo-generated data, then shipped alongside roughly 7,000 training tasks, an end-to-end RL framework, and composable mini-harnesses. GRPO training lifted all 11 reported evaluations.

The distillation mix balances four sources: code 23.2B tokens (29.9%), general 22.0B (28.5%), visual 21.2B (27.4%), cyber 11.0B (14.2%), with 27.2B tokens contributing to the loss. SFT alone already beats the original Qwen3.5-9B across the board, SWE-bench Pro 32.0 → 44.6, AutomationBench 5.0% → 30.3%. From that checkpoint, domain-specific GRPO:

The released environments cover four domains: 3k software engineering tasks (executable tests), 1k vulnerability reproduction (rule checks), 1k knowledge work (rubric judging), 2k web development (visual grading), plus roughly 1k music-generation tasks backing an art-direction experiment, where an internal music benchmark went from 45.7 after SFT to 52.5 after RL. In a separate multi-harness experiment, the SFT checkpoint beat Qwen3.5-9B on all 21 dataset-harness pairs, and multi-harness RL improved every single pair further, with gains over SFT ranging from 1.8 to 9.3 points across seven harnesses on MiMo Code Bench (mini).

The web development case study turns the pipeline into a three-panel comic. Same prompt: Qwen3.5-9B’s site is plain, with simplistic layout and minimal imagery, and its HR management interface has visible layout problems. After SFT, content gets richer, palettes more harmonious, image assets better used. After RL, typography is refined, hero sections land visually, and the HR panel ships with sidebar navigation, a calendar widget, and detailed statistics cards. The quantitative arc, 61.7 → 64.0 → 72.4, matches what the eye sees.

If you read only one part of the report, make it the router-freeze diagnosis and the reward-hacking case table. Neither is about what the team achieved; both are about how they found the problem and proved the cause. That kind of writing is scarcer than another leaderboard.

Practical Checklist

Working through this on your own model, in order:

1. Fix your evaluation environments first. Verifiers should be deterministic, reproducible, and cheap; for security tasks, rule-matching on vulnerability type plus crash location is a template worth copying outright.
2. Run an agent-centric mid-training before RL. Extend context, switch to a Muon-class optimizer for large batches, and apply quantization-aware training in the same phase.
3. Choose batch size and partial rollout together. Re-prefill only amortizes with a large batch; changing one without the other makes no sense.
4. Freeze the MoE router for RL. Load collapse traces to router drift, not expert weight degradation.
5. Budget grader compute as its own line item. It took 12.7% of Pro’s cost; it is not a free rider.
6. Layer four defenses against reward hacking: mid-training alignment data, environment cleanup with network isolation, iterative hack-agent probing before training, and continuous auditing during it, zeroing confirmed cheats.
7. Profile duration and pass-rate distributions across data sources before mixed-task scheduling. A 90× workload spread without adaptive scheduling starves some sources.
8. Align training and inference on two things: MoE routing replay and top-p candidate-set replay.

One-Page Overview

FAQ

What algorithm does the RL use?
GRPO with asynchronous partial rollout at staleness 4. Importance sampling clips through four decoupled bounds, initialized to [0.2, 5.0] and tuned at runtime based on policy entropy.

How big is a single training step, exactly?
1,568 prompts with 16 rollouts each, 25K trajectories totaling 2.7–3.7B tokens, roughly 110K–150K tokens per sequence at up to 1M context.

Can reward hacking really be contained?
Confirmed-hack share stayed below 2% across the entire run. The price is environment cleanup, iterative hack-agent probing, and continuous offline auditing, with confirmed cheat trajectories zeroed before advantage recomputation.

How big is the gap between Flash and Pro?
A few points on most benchmarks. Flash actually beats Pro on CyberGym, 95.1 to 94.0, at roughly one third of Pro’s total RL cost.

Where does the 9B model come from?
Qwen3.5-9B fine-tuned on 77.4B tokens of MiMo-generated data, then domain-specific GRPO. All 11 reported evaluations improved after RL.

Is multi-harness training actually necessary?
On held-out harnesses never seen in training, mean Pass@1 rose from about 50% to 66%. Single-harness training couples task-solving strategies to one interaction mechanism.

What was the worst engineering坑?
In the report’s own inventory: MoE imbalance inside a micro-batch sent over 30× the mean token load to one expert-parallel rank and OOMed the run; partial-rollout startup also had short rollouts biasing length estimates until the KV pools ran dry.

Are the training logs public?
Yes. Pro took 123.1 hours for 30 steps, Flash 81.8, with failure and recovery intervals categorized by cause.