Step3: How a 321-Billion-Parameter Model Runs Cheaper Than a 37-Billion One

A Plain-English Guide for Developers, Students, and Curious Minds

Quick Takeaways

If you only remember three things, remember those.

1. What Exactly Is Step3?

Step3 is a vision-language model with 321 billion total parameters, but only 38 billion are active for each token.
Think of it like a huge library where the librarian only opens the exact shelves you need—so the place is massive, yet you pay only for the books you actually read.

1.1 Model Card at a Glance

2. The Two Secret Ingredients

2.1 MFA—Smarter Attention, Smaller Memory

Traditional attention keeps every Key-Value pair in memory, which explodes as your text gets longer.
MFA factorizes the Query matrix into two smaller matrices:

• Less KV to store (only 90 % of DeepSeek-V3’s KV budget).
• Fewer FLOPs per layer (¼ of GQA).
• Same expressive power—effective rank stays at 16 384.

Imagine compressing a 4 K photo into a 1 K thumbnail that still looks crisp.

2.2 AFD—Splitting Work Like a Factory Line

Most systems run Attention and Feed-Forward Network (FFN) on the same GPUs.
Step3 disaggregates them:

• Attention GPUs handle memory-heavy KV cache.
• FFN GPUs handle compute-heavy experts.
• Data flows over a fast network in a three-stage pipeline, so communication hides behind computation.

Result: each side runs at high hardware utilization instead of waiting for the other.

3. Cost Breakdown: Where Do the Savings Come From?

3.1 Memory Traffic (8 K context, per token)

Step3 already moves less data and does fewer math.

3.2 Price Tag (USD per 1 M tokens, 32 K context)

AFD lets us pick cheapest-fit hardware for each stage.

4. Hands-On: Running Step3 Yourself

4.1 Grab the Weights

• Hugging Face: stepfun-ai/step3
• ModelScope: mirror available
• Precision:

  • BF16 (universal)
  • Block-FP8 (smaller, needs Hopper/Ada)

4.2 Supported Engines

• vLLM
• SGLang

Both have ready-made Docker images and OpenAI-compatible REST endpoints.

4.3 Minimal Production Setup (32 K context)

Total: 32 GPUs instead of 128–320.

4.4 Quick Start with vLLM

# 1. Clone weights
git lfs install
git clone https://huggingface.co/stepfun-ai/step3

# 2. Start server
docker run --gpus all -p 8000:8000 \
  -v $(pwd)/step3:/model \
  vllm/vllm:latest \
  python -m vllm.entrypoints.openai.api_server \
  --model /model \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 4 \
  --max-model-len 32768

Client call (identical to OpenAI):

import openai
openai.api_base = "http://localhost:8000/v1"
openai.ChatCompletion.create(
  model="step3",
  messages=[{"role": "user", "content": "Explain quantum tunneling simply."}]
)

5. Performance in the Wild

Figures measured on identical hardware under 20 tokens/s latency SLA.

6. Common Questions

Q1: My lab only has A800 or 910B GPUs. Will it work?

Yes. Step3’s MFA is memory-bandwidth friendly:

• On A800, attention cost only rises 0.01 USD/1 M tokens.
• DeepSeek-V3’s MLA on A800 triples the cost.

Q2: Is the model truly open-source?

• Code: Apache 2.0
• Weights: Apache 2.0
• Commercial use: Allowed without restrictions.

Q3: How much VRAM do I really need?

Q4: What about training?

This article focuses on inference. Training costs follow classic scaling laws; Step3’s activated parameters (38 B) keep training within reach of most research groups.

7. Roadmap from the Authors

• Multi-Token Prediction (MTP): +50 % throughput on long context.
• 4-bit KV cache: Another halving of memory traffic.
• 800 Gbps domain networks: Unlocking sparser MoE in future releases.

8. When Should You Pick Step3?

9. References

• Technical report: arXiv:2507.19427
• Official blog: https://stepfun.ai/research/step3
• Weights & code: https://huggingface.co/collections/stepfun-ai/step3