Understanding AI Agent Terminology: Harness, Scaffold, and What They Really Mean

Core question this article answers: In the fast-moving field of AI agents, what do “harness” and “scaffold” actually mean? How do they relate to the model and the agent itself? And why does getting these terms right matter for building real systems?

If you’re new to AI agents—or even if you’re already using tools like Claude Code or Codex—you’ve probably run into a wall of confusing terms. At ICLR 2026, researchers were still asking: “What do you mean by ‘harness’ and ‘scaffold’? I’ve heard a dozen different explanations.”

This confusion isn’t accidental. When a field moves as fast as AI agents, vocabulary often outpaces shared understanding. Terms get reused in different contexts, become shorthand for half‑explained ideas, or disappear after a few months.

This glossary isn’t meant to be the final word. Instead, it gives you a practical mental model—one that works whether you’re building, deploying, or just using agents. We’ll start with the most basic piece (the model), then build up through scaffolding, harness, and finally the agent itself. After that, we’ll cover context engineering, policy, tool use, skills, sub‑agents, and the key terms used in training (RL environment, trainer, rollout, reward).

A quick reflection
When I first started building agents, I spent all my time on model selection and prompt tuning. The real breakthroughs came only after I understood that the harness—the execution loop—was just as important as the model. This glossary is what I wish I’d had on day one.

Table of Contents

1. Model: The starting point, but it doesn’t do anything by itself

Core question: Is a raw LLM already an agent?

No. A pure LLM (Claude, Qwen, GPT, DeepSeek, etc.) is a text generator: you give it input, it produces output. It has no memory between calls, no loop, no ability to execute tools on its own. It can express the intent to use a tool (for example, by outputting a structured JSON block), but it cannot act on that intent.

A model answers one prompt and then stops. Wrap it in scaffolding and a harness, and it becomes an agent.

Real‑world analogy
Think of the model as a brilliant intern who knows everything but only works from a single note. You hand them a question, they write an answer, and then they stop. They won’t look anything up, run a command, or ask a clarifying question unless you give them a process (scaffolding) and a workspace (harness).

Example
In Claude Code, the underlying Claude model generates code and reasoning. By itself, the model cannot even list the files in your current directory. That ability comes from the scaffolding and harness that surround it.

2. Scaffolding: The model’s behavior manual

Core question: What exactly does scaffolding include, and how is it different from a harness?

Scaffolding is the behavior‑defining layer around the model. It includes:

  • System prompt (role, boundaries, persona)
  • Tool descriptions (what each tool does, what arguments it expects)
  • Parsing rules (how to extract tool calls from the model’s output)
  • Context management (what the model remembers across steps)

Scaffolding shapes how the model sees the world and how it acts, whether you’re training or running inference.

Products like Claude Code, Codex, and Antigravity CLI often call the whole non‑model part a harness. Claude Code’s own documentation says: “Claude Code serves as the agentic harness around Claude.” That broad usage is fine most of the time. The scaffold/harness distinction becomes important when you need to reason about them separately—for example, in a training pipeline where you might change the scaffolding without changing the execution logic.

Example from a customer support agent

  • System prompt: “You are a polite, professional support agent. Only answer questions about warranty. For anything else, guide the user to the website.”
  • Tool descriptions: search_kb(query) – search the knowledge base; create_ticket(title, desc) – create a support ticket.
  • Parsing rule: when the model outputs <tool_call>search_kb('warranty period')</tool_call>, the harness extracts the tool name and argument.

Reflection
In my first agent, I focused almost entirely on the model and the prompt. The agent was still brittle. Only after I added proper parsing and error handling did it become reliable. A good scaffold can make an average model behave like an expert; a bad scaffold can make a great model useless.

3. Harness: The engine that makes the model act

Core question: What specific execution tasks does the harness handle, and why is harness engineering a distinct discipline?

If scaffolding is the what and how (instructions, format), the harness is the do: it runs the loop that turns a model into an agent. The harness:

  • Calls the model with the current context
  • Handles tool calls returned by the model (executes functions, makes API requests)
  • Feeds tool results back to the model
  • Decides when to stop (task complete, max steps reached, error state)

Harness engineering is the discipline of designing this layer well. It applies to both training and inference. Key decisions include: stop conditions, error recovery, guardrails, and state management.

Real example: Cursor’s iterative harness
Cursor’s team wrote about how they continually improve their agent harness. In early versions, if the model requested to open a file that didn’t exist, the harness simply returned “file not found.” The model might try the same wrong action repeatedly. The improved harness now detects repeated errors, suggests alternatives, and stops after too many failures.

Eval harness (evaluation harness)
At evaluation time, the same pattern appears as an eval harness. Instead of collecting training data, it runs a fixed set of scenarios against a model checkpoint and records metrics. The canonical example is EleutherAI’s lm-evaluation-harness.

Scenario comparison
Same model (GPT‑4) + two different harnesses:

  • Harness A: max 10 steps, stop on any error, no retry
  • Harness B: max 30 steps, automatic retries (2 attempts), intermediate state saving

The two agents will behave completely differently. Product experience is often determined more by the harness than by the model.

4. Agent: Model + harness as a complete system

Core question: What is the most widely accepted formula for an agent in the LLM community?

The term agent comes from reinforcement learning: an agent is a function that takes an observation and returns an action. The environment takes the action and returns a new observation, and the loop repeats. LLM agents work the same way.

In the LLM world, the definition has expanded: an agent is the model plus everything around it that lets it act, not just respond. The community has largely converged on Agent = Model + Harness (see discussions from @Vtrivedy10 and Will Brown). If it’s not the model, it’s part of the harness.

Diagram (mental model)

Agent
├── Harness (execution layer)
│   ├── Model invocation
│   ├── Tool execution
│   ├── Loop control
│   └── Error handling
└── Scaffolding (behavior definition)
    ├── System prompt
    ├── Tool descriptions
    ├── Parsing rules
    └── Context management

Product examples

  • Claude Code: harness tightly coupled to Claude, optimised for file reading, command execution, and code editing.
  • Cursor: harness integrated with the editor UI (e.g., one‑click diffs).
  • Hermes Agent: lets you plug in any model (local or cloud) with a generic harness.

Key insight: two products using the same underlying model can feel completely different because their harnesses make different choices. And swapping a better model into the same harness also changes the experience. The model, the harness, and the product are three different things.

Reflection
I used to think “agent framework” meant LangChain or AutoGPT. But a real agent is the runtime entity (model + harness). Frameworks are just tools to build the harness. Once I separated these concerns, debugging became much easier: is it a scaffolding problem (bad prompt) or a harness problem (broken loop logic)?

5. Context Engineering: The agent’s eyes and memory

Core question: What exactly does context engineering manage, and how are short‑term and long‑term memory different?

Context engineering is designing what goes into the model’s context window at each step: system prompt, tool descriptions, conversation history, retrieved knowledge. It’s not a one‑time decision. As the agent runs, previous turns shape future calls, and the harness actively manages this throughout the run.

Short‑term vs long‑term memory

  • Short‑term memory: what stays in the context window during a single run: conversation history, tool results, previous reasoning. It disappears when the run ends or when the context window fills up.
  • Long‑term memory: persists across sessions, stored externally (vector database, relational database), retrieved on demand, and injected back into the context when relevant.

Example: a code review agent

  • Short‑term memory: files already reviewed in this session, errors found so far.
  • Long‑term memory: team coding preferences, past bug fix patterns. The agent retrieves these before starting the review and adds them to the system prompt.

Different costs in training vs inference
At inference, changing the context is just editing text: you can redeploy immediately. At training, the context the model sees shapes what gets learned. Get it wrong and you have to retrain—much more expensive.

Reflection
“Context engineering” sounds fancy. In practice, it often means: what do I put in the prompt, in what order, and what do I drop when the window is full? Adding a simple monitor for context length and an auto‑truncation strategy will save you more headaches than any advanced technique.

6. Policy: The agent’s decision rule

Core question: Is policy the same as an agent?

No. A policy is the behavior an agent follows: given any situation, it defines the probability of taking each possible action. In LLM systems, part of that policy is learned in the model weights, but it also depends on the scaffolding and harness. The same model can behave very differently depending on prompts, tools, memory, and the execution loop.

A policy is not an agent. The policy defines behavior; the agent is the full system that acts in an environment. You deploy an agent and observe its policy.

Example
A math problem solver agent. With a system prompt that says “reason step by step, then give the answer,” the policy is cautious and deliberative. Change the prompt to “guess the answer immediately,” and the policy becomes reckless—even though the underlying model weights haven’t changed.

7. Tool Use and Skills: From single actions to reusable capabilities

Core question: What is the essential difference between a tool and a skill, and why does it matter?

Tool: a single, callable function or API. Examples: run_command(cmd), http_request(url), query_database(sql). The model expresses intent to use a tool in a structured format; the harness executes it and feeds the result back.

Skill: a reusable, structured package of knowledge that enables multi‑step tasks. Where a tool is an action (“run this command”), a skill bundles everything needed to accomplish a goal (“investigate this bug, form a hypothesis, write a fix, verify”).

Example from customer support

  • Tools: get_order_status(order_id), refund(order_id), send_email(to, subject, body)
  • Skill: handle_refund_request(order_id, reason) – internally checks order status, verifies eligibility, executes refund, sends confirmation email. This skill can be reused across different agents.

Different frameworks draw the line between tool, skill, and sub‑agent differently. Focus on the engineering concept, not the label.

8. Sub‑agents: Divide and conquer

Core question: How is a sub‑agent fundamentally different from a regular tool, and when should you use one?

A sub‑agent is an agent called by another agent to handle a specific subtask. It has its own model and scaffolding, reasons independently, uses tools, and returns a result. The calling agent doesn’t need to know how it works internally.

Sub‑agent vs tool vs skill

  • Tool: synchronous function call, stateless, returns quickly.
  • Skill: a packaged workflow, but still interpreted by the main agent’s harness (no independent “brain”).
  • Sub‑agent: has its own model, context, and execution loop. It can handle multi‑step reasoning and unexpected situations.

Example
A main agent responsible for “develop a new feature.” It delegates “write unit tests” to a dedicated test sub‑agent. That sub‑agent analyzes the code, generates tests, runs them, and iterates based on failures—all autonomously. It returns only the passing test code to the main agent.

Why use sub‑agents

  • Encapsulation: the main agent’s prompt doesn’t need to include testing knowledge.
  • Independent context: the sub‑agent’s conversation history doesn’t pollute the main agent’s context.
  • Reusability: the same test sub‑agent can be used by many main agents.

Reflection
I first thought sub‑agents were over‑engineering. Then I tried to stuff a 20‑step task into a single agent—context exploded, errors accumulated. Splitting the task into a main agent and a few sub‑agents improved success dramatically. The trade‑off is communication overhead and harder debugging. Use sub‑agents only when the subtask has clear boundaries and requires its own multi‑step reasoning.

9. Training terms: RL Environment, Trainer, Rollout, and Reward

Core question: What are the core components needed to train an RL agent, and what does each one do?

These terms are specific to training (updating model weights), not just deployment. Every RL training system for LLMs uses the same basic pipeline.

RL Environment

The environment is anything the agent can interact with: a stateful object that takes an action, updates its internal state, and returns an observation. In LLM contexts, actions are typically tool calls. A simple example: a filesystem. The action touch foo.txt updates the state (creates the file), and the observation might be the updated file listing.

A dedicated guide on RL environments exists (referenced in the original material), so we won’t compress it further here.

Trainer

The trainer is what makes the agent better. It runs many agent episodes, scores the results, and uses those scores to update the inner model’s weights. TRL’s GRPOTrainer is a concrete example: a single class that handles episode generation, reward scoring, and weight updates.

Rollout

A rollout is one full agent run from start to finish: what the agent saw, what it did, and what reward it got at each step. It’s also called a trajectory or a trace. This is the raw data RL algorithms learn from.

Reward and Rubrics

The reward is the score that tells the training algorithm whether the model is improving. It can be:

  • Verifiable: tests pass/fail, answer matches a known correct answer.
  • Learned: from human preferences or an LLM acting as a judge.
  • Sparse: one score at the end of an episode.
  • Dense: a score at each step.

Rubrics break the reward into explicit dimensions with weights, rather than a single number. For example, a code generation agent’s rubric might be: correctness (60%), code style (20%), performance (20%). Libraries like OpenEnv and Verifiers provide composable objects (WeightedSum, Sequential, Gate) to build rubrics.

Example from training a shopping assistant

  • Sparse reward: did the agent successfully complete an order?
  • Dense reward: +1 for each correct product recommendation, -1 for each misleading suggestion.
  • Verifiable reward: order total matches cart, address format is valid.
  • Learned reward: user thumbs‑up/down after the interaction.

10. Practical summary and checklist

Core question: What should an engineer or product manager take away from this article?

Key conclusions

  1. Model ≠ Agent. A model is a text generator; an agent is model + scaffolding + harness.
  2. Scaffolding defines behavior (prompts, tool descriptions, parsing); harness executes (calls model, runs tools, controls the loop).
  3. The same model with different harnesses can produce wildly different products.
  4. Context engineering is a continuous process. Separate short‑term memory (in‑context) from long‑term memory (external storage).
  5. Training an agent requires understanding Environment, Trainer, Rollout, and Reward.

Actionable checklist for a new agent project

  • [ ] Choose your model(s) – one foundation model? multiple? local or API?
  • [ ] Design scaffolding

    • [ ] Write system prompt (role, boundaries, prohibited actions)
    • [ ] List all tools with natural language descriptions
    • [ ] Define model output format (JSON, XML, function call)
  • [ ] Implement harness

    • [ ] Function to call the model
    • [ ] Parser to extract tool calls
    • [ ] Tool executor
    • [ ] Loop stop conditions (max steps, completion keyword, error)
    • [ ] Error handling and retry strategy
  • [ ] Context management

    • [ ] What history to keep? Truncation strategy when window fills?
    • [ ] Long‑term memory? How to retrieve and inject?
  • [ ] Evaluation / training (if needed)

    • [ ] Build an eval harness with fixed scenarios
    • [ ] Define reward (verifiable / learned / sparse / dense)
    • [ ] Choose a trainer (e.g., TRL, OpenEnv)

11. One‑page reference table

Term One‑sentence definition Analogy
Model Text generator, no memory, no loop A brilliant scholar who never acts
Scaffolding Behavior‑defining layer: prompts, tool descriptions, parsing rules Employee manual + workflow
Harness Execution layer: calls model, runs tools, controls the loop Office, computer, ticket system
Agent Model + harness as a complete system A researcher who actually works
Context engineering Managing what goes into the context window (short‑term + long‑term memory) What’s on the whiteboard vs what’s in the archive
Policy The agent’s behavior pattern, shaped by weights + scaffolding Work style
Tool Single action function A wrench or screwdriver
Skill Packaged multi‑step knowledge “Change a tire” standard procedure
Sub‑agent Independent agent called to handle a subtask Outsourcing to a specialist
RL Environment Stateful object that takes actions, returns observations A game world
Trainer Runs rollouts, computes rewards, updates weights Coach + gym
Rollout One complete agent run (trajectory) A chess game replay
Reward Score that tells the training algorithm if the agent is improving Scoreboard

12. FAQ (Frequently Asked Questions)

Q1: What is the exact relationship between harness and agent?
A: Agent = model + harness. The harness is the execution engine inside the agent that calls the model, handles tool calls, and controls the loop. Without the harness, the model is just a one‑shot text generator.

Q2: Where is the boundary between scaffolding and harness?
A: Scaffolding defines what to do and in what format (prompts, tool descriptions, parsing rules). The harness does the actual execution (calling the model, running functions, deciding when to stop). In product documentation, they are often lumped together as “harness.” The distinction matters most in training pipelines.

Q3: Can I take Claude Code’s harness and use it with a different model?
A: Some products (like Hermes Agent) allow model swapping, but Claude Code’s harness is tightly coupled to Claude. You could theoretically implement a compatibility layer, but it’s non‑trivial because output formats and tool‑calling schemas differ across models.

Q4: How do I implement short‑term and long‑term memory in practice?
A: Short‑term memory is simply the conversation history kept in the context window (e.g., a list of messages). Long‑term memory requires an external store (vector database, key‑value store). At runtime, retrieve relevant items (e.g., by semantic similarity) and inject them into the system prompt or user message.

Q5: When should I use a sub‑agent instead of a regular tool?
A: Use a sub‑agent when the subtask requires multi‑step reasoning, may encounter unexpected situations, and can be cleanly isolated from the main agent’s context. If the subtask is a single function call, stick with a tool.

Q6: How should I design rewards when training an agent?
A: Start with verifiable rewards (e.g., test pass/fail). If the task is open‑ended, consider learned rewards (LLM as judge). Use rubrics to break the reward into multiple weighted dimensions—this makes debugging much easier.

Q7: Do I have to train my own model to build a useful agent?
A: No. Most production agents use off‑the‑shelf models (Claude, GPT, Qwen) with carefully designed scaffolding and harness. Training (fine‑tuning or RL) is something you do after you already have a working pipeline and need to improve performance on a specific domain.

Q8: Will these terms ever converge?
A: Over time, core terms will become more stable, but different frameworks and products will always have their own nuances. The practical advice: agree on a shared vocabulary inside your team, and understand the engineering concepts behind the labels rather than memorising definitions.

If any definition feels imprecise or you’ve encountered a term we missed, we’d love to hear from you.

Thanks to the original authors and reviewers for building a shared reference for the community.