Microsoft Open-Sources Orchard: A Standardized “Workspace” for AI Agents

AI can chat, write poems, and draw pictures. But ask it to actually operate software, navigate a browser, or submit a pull request on GitHub, and things get messy. The problem isn’t the model—it’s the environment. An agent needs a space where it can run commands, read and write files, and interact with a browser, all in a safe, isolated way. In the past, every new project meant rebuilding that space from scratch. Datasets weren’t reusable, training code wasn’t portable, and evaluation methods were tied to a specific setup. Switch research directions, and you threw everything away.

Microsoft’s newly open‑sourced Orchard framework aims to fix that.

Orchard provides a standardized backend environment service—Orchard Env. Think of it as a unified workshop where any agent can practice. All training data, training scripts, and evaluation pipelines run on the same base, so you don’t have to start over when you change projects.

I’ve gone through the paper and the technical docs, and this isn’t just a theoretical proposal. It already produces solid results: 73.0% pass rate on SWE‑bench Verified for software engineering tasks, 68.4% average on live‑web browser tasks, and for general computer use, swapping to a stronger harness lifts performance from 59.6% to 73.9%. Those numbers come from a reproducible, deployable system that you can use today.

This article breaks down what Orchard is, how to use it, and what you’ll run into when deploying and training with it. No fluff—just the practical details.

1. The Core Question: What Problem Does Orchard Actually Solve?

Straight answer: It turns the agent runtime environment into a standardized, massively scalable cloud service, so researchers can stop wrestling with infrastructure and focus on algorithms and models.

Why do we need this?

Training a “hands‑on” agent means letting it try things in an environment: run a command, read the output, decide the next action, repeat. Every trial is an interaction. If one interaction takes one second longer, a hundred thousand interactions cost you an extra 27 hours. If the environment crashes mid‑training, you lose all that compute. If data formats vary, you have to rewrite your data pipeline for every new task. These aren’t algorithmic problems—they’re engineering headaches.

Orchard Env standardizes that grunt work.

You deploy it on your own Kubernetes cluster, then call it via a Python SDK or REST API. Each create_sandbox() call spins up an isolated container in seconds. The container comes with built‑in capabilities for executing commands, reading/writing files, network access, and Git operations. The agent talks to the container over HTTP. When the task finishes, delete() cleans everything up.

One detail I appreciate: Orchard Env works with any Docker image as the base environment. Whether you want python:3.11‑slim, ubuntu:22.04, or a CUDA‑enabled deep learning image, it just works. How? It injects its own agent helper via an init container—that helper bundles a standalone Python interpreter, so the target image doesn’t even need Python. You don’t have to customize your images for Orchard.

2. The Three‑Layer Architecture

Orchard is split into three layers, each with a clear responsibility:

Bottom and middle layers are infrastructure; the top layer is research output. You can take just the environment service and run your own training code, or you can reproduce the published recipes. Both are open source.

3. Getting Started with Orchard Env

I’ve pulled the exact steps from the docs. Follow these and you should have your first sandbox running in about ten minutes.

Step 1: Install the Python package

pip install -e "orchard_env[dev]"

This installs the Python SDK. The [dev] flag adds development dependencies—omit it if you’re just using the client.

Step 2: Set environment variables

Tell the SDK where your Orchard service lives:

export SANDBOX_BASE_URL="http://your-orchestrator-host"
export SANDBOX_API_KEY="your-api-key"

Don’t have your own orchestrator yet? The docs include scripts to deploy one on Azure AKS—more on that later.

Step 3: Write Python code

from orchard_env import SandboxClient

with SandboxClient() as client:
    with client.create_sandbox("python:3.11-slim") as sandbox:
        result = sandbox.exec("echo 'Hello, Orchard!'")
        print(result.stdout)

What this does:

1. Connects to the Orchard service.
2. Creates a sandbox from python:3.11‑slim.
3. Runs echo 'Hello, Orchard!' inside it.
4. Prints the output, then automatically destroys the sandbox (thanks to the with statements).

Why use with? Manual cleanup is error‑prone—if an exception is thrown, delete() may never be called, causing resource leaks. Context managers guarantee cleanup whether the code succeeds or fails. I strongly recommend using them everywhere; don’t cut corners.

4. Deploying Your Own Orchard Cluster (Practical Steps)

The docs include complete deployment scripts for Azure AKS. The whole process takes about 20 minutes. I’ve broken it into four steps, each running one script.

Prerequisites

• An Azure subscription
• az CLI installed and logged in
• kubectl installed

Step‑by‑Step

1. Set environment variables

Define your resource group, cluster name, VM size, node count, etc. Defaults are provided; adjust as needed:

export RESOURCE_GROUP="orchard-rg"
export CLUSTER_NAME="orchard-aks"
export VM_SIZE="Standard_D4s_v3"   # 4 vCPU, 16 GB RAM
export NODE_COUNT="3"
export REDIS_SKU="Standard_B2s"

2. Run provision‑aks.sh

./provision-aks.sh

This creates the AKS cluster and a Redis instance (used for state storage).

3. Run build‑and‑push.sh

./build-and-push.sh

Builds the Orchard Env Docker images and pushes them to your Azure Container Registry. This takes a few minutes.

4. Run deploy‑orchard.sh

./deploy-orchard.sh

Deploys the service to your AKS cluster.

5. Verify

./test-deployment.sh

Runs smoke tests to confirm everything works.

If You’re Not on Azure

The repo provides manual Kubernetes YAML files under orchard_env/deploy/kubernetes/. Adjust StorageClass, Service types, and other parameters to match your cluster, then run kubectl apply -f.

Deployment Pitfalls to Watch For

• Redis is mandatory: Orchard Env uses Redis for distributed locks and state sync across replicas. Without it, the service won’t start.
• Registry permissions: build‑and‑push.sh pushes to ACR—make sure your az is logged into the right subscription and that the ACR exists.
• Network policies: The default Calico policy is deny‑egress—sandboxes can’t reach the internet. If your agent needs to download packages (e.g., pip install), you’ll need to whitelist egress rules. This is easy to overlook and can cause mysterious training failures.

5. The Three Published Recipes: Results and Data

The documentation details three completed research projects. Here are the hard numbers.

Orchard‑SWE (Software Engineering)

The cross‑harness number is the one that stands out to me. Many models score well on their training benchmark but collapse when the tooling changes. Orchard‑SWE keeps 45.0% with an unseen harness—it learned real problem‑solving, not muscle memory for a specific interface.

Orchard‑GUI (Browser Navigation)

Counter‑intuitive finding: with just 400 distilled trajectories for initialization, online RL pushed performance surprisingly high. Stable, realistic environments make online learning more data‑efficient than you might expect.

Orchard‑Claw (General Computer Use)

Among all models tested, Orchard‑Claw improved the most when switching to a stronger harness. That validates the core design: decouple the environment from the harness. What the model learns in one harness transfers to another.

6. Open Datasets: 107K SWE Trajectories + 3K GUI Trajectories

Microsoft also released the datasets generated on Orchard Env. They’re on Hugging Face under microsoft/Orchard.

SWE subset

• 107,185 multi‑turn trajectories
• 2,788 GitHub repositories
• 19,287 unique task instances
• Average 47.5 steps per trajectory
• Labels: 74,649 resolved, 32,536 unresolved

This is the largest open‑source, process‑annotated dataset for software engineering agents. It doesn’t just give you “question” and “final answer”—it records the entire solution path.

GUI subset

• 3,070 judge‑verified successful trajectories
• Each includes a screenshot (multimodal)
• Based on 409 WebVoyager‑style tasks

If you’re building multimodal browser agents, this is ready‑to‑use for SFT initialization.

7. The Road Ahead: Stateful Sandboxes

The docs mention a feature under development that I think deserves its own section: stateful sandboxes.

Currently, Orchard Env is linear: create sandbox, run N steps, destroy. With an average of 47.5 steps per trajectory and only a single success/fail signal at the end, you can’t tell which step made the difference. That’s the classic credit assignment problem in RL.

Planned capabilities:

• Pause / resume: Save the full sandbox state (filesystem, processes, environment variables) and restore later.
• Branching: Fork multiple continuations from the same checkpoint and explore different paths in parallel.
• Prefix sharing: Reuse common prefixes across branches to avoid redundant computation.

Once these land, you can branch at step 5, try ten different subsequent paths, and directly measure which actions actually mattered. This moves credit assignment from guessing to measurement. And because it lives in the environment layer, all training code benefits without modification.

My take: this could be a game‑changer for training efficiency and effectiveness.

8. Practical Checklist: Common Pitfalls

Based on what the docs reveal, here are the things that will trip you up if you’re not careful:

1. Clean up sandboxes: Always use with statements or try/finally. Leaked sandboxes accumulate and exhaust cluster resources.
2. Network isolation: Default is deny‑egress. If your agent needs pip install or apt‑get update, configure egress rules in NetworkPolicy.
3. API key hygiene: Don’t hard‑code keys. Use environment variables or a secret manager.
4. Base image size: Orchard works with any image, but larger images mean longer create_sandbox times. Prefer ‑slim or ‑alpine variants.
5. Harness consistency: The sandbox comes with multiple harnesses on PATH (codex, claude, pi, opencode, hermes). I recommend using the same harness for training and evaluation, or training on multiple harnesses to catch adaptation issues early.

Summary at a Glance

• What Orchard is: An open‑source framework for agentic modeling, built around a standardized environment service (Orchard Env) that lets agents run commands, read/write files, and operate browsers at scale.
• What it solves: Eliminates the engineering overhead of rebuilding environments for every project, so researchers focus on algorithms and models.
• Key results: 73.0% on SWE‑bench Verified, 68.4% average on live‑web benchmarks, and strong cross‑harness generalization.
• How to start: pip install, set two environment variables, and write a few lines of Python to launch a sandbox.
• How to deploy: One‑click scripts for Azure AKS; manual YAML for other Kubernetes clusters.

One‑Page Quick Reference

Frequently Asked Questions

Q: Does Orchard only run on Azure?
No. Azure AKS is just the platform with ready‑to‑run scripts. Orchard Env ships with Helm charts and manual YAML, so you can deploy it on any Kubernetes cluster—on‑prem or any cloud.

Q: I don’t do RL. Is Orchard still useful?
Absolutely. Orchard Env is a generic sandbox orchestration service. You can use it for batch evaluation, data collection, CI/CD pipelines, or any workload that needs isolated execution environments.

Q: How does Orchard compare to E2B or Modal?
Orchard is open‑source and fully self‑hostable – no vendor lock‑in. The docs show command latency of 0.28s vs. 0.747s for E2B and 2.046s for Modal. Cost‑wise, self‑hosted on spot instances can be ~10× cheaper than managed services.

Q: Why does the average trajectory have 47.5 steps?
That’s a long sequence, which makes credit assignment hard – a single reward at the end tells you very little about which step contributed. That’s exactly why Orchard is adding stateful sandboxes with branching, so each step’s value can be measured independently.

Q: I already have my own training framework. Can I still use Orchard Env?
Yes. Orchard Env exposes a REST API, not a framework‑specific plugin. Your training code just needs to replace the “execute command” action with an HTTP call. The Python SDK is a convenience, not a requirement.