Odyssey: Empowering Minecraft Agents with Open-World Skills

The Revolutionary Breakthrough in Minecraft AI Agents

Imagine an AI agent that autonomously explores Minecraft worlds, crafts diamond swords, battles monsters, and manages farms – no longer science fiction! The Odyssey Framework developed by Zhejiang University’s VIPA Lab makes this reality possible. This groundbreaking technology equips Minecraft agents with true open-world survival capabilities. In this comprehensive analysis, we’ll explore this cutting-edge innovation.

“

📌 Core Value: Odyssey solves the limitations of existing Minecraft agents that can only perform basic tasks (like collecting materials) through three key innovations enabling authentic open-world interactions.

Comprehensive Technical Architecture

Three Core Technical Pillars

1. Open-World Skill Library

   • 40 primitive skills (mining/building etc.)
   • 183 compositional skills (e.g., “craft weapons → combat”)
   • Sample skill chain: Gather wood → Craft workbench → Make wooden pickaxe → Mine stone → Create stone sword

2. Specialized Minecraft Knowledge Engine

   • Fine-tuned LLaMA-3 based model
   • Trained on 390,000+ Minecraft Wiki Q&A entries
   • Answers complex queries like “How to find fortress remnants in the Nether?”

3. Tri-Dimensional Capability Benchmark

   Task Type	Evaluation Focus	Example Challenge
   Long-Term Planning	Multi-stage goal management	Mine diamonds starting from scratch
   Dynamic Planning	Real-time response capability	Handle multiple monsters simultaneously
   Autonomous Exploration	Environmental cognition	Discover resources in unknown regions

Real-World Demonstrations

Mining Diamonds from Scratch

Weapon Crafting and Combat

Farm Management

Quick Start Guide

4-Step Environment Setup

# 1. Install Python dependencies
cd Odyssey
pip install -e .
pip install -r requirements.txt

# 2. Configure Node.js environment
npm install -g yarn
cd Odyssey/odyssey/env/mineflayer
yarn install

# 3. Deploy Minecraft server (Docker recommended)
# See official docs: /Odyssey/docs/run_using_docker.md

# 4. Download embedding model
git lfs install
git clone https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2.git

Configuration Template

Create config.json with these key parameters:

{
  "server_host": "LLaMa_service_IP",
  "server_port": "LLaMa_service_port",
  "NODE_SERVER_PORT": "Node_service_port",
  "SENTENT_EMBEDDING_DIR": "embedding_model_path",
  "MC_SERVER_HOST": "Minecraft_server_IP",
  "MC_SERVER_PORT": "Minecraft_server_port"
}

Four Practical Task Implementations

1. Subgoal Chaining Task

def test_subgoal():
    odyssey = Odyssey(environment='subgoal')
    test_sub_goals = ["Craft crafting table", "Craft wooden pickaxe", "Craft stone pickaxe", "Craft iron pickaxe", "Mine diamond"]
    odyssey.inference_sub_goal(sub_goals=test_sub_goals)

2. Long-Term Planning (Combat)

def test_combat():
    odyssey = Odyssey(environment='combat')
    combat_tasks = ["1 enderman", "3 zombies", "1 zombie + 1 skeleton"]
    for task in combat_tasks:
        odyssey.inference(task=task)

3. Dynamic Planning (Farming)

def test_farming():
    odyssey = Odyssey(environment='farming')
    farming_tasks = [
        "Shear 1 sheep for wool",
        "Collect 1 bucket of milk",
        "Cook 1 piece of meat",
        "Collect and plant 1 seed"
    ]
    for task in farming_tasks:
        odyssey.learn(goals=task)

4. Autonomous Exploration

def explore():
    odyssey = Odyssey(environment='explore')
    odyssey.learn()  # Fully autonomous exploration mode

Technology Evolution Timeline

Academic Impact

25 Related Studies advancing the field:

1. MineRL (IJCAI 2019) – First large-scale dataset
2. Voyager (arXiv 2023) – Open exploration pioneer
3. JARVIS-1 (arXiv 2023) – Multi-task processing breakthrough
4. VillagerAgent (ACL 2024) – Multi-agent collaboration
5. GROOT-2 (arXiv 2024) – Weakly supervised solution

“

💡 Note: Complete reference list available in GitHub’s Related Works section

Resource Access

All components are open-source:

• Codebase: GitHub Repository (MIT License)
• Dataset: HuggingFace Repository (CC BY-NC-SA 3.0)
• Model Weights: Included in code repository

Frequently Asked Questions

Q: What hardware configuration is required?

A: Recommended setup:

• GPU: ≥12GB VRAM (for LLaMA-3 8B model)
• RAM: ≥32GB
• Storage: ≥50GB (including Minecraft server)

Q: Can I create custom skills?

A: Yes! Modify JSON files in the skill_library directory:

{
  "Skill name": "Tame wolf",
  "Prerequisites": ["Raw meat", "Bone"],
  "Action sequence": ["Find wolf pack", "Hold bone", "Right-click wolf"]
}

Q: Key differences from Voyager framework?

A: Three core distinctions:

1. Odyssey’s skill library is 3x larger
2. Dedicated Minecraft knowledge model (not general-purpose LLM)
3. Dynamic environment response (e.g., sudden monster spawns)

Conclusion and Future Directions

Odyssey represents a major breakthrough in Minecraft agent research, enabling long-term survival in complex open worlds for the first time. The team plans:

1. Skill library expansion to 300+ actions
2. Multi-agent collaboration ecosystem
3. Real-world physics environment migration

“

Project Authors: Shunyu Liu, Yaoru Li et al. (Zhejiang University VIPA Lab)
Citation:

@inproceedings{liu2025odyssey,
  title={Odyssey: Empowering Minecraft Agents with Open-World Skills},
  author={Shunyu Liu et al.},
  booktitle={IJCAI},
  year={2025}
}