Google Open-Sources Artemis: Let AI Operate Android Devices Directly for Automated Testing

Testing an app means tapping the phone repeatedly. Moving data across apps means switching back and forth manually. Writing an automation script that hits a custom-drawn UI fails outright, and you end up filling the gaps by hand. Google’s open-source Artemis takes a different approach. It lets AI assistants and test suites operate a real Android device the way a person would. Plug in a USB cable, enable USB debugging, and it can see the screen, swipe, tap, type, and switch apps. No root required.

Artemis demo

Live demo: Set a driving route in Google Maps and calculate total time, then open YouTube and play a Coldplay song.

What Is Artemis and What Operations Can It Handle?

It connects AI assistants and test suites to real Android devices. It handles perception and operations itself; it doesn’t organize test cases for you. The README lists several core capabilities.

Cross-app automation is the main thread. You give it a natural-language instruction, and it executes a test flow or daily task on Android. You don’t need to write a separate script for each app.

Element location works in two layers. For controls that expose an accessibility tree, it prioritizes element indexing. For custom-drawn UIs like Canvas, Compose, and Flutter, where the accessibility tree has no usable nodes, it falls back to coordinate and visual localization. The README summarizes this as “dynamic first, coordinates as fallback.”

It can also act as a standard MCP server and connect to Antigravity, Claude Code, Windsurf, and other development environments, driving test devices directly from the IDE and collecting Logcat output and screenshots. After you change code, you can tell it to build and install the app on the phone, run tests, and send screenshots back. No need to switch windows.

The web console offers another view. After startup, open http://localhost:8000 in a browser. The left side shows a real-time device mirror, and the right side has a task status and replay panel. You can issue test cases in natural language and watch reasoning steps and operation traces.

The tweet adds two boundaries. Banking and payment apps have strict risk controls and may trigger CAPTCHAs. WeChat automatic messaging is not supported yet. Keep that in mind before you run it.

Flash vs. Pro: Which Mode Should You Choose?

Simple tasks go to Flash; complex flows go to Pro. The differences in response speed, verification strength, and available tools determine when you must switch to Pro.

Flash uses an observe-execute loop with asynchronous history summarization. A single step usually takes 3 to 5 seconds. Its history is compressed and shares the same session ledger as Pro. Relative test time is recorded as T+mm:ss, screenshots are folded into visual summaries, and earlier steps are chunked and archived. When needed, use search_history or replay_steps to recall them, or call video_analyzer to analyze the entire session recording. For transient controls like auto-dismissing control bars and toasts, use click_sequence to chain multiple clicks into one atomic sequence. By default, there is no step limit. Setting agent.flash.max_turns to 0 means unlimited.

Flash’s limitations are stated plainly in the README. No task plan or notes, no pre-execution safety net, no checkpoint verification or final report, and no ADB commands. It suits routine, deterministic UI operations.

Pro is a different orchestration. A single step takes about 15 to 40 seconds and is orchestrated by a multi-agent graph. The Planner maintains a Markdown task plan with milestones and verify / assert check items. The Operator executes according to the plan and has the full toolset, including Explorer element location, notes, history recall, video analysis, and ADB diagnostics.

Explorer has three perception depths: flash, pro, and ultra. The README specifically notes that this is a user configuration set by run mode. Configure it in config/artemis.jsonc via pro.explorer.mode or flash.explorer_mode, or specify it with --explorer-pro-mode. The agent itself does not choose the depth.

Before a single action executes, it must pass Safety Net verification, with XML first and pixel fallback. Multiple actions in one round are sent as a fast-action burst to complete operations before transient controls disappear. When an action is blocked or fails, an execution incident opens and stays in the Operator’s context until a subsequent action succeeds. Recovery is handled by the Operator itself; there is no separate repair agent. A read-only Checker verifies checkpoints in the plan and performs a final review against the original goal before exit. --verification-level can be set to off / final (default) / checkpoints / strict. Pro supports long-horizon business flows of 100+ steps, [Loop:continuous] monitoring, and optional written reports.

Comparison	Flash	Pro
Single-step time	About 3 to 5 seconds	About 15 to 40 seconds
Orchestration	Reactive loop	Multi-agent graph
Task plan and notes	No	Planner maintains a Markdown plan
Pre-execution safety net	No	Safety Net verification
Checkpoint verification and final review	No	Checker verification
ADB commands	Cannot execute	Can execute
Use cases	Routine deterministic UI operations	Long-horizon complex business flows, stability testing

Complete Steps to Start Artemis from Scratch

As long as your computer is connected to an Android device with USB debugging enabled or an emulator, running the startup script once installs the dependencies. The one-click startup script automatically detects and installs ADB, scrcpy, FFmpeg, and the Python (uv) runtime and project dependencies. It also asks whether to mount the global MCP service and testing rules into your AI IDE.

On macOS and Linux, run these two commands.

git clone https://github.com/google/artemis.git && cd artemis
./start.sh

On Windows PowerShell, write them as two lines.

git clone https://github.com/google/artemis.git
cd artemis
.\start.bat

PowerShell does not look for executable scripts in the current directory by default, so you must write .\start.bat and do not add a backslash at the end. If you use the traditional Command Prompt (CMD), run start.bat instead.

After startup, the browser opens the web console automatically. If you prefer not to open a browser, you can run a task directly from the command line.

uv run artemis run "Open system settings, find the battery option, and tell me the current battery level" --profile flash

If you want to call the artemis command from any directory without uv run, run uv tool install -e . once in the project root.

When you first run the uv run artemis init configuration wizard, you can interactively complete the IDE’s MCP auto-mount instead of configuring it separately afterward.

How to Fill in the MCP Configuration for IDE Integration

Artemis has a built-in native Model Context Protocol service. If you have run the startup script, it will ask whether to mount it. You can also run the install command manually to mount it to one IDE or all supported IDEs.

# Install to Antigravity / Jetski
uv run artemis mcp --install antigravity

# Install to all supported AI IDEs, including Codex
uv run artemis mcp --install all

To configure manually, first generate the configuration template for your client.

uv run artemis mcp --generate-config codex
uv run artemis mcp --generate-config antigravity

Two parameters matter most. Set command to the absolute path of the Python interpreter in the project’s .venv virtual environment, and replace /path/to/artemis with the actual project path. Below are the formats for Codex and Antigravity.

Codex configuration goes in ~/.codex/config.toml.

[mcp_servers.artemis]
command = "/path/to/artemis/.venv/bin/python"
args = ["-m", "mcp_server"]
cwd = "/path/to/artemis"

[mcp_servers.artemis.env]
PYTHONUNBUFFERED = "1"
PYTHONPATH = "/path/to/artemis"

Antigravity configuration goes in ~/.gemini/jetski/mcp_config.json.

{
  "mcpServers": {
    "artemis": {
      "command": "/path/to/artemis/.venv/bin/python",
      "args": ["-m", "mcp_server"],
      "cwd": "/path/to/artemis",
      "env": {
        "PYTHONUNBUFFERED": "1"
      },
      "tools": {
        "mobile_run_task": { "eager": true },
        "mobile_manage_task": { "eager": true },
        "mobile_get_device_state": { "eager": true },
        "mobile_inspect_trace": { "eager": true },
        "mobile_diagnose": { "eager": true }
      }
    }
  }
}

Claude Desktop configuration goes in claude_desktop_config.json.

{
  "mcpServers": {
    "artemis": {
      "command": "/path/to/artemis/.venv/bin/python",
      "args": ["-m", "mcp_server"],
      "cwd": "/path/to/artemis"
    }
  }
}

Mounting MCP alone is not enough. The README specifically emphasizes mounting mcp_server/rules.md, which contains the Artemis mobile testing mindset guidelines. It covers runnable code principles and real-device exploration, Flash vs. Pro task routing strategy, latency and time compensation mechanisms, and the “dynamic first, coordinates as fallback” localization pattern. Once mounted, the AI programming assistant follows the mindset of a senior mobile test engineer and does not guess UI interactions out of thin air.

The mount location differs by IDE. For Antigravity, add it to workspace rules or global rules. For Claude Code, use artemis mcp --install claude to automatically install it to ~/.claude/rules/artemis.md. For Cursor, copy it to .cursorrules or create .cursor/rules/artemis.mdc. For Codex, add it to ~/.codex/AGENTS.md (or the currently effective AGENTS.override.md). For Windsurf and OpenClaw, add it to workspace rules or the global System Prompt.

There is a pitfall with Claude Code. It loads both ~/.claude/CLAUDE.md and ~/.claude/rules/*.md. Duplicate rule installation wastes context, so install to only one location.

After configuration, issue instructions directly in the IDE chat. For example, ask the assistant to compile the code you just changed into an APK, install it on the phone, open the login page, enter the test account, verify whether an abnormal popup appears after login, and send back a screenshot of the final page.

What Does Artemis Install on the Phone?

The first time it executes a task on a device, Artemis installs the Artemis Accessibility Helper. It is a small accessibility service that reads the screen layout and does not occupy the UiAutomation connection. The README explains that tools using UiAutomation need FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES enabled, otherwise the service may be paused.

A collapsed “Artemis test helper is running” notification appears on the phone, and a new entry appears under Settings > Accessibility. It only listens on the phone’s local loopback address and does not send any data out.

To install it in advance and avoid about 3 seconds of extra waiting on the first task, run the install command separately.

uv run artemis helper install

Check status with either of the following.

uv run artemis helper status
uv run artemis doctor

Remove it at any time.

uv run artemis helper uninstall

To switch to UIAutomator2, set ARTEMIS_HIERARCHY_BACKEND=uiautomator in .env. To prevent automatic helper installation, set ARTEMIS_HELPER_AUTO_INSTALL=false.

If the helper fails mid-task, Artemis falls back to UIAutomator2 and states this clearly in the task timeline, mobile_manage_task status, and final report.

Benchmark Results and Roadmap

Artemis achieves a task completion rate of over 99% on the AndroidWorld benchmark released by Google Research. The benchmark covers 20+ apps and 100+ multi-step tasks. The tweet adds that custom-drawn UIs, delayed popups, and animation stutters are all tolerated.

AndroidWorld benchmark comparison

The README’s architecture notes mention three things. For pre-execution verification and action sequences, Pro verifies the target via the real-time UI tree and pixels before dispatching a single action. Fast action sequences handle transient controls without waiting for the next model response. Element location combines the accessibility tree, OCR, and visual models to support custom-drawn UIs such as Canvas, Compose, and Flutter. For shared history compression, Flash and Pro replace earlier screenshots with visual summaries and compress completed steps into retrievable history chunks. A context threshold determines when to replace raw records.

ARTEMIS architecture diagram

The roadmap lists four items, none of which are done yet. Deep Android Studio integration, an official IDE plugin, and collaborative workflows that support automated testing, device interaction, and breakpoint debugging directly inside Android Studio. iOS cross-platform support, extending the visual perception and automation execution engine to real iOS devices and simulators. On-device lightweight models, supporting offline lightweight Edge VLM for low-latency and privacy-safe local automation. Real-time voice duplex interaction, supporting natural voice task dispatch and real-time barge-in control.

Artemis is open-sourced under the Apache License 2.0 and includes source code developed by Minitap, Inc. The tweet summarizes it as “completely free and open source, no ads, no payment.”

Quick Operation Checklist

Follow these steps to get it running. After installation, keep the commands handy for reference.

Enable USB debugging on your phone, connect it to your computer with a USB cable, or start an Android emulator.
On macOS or Linux, run git clone https://github.com/google/artemis.git && cd artemis, then ./start.sh.
On Windows PowerShell, run git clone, enter the directory, then run .\start.bat (do not add a backslash at the end).
The startup script automatically installs ADB, scrcpy, FFmpeg, Python, and project dependencies. Follow the prompts to decide whether to mount MCP and Rules.
Open http://localhost:8000 in a browser to see the mirror and console, or run tasks directly with the CLI.
To try a run from the command line, use uv run artemis run "task description" --profile flash or --profile pro.
To make the artemis command globally available, run uv tool install -e . in the project root.
To install MCP into an IDE, use uv run artemis mcp --install antigravity or --install all.
To install the helper in advance, use uv run artemis helper install. To check status, use uv run artemis doctor.
For service management, use uv run artemis restart, uv run artemis stop, and uv run artemis status.

FAQ

Does Artemis require root?

No. Plug in a USB cable, enable USB debugging, and you can use it.

How much time does Flash differ from Pro?

Flash takes about 3 to 5 seconds per step. Pro takes about 15 to 40 seconds per step.

Does Flash limit the number of steps by default?

No. Setting agent.flash.max_turns to 0 means unlimited, because history is compressed rather than truncated.

Who decides the three Explorer perception depths in Pro?

The user configures pro.explorer.mode or flash.explorer_mode in config/artemis.jsonc, or specifies it with --explorer-pro-mode. The agent itself does not choose the depth.

What is the “Artemis test helper is running” notification on the phone?

It is the Artemis Accessibility Helper, a small accessibility service that reads the screen layout and only listens on the local loopback address. It does not send data out.

What happens if the helper fails?

Artemis falls back to UIAutomator2 and states this in the task timeline, mobile_manage_task status, and final report.

Which scenarios are not supported yet?

Banking and payment apps have strict risk controls and may trigger CAPTCHAs. WeChat automatic messaging is not supported yet.

Why should Claude Code rules be installed in only one location?

Claude Code loads both ~/.claude/CLAUDE.md and ~/.claude/rules/*.md. Duplicate installation wastes context.

Google Artemis: AI-Powered Android Automation for Direct Device Testing