How We Slashed AI Coding Costs by 52% Per Session While Scaling Usage 7x at Uber

By August 2026, our weekly active users for AI coding tools grew 7x, and agentic requests grew 9.4x. Yet, our total AI spend has been flat since April.

The core strategy wasn’t negotiating lower API prices. It was eliminating zero-value token consumption while maintaining output quality. Here is exactly how we measured and optimized every layer of our AI Software Factory.

The Four Layers of Our AI Software Factory

We organize AI usage into four distinct layers. The higher the layer, the more control we have over cost, quality, and model selection.

Layer 1: Specialized Agents
These handle specific tasks: code review (uReview), self-healing CI failures, on-call alert triage, bug debugging, and maintenance tasks. Each agent focuses on one job, making benchmarking precise and model selection straightforward.

Layer 2: Generalist Agents
Tools like Claude Code fall here. Engineers trigger these for broader, less-defined tasks.

Layer 3: Context Retrieval
This layer handles information fetching across our codebase, data tables, logs, and monitoring systems. We built an AI Context Graph to power this—more on that below.

Layer 4: Base Model APIs
The raw foundational models.

Every session across these layers is broken down by the same cost formula, which gave us our optimization roadmap.

The Cost Equation: Six Numbers That Tell the Whole Story

We decomposed the cost of any agentic session into these six multiplying terms:

Here’s what each means:

• Weekly Active Users × Sessions per User: We want these to grow. This is healthy adoption.
• Requests per Session × Tokens per Request: This is the “work” the agent does on behalf of the user. This is where most of our optimization effort goes.
• Price per Token: Set by vendors, but we control which model runs which workload.

If usage grows, spend grows—that’s expected. But if Tokens per Request or Requests per Session balloons, that’s waste. We engineered our way out of it.

The Metrics We Track Weekly

Here is the full table of metrics we monitor to forecast short-term capacity and long-term infrastructure needs.

The rule is simple: isolate each variable. If “Requests per Session” spikes, we investigate agent looping. If “Tokens per Request” jumps, we check for context bloat from tool schemas.

Our Optimization Levers

We attacked three multipliers simultaneously: Price per Token, Tokens per Request, and Requests per Turn. Here is the playbook for each.

1. Optimizing Price per Token (Model Selection)

Vendors set the price. We pick which model runs which job. We define “Pareto-efficient” as the model that gives the best balance of cost per completed task, output quality, and reliability.

Our Four-Step Selection Process
For every managed agent we deploy:

1. Build a benchmark using the agent’s real-world work (e.g., actual PRs with known bugs).
2. Run the agent on a harness that supports any frontier or open-weight model via a unified interface.
3. Find the configuration on the Pareto frontier.
4. Keep moving—the frontier shifts every few weeks.

Case Study: uReview (AI Code Review)

We benchmarked uReview using thousands of real PRs from our monorepos, grading bugs as easy, medium, or hard. We scored precision, recall, F1, and tracked cost per review, latency, and timeouts.

The dashed line is the Pareto frontier. Every dot below or left of it is beaten by something cheaper or better. By switching models, we improved F1 while dramatically cutting cost per PR.

Default Model Routing Strategy

We default to weaker, cheaper models for subagents. Why? Subagents perform narrowly defined tasks with specific inputs—they don’t need frontier-level reasoning for most jobs. The primary model handles decomposition and evaluation while subagents execute.

This single default (setting subagent models to the cost-effective tier) was our most impactful lever. As model capabilities improve, multi-agent orchestration has only grown, amplifying these savings.

2. Optimizing Tokens per Request

Every turn re-sends the full conversation history, project context, and tool results. Anything you cut from the per-request payload compounds multiplicatively across the session.

Two Baseline Configuration Rules

We applied these defaults across all interactive harnesses:

• Automatic compaction at 400k tokens, even for 1M context window models. This balances performance against cache burst costs. Fleet-wide input tokens per request dropped meaningfully.
• Reasoning effort set to Medium. Output tokens (including internal reasoning) are billed at multiples of input token rates. Medium hits the sweet spot for a broad category of tasks.

Our Prompt Caching Strategy

Since every turn re-transmits history, caching the prefix turns subsequent reads into just 0.1x the standard input token rate. However, writing to cache has premiums: 5-minute TTL costs 1.25x, while 1-hour costs 2x.

Engineers often leave interactive sessions idle for more than 5 minutes (checking docs, responding to Slack). When we defaulted to 5-minute TTL, those idle gaps invalidated caches, forcing full-price rebuilds. We switched interactive sessions to 1-hour TTL and kept subagents at 5-minute TTL (since they run short, continuous tasks).

MCP Context Bloat: Moving Tools from Context to CLI

Our internal MCP gateway routes over 1,000 MCP servers (internal and third-party SaaS).

Standard MCP loads every tool schema into the session context. With 100+ tools installed, this added 50k–70k tokens of schema overhead to the initial prompt—re-sent on every turn.

Solution 1: CLI Tool Resolution
Instead of direct MCP integration, we let the model execute shell commands. The CLI resolves the tool against the gateway dynamically at call time. Result: MCP schemas are entirely eliminated from the session context. All 1,000+ MCP tools are projected as CLI commands.

Solution 2: Tool Search
For scaling to thousands of tools, we let the model search the tool catalog and load only what it needs on demand. This maintains high selection accuracy while preventing context degradation.

Code-Mode: Removing the Model from the Polling Loop

When tools act as shell commands, models can batch multiple actions into a single script. This is a game-changer for “chatty” protocols (e.g., SQL queries requiring submit, poll 2-5 times, fetch results).

Standard MCP: The model participates in every step, loading each polling response into context.

Code-Mode: The polling loop runs in a subprocess. Only the final summary comes back to the model.

We ran 5 identical SQL queries through both paths in the same session:

Even for small result sets, Code-Mode cut token usage by >50%. The savings aren’t from bypassing large payloads—they come from eliminating schema init, multi-turn polling, and redundant step-by-step reasoning.

For bulk workflows, where N model turns become one script, savings compound to >90%. We’ve deployed 25+ pre-built Code-Mode skills for our most accessed MCP servers.

Tackling SaaS MCPs

Third-party vendors ship massive tool bundles. A workspace suite bundles 49 tools (~22k tokens of schema). A messaging tool ships 34, project management ships 46. Loading just two vendor servers adds more schema overhead than the file being edited.

We route every SaaS MCP through our gateway, expose them as CLIs, and author dedicated Code-Mode skills for common workflows. This unifies access and cuts waste.

3. Optimizing Requests per Turn

An ungrounded agent fails slowly. It keeps sending expanding context windows to “search one more location.” Providing rich upfront context is the most powerful lever to reduce this.

The AI Context Graph

Uber has hundreds of millions of lines of code and thousands of data tables. Agents were spending most of their turns locating information instead of generating code.

We built the AI Context Graph: a unified network with 24 million nodes and 80 million edges across 86 node types and 117 edge types. It ingests 30+ internal systems (services, teams, incident logs, PRs, design docs, deployments, datasets, historical query logs). Agents can query it in natural language.

The Difference is Night and Day

• Grounded Agent: Queried historical usage, identified a table used by 50+ analysts, and returned the answer in 38 seconds.
• Ungrounded Agent: Couldn’t see that table. Spent 20 minutes inspecting service code, spawning 2 subagents, hitting 3 errors, and concluding the dataset was unqueryable.

38 seconds vs. 20 minutes. That’s what happens when you feed the agent the right information on turn one.

Visibility and Education: The Human Feedback Loop

You can’t optimize what you can’t see. We built tools to make every session cost visible and actionable.

The Live Status Line

Every interactive terminal shows a live cost counter for the current session and cumulative usage across all harnesses for that user.

Progressive Spend Tiers (No Hard Caps)

Instead of rigid quotas, we implemented real-time tracking and automated nudges:

• Live counter in the status line.
• One shared tier pool across all interactive harnesses (not per-tool budgets). Separate pools for managed agents.
• Slack nudges at 50%/80%/100% of expected spend.
• Manager approval flows for tier upgrades with instant propagation.

Session Analysis Dashboard

The status line shows total cost, but not why it’s high. The session analysis dashboard inspects session artifacts directly. With zero setup, it flags 16 distinct anti-patterns across sessions, pairing each with its financial impact and remediation.

Example flags:

• Suboptimal model routing: Running easy multi-turn sessions on Opus when Sonnet would suffice.
• Context bloat: Large MCP payloads (e.g., 40KB responses) persisting across turns.
• Cache expiration: Resuming sessions after long breaks, forcing full-price rebuilds.
• Initialization overhead: Pre-loading 100k tokens of system instructions before the user types anything.

What’s Next on Our Roadmap

We are not stopping here. Current initiatives include:

• Growing the managed agent fleet: Every new SDLC agent follows the same roadmap—target metrics, benchmark assembly, Pareto-optimal model selection.
• Dynamic Model Routing: Expanding benchmark coverage across languages, repos, and modalities to feed smarter routing decisions.
• Deeper Context-Graph Integration: Unlocking graph queries for a wider array of autonomous agents.
• Real-Time Developer Guidance: Shifting from batch detection of anti-patterns to continuous, in-terminal recommendations.
• Automated Skill Improvement: Recording “papercuts” from agent executions and auto-generating skill updates from the collected traces.

Quick Operational Checklist

• [ ] Do you know the actual Pareto-efficient model for your specific tasks, not just benchmark scores?
• [ ] Have you moved tool schemas and system instructions out of your per-request context?
• [ ] Is your cache TTL aligned with your users’ actual idle gaps, not just set to maximum?
• [ ] Are you feeding your agents rich context upfront (code graphs, table schemas) so they don’t waste turns searching?
• [ ] Is every session’s cost visible to the engineer running it?

Frequently Asked Questions

Q: How much of the savings came from switching to cheaper models?
Some, but that’s not the headline. We focus on “good enough” for the job. The real savings came from cutting context bloat and reducing unnecessary turns—a multiplicative effect.

Q: What exactly is Code-Mode, and can anyone use it?
Code-Mode moves multi-step polling operations into a subprocess script and returns only the result to the model. We’ve pre-built this for 25+ internal MCP operations. Users call these skills directly without writing scripts.

Q: How do you choose between 5-minute and 1-hour cache TTL?
Look at your users’ typical idle time between requests. If they often step away for a few minutes, use 1-hour. For continuous automation (subagents), 5-minutes is fine. We use both configurations depending on the workload.

Q: What can the Context Graph actually query?
Code, services, tables, teams, incident records, PRs, design docs, deployments, and historical query logs. Agents query it using natural language.

Q: Is the Session Analysis Dashboard opt-in?
No. It’s built directly into the runtime with zero setup required. Just run the cost dashboard skill to see your traces and waste markers.

Q: Are these optimizations applicable to small teams or open-source projects?
The specific tools are Uber-scale, but the methodology is universal: decompose your cost equation, measure each variable independently, attack the largest waste sources, and make costs visible. This works whether you have 10 engineers or 10,000.