Agentic Vision in Gemini 3 Flash: How Visual Reasoning and Code Execution Redefine Image Understanding

In the rapidly evolving field of artificial intelligence, particularly within large vision models, we have long faced a fundamental challenge: models typically process the world in a single, static glance. They act like a casual observer scanning a photograph; if they miss a fine-grained detail—such as a serial number on a microchip, a distant street sign, or a specific line in a complex blueprint—they are forced to guess.

This “one-shot” processing method often reveals its limitations when faced with tasks requiring extreme precision and complex logical reasoning. However, with the release of Gemini 3 Flash, a new capability known as Agentic Vision is transforming this landscape. It is no longer just about “looking” at an image; it is about “investigating” it.

This article delves into the workings of Agentic Vision, its core mechanisms, practical real-world applications, and how the integration of code execution turns image understanding from a static perception into a dynamic, evidence-based reasoning process.

What is Agentic Vision?

Agentic Vision is a frontier capability introduced in Gemini 3 Flash that fundamentally changes how the model handles and interprets images. While traditional vision models operate largely as passive observers, Agentic Vision transforms visual processing into an active, agentic process.

From Static Perception to Active Investigation

Conventional models process an image’s pixel data in a single pass. While efficient, this approach has limitations when dealing with high-resolution images or capturing minute details. If critical information occupies a small portion of the image, the model is likely to overlook it.

The core philosophy behind Agentic Vision is treating vision as an “active investigation.” Instead of being satisfied with a one-time holistic scan of the image, it combines visual reasoning with code execution. This allows the model to formulate plans to zoom in, inspect, and manipulate the image step-by-step. This means the model can autonomously decide to “look closer” at a specific region or rotate an image to gain a better perspective, thereby grounding its answers in concrete visual evidence rather than speculation.

Core Improvement: Quality and Accuracy

Based on internal testing data, enabling code execution within Gemini 3 Flash delivers a consistent 5% to 10% quality boost across most vision benchmarks. This is not a trivial improvement; it represents a significant reduction in the model’s “hallucination” rate and an increase in reliability, achieved by introducing deterministic computation and active visual interaction.

The Mechanism: The Think, Act, Observe Loop

The power of Agentic Vision lies in its introduction of a reasoning loop similar to an intelligent agent, referred to as “Think, Act, Observe.” This loop breaks down the image understanding task into three distinct steps, ensuring every conclusion is supported by evidence.

1. Think: Analysis and Planning

When a user submits a query along with an image, the model first enters the “Think” phase. Here, the model deeply analyzes the user’s intent and the initial image content.

This involves more than simple recognition; it is a complex planning process. The model considers questions such as:

• What is the user’s actual need?
• Which regions of the image are relevant to the answer?
• Does the image need pre-processing to reveal details?
• Are mathematical calculations required to verify a hypothesis?

Based on this analysis, the model formulates a multi-step plan. For example, if asked about a component on a circuit board, the model might plan to first locate the component, then crop a close-up of that area, and finally read the text on it.

2. Act: Code Generation and Execution

The “Act” phase is where Agentic Vision diverges most sharply from traditional models. In this stage, the model does not limit itself to generating text descriptions. Instead, it generates and executes Python code.

Through the tool of code execution, the model can actively manipulate the image in various ways, including:

• Cropping: Extracting specific regions of the image.
• Rotating: Adjusting the image angle for the optimal viewing perspective.
• Annotating: Drawing bounding boxes, arrows, or text labels directly on the image.
• Calculating: Analyzing, counting, or performing mathematical operations on data within the image.

This capability gives the model the ability to “use tools” rather than just “talk.” It moves beyond describing “I see a number” to executing code that precisely locates and processes the region containing that number.

3. Observe: Context Update and Re-analysis

Once the model executes the code and generates a new, processed image (such as a cropped close-up), this new image data is appended to the model’s context window.

The context window serves as the model’s “short-term memory.” By adding these processed images to its memory, the model can now “observe” brand new data. This data carries better contextual information and is free from background noise. The model then performs a final analysis based on this clearer evidence to generate a response.

This process may repeat multiple times within the Think-Act-Observe loop until the model is confident it has found sufficient evidence to answer the user’s question accurately.

Practical Use Cases: Agentic Vision in Action

By enabling code execution in the API, developers have unlocked many unprecedented behaviors. From major products to smaller startups, various industries are leveraging this capability to solve real-world problems. Below are several typical application examples.

1. Zooming and Inspection: Building Plan Validation

When processing high-resolution inputs, Gemini 3 Flash is trained to implicitly zoom in to detect fine-grained details.

Case Study: PlanCheckSolver.com
PlanCheckSolver.com is an AI-powered building plan validation platform. In dealing with complex architectural blueprints, even minute errors can lead to serious compliance issues. By enabling code execution with Gemini 3 Flash, the platform allows for iterative inspection of high-resolution inputs, improving accuracy by 5%.

Workflow Analysis:

1. Input: A massive architectural floor plan.
2. Think: The model analyzes the need to check if roof edges comply with building codes.
3. Act: The model generates Python code to specifically crop image patches of “roof edges” or specific building sections.
4. Observe: These cropped new images are fed back into the model’s context.
5. Verification: The model now has clear local close-ups, allowing it to check lines and annotations pixel-by-pixel to confirm compliance with complex building codes.

This “divide and conquer” strategy allows the model to maintain a macro view while possessing micro-level precision when processing extremely large images.

2. Image Annotation: The Visual Scratchpad

Agentic Vision allows the model to interact with its environment by annotating images. This goes beyond just describing what it sees; the model executes code to draw directly on the canvas to ground its reasoning.

Case Study: Counting Fingers
In the Gemini app, if a simple task is “count how many fingers are on a hand,” traditional models might make errors due to overlapping fingers or lighting issues. However, with Agentic Vision enabled, the process changes completely.

Workflow Analysis:

1. Identify: The model identifies the hand in the image.
2. Act: To avoid counting errors, the model executes Python code to draw bounding boxes and numeric labels (e.g., 1, 2, 3…) over each finger it identifies.
3. Visual Scratchpad: This image with labels acts as a scratchpad. The model uses it to verify its counting logic.
4. Result: The final answer is not a guess but a precise count based on pixel-perfect understanding.

This method significantly enhances the interpretability of the task. Users don’t just get the result; they can see how the model arrived at it.

3. Visual Math and Plotting: From Tables to Charts

Standard Large Language Models (LLMs) often “hallucinate” during multi-step visual arithmetic—they might make up numbers or calculation results. Agentic Vision bypasses this issue by offloading computation to a deterministic Python environment.

Case Study: High-Density Table Data Visualization
When presented with a table containing high-density data, the model needs to extract data and generate a chart.

Workflow Analysis:

1. Extract: The model identifies raw data from the image.
2. Act: The model writes Python code to normalize the extracted data (e.g., normalizing prior SOTA values to 1.0).
3. Plot: The code calls professional plotting libraries (like Matplotlib) to generate a bar chart.
4. Verify: The generated chart is a direct result of data-driven execution, not probabilistic guessing.

This process replaces “probabilistic guessing” with “verifiable execution.” Whether it is financial report analysis or scientific experiment data recording, this capability ensures the professionalism and accuracy of the results.

Technical Advantages and Future Outlook of Agentic Vision

Summary of Technical Advantages

To better understand the changes brought by Agentic Vision, we can compare traditional static vision with Agentic Vision in the following table:

Feature	Traditional Static Vision	Agentic Vision
Observation Method	Passive, one-time holistic scan	Active, multi-step iterative investigation
Detail Handling	Easily misses minute details; relies on guessing	Actively focuses on details via code cropping and zooming
Tool Usage	Relies solely on internal weights for reasoning	Integrates Python code execution for calculation and plotting
Accuracy	Prone to hallucinations, especially in complex math	Based on deterministic calculation and solid evidence
Interactivity	Outputs only text descriptions	Can output annotated images, charts, etc.

Future Development Directions

While Agentic Vision has already demonstrated powerful capabilities, this is just the beginning. According to the roadmap, future updates will focus on the following directions:

1. More Implicit Code-Driven Behaviors
   Currently, Gemini 3 Flash excels at implicitly deciding when to “zoom in” on small details. However, other capabilities, such as rotating images or performing visual math, currently require an explicit prompt nudge to trigger.
   Goal: Future updates aim to make these behaviors fully implicit. The model will be able to autonomously judge when an image needs rotation or when calculation is necessary without the user explicitly instructing it to do so.

2. Integration of More Tools
   Currently, Python code execution is the primary tool supported.
   Goal: The plan is to equip Gemini models with even more tools, including web search and reverse image search. This will further enhance the model’s understanding of the world, allowing it to verify and supplement visual information with external knowledge bases.

3. Support for More Model Sizes
   Agentic Vision is currently rolling out primarily on the Flash model.
   Goal: The capability is planned to expand to other model sizes, ensuring that users with different computing power requirements and application scenarios can benefit from agentic vision.

How to Get Started with Agentic Vision

For developers and general users, Agentic Vision is not a distant concept; it is available today. Whether accessed via API or directly in an app, it is very convenient to use.

1. For Developers: API Integration

Developers can access Agentic Vision through two primary platforms:

• Google AI Studio
• Vertex AI

In the API call, the key is enabling the code execution feature. Once enabled, the model will automatically decide whether to use Python code to assist with image understanding based on the complexity of the task.

Developer Docs and Resources:

• Google AI Studio provides detailed developer documentation on how to enable code execution when processing images.
• Vertex AI users can consult the corresponding development documentation.

2. For General Users: AI Studio Playground and Gemini App

If you are not a developer but just want to experience this technology, you can do so through the following methods:

Method A: AI Studio Playground

This is a very intuitive experimental environment.

1. Visit the Prompts/New Chat page in Google AI Studio.
2. Select gemini-3-flash-preview in the model selector.
3. Locate the Tools settings section.
4. Toggle on “Code Execution”.
5. Upload an image and ask a question, then observe whether the model automatically runs code to analyze the image.

Method B: Gemini App (Mobile or Web)

The Agentic Vision feature is starting to roll out to the Gemini app.

1. Open the Gemini app.
2. Select “Thinking” mode from the model drop-down menu.
3. In this mode, you can try uploading complex images to ask questions and experience the reasoning process behind the model.

3. Experience the Demo

To直观展示 Agentic Vision 的能力，官方提供了一个 Demo 应用。在这个 Demo 中，你可以亲眼看到模型如何生成代码来裁剪图片、绘制图表，并将这些步骤可视化。

---

Frequently Asked Questions (FAQ)

Below are some common questions about Gemini 3 Flash’s Agentic Vision and their detailed answers.

Q: Does Agentic Vision only work with photos?
A: Not exactly. While the examples mention photos, Agentic Vision applies equally to digital documents, charts, hand-drawn sketches, and architectural blueprints. Any visual input that allows for pixel analysis and logical reasoning can be processed.

Q: Will enabling code execution make the response slower?
A: Because Agentic Vision involves a “Think-Act-Observe” loop and includes the generation and execution of code, the processing time may be slightly longer compared to simple static image recognition. However, this investment in time is exchanged for higher accuracy and more reliable results, especially in complex tasks.

Q: Do I need to know how to program to use this feature?
A: For end-users, no. The model automatically generates and executes the Python code. You simply need to upload an image and ask a question; the model handles all the complex code logic in the background. Of course, if you are a developer, you need to configure the relevant parameters correctly when calling the API.

Q: What happens if the model generates incorrect code?
A: The model possesses self-correction capabilities. In the “Observe” phase, if the generated image or calculation result does not meet expectations, the model can re-enter the “Think” phase, adjust the code, and re-execute it until a satisfactory result is obtained. This iterative mechanism significantly improves the final correctness rate.

Q: Why does Agentic Vision improve accuracy in building plan verification?
A: Architectural plans are typically extremely high-resolution and dense with detail. Static models can easily get lost in the details. Agentic Vision uses code to slice large images into small blocks (like checking roof edges), allowing the model to inspect each part with precision, similar to a human expert, thereby avoiding missed key violations.

Q: How does the visual math feature avoid hallucinations?
A: Standard models perform mathematical operations based on language probability, predicting the next number, which is prone to error. Agentic Vision extracts data from the image and hands it directly to a Python environment for deterministic mathematical operations (addition, subtraction, multiplication, division, normalization). The results of Python’s calculations are based on logic, not probability, thus eliminating mathematical hallucinations.

Q: Where can I find documentation for this feature in Vertex AI?
A: You can consult the Vertex AI Generative AI documentation under the section related to “Multimodal Code Execution,” which contains specific configuration examples and best practice guides.

Q: What is a “Visual Scratchpad”?
A: This refers to the intermediate images generated by the model during the code execution process, such as a hand image with annotated boxes. These scratchpad images help the model “visualize” its thought process, ensuring that what it understands is correct, and also allow users to see the model’s reasoning path.

Conclusion

The introduction of Agentic Vision marks a significant leap from “perception” to “cognition” in AI visual understanding. By combining visual reasoning with code execution, Gemini 3 Flash is no longer just an observer capable of describing images, but an intelligent agent capable of actively analyzing, verifying, and solving problems.

Whether it is checking minute details for building compliance or performing precise data visualization in complex tables, Agentic Vision demonstrates the immense potential of AI in handling real-world complex tasks. As more tools are integrated and the model continues to iterate, we have reason to believe that this evidence-based, active visual intelligence will become a standard configuration in future AI applications.