Stop Guessing File Types: Google’s Magika Uses AI to Identify Files Accurately

Ever received a file with no extension? Or extracted a batch of .dat files from an old system and had no idea what’s inside? You try opening it with a text editor – garbled mess. You run the system file command – vague answer. Then you start dragging it into different applications, hoping one of them will open it. Minutes turn into hours.

Google has open‑sourced a tool called Magika that solves exactly this problem. It uses a deep‑learning model to detect file types – not by looking at the filename or extension, but by reading the actual content. According to the official announcement, Magika already runs inside Gmail, Google Drive, and Safe Browsing, handling hundreds of billions of samples every week.

I installed it and ran it on a few tricky files myself – it works impressively well. In this post, I’ll walk you through what it is, how to install it, and how to use it effectively.

How Accurate Is It? Nearly 99% on the Test Set

Magika is essentially a trained AI model that answers one question: what format is this file?

Unlike traditional tools that rely on fixed “magic bytes” at the file header, Magika uses deep learning. You can think of it as a model that has seen a massive number of samples – roughly 100 million files across over 200 content types – and learned the distinguishing patterns by itself. That covers most of the formats you’ll encounter in daily work.

On their test set, the model achieves about 99% average precision and recall. I tested it on a dozen less common formats, including extension‑less text files, and it got almost all of them right.

Speed is another highlight. The model loads once (that’s a one‑time overhead), and after that, each inference takes about 5 milliseconds on a single CPU. Five milliseconds is roughly the time it takes you to blink once – and in that same blink, Magika can run 20–30 identifications. And the file size barely matters, because it only reads a small portion of the file (usually the first few kilobytes), not the whole thing.

One design choice I find particularly smart: Magika uses a per‑content‑type threshold system. If the model isn’t confident enough about its prediction, it doesn’t force a specific label. Instead, it returns a generic one like “Generic text document” or “Unknown binary data.” In security‑sensitive scenarios, admitting uncertainty is far better than giving a false concrete answer.

Installation: Three Ways, Under Three Minutes

Depending on your toolchain, pick the one that suits you.

Option 1 – if you’re a Python user:

pipx install magika

Or with plain pip (preferably inside a virtual environment):

pip install magika

Option 2 – macOS or Linux with Homebrew:

brew install magika

This puts the magika command directly in your PATH.

Option 3 – universal installer script (no language runtime required):

macOS / Linux:

curl -LsSf https://securityresearch.google/magika/install.sh | sh

Windows PowerShell:

powershell -ExecutionPolicy Bypass -c "irm https://securityresearch.google/magika/install.ps1 | iex"

These scripts download a pre‑built binary and install it globally. After installation, run magika --version to confirm it works.

Option 4 – if you’re a Rust developer:

cargo install --locked magika-cli

Command‑Line Basics: The Most Useful Patterns

The simplest usage is to pass a file path:

magika myfile

It prints the path and the detected type description.

But you’ll soon want more control – here are the flags I use most often.

Short label only (-l)

magika -l myfile

Outputs a compact label like python, jpg, or pdf. Great for scripting.

MIME type only (-i)

magika -i myfile

Handy when you need to set Content‑Type dynamically in a web application.

Show confidence score (-s)

magika -s myfile

The score ranges from 0 to 1 – the closer to 1, the more certain the model. Most normal files score above 0.98, but I’ve seen a few edge cases around 0.7. That’s a useful signal that the model is hedging.

Recursive directory scan (-r)

magika -r ./some_folder/

It walks through every file inside the directory and outputs results for each. The official example shows:

cd tests_data/basic && magika -r *

It correctly identifies Assembly code, batch files, C source, CSS, CSV, Dockerfiles, Word documents, email files, and even an empty file – all at once.

JSON output (--json)

magika --json myfile

This returns a structured object with the full result: path, status, label, description, MIME type, group, extensions, and score. Perfect for integrating into pipelines or backend services.

[
  {
    "path": "./tests_data/basic/python/code.py",
    "result": {
      "status": "ok",
      "value": {
        "dl": {
          "description": "Python source",
          "extensions": ["py", "pyi"],
          "group": "code",
          "is_text": true,
          "label": "python",
          "mime_type": "text/x-python"
        },
        "output": {
          "description": "Python source",
          "extensions": ["py", "pyi"],
          "group": "code",
          "is_text": true,
          "label": "python",
          "mime_type": "text/x-python"
        },
        "score": 0.996999979019165
      }
    }
  }
]

Custom output format (--format)

You can build your own output using placeholders:

• %p – file path
• %l – label
• %d – description
• %g – group (e.g., code, document, image)
• %m – MIME type
• %e – possible extensions
• %s – confidence score
• %S – score as percentage

Example:

magika --format "%p => %l (%s)" myfile

Read from standard input

Use a dash - to pass content via stdin:

cat myfile | magika -

Or:

echo "function log(msg) {console.log(msg);}" | magika -

This works even when the file isn’t saved on disk.

Python and JavaScript Integration

For developers, the SDKs are straightforward.

Python API

Install the package:

pip install magika

Then:

from magika import Magika

m = Magika()

# From bytes
res = m.identify_bytes(b'function log(msg) {console.log(msg);}')
print(res.output.label)  # javascript

# From file path
res = m.identify_path('./tests_data/basic/ini/doc.ini')
print(res.output.label)  # ini

# From an open stream
with open('./tests_data/basic/ini/doc.ini', 'rb') as f:
    res = m.identify_stream(f)
print(res.output.label)  # ini

One important note: initialising Magika() loads the model into memory – that’s a one‑off cost. In a server application, reuse the same instance instead of creating a new one for every request.

JavaScript / TypeScript (experimental)

There’s also an npm package:

npm install magika

It runs fully in the browser – no file upload to any server. You can even try the official web demo without installing anything.

Who’s Using Magika in Production?

This isn’t a lab toy. Google runs it at scale:

• Gmail – attachments are classified by Magika to route them to the appropriate security scanners.
• Google Drive – uploaded files go through Magika to enforce content policies.
• Safe Browsing – Magika helps identify file types among hundreds of millions of potentially malicious samples.

Security platforms like VirusTotal and abuse.ch have also integrated Magika. Analysts upload suspicious files, and Magika quickly tells them what type they’re dealing with – a critical first step in incident response.

These use cases share two requirements: massive throughput and high accuracy. The fact that Magika holds up under those conditions says a lot about its real‑world reliability.

How Does It Compare to the Classic file Command?

The venerable file command on Linux relies mainly on two techniques: magic byte matching (looking for fixed patterns like %PDF at the beginning) and extension mapping. That works well for many binary formats, but it struggles with plain‑text files – a Python script and a JavaScript file look almost identical from the header.

Magika’s deep‑learning approach doesn’t need hand‑crafted rules. It learns patterns from the data itself, which gives it an edge especially on textual content types – source code, config files, email, markup, and more.

That said, file isn’t obsolete. For well‑known binary formats, it’s still fast and reliable. Magika is a complement: when file gives you a vague or wrong answer, Magika often resolves the ambiguity.

Common Pitfalls (and How to Avoid Them)

1. Model loading overhead

As mentioned, Magika() loads the model on first instantiation. For CLI usage it doesn’t matter, but for web services, do not create a new instance per request – reuse it.

2. You don’t need to pass the entire file

Magika reads only a small chunk internally. You don’t have to truncate the file yourself. But knowing this can help you avoid reading huge files into memory unnecessarily – just pass the path or a stream and let Magika handle it.

3. Low confidence scores

If you see a score around 0.6 or 0.7, don’t treat it as a definitive answer. The model is uncertain. In such cases, consider a manual check or combine it with other signals. Magika offers different prediction modes – high-confidence returns a specific label only when the score is high, otherwise falls back to a generic one; best-guess always returns the most probable label. Choose based on whether you prioritise low false positives or low false negatives.

Practical Summary

• Magika is an open‑source AI tool for file‑type detection, with ~99% accuracy across 200+ types.
• Install via pipx, brew, or the official installer script.
• Basic usage: magika <path>. Add -r for recursive directories, --json for structured output.
• Python API: identify_path(), identify_bytes(), identify_stream().
• Speed: ~5 ms per file, independent of file size.
• Works best for text‑based formats where traditional tools struggle.
• Reuse the model instance in server environments to avoid reload overhead.

One‑Page Quick Reference

Frequently Asked Questions

Does Magika upload my files to the cloud?

No. The model runs locally. No data leaves your machine.

Does it work on Windows?

Yes – Windows, macOS, and Linux are all supported.

How many file types can it recognise?

Over 200 content types, including common binary formats (images, audio, video, archives, executables) and a wide range of text formats (source code, configs, documents, email, etc.). Check the official model README for the full list.

Why not just use the file command?

file relies on magic bytes and extensions, which often fail on text files. Magika uses deep learning and handles text formats much more accurately. They complement each other.

Is this an official Google product?

The disclaimer says it’s not an official Google project and isn’t supported by Google. But it’s used internally at Google, and the code quality is solid.

What’s the license?

Apache 2.0 – business‑friendly. See the LICENSE file in the repository for details.