Gmailtail: A Powerful Gmail Monitoring Tool

In the digital age of work, a large number of emails flood in from various channels every day. However, manually checking each email is inefficient and prone to missing important information. The good news is that there is a command-line tool called Gmailtail that can help us monitor Gmail emails in real-time and output them in JSON format, making it convenient for integration and automated processing with other tools.

What is Gmailtail?

Gmailtail is a Gmail monitoring tool designed for automation, monitoring, and integration. It features:

  • Real-time monitoring: With the --tail mode, it continuously monitors new emails to ensure no important information is missed.
  • Flexible filtering: Filters emails by sender, subject, labels, attachments, etc.
  • Checkpoint support: Resumes monitoring from where it left off, avoiding reprocessing of viewed emails.
  • Multiple output formats: Supports JSON, JSON Lines, and compact formats.
  • Configuration files: Uses YAML configuration for complex setups.
  • Gmail search syntax: Fully supports Gmail’s search queries.
  • Easy authentication: Supports OAuth2 and service accounts.

Quick Start

Installing Gmailtail

It is recommended to install using uv. Here are the steps:

  1. If you haven’t installed uv yet, you can do so with the following command:

curl -LsSf https://astral.sh/uv/install.sh | sh


2. Clone the Gmailtail project and set it up:
   ```bash
git clone https://github.com/c4pt0r/gmailtail.git
cd gmailtail
uv sync

Configuring Google API Credentials

To use Gmailtail, you need to configure Google API credentials:

  1. Log in to the Google Cloud Console.
  2. Create a new project or select an existing one.
  3. Enable the Gmail API.
  4. Create credentials (OAuth 2.0 Client ID for desktop applications).
  5. Download the credentials JSON file.

Running Gmailtail

After completing the above steps, you can run Gmailtail:

uv run gmailtail --credentials credentials.json --tail

Example Use Cases

Basic Monitoring

  • Monitor all new emails: Run gmailtail --tail to view all newly received emails in real-time.
  • Monitor emails from specific senders: If you only want to monitor emails from a specific sender (e.g., GitHub notifications), use gmailtail --from "noreply@github.com" --tail.
  • Monitor emails with Gmail search queries: For example, use gmailtail --query "subject:alert OR subject:error" --tail to monitor emails with “alert” or “error” in the subject.

Filtering Options

  • Monitor unread emails only: Use gmailtail --unread-only --tail to view only unread emails.
  • Monitor emails with attachments and include attachment information: Use gmailtail --has-attachment --include-attachments --tail to retrieve emails with attachments and their details.
  • Monitor emails with specific labels: If you only want to monitor emails labeled as “important” and “work,” run gmailtail --label important --label work --tail.
  • Monitor emails since a specific date: To view emails received after a certain date (e.g., January 1, 2025), use gmailtail --since "2025-01-01T00:00:00Z" --tail.

Output Formats

  • Pretty JSON output: Use gmailtail --format json --pretty --tail to view JSON data in an easy-to-read format.
  • JSON Lines format (one JSON per line): Run gmailtail --format json-lines --tail to generate data suitable for stream processing.
  • Compact format: Use gmailtail --format compact --tail for a concise display of email information.
  • Include email body: If you need to access the email body, add the --include-body parameter and control the body length with --max-body-length, such as gmailtail --include-body --max-body-length 500 --tail.
  • Custom fields output: If you only want to view specific fields (e.g., email ID, subject, sender, timestamp), use gmailtail --fields "id,subject,from,timestamp" --tail.

Checkpoint Management

  • Resume monitoring from the last checkpoint: Use gmailtail --resume --tail to continue monitoring from the last stop point.
  • Reset the checkpoint and start fresh: To clear the previous checkpoint and restart monitoring, use gmailtail --reset-checkpoint --tail.
  • Customize the checkpoint file: You can specify the checkpoint file path, such as gmailtail --checkpoint-file ./my-checkpoint --tail.

Using Configuration Files

Create a configuration file (e.g., gmailtail.yaml), place the relevant settings in it, and then run Gmailtail with gmailtail --config-file gmailtail.yaml. This is very convenient for setting up complex monitoring rules.

Advanced Usage with jq

jq is a lightweight command-line JSON processor that, when combined with Gmailtail, enables powerful data analysis. Here are some examples:

  • Extract sender email and subject: gmailtail --format json-lines --tail | jq -r '.from.email + ": " + .subject quickly retrieves the sender and subject of each email.
  • Filter emails by specific sender and get timestamps: gmailtail --format json-lines --tail | jq -r 'select(.from.email == "noreply@github.com") | .timestamp views the timestamps of emails from GitHub.
  • Count emails by sender: gmailtail --format json-lines --once | jq -r '.from.email' | sort | uniq -c | sort -nr counts the number of emails sent by each sender.
  • Get all unique labels: gmailtail --format json-lines --once | jq -r '.labels[]?' | sort | uniq views all unique labels in the emails.
  • Extract emails with attachments and display attachment information: gmailtail --format json-lines --include-attachments --tail | jq 'select(.attachments | length > 0) | {subject, from: .from.email, attachments: [.attachments[].filename]}' retrieves emails with attachments and their file names.
  • Monitor urgent emails and send desktop notifications (macOS): gmailtail --query "label:urgent OR subject:urgent" --format json-lines --tail | jq -r '.subject' | while read subject; do osascript -e "display notification \"$subject\" with title \"Urgent Email\""; done displays desktop notifications for urgent emails on macOS.
  • Extract email body and save to files: gmailtail --include-body --format json-lines --once | jq -r '"\(.id).txt|\(.body // .snippet)"' | while IFS='|' read filename content; do echo "$content" > "$filename"; done saves email bodies or snippets to text files named after the email IDs.
  • Monitor GitHub notifications and extract PR/issue numbers: gmailtail --from "notifications@github.com" --format json-lines --tail | jq -r 'select(.subject | test("Pull Request|Issue")) | .subject | capture(".*#(?<number>[0-9]+).*") | .number extracts pull request or issue numbers from GitHub notification emails.
  • Create a daily email activity summary: `gmailtail –since “$(date -d ‘today’ ‘+%Y-%m-%dT00:00:00Z’)” –format json-lines –once | jq -r ‘[group_by(.from.email) | .[] | {sender: .[0].from.email, count: length}] | sort_by(.count) | reverse’ generates a daily summary of email activities grouped by sender.
  • Monitor emails with specific keywords in the body and trigger alerts: `gmailtail –include-body –format json-lines –tail | jq -r ‘select(.body | test(“error|fail|alert”; “i”)) | “ALERT: (.from.email) – (.subject)”‘ outputs alerts when specific keywords appear in email bodies.
  • Extract and format meeting invitations: `gmailtail –query “has:attachment filename:ics” –include-attachments –format json-lines –tail | jq ‘meeting: .subject, organizer: .from.email, time: .timestamp, location: (.body capture(“Location:.(?.)”)? | .loc // “N/A”)‘ extracts and formats information from meeting invitation emails.

Command-Line Options

Authentication

  • --credentials PATH: Specifies the path to the OAuth2 credentials file.
  • --auth-token PATH: Specifies the path to the service account authentication token file.
  • --cached-auth-token PATH: Specifies the path to the cached authentication token file (default: ~/.gmailtail/tokens).

Filtering

  • --query QUERY: Filters using Gmail search syntax.
  • --from EMAIL: Filters by sender email address.
  • --to EMAIL: Filters by recipient email address.
  • --subject PATTERN: Filters by subject (supports regular expressions).
  • --label LABEL: Filters by label (can be used multiple times).
  • --has-attachment: Filters emails with attachments.
  • --unread-only: Filters unread emails only.
  • --since DATETIME: Filters emails from the specified ISO 8601 time.

Output

  • --format FORMAT: Specifies the output format (json, json-lines, compact).
  • --fields FIELDS: Specifies a comma-separated list of fields to include.
  • --include-body: Includes the email body in the output.
  • --include-attachments: Includes attachment information.
  • --max-body-length N: Sets the maximum body length (default: 1000).
  • --pretty: Pretty-prints JSON output.

Monitoring

  • --tail, -f: Enables continuous monitoring mode.
  • --once: Runs once and exits.
  • --poll-interval N: Sets the polling interval in seconds (default: 30).
  • --batch-size N: Sets the number of messages per batch (default: 10).

Checkpoint

  • --checkpoint-file PATH: Specifies the checkpoint file path.
  • --checkpoint-interval N: Sets the checkpoint save interval in seconds (default: 60).
  • --resume: Resumes from the last checkpoint.
  • --reset-checkpoint: Resets the checkpoint.

Other

  • --verbose, -v: Enables verbose output mode.
  • --quiet: Quiet mode, outputs only email JSON data.
  • --log-file PATH: Specifies the log file path.
  • --config-file PATH: Specifies the configuration file path.
  • --dry-run: Simulates a run without actual processing.

Output Format Example

Here is an example of the JSON format output by Gmailtail:

{
  "id": "18234567890abcdef",
  "threadId": "18234567890abcdef",
  "timestamp": "2025-07-01T10:30:00Z",
  "subject": "GitHub notification",
  "from": {
    "name": "GitHub",
    "email": "noreply@github.com"
  },
  "to": [
    {
      "name": "John Doe",
      "email": "john@example.com"
    }
  ],
  "labels": ["INBOX", "UNREAD"],
  "snippet": "You have a new pull request...",
  "body": "Full email body here...",
  "attachments": [
    {
      "filename": "report.pdf",
      "mimeType": "application/pdf",
      "size": 1024
    }
  ]
}

Application Scenarios

  • Monitoring systems: Set up Gmailtail to monitor specific email patterns, such as server failure alerts or updates on business-critical metrics, to enable timely notifications and responses.
  • Automation workflows: Trigger automated actions based on email content. For example, automatically update the inventory management system upon receiving order confirmation emails, or automatically create tickets when emails with specific keywords are received.
  • Data analysis: Collect email data for analysis to understand email traffic trends, average response times, common subjects, etc., to optimize business processes or customer service.
  • System integration: Integrate Gmail data into other tools and systems, such as customer relationship management (CRM) systems or project management tools, to achieve unified data management and utilization.
  • Email backup: Archive important emails in a structured JSON format to ensure the safe storage and long-term accessibility of email data.
  • CI/CD pipelines: Monitor build notifications and alert emails in CI/CD processes to promptly identify and address build failures or deployment issues.

Configuration File Example

Create a gmailtail.yaml configuration file with the following content:

# Authentication settings
auth:
  credentials_file: ~/.config/gmailtail/credentials.json
  # auth_token: ~/.config/gmailtail/service-account.json
  cached_auth_token: ~/.config/gmailtail/tokens

# Email filtering settings
filters:
  query: "label:important"
  # from: "noreply@github.com"
  # to: "me@example.com"
  # subject: "alert|error|warning"
  # labels: ["important", "inbox"]
  # has_attachment: true
  unread_only: true
  # since: "2025-01-01T00:00:00Z"

# Output formatting
output:
  format: json-lines
  include_body: true
  include_attachments: true
  max_body_length: 500
  pretty: false
  # fields: ["id", "subject", "from", "timestamp", "labels"]

# Monitoring behavior
monitoring:
  poll_interval: 60
  batch_size: 20
  tail: true
  # max_messages: 1000

# Checkpoint settings
checkpoint:
  checkpoint_file: ~/.config/gmailtail/checkpoint
  checkpoint_interval: 120
  resume: true

# Logging
verbose: false
quiet: false
# log_file: ~/.config/gmailtail/gmailtail.log

Authentication Setup

OAuth2 (Recommended for personal use)

  1. Log in to the Google Cloud Console.
  2. Create or select a project.
  3. Enable the Gmail API.
  4. Go to “Credentials” → “Create Credentials” → “OAuth 2.0 Client ID”.
  5. Select “Desktop application”.
  6. Download the JSON file.
  7. Use it with --credentials path/to/credentials.json.

Service Account (For server/automated environments)

  1. In the Google Cloud Console, go to “Credentials”.
  2. Create a “Service Account”.
  3. Download the JSON key file.
  4. Use it with --service-account path/to/service-account.json.

Note: Service accounts require domain-wide delegation for Gmail access.

Common Questions (FAQ)

Does Gmailtail support monitoring multiple email accounts simultaneously?

Currently, Gmailtail is primarily designed for monitoring a single Gmail account. If you need to monitor multiple email accounts, you can set up separate instances of Gmailtail for each account, each with its own configuration file and credentials.

How can I process the JSON data output by Gmailtail?

You can process JSON data using various programming languages and tools. For example, use Python’s json module to parse the data, or use the jq command-line tool for simple data extraction and transformation, as demonstrated in the advanced usage examples.

Does Gmailtail support automatic email reply functionality?

Gmailtail itself does not directly support automatic email replies. However, you can combine it with other tools and APIs to implement automatic reply logic based on the email data output by Gmailtail. For instance, use the Gmail API’s email-sending functionality to send automatically generated replies upon receiving emails that meet specific criteria.

How can I ensure the stable operation and data security of Gmailtail?

To ensure stable operation, regularly check the Gmailtail log files, monitor its operational status, and set up appropriate error handling and recovery mechanisms. For data security, properly safeguard authentication credentials and prevent unauthorized access to sensitive information. Additionally, follow Google’s API usage policies and best practices, and reasonably configure monitoring frequency and data processing logic to protect user privacy and data security.

Can Gmailtail be integrated with message queues or stream processing systems?

Yes, since Gmailtail outputs email data in JSON format and supports stream-friendly formats like JSON Lines, it can easily be integrated with message queues (e.g., RabbitMQ, Kafka) or stream processing systems (e.g., Apache Flink, Apache Storm). By sending the data output by Gmailtail to message queues or stream processing systems, you can achieve more complex data processing and analysis workflows, such as real-time email content analysis, email traffic monitoring, and event-driven automated workflows.