Combatting Shadow AI in Enterprises: An Open-Source Detection System in Action

The Silent Threat in Modern Organizations

As large language models (LLMs) like ChatGPT become workplace staples, a hidden vulnerability emerges—Shadow AI. This term describes employees’ unauthorized use of external AI tools to process company data. Recent technical analysis reveals alarming patterns: during simulated enterprise testing, an open-source detection system intercepted 36% of LLM requests as high-risk, involving potential data leaks and compliance violations. This invisible threat is compelling organizations to reevaluate their AI governance strategies.

Inside the Real-Time Detection Architecture

The FlagWise open-source system (GitHub: bluewave-labs/flagwise) delivers a comprehensive security framework through its multilayered approach:

  1. Full Traffic Capture
    LLM requests flow through Apache Kafka message streams (topic: llm-traffic-logs), recording prompts, responses, and metadata at 2-3 requests/second without impacting business operations.

  2. Tri-Layer Detection Engine

  • Pattern Recognition: Regex rules identify sensitive data (e.g., credit card numbers \b(?:\d{4}[-\s]?){3}\d{4}\b)
  • Dynamic Risk Scoring: Algorithms assign threat levels (0-100 scale) based on request context
  • Access Control: Model/IP allowlists block unauthorized providers
  1. Military-Grade Data Protection
    Sensitive fields use AES-256 encryption with PBKDF2 key derivation. Even database breaches can’t expose original content due to Fernet encryption implementation.

Live Test Data Reveals Critical Insights

In a simulation processing 3,000+ enterprise requests (August 2025 data):

  • 36% of requests triggered security flags
  • Threat distribution showed:

    • Data exposure risks (22%)
    • Unauthorized model usage (9%)
    • Prompt injection attempts (5%)
  • Peak business hours amplified threats: 3x more incidents during high-activity periods

Four Pillars of Enterprise Protection

1. Dynamic Risk Visualization

Real-time dashboards map request origins (HQ/remote workers/contractors) and automatically highlight anomalous IP addresses. The React-based interface displays:

  • Live traffic statistics
  • Risk score distributions
  • Top AI providers/models in use

2. Granular Access Control

JWT token authentication enables role-based permissions:

  • Admin roles: View raw encrypted data, configure detection rules
  • Read-only roles: Access sanitized request previews only
    Password security uses bcrypt hashing with self-service resets.

3. Intelligent Alert Protocols

When risk scores exceed 70:

  • Alerts trigger in <3 seconds via Slack/email integrations
  • Configurable conditions target specific threat profiles:
{ "alert_type": "slack",  
  "conditions": { "risk_score": {"min": 70}, "is_flagged": True } }

4. Resource Abuse Prevention

Tracking modules monitor:

  • Model-specific usage frequency
  • Response latency metrics
  • Cost-per-request calculations
    This prevents expensive model misuse (e.g., GPT-4 for simple tasks).

Deployment Roadmap for Technical Teams

Infrastructure Setup

Launch the complete system in 60 seconds:

docker-compose up -d  # Starts PostgreSQL, FastAPI, React services

Access points:

  • Dashboard: http://localhost:3000 (admin/admin123)
  • API Docs: http://localhost:8000/docs

Data Pipeline Configuration

  1. Route organizational LLM traffic to Kafka
  2. In FlagWise dashboard: Settings → Data Sources → Kafka Topic
  3. Set topic name to llm-traffic-logs

Rule Configuration Example

Create custom detection protocols:

{ "name": "Customer Privacy Shield",  
  "rule_type": "keyword",  
  "pattern": ["ID number", "bank card"],  
  "severity": "critical" }

Measurable Business Value

Regulatory Compliance

Automated audit trails satisfy GDPR/CCPA requirements with exportable reports showing:

  • Prompt/response timelines
  • Risk score evolution
  • Action histories

Cost Optimization

Identify resource waste:

  • High-cost models used for trivial tasks
  • Duplicate requests from same users
  • Performance-lagging providers

Threat Forensics

Session correlation analysis reveals:

  • Employee-specific risk patterns
  • Department-level vulnerability hotspots
  • Recurring threat vectors

Implementation Best Practices

  1. Phased Monitoring Rollout
  • Begin with legal/finance departments
  • Start in detection-only mode (no blocking)
  • Gradually expand to R&D teams
  1. SIEM Integration
  • Feed risk scores into Splunk/Sentinel platforms
  • Correlate AI threats with existing security events
  • Create unified incident response playbooks
  1. Progressive Rule Deployment
  • Initial phase: Basic keyword detection
  • Intermediate: Add regex pattern matching
  • Advanced: Implement behavioral analysis

Technical Evolution & Solutions

Recent system enhancements addressed critical challenges:

  • MacOS Silicon Compatibility: Upgraded cryptography dependencies to resolve build failures
  • Data Validation: Migrated from Pydantic regex to pattern parameters
  • API Connectivity: Fixed container communication by changing React proxy to api:8000
  • Encryption Handling: Implemented dual-field storage (prompt + prompt_preview) for admin/viewer segregation

Conclusion: From Vulnerability to Vigilance

Shadow AI represents more than a technical nuisance—it’s a tangible business risk. With solutions like FlagWise now overcoming core challenges (cross-platform compatibility, real-time detection accuracy, and scalable encryption), organizations can transform reactive security into proactive defense. The system’s open-source nature allows continuous refinement, while Docker-based deployment eliminates infrastructure barriers. As LLMs become further embedded in workflows, establishing monitored AI usage channels isn’t optional—it’s foundational to enterprise resilience.