Gabber: Building Real-Time AI Applications Across Voice, Text, and Video

Have you ever wondered how developers create those seamless AI experiences that understand your voice, analyze your emotions, and respond in real time? What if you could build applications that handle multiple forms of communication simultaneously—processing speech while analyzing facial expressions and generating thoughtful responses—all without drowning in complex code? This is where Gabber comes in, offering a powerful yet accessible solution for creating the next generation of AI applications.

What Exactly Is Gabber?

Gabber is an engine specifically designed for building real-time AI applications that work across all communication modalities—voice, text, video, and beyond. Unlike traditional AI development tools, Gabber enables developers to create graph-based applications that can handle multiple participants and simultaneous media streams with remarkable efficiency.

At its core, Gabber’s mission is simple but ambitious: to provide developers with the most powerful, yet developer-friendly AI application builder available. Whether you’re building a simple voice assistant or a complex multi-user interactive system, Gabber offers the tools to bring your vision to life without getting bogged down in technical complexities.

Why Gabber Stands Out in the AI Development Landscape

The AI development space is crowded with tools and frameworks, so what makes Gabber different? Let’s break it down:

1. True Multi-Modal Support

Many AI tools focus on a single modality—either text, voice, or vision. Gabber breaks this mold by providing native support for processing and integrating multiple modalities simultaneously. This means your application can:

• ✦ Listen to and process speech in real time
• ✦ Analyze text inputs and generate appropriate responses
• ✦ Process video streams to detect emotions or gestures
• ✦ Combine these inputs to create richer, more contextual interactions

This multi-modal approach reflects how humans actually communicate, making for more natural and effective AI interactions.

2. Graph-Based Application Design

Gabber uses a visual, graph-based approach to application development. Instead of writing thousands of lines of code, you connect functional building blocks (called “nodes”) to create your application flow. This visual approach offers several advantages:

• ✦ Intuitive workflow design: See exactly how data flows through your application
• ✦ Rapid prototyping: Test ideas quickly without extensive coding
• ✦ Easy debugging: Visually trace where issues might be occurring
• ✦ Collaborative development: Share visual representations that team members can understand

3. Real-Time Processing Capabilities

Many AI tools process data in batches or with noticeable delays. Gabber is engineered from the ground up for real-time performance, ensuring that interactions feel natural and immediate—critical for applications like voice assistants, live translation services, or interactive customer support systems.

Getting Started with Gabber: Installation and Setup

Let’s walk through the practical steps to get Gabber up and running on your system. Don’t worry if you’re not a terminal expert—I’ll explain each step clearly.

Step 1: Install Required Dependencies

Before you can use Gabber, you’ll need to install two essential components:

LiveKit: The Media Transport Layer

LiveKit handles the real-time media transmission between your frontend interface and backend services. It uses WebRTC technology to ensure smooth, low-latency communication.

To install LiveKit on macOS (using Homebrew):

brew install livekit

For Windows or Linux users, check the LiveKit documentation for platform-specific installation instructions.

uv: Python Dependency Management

uv is a fast Python installer and resolver that will manage the Python dependencies required by Gabber.

Install uv with this command:

curl -LsSf https://astral.sh/uv/install.sh | sh

This command downloads and runs the installation script, setting up uv on your system.

Step 2: Launch the Entire System

Once your dependencies are installed, starting Gabber is remarkably simple. Just run:

make all

This single command does the heavy lifting for you—it starts all the necessary services including:

• ✦ The frontend interface
• ✦ The editor backend
• ✦ The application engine
• ✦ The repository service

After running this command, you’ll see various services starting up in your terminal. Don’t worry if you see some log messages—this is normal as the system initializes.

Step 3: Configure Your Secrets Securely

Many AI features require API keys or other sensitive information. Gabber provides a secure way to manage these without risking accidental exposure.

1. Create a file named .secret in your project directory
2. Add your API keys and other sensitive information to this file in a key-value format
3. Restart Gabber if it was already running

Once configured, you’ll see these secrets appear in dropdown menus when configuring nodes that require them. Importantly, the actual secret values are never stored in your application graphs, ensuring safe sharing of your work without risking security breaches.

Understanding Gabber’s Core Concepts

To truly harness Gabber’s power, you need to understand its fundamental building blocks. Let’s explore these concepts in plain language.

1. Applications: Your Complete AI Solution

In Gabber, an Application (or “App” for short) represents your entire AI solution. Think of it as a complete workflow that processes inputs and generates outputs.

Technically, an App is a graph consisting of interconnected nodes and their connection points (called “Pads”). It’s the highest-level object in Gabber—you build everything within the context of an Application.

For example, a simple voice assistant application might include:

• ✦ A node that captures audio input
• ✦ A node that converts speech to text
• ✦ A node that processes the text request
• ✦ A node that generates a response
• ✦ A node that converts text back to speech

All these components connected together form your complete Application.

2. Nodes: The Functional Building Blocks

Nodes are the individual components that perform specific tasks within your Application. Each node has a specialized function, such as:

• ✦ Ingesting media (audio, video, text)
• ✦ Transcribing speech to text
• ✦ Analyzing emotions in voice or facial expressions
• ✦ Calling external APIs
• ✦ Generating responses
• ✦ Processing data in various ways

Nodes are designed to be composable—you connect them together to create complex processing flows. This modular approach means you can mix and match functionality without rewriting everything from scratch.

Each node has configurable properties that determine its behavior. For instance, a speech-to-text node might let you select the language model or adjust sensitivity settings.

3. Pads: The Connection Points

Pads are the connection points on nodes that allow data to flow between them. Understanding Pads is crucial to designing effective applications.

Pad Types

There are two fundamental types of Pads:

Type	Purpose	Real-World Analogy
Sink Pads	Receive data from upstream nodes	Like an electrical socket receiving power
Source Pads	Send data to downstream nodes	Like an electrical plug providing power

Pad Modes

Pads also operate in different “modes” that affect how they handle data:

• ✦ Property Mode: Always maintains a value (either an initial value or the last value streamed through it)
• ✦ Stateless Mode: Only transmits values as they happen, without maintaining state

Pads are also type-specific, meaning only compatible types can connect. When a node emits data through a Pad, any connected downstream nodes can immediately process that data in real time.

4. SubGraphs: Reusable Components

SubGraphs function like mini-applications that can be embedded within larger applications. They’re essentially collections of nodes and their Pad connections, designed to be treated as single units.

The real power of SubGraphs comes from Proxy nodes, which create entry and exit points that appear in your main application. This allows you to:

• ✦ Package complex functionality into reusable components
• ✦ Hide implementation details while exposing necessary interfaces
• ✦ Build hierarchical applications with clear separation of concerns

For example, you might create a SubGraph for “user authentication” that handles all the details internally, exposing only “login” and “status” outputs to your main application.

5. State Machines: Managing Complex Logic

State Machines provide a structured way to handle complex decision-making within your applications. They define how your application transitions between different states based on conditions and inputs.

A State Machine consists of:

• ✦ Parameters: Variables the state machine monitors
• ✦ States: Distinct phases in your application flow (starting from an initial state)
• ✦ State Transitions: Nodes that determine when to move between states
• ✦ Transition Logic: Rules that govern state changes (acting as AND gates, combinable for OR logic)

This approach is invaluable for applications that need to respond differently based on context, such as:

• ✦ Customer service bots that change behavior based on user frustration levels
• ✦ Interactive tutorials that adapt to user progress
• ✦ Voice assistants that maintain conversation context

Gabber’s System Architecture: How It All Fits Together

Gabber consists of four main components that work together seamlessly. Understanding this architecture helps you troubleshoot issues and optimize your development workflow.

1. Frontend: Your Visual Interface

The Frontend is a NextJS application that serves as your primary interface for interacting with Gabber. After running make all, you can access it at:

http://localhost:3000

This is where you’ll:

• ✦ Design your application graphs
• ✦ Configure nodes and connections
• ✦ Monitor running applications
• ✦ Access example applications

The Frontend provides an intuitive, drag-and-drop interface that makes visualizing and building your AI applications straightforward.

2. Editor: The Backend Brain

The Editor is a backend service that processes requests from the Frontend. When you click buttons or make changes in the visual interface, the Editor handles those operations behind the scenes.

Key Editor responsibilities include:

• ✦ Managing the creation and editing of applications
• ✦ Saving your work to the repository
• ✦ Validating your application structure
• ✦ Providing the API that the Frontend consumes

You don’t interact with the Editor directly—it works silently in the background to make your development experience smooth.

3. Engine: The Execution Powerhouse

The Engine is responsible for actually running your applications. When you click “Run” in the Frontend, the Engine takes over and processes the data flows you’ve designed.

The Engine’s critical functions:

• ✦ Managing real-time data flow between nodes
• ✦ Handling media processing (audio, video, text)
• ✦ Executing state machine logic
• ✦ Coordinating multiple simultaneous interactions

This component is optimized for performance, ensuring your applications run with minimal latency—essential for real-time AI interactions.

4. Repository: Your Application Storage

The Repository is a lightweight HTTP server that handles saving and retrieving your applications and SubGraphs. It runs on port 8001 and stores everything in the .gabber directory.

The Repository enables:

• ✦ Persistent storage of your application designs
• ✦ Version control (through standard file management)
• ✦ Sharing applications between team members
• ✦ Loading example applications included with Gabber

All your work is stored locally in an organized structure, making it easy to back up or transfer between development environments.

Gabber SDKs: Extending Your Development Options

Gabber provides several Software Development Kits (SDKs) to help you integrate its capabilities into your projects. These SDKs cater to different development environments and preferences.

JavaScript/TypeScript SDK

This framework-agnostic client library works across multiple environments:

• ✦ Node.js backend services
• ✦ Web browsers
• ✦ Bun runtime
• ✦ Deno runtime

Ideal for:

• ✦ Backend services requiring AI capabilities
• ✦ Non-React frontend integrations
• ✦ Projects already using JavaScript/TypeScript

React SDK

Specifically designed for React and React Native developers, this SDK includes:

• ✦ Prebuilt hooks for common functionality
• ✦ Ready-to-use provider components
• ✦ UI components that integrate seamlessly

Benefits for React developers:

• ✦ Dramatically reduced setup time
• ✦ Consistent integration patterns
• ✦ Easier maintenance through standardized components

Python SDK

Perfect for:

• ✦ Backend integrations where Python is preferred
• ✦ Rapid prototyping of AI features
• ✦ Scripting and automation tasks

The Python SDK makes it easy to incorporate Gabber’s capabilities into existing Python workflows or to build new applications using Python’s extensive AI ecosystem.

All SDKs follow consistent design principles, making it relatively straightforward to switch between them if your project requirements change.

Practical Application: Exploring Gabber’s Examples

One of the best ways to learn Gabber is by examining and modifying the example applications included in the repository. Here’s how to get started:

1. Ensure all services are running (make all)
2. Open your browser and navigate to http://localhost:3000
3. In the dashboard, locate and click the “Examples” tab
4. Browse the available examples and select one that interests you
5. Follow the specific instructions provided for that example

These examples showcase real-world usage patterns across different modalities:

• ✦ Voice Processing Examples: Demonstrate speech-to-text, voice analysis, and text-to-speech workflows
• ✦ Text Interaction Examples: Show how to build conversational interfaces and text processing pipelines
• ✦ Multi-Modal Examples: Illustrate how to combine voice, text, and video processing for richer interactions

Each example serves as both a learning tool and a starting point for your own projects—simply modify the example to suit your specific needs rather than building from scratch.

Joining the Gabber Community

Gabber is designed as a collaborative, developer-first project. The team actively encourages community involvement and welcomes contributions of all kinds.

Ways to Engage

• ✦ Questions and Feedback: Start a discussion on GitHub or open an issue for specific problems
• ✦ Contributions: Submit new nodes, fix bugs, or improve documentation
• ✦ Enterprise Needs: Contact brian@gabber.dev or label issues with “enterprise”
• ✦ Stay Updated: Follow @gabberdev on Twitter/X or join the Discord community

The Discord community (https://discord.gg/hJdjwBRc7g) is particularly valuable for:

• ✦ Getting real-time help from the development team
• ✦ Connecting with other developers building similar applications
• ✦ Sharing your own creations and getting feedback
• ✦ Staying informed about upcoming features

This community focus ensures Gabber continues to evolve based on real developer needs rather than theoretical use cases.

Understanding Gabber’s Licensing Model

Gabber uses a thoughtful licensing approach that balances open development with sustainable project maintenance.

Core Engine and Frontend

The main Gabber engine and frontend code follow a “fair-code” distribution model under two licenses:

• ✦ Sustainable Use License: For standard usage
• ✦ Gabber Enterprise License: For enterprise deployments

This model, similar to n8n’s approach, offers several important benefits:

• ✦ Source Available: You always have visibility into the source code
• ✦ Self-Hostable: Deploy Gabber on your own infrastructure
• ✦ Extensible: Add your own nodes and functionality

Non-Core Components

Additional components like examples and SDKs use the more permissive Apache 2.0 license, as indicated by LICENSE files in their respective directories.

This licensing structure ensures:

• ✦ Transparency for developers evaluating the technology
• ✦ Flexibility for different deployment scenarios
• ✦ Sustainability for the project’s long-term development

Frequently Asked Questions

What makes Gabber different from other AI development frameworks?

Gabber stands out through its focus on real-time, multi-modal applications with a visual, graph-based development approach. While many frameworks handle single modalities or require extensive coding, Gabber lets you build complex AI interactions through intuitive node connections, with special attention to low-latency performance.

Do I need advanced programming skills to use Gabber?

Gabber is designed to be accessible to developers with varying skill levels. If you understand basic programming concepts and can follow technical documentation, you should be able to start building with Gabber. The visual interface reduces the need for extensive coding, though some technical knowledge is required for more complex applications.

How does Gabber handle real-time performance?

Gabber’s architecture is optimized for real-time processing at every level. The Engine component is specifically designed to minimize latency in data flow between nodes, and the underlying media transport (via LiveKit) uses WebRTC technology for efficient, low-latency communication.

Can I integrate Gabber with existing AI models and services?

Absolutely. Gabber functions as an integration framework rather than a provider of AI models. You can connect to various AI services through appropriate nodes—like using the OpenAICompatibleLLM node to interface with different language model providers. This flexibility lets you leverage the AI services that best fit your needs.

How secure is Gabber with sensitive information?

Gabber takes security seriously. The system never stores actual secret values (like API keys) in application graphs. Instead, it references them securely from your .secret file, ensuring that even if you share your application design, your credentials remain protected. Additionally, being self-hostable gives you complete control over your data environment.

What kind of applications can I build with Gabber?

The possibilities are diverse:

• ✦ Voice assistants with emotional intelligence
• ✦ Real-time translation services
• ✦ Interactive customer support systems
• ✦ Multi-user collaborative AI tools
• ✦ Educational applications with adaptive feedback
• ✦ Any application requiring real-time processing of multiple input types

How does Gabber handle multiple users or participants?

Gabber’s architecture supports multiple participants interacting simultaneously. The system can manage separate media streams for each participant while maintaining coherent application state—essential for applications like multi-user virtual assistants or collaborative tools.

Can I customize or extend Gabber’s functionality?

Yes, one of Gabber’s strengths is its extensibility. You can create custom nodes to handle specialized functionality, develop your own SubGraphs for reuse, and integrate with virtually any external service through the available SDKs.

Is Gabber suitable for production deployments?

Gabber is designed with production use in mind. Its self-hostable nature gives you complete control over your deployment environment, and the architecture supports the performance requirements of real-world applications. Many of the example applications demonstrate patterns suitable for production use.

How does the community influence Gabber’s development?

The Gabber team actively incorporates community feedback into their development roadmap. Through GitHub discussions, issues, and Discord conversations, users directly shape the project’s direction, ensuring it addresses real-world needs rather than theoretical scenarios.

Building Your First Gabber Application: A Practical Walkthrough

Let’s walk through creating a simple voice-to-text application—perfect for getting familiar with Gabber’s workflow.

Step 1: Set Up Your Environment

Ensure you’ve completed the installation steps:

brew install livekit
curl -LsSf https://astral.sh/uv/install.sh | sh
make all

Step 2: Access the Frontend

Open your browser and navigate to http://localhost:3000.

Step 3: Create a New Application

1. Click “New Application” in the dashboard
2. Give your application a name (e.g., “Voice Transcriber”)
3. Select a template if available, or start from scratch

Step 4: Add Necessary Nodes

For a basic voice transcriber, you’ll need:

1. Audio Input Node: Captures microphone input
2. Speech-to-Text Node: Converts audio to text
3. Text Output Node: Displays the transcribed text

Drag each of these nodes onto your workspace.

Step 5: Connect the Nodes

1. Locate the Source Pad on the Audio Input Node (typically on the right side)
2. Click and drag from this pad to the Sink Pad on the Speech-to-Text Node
3. Connect the Speech-to-Text Node’s output to the Text Output Node

Step 6: Configure Node Settings

1. Click on the Speech-to-Text Node
2. In the configuration panel, select your preferred language model
3. If required, select your API key from the secrets dropdown
4. Configure any additional parameters as needed

Step 7: Run Your Application

Click the “Run” button in the top toolbar. Your application should now:

• ✦ Activate your microphone
• ✦ Process incoming audio
• ✦ Display transcribed text in real time

Step 8: Test and Refine

Speak into your microphone and watch the transcription appear. If you encounter issues:

• ✦ Check node connections
• ✦ Verify your API keys are properly configured
• ✦ Review any error messages in the logs

This simple example demonstrates Gabber’s core workflow. As you become more comfortable, you can add complexity—like emotion analysis, response generation, or multi-language support—by incorporating additional nodes.

Best Practices for Effective Gabber Development

To get the most out of Gabber, consider these practical recommendations:

1. Start Simple, Then Iterate

Begin with a minimal viable application that demonstrates core functionality. Once that works reliably, gradually add complexity. This approach makes debugging easier and helps you understand how each component contributes to the whole.

2. Organize with SubGraphs Early

As your application grows, use SubGraphs to group related functionality. This keeps your main workspace clean and makes your application more maintainable. Good candidates for SubGraphs include:

• ✦ Authentication flows
• ✦ Common processing pipelines
• ✦ Reusable UI components

3. Document Your Node Configurations

While Gabber’s visual interface is intuitive, complex applications can become difficult to understand. Add comments to your nodes and connections to explain their purpose, especially for non-obvious configurations.

4. Manage Secrets Carefully

Always use the .secret file for sensitive information rather than hard-coding values. Before sharing any application designs, double-check that no secrets have been accidentally included.

5. Leverage Example Applications

The provided examples aren’t just for learning—they’re practical starting points. Don’t hesitate to copy and modify existing examples rather than building everything from scratch.

6. Test in Stages

Validate each component of your application separately before connecting everything. Test individual nodes, then small groups of nodes, before running the complete application. This makes identifying issues much easier.

7. Monitor Performance

Pay attention to latency and resource usage, especially for real-time applications. If you notice performance issues, consider:

• ✦ Optimizing node configurations
• ✦ Reducing unnecessary processing
• ✦ Breaking complex operations into asynchronous tasks

The Future of Gabber and Real-Time AI Development

Gabber represents an important evolution in AI application development—one that prioritizes practical usability alongside technical capability. As the project continues to mature, we can expect to see:

• ✦ Expanded modality support: Integration with additional input/output types
• ✦ Enhanced performance: Continued optimization for lower latency
• ✦ More pre-built nodes: Reducing the need for custom development
• ✦ Improved collaboration features: Better tools for team-based development
• ✦ Advanced debugging tools: Making complex applications easier to troubleshoot

What makes Gabber particularly promising is its focus on solving real developer pain points rather than chasing AI hype. By providing a practical framework for building genuinely useful AI applications, Gabber helps move the field forward in a meaningful way.

Conclusion: Building the Future of AI Interactions

Gabber offers something special in the crowded AI development landscape—a practical, accessible framework for building truly interactive AI applications. Its visual, graph-based approach lowers the barrier to creating sophisticated multi-modal experiences while maintaining the flexibility needed for complex real-world applications.

Whether you’re an experienced AI developer looking to streamline your workflow or someone just starting to explore AI application development, Gabber provides the tools to bring your ideas to life. The project’s commitment to developer experience, combined with its thoughtful architecture and active community, makes it a compelling choice for anyone building the next generation of AI interactions.

The true power of Gabber isn’t just in its technical capabilities—it’s in how it empowers developers to focus on what matters most: creating meaningful, useful AI experiences that solve real problems. As you begin your journey with Gabber, remember that the most impactful AI applications aren’t the most technically impressive, but those that genuinely enhance how we interact with technology and each other.

Ready to start building? With Gabber installed and these concepts in mind, you’re well-equipped to create your first real-time, multi-modal AI application. The only limit is your imagination—and with tools like Gabber making the technical implementation more accessible, that’s exactly where your focus should be.