🤖 AI Voice Agent with Virtual Avatar

📸 Screenshots

🖥️ Main Interface

Clean and modern interface with real-time connection status

🎥 Video Call with AI Avatar

Full-screen video experience with virtual avatar interaction

✨ Features

🎭 Virtual Avatar Integration

• Tavus-powered avatars for lifelike video interactions
• Real-time lip sync with voice responses
• Customizable avatar personas and appearances
• Seamless video streaming through LiveKit

🗣️ Advanced Voice Capabilities

• Real-time speech-to-text with high accuracy
• Natural voice synthesis with multiple voice options
• Low-latency audio processing for smooth conversations
• Noise cancellation and audio enhancement

🧠 AI-Powered Intelligence

• OpenAI GPT integration for intelligent responses
• Context-aware conversations with memory
• Custom prompt engineering for specific use cases
• Multi-turn dialogue support

🔧 MCP Protocol Integration

• Extensible tool system via Model Context Protocol
• Custom function calling capabilities
• Dynamic tool registration and management
• Secure tool execution environment

🌐 Modern Web Interface

• Responsive design for all devices
• Real-time connection status indicators
• Beautiful UI components with smooth animations
• Dark/light theme support

📁 Project Structure

📦 ai-voice-agent/
├── 🎨 frontend/                    # Next.js web application
│   ├── 📱 app/                     # Next.js App Router
│   │   ├── (app)/                  # Main app routes
│   │   ├── api/                    # API endpoints
│   │   └── components/             # Page components
│   ├── 🧩 components/              # Reusable UI components
│   │   ├── livekit/               # LiveKit-specific components
│   │   ├── ui/                    # Base UI components
│   │   └── session-view.tsx       # Main session interface
│   ├── 🎣 hooks/                   # Custom React hooks
│   │   ├── useChatAndTranscription.ts
│   │   ├── useConnectionDetails.ts
│   │   └── useDebug.ts
│   └── 📦 package.json             # Frontend dependencies
├── 🤖 agent.py                     # Main AI agent server
├── 🔧 mcp_client/                  # MCP client implementation
│   ├── agent_tools.py              # Agent tool definitions
│   ├── server.py                   # MCP server interface
│   └── util.py                     # Utility functions
├── 💬 prompts.py                   # AI prompt templates
├── 🛠️ tools.py                     # Agent tools and functions
├── 📋 requirements.txt             # Python dependencies
└── 🔐 SSL certificates/            # Local development certificates

🚀 Quick Start

Prerequisites

• Python 3.11+ with pip
• Node.js 18+ with npm/pnpm
• LiveKit account and credentials
• OpenAI API key
• Tavus API key (for avatar features)

🛠️ Installation

1. Clone the Repository

git clone https://github.com/tr00x/mamut.git
cd mamut

3. Frontend Setup

# Navigate to frontend directory
cd frontend

# Install Node.js dependencies
npm install
# or
pnpm install

4. Environment Configuration

Create .env file in the root directory:

# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here

# LiveKit Configuration
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

# Tavus Configuration (for avatar)
TAVUS_API_KEY=your_tavus_api_key
TAVUS_REPLICA_ID=your_replica_id

# MCP Configuration
MCP_SERVER_URL=http://localhost:8080

Create frontend/.env.local:

# LiveKit Frontend Configuration
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

🎯 Running the Application

Terminal 1 - Start the AI Agent

# Activate virtual environment
source venv/bin/activate

# Start agent in development mode
python agent.py dev

Terminal 2 - Start the Frontend

# Navigate to frontend
cd frontend

# Start development server
npm run dev

🌐 Access the Application

Open your browser and navigate to: http://localhost:3000

🎮 Demo

🖥️ Desktop Experience

• Full-screen video interface with virtual avatar
• Real-time voice interaction with visual feedback
• Responsive controls and status indicators

🙏 Acknowledgments

• LiveKit - Real-time communication platform
• voice-agent-ui - Frontend foundation
• OpenAI - AI language model
• Tavus - Virtual avatar technology
• MCP - Model Context Protocol