VTuber Game - Live2D Framework Documentation

Welcome to the comprehensive documentation for the VTuber Game project - a powerful Live2D framework with enhanced lipsync capabilities built on PixiJS.

🎯 What is This Project?

This project is a universal Live2D framework that demonstrates advanced integration between Live2D models and web technologies, featuring:

Universal Model Support: Compatible with Cubism 2.1, 3, and 4 models
Advanced Lipsync: Real-time lip synchronization with audio using a patched library
Text-to-Speech Integration: Browser TTS with motion synchronization
Interactive Controls: Comprehensive model control interface
Performance Optimized: Efficient rendering and memory management

🏁 Getting Started

Set up the project and run your first Live2D model

🏗️ Architecture

Understand the system design and component structure

📚 API Reference

Complete API documentation with examples

⚙️ Development

Developer guides and contribution instructions

🌟 Key Features

Core Live2D Capabilities

Multi-Version Support: Seamless handling of all Cubism versions
Universal API: Simplified, unified interface for all model types
Interactive Events: Touch, drag, and gesture support
Expression Management: Dynamic facial expression control
Motion Control: Comprehensive animation system

Enhanced Audio Features

Lipsync Technology: Real-time mouth movement synchronization
TTS Integration: Browser text-to-speech with motion triggers
Audio Processing: Advanced audio analysis for natural lip movement
CORS Support: Cross-origin audio file handling

Developer Experience

TypeScript Support: Full type definitions for better development
Modern Build Tools: Vite-powered development and building
Comprehensive Testing: Model testing interface
Debug Tools: Built-in debugging and diagnostic functions

🔧 The Lipsync Patch

This project uses a custom-patched version of the pixi-live2d-display library that adds advanced lipsync capabilities:

// Enhanced speak function with lipsync
model.speak(audioPath, {
  volume: 1.0,
  expression: 4,
  resetExpression: true,
  crossOrigin: "anonymous",
  onFinish: () => console.log("Speech completed"),
  onError: (err) => console.error("Audio error:", err),
});

The patch enables:

Real-time audio analysis
Automatic mouth parameter adjustment
Expression preservation during speech
Error handling and callbacks

📖 Documentation Structure

This documentation is organized into several key sections:

Getting Started - Quick setup and basic usage
Architecture - System design and components
API Reference - Complete method and property documentation
Development - Advanced development topics
Examples - Practical implementation examples
Troubleshooting - Common issues and solutions
Expression Fix - 🆕 Cubism version compatibility fixes

🔧 Recent Updates

Expression System Fix (Latest)

Fixed cross-version compatibility for Live2D expressions:

✅ Cubism 2.1 Support: Shizuku model expressions now working
✅ Cubism 4.0 Support: Haru and Cyan model expressions maintained
✅ Automatic Detection: System auto-detects and handles both versions
✅ Comprehensive Testing: Full test suite for validation

Learn more about the expression fix →

🤝 Contributing

This project welcomes contributions! Whether you’re:

Adding new features
Improving documentation
Reporting bugs
Creating examples

Check our Development Guide for contribution guidelines.

📄 License

This project is for testing and demonstration purposes. Live2D models are redistributed under Live2D’s Free Material License.