One Open Source Project a Day (Part 60): IndexTTS - Bilibili's Industrial Zero-Shot Text-to-Speech System

Introduction

"The Future of Voice, Now Generating."

This is Part 60 of the "One Open Source Project a Day" series. Today's project is IndexTTS (GitHub).

Want industrial-grade, controllable, efficient zero-shot text-to-speech? IndexTTS is Bilibili IndexTeam's open-source zero-shot TTS system: voice cloning, emotion control, duration control (IndexTTS2). Supports emotion reference audio, emotion vectors, text emotion descriptions; decouples emotion and timbre; multilingual (Chinese, English, etc.); WebUI and Python API; uv for dependency management.

What You'll Learn

IndexTTS core capabilities and architecture
IndexTTS2 emotion and duration control
Voice cloning, emotion reference, text emotion usage
Environment setup and WebUI quick start
Comparison with other TTS systems

Prerequisites

Basic understanding of TTS and speech synthesis
NVIDIA GPU with CUDA 12.8+ (recommended)

Project Background

Project Overview

IndexTTS is Bilibili IndexTeam's open-source industrial-grade controllable zero-shot TTS system. Users provide a reference audio clip to clone voice; emotion can be controlled independently via emotion reference audio, 8‑dimensional emotion vectors, or text descriptions. IndexTTS2 adds precise duration control for autoregressive TTS (first of its kind), supports controllable and uncontrollable modes, decouples emotion and timbre, and excels in multilingual synthesis.

Team

Team: Bilibili IndexTeam (indexspeech@bilibili.com)
Core authors: Wei Deng, Siyi Zhou, Jingchen Shu, Xun Zhou, Jinchao Wang, and others
Community: QQ groups, Discord

Project Data

⭐ GitHub Stars: 19.4k+
🍴 Forks: 2.4k+
📄 License: See repository LICENSE
🌐 Demo: IndexTTS2 demo page
📦 Models: HuggingFace, ModelScope

Core Features

Main Capabilities

Capability	Description
Zero-shot voice cloning	Single reference audio clone
Emotion control	Emotion reference audio, 8‑dim emotion vector, text emotion description
Emotion–timbre decoupling	Timbre from timbre prompt, emotion from emotion prompt; independent control
Duration control	IndexTTS2 precise duration control (first for autoregressive TTS)
Multilingual	Chinese, English, etc.; IndexTTS1.5 improves English significantly
Pinyin control	Pinyin annotations for precise pronunciation

Use Cases

Video dubbing: Strict audio–visual sync with duration control
Audiobooks / podcasts: Expressive narration
Games / virtual characters: Voice cloning + emotion control
Multilingual content: Chinese–English synthesis

Quick Start

Requirements: git, git-lfs, uv, CUDA 12.8+ (recommended)

Install:

git clone https://github.com/index-tts/index-tts.git && cd index-tts
git lfs pull
pip install -U uv
uv sync --all-extras

Download model (IndexTTS-2):

uv tool install "huggingface-hub[cli,hf_xet]"
hf download IndexTeam/IndexTTS-2 --local-dir=checkpoints

Start WebUI:

uv run webui.py

Visit http://127.0.0.1:7860.

Python example:

from indextts.infer_v2 import IndexTTS2
 
tts = IndexTTS2(cfg_path="checkpoints/config.yaml", model_dir="checkpoints",
                use_fp16=True, use_cuda_kernel=False, use_deepspeed=False)
 
# Basic voice cloning
tts.infer(spk_audio_prompt='examples/voice_01.wav', text="Translate for me, what is a surprise!",
          output_path="gen.wav", verbose=True)
 
# Emotion reference audio
tts.infer(spk_audio_prompt='examples/voice_07.wav', text="酒楼丧尽天良，开始借机竞拍房间，哎，一群蠢货。",
          output_path="gen.wav", emo_audio_prompt="examples/emo_sad.wav", verbose=True)
 
# Emotion vector [happy, angry, sad, afraid, disgusted, melancholic, surprised, calm]
tts.infer(spk_audio_prompt='examples/09.wav', text="对不起嘛！我的记性真的不太好~",
          output_path="gen.wav", emo_vector=[0, 0, 0.8, 0, 0, 0, 0, 0], verbose=True)
 
# Text emotion description
tts.infer(spk_audio_prompt='examples/voice_12.wav', text="快躲起来！是他要来了！",
          output_path="gen.wav", use_emo_text=True, emo_alpha=0.6, verbose=True)

Emotion Control Methods

Method	Parameter	Description
Emotion reference	`emo_audio_prompt`	Extract emotion from reference audio
Emotion vector	`emo_vector`	8 dims: [happy, angry, sad, afraid, disgusted, melancholic, surprised, calm]
Text emotion	`use_emo_text=True` or `emo_text`	Generate emotion from text or description
Emotion strength	`emo_alpha`	0.0–1.0, default 1.0

Project Advantages

Comparison	IndexTTS2	Traditional TTS	Other zero-shot TTS
Duration control	Precise (first for autoregressive)	Usually none	Often none
Emotion control	Multi-modal (audio/vector/text)	Limited	Varies
Emotion & timbre	Decoupled	Often coupled	Often coupled
Zero-shot	Single reference	Needs training	Varies
Multilingual	Chinese, English, etc.	Varies	Varies

Deep Dive

Version History

Version	Release	Notes
IndexTTS-1.0	2025/03	Model weights and inference
IndexTTS-1.5	2025/05	Stability, better English
IndexTTS-2	2025/09	Duration control, emotion decoupling, multi-modal emotion

Technical Highlights

Autoregressive + duration control: First precise duration control for autoregressive TTS; suited for video dubbing
GPT latent representations: Improve clarity in high-emotion speech
Three-stage training: Better generation stability
Soft instruction: Qwen3 fine-tuning for text-based emotion guidance

Recommended Settings

FP16: Lower VRAM, faster inference, minimal quality loss
DeepSpeed: May speed up on some hardware; test empirically
CUDA 12.8+: Ensure correct installation

Dependency Management

uv only: Official requirement; conda/pip may cause dependency and GPU issues
China mirrors: uv sync --all-extras --default-index "https://mirrors.aliyun.com/pypi/simple"

Project Links and Resources

Official Resources

🌟 GitHub: https://github.com/index-tts/index-tts
🌐 Demo: IndexTTS2 demo page
📦 HuggingFace: IndexTeam/IndexTTS-2
📦 ModelScope: IndexTeam/IndexTTS-2
📧 Business: indexspeech@bilibili.com
💬 Discord: discord.gg/uT32E7KDmy
🐛 Issues: GitHub Issues

Target Audience

Developers needing high-quality zero-shot TTS
Video dubbing, audiobook, podcast creators
Use cases requiring independent emotion and timbre control
Multilingual speech synthesis needs

Visit my homepage for more useful knowledge and interesting products