# VoxCPM 1.5 Setup

Generate MP3 dialogue audio from a text script with cloned voices using
VoxCPM 1.5 Continuation cloning mode.

VoxCPM 1.5 uses **Continuation cloning**: provide a reference `.wav` file
AND its exact transcript (`.txt`), and the model continues from that audio,
preserving timbre, rhythm, emotion, and style.

**Why VoxCPM 1.5:** Apache 2.0 license, low VRAM (~6 GB), works with
PyTorch 2.5+, Python 3.10+.

## Prerequisites

- NVIDIA GPU with ~6 GB free VRAM
- NVIDIA driver ≥ 550 (CUDA 12.4+). Check with `nvidia-smi` — the "CUDA Version"
  column shows your driver's max supported CUDA version.
- `uv` installed, Python ≥ 3.10

## Installation

```bash
uv venv ~/voxcpm-1.5 --python 3.12
source ~/voxcpm-1.5/bin/activate
uv pip install voxcpm soundfile

# Force PyTorch with matching CUDA runtime (cu124 for CUDA 12.4)
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# ffmpeg for concatenation and speed adjustment
sudo apt-get install -y ffmpeg
```

| System CUDA | PyTorch index URL |
|---|---|
| CUDA 12.4+ | `https://download.pytorch.org/whl/cu124` |
| CUDA 12.1–12.3 | `https://download.pytorch.org/whl/cu121` |

## File Structure

```
~/voxcpm-1.5-setup/
├── dialog.py           # Main script
├── dialog.txt          # Example dialogue (single-file mode)
├── alan.wav / alan.txt # Voice sample + transcript for "alan"
├── bruin.wav / bruin.txt
└── ai-chat/            # Example batch folder
    ├── 00-....txt
    ├── 01-....txt
    └── ...
```

Audio files must be WAV, mono, 16 kHz, PCM 16-bit (check with `file *.wav`).

## Quick Start

**Single file:**

```bash
source ~/voxcpm-1.5/bin/activate
python ~/voxcpm-1.5-setup/dialog.py ~/voxcpm-1.5-setup/dialog.txt
```

First run downloads ~1.8 GB model from HuggingFace. Output: `dialogue_final.mp3`.

**Batch (folder):**

```bash
# TTS all .txt files in ai-chat/, output MP3s alongside each .txt
python ~/voxcpm-1.5-setup/dialog.py ~/voxcpm-1.5-setup/ai-chat/ \
    --samples-dir ~/voxcpm-1.5-setup/
```

This generates one `.mp3` per `.txt` in the same folder (e.g. `ai-chat/01-conjugate-untwisting-probe-perspective.mp3`).

## Usage

```
python dialog.py <file_or_directory> [options]
```

Accepts either a single `.txt` dialogue file or a directory containing `.txt` files.

| Option | Default | Description |
|---|---|---|
| `-o FILE` | `dialogue_final.mp3` | Output MP3 (single-file mode only) |
| `--samples-dir DIR` | same as input | Directory with `.wav` + `.txt` voice samples |
| `--timesteps N` | 10 | Inference timesteps (5–20). Higher = better quality, slower |
| `--cfg FLOAT` | 2.0 | CFG value (1.0–4.0). Higher = tighter to reference voice |
| `--speaker-speeds STR` | _(none)_ | Per-speaker playback speed, e.g. `bruin=1.2,alan=0.9` |
| `--pause SEC` | 0.5 | Pause between dialogue parts in seconds |
| `--dry-run` | — | Show parsed lines without generating audio |

### Batch mode

When a directory is passed, `dialog.py` processes all `.txt` files in sorted
order. The VoxCPM model is loaded once and shared across all files. Output
`.mp3` files are placed in the same directory as each `.txt`.

If any single file fails (e.g. CUDA OOM), processing continues with the next
file. A summary shows how many files succeeded.

```bash
# Preview before generating
python dialog.py ai-chat/ --samples-dir . --dry-run

# Generate with custom speed
python dialog.py ai-chat/ --samples-dir . --speaker-speeds bruin=1.2,alan=1.0
```

### Dialogue file format

```
[speaker1]: First line of dialogue.
[speaker2]: Response.
[speaker1]: Another line.
```

Lines without `[speaker]:` prefix are appended to the previous speaker's text
as multi-paragraph speech.

### Per-speaker speed adjustment

Uses ffmpeg `atempo` to adjust playback speed of each speaker independently:

```bash
python dialog.py dialog.txt --speaker-speeds bruin=1.2,alan=0.85
```

Speed 1.0 = normal, 0.5 = half, 2.0 = double. Values outside 0.5–2.0 are
supported via chained filters.

## Voice Samples & Transcripts

- **`.wav`**: 10–30 seconds, mono 16 kHz, one speaker, no background noise/music.
- **`.txt`**: Exact transcript of the corresponding `.wav` file — **required** for
  VoxCPM 1.5 Continuation cloning. Plain text, no timestamps or metadata.

## Troubleshooting

**Running on CPU:** Check `nvidia-smi` for driver ≥ 550. Or reinstall PyTorch
with the correct CUDA index (see installation table).

**`libcudart.so.13: cannot open shared object file`:** PyTorch default is
CUDA 13 wheels. Reinstall with a matching index:
```bash
source ~/voxcpm-1.5/bin/activate
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
```

**CUDA out of memory:** Reduce timesteps: `--timesteps 5`.

**Robotic / wrong pitch output:**
- Ensure reference WAV is clean 16 kHz mono
- Ensure transcript matches audio exactly
- Try `--cfg 3.0` or `--timesteps 15`

**Model download fails mid-way:**
```bash
rm -rf ~/.cache/huggingface/hub/models--openbmb--VoxCPM1.5
huggingface-cli download openbmb/VoxCPM1.5 --local-dir ~/.cache/huggingface/hub/models--openbmb--VoxCPM1.5
```

## Audio Quality Tools

Two Python scripts for podcast audio quality assessment and enhancement.

### audio-quality-metrics.py — 音质评估

客观评估 WAV/MP3 文件的播客就绪度，输出三项核心指标：

| 指标 | 说明 | 阈值 |
|------|------|------|
| Clipping / Headroom | 削波检查 + 峰值余量 | Peak ≤ -1 dBFS |
| Spectral Balance | 频谱重心 (归一化到 Nyquist) | ≥ 22% |
| Noise / SNR | 非侵入式信噪比估计 | ≥ 20 dB |

**用法:**
```bash
# 单文件评估 (非侵入式)
python3 audio-quality-metrics.py input.wav

# 双文件对比 (侵入式: PESQ + STOI + SNR)
python3 audio-quality-metrics.py reference.wav enhanced.wav
```

**依赖:** `uv pip install soundfile librosa pystoi pesq pyloudnorm numpy` (缺失时自动回退到 stdlib/scipy)

---

### audio-quality-enhance.py — 音频增强

完整的 DSP 处理链，替代 Adobe Podcast Enhance：

1. **Denoise** — noisereduce 谱门限降噪 (stationary/nonstationary/both)
2. **High-pass filter** — 切除低频隆隆声 (默认 80Hz)
3. **Parametric EQ** — 低频衰减 + 中频提升
4. **Noise gate** — 静音段压制残留噪声
5. **Loudness normalize** — ITU-R BS.1770 归一化到 -16 LUFS
6. **Lookahead limiter** — 透明 brickwall 限制器 (无硬削波)

**用法:**
```bash
# 默认增强
python3 audio-quality-enhance.py input.wav

# 平稳噪声模式 (风扇/空调)
python3 audio-quality-enhance.py input.wav --denoise-mode stationary

# 跳过降噪 (源文件已干净)
python3 audio-quality-enhance.py input.wav --no-denoise

# 自定义参数
python3 audio-quality-enhance.py input.wav --bass-cut -6 --presence-boost 4 --target-lufs -18
```

**依赖:** `uv pip install noisereduce numpy scipy` (缺失时自动回退到 stdlib wave)

---

### 工作流

```
录音 → audio-quality-metrics.py 评估
     → 不达标 → audio-quality-enhance.py 增强
     → audio-quality-metrics.py 复评
     → 达标 → 用于 VoxCPM / 播客发布
```