feat(tts): Inflect v2 (Micro/Nano) VITS CoreML conversion + MiniMax benchmark

models/tts/inflect-v2/coreml/.gitignore

@@ -0,0 +1,2 @@
+checkpoints/
+build/

models/tts/inflect-v2/coreml/README.md

@@ -0,0 +1,138 @@
+# Inflect v2 (Micro / Nano) → CoreML
+
+Conversion of [owensong/Inflect-Micro-v2](https://huggingface.co/owensong/Inflect-Micro-v2) (9.36M params)
+and [owensong/Inflect-Nano-v2](https://huggingface.co/owensong/Inflect-Nano-v2) (3.97M params) —
+ultra-tiny VITS-family end-to-end TTS, 24 kHz mono, Apache-2.0.
+
+## Pipeline split
+
+The VITS inference graph is split into two fixed-shape, fully deterministic CoreML models.
+Everything stochastic or dynamically shaped runs on the host:
+
+```
+text ── espeak frontend ── tokens (interspersed with blanks, pad to T_text=256)
+                              │
+              ┌───────────────▼────────────────┐
+              │ encoder.mlpackage              │  tokens + x_mask
+              │ TextEncoder + DurationPredictor│  → m_p, logs_p, logw   [1, C, 256]
+              └───────────────┬────────────────┘
+                              │ host: w = ceil(exp(logw))·length_scale
+                              │ host: repeat-expand m_p/logs_p to frames
+                              │ host: z_p = m_p + randn·exp(logs_p)·noise_scale
+              ┌───────────────▼────────────────┐
+              │ synthesizer.mlpackage          │  z_p + y_mask  [1, C, T_frames]
+              │ reverse coupling flow + HiFiGAN│  → waveform    [1, 1, T_frames·256]
+              └────────────────────────────────┘
+                              │ host: trim to y_len·256 samples
+```
+
+- `use_sdp: false` in both configs → deterministic duration predictor, no noise inside either model.
+- All masking is host-provided (`x_mask`, `y_mask`), so padded positions stay zero through the
+  masked encoder/flow. Only the unmasked HiFiGAN decoder sees pad bleed-through, limited to its
+  receptive field at the valid/pad boundary (audio parity below includes this effect).
+- Config-driven: Micro and Nano share runtime code; only channel widths differ.
+
+## Commands
+
+```bash
+uv sync
+# both variants, fp16, T_text=256 tokens / T_frames=1024 frames (10.9 s audio budget)
+uv run python convert-coreml.py
+uv run python compare-models.py --variant micro
+uv run python compare-models.py --variant nano
+```
+
+## Parity (fp16, M5 Pro, macOS 26.6, seed 0, "The quick brown fox…")
+
+| stage | metric | Micro | Nano |
+|---|---|---|---|
+| encoder m_p | max_abs / rel_l2 | 0.0032 / 0.0005 | 0.0034 / 0.0006 |
+| encoder logw | max_abs / rel_l2 | 0.0018 / 0.0008 | 0.0020 / 0.0008 |
+| durations | frame diff vs torch | 0 | 0 |
+| audio (same z_p) | corr / rel_l2 | 0.99991 / 0.0136 | 0.99996 / 0.0096 |
+
+## Profiling (coreml-cli, warm predicts, Micro)
+
+| model | shape | best unit | predict | CPU+NE split |
+|---|---|---|---|---|
+| encoder | T_text=256 | any (~1 ms) | 1.2 ms | 26% ANE |
+| synthesizer | T_frames=1024 (10.9 s) | GPU | 33.4 ms (~330× RT) | 17% ANE / 83% CPU |
+| synthesizer | T_frames=256 (2.7 s) | GPU | 9.9 ms (~275× RT) | **77% ANE** / 23% CPU |
+
+### ANE findings
+
+- At `T_frames=1024` the waveform-rate tensors hit the ANE tensor-width limit
+  (`W ≤ 65536`; final stages are W=131072/262144) → almost everything falls back.
+- At `T_frames=256` (2.7 s bucket, final W=65536 exactly at the limit) residency jumps to 77%.
+  Remaining fallbacks: 8 dilated resblock convs ("space-to-batch exceeds L2 DMA buffer") and
+  2 convs at W=65538 (same-pad overshoot past the limit).
+- GPU is fastest at every size; ANE only matters for power. Full-ANE would need the
+  kokoro-ane treatment (reshape waveform rate to 2D, split dilated convs) — not pursued here.
+
+## Trace notes (future VITS conversions)
+
+- `attentions.MultiHeadAttention` relative-position helpers do Python `max()` over traced sizes →
+  coremltools `aten::int` crash. Patched at conversion time to coerce lengths with `int()`
+  (static shapes make them constants). See `_static_relative_attention`.
+- `commons.fused_add_tanh_sigmoid_multiply` indexes an `IntTensor` channel count →
+  `intimplicit` not implemented. Patched to a Python-int slice. See `_patch_fused_activation`.
+- Weight norm must be removed (`dec` + flow WN layers) before tracing, matching the
+  reference `optimize_for_inference`.
+
+## Host responsibilities (Swift port checklist)
+
+1. espeak-ng en-us phonemization with stress (same frontend family as existing G2P work),
+   keithito symbol table, blank interspersal (`add_blank: true`).
+2. Duration expansion: `repeat` each token column `ceil(exp(logw)/speed)` times.
+3. `z_p = m_p + N(0,1)·exp(logs_p)·noise_scale` (default noise_scale 0.667).
+4. Trim output to `y_len·256` samples; optional 5 ms edge fade + inter-chunk pauses
+   (see upstream `inference.py`).
+
+## MiniMax-English benchmark vs FluidAudio TTS backends
+
+`benchmark-minimax.py` runs the FluidAudio `tts-benchmark` methodology (100
+MiniMax-English phrases, warm one-shot latency, agg RTFx) over the CoreML
+pipeline: persistent espeak G2P → t512 encoder → host expansion/noise →
+smallest fitting synthesizer bucket {256, 512, 1024, 2048 frames}. WER/CER
+scored with the identical Parakeet TDT + TextNormalizer path via
+`fluidaudio tts-asr-verify --score-only` (added for this purpose).
+
+M5 Pro, macOS 26.6, `ComputeUnit.ALL` (routes to GPU), fp16, one bucket set
+preloaded (cold start = first compile of all buckets). FluidAudio rows from
+`Documentation/TTS/Benchmarks.md` (Kokoro re-baselined same day, same host):
+
+| Backend | Weights | Synth p50/p95 | Agg RTFx | WER | CER |
+|---|---|---|---|---|---|
+| **Inflect Micro CoreML** | 19 MB fp16 (9.4M) | **25.7 / 35.5 ms** | **245×** | 1.43% | 0.44% |
+| **Inflect Nano CoreML** | 8 MB fp16 (4.0M) | **13.2 / 16.9 ms** | **460×** | 1.92% | 0.57% |
+| Kokoro ANE (en) | ~330 MB (82M) | 241 / 305 ms | 31× | 0.68% | 0.15% |
+| PocketTTS (en, streaming) | ~330 MB | 933 / 1233 ms (TTFT 26 ms) | 6.5× | 0.51% | 0.08% |
+| Supertonic-3 (int4) | ~100 MB | 81 / 120 ms | 94× | 1.02% | 0.31% |
+| StyleTTS2 (M2 numbers) | ~670 MB | 1574 / 3088 ms | 4.6× | 9.4% | 4.1% |
+
+Notes:
+- Synthesizer cost is bucket-quantized and ~linear in frames (micro ≈
+  0.033 ms/frame on GPU). The first run used coarse buckets {256, 512, 1024,
+  2048}; 75% of phrases (p50 = 621 frames) overpaid in f1024 (p50 37 ms /
+  192×). The shipped set {256, 384, 512, 640, 768, 896, 1024, 2048} tracks
+  the corpus distribution — at 128-frame granularity, padding overhead is
+  within ~5% of exact-shape synthesis, so dynamic shapes buy nothing more.
+- fp16 model I/O and `CompiledMLModel` dispatch were A/B'd at f768: both
+  within noise (~25.3 ms either way) — the GPU is compute-bound, casts and
+  dispatch don't matter. Kept fp32 I/O.
+- Upstream claims 6.28× realtime on 4-thread CPU for Micro; the CoreML GPU
+  pipeline is ~30× that.
+- The upstream Python frontend costs ~450 ms/phrase constructing a fresh
+  espeak backend per `phonemize()` call; the runner uses one persistent
+  `EspeakBackend` (byte-identical phonemes, ~0.4 ms) — this matches what an
+  in-process Swift port would do. First runs with the stock frontend read
+  p50 457 ms / RTFx 13.8× — that artifact was 85% of the latency.
+- Peak RSS (Micro 628 MB / Nano 587 MB) is the whole Python process
+  (interpreter + numpy + 4 synthesizer buckets) — not comparable to the
+  Swift harness numbers.
+- Quality-per-byte is the headline: Nano is 1/40th the weights of Kokoro at
+  ~3× the (still very low) WER; Micro at 19 MB beats Supertonic-3's WER.
+
+## Assets
+
+`checkpoints/` (downloaded via `hf download`, not committed) and `build/` outputs are local-only.

models/tts/inflect-v2/coreml/benchmark-minimax.py

@@ -0,0 +1,170 @@
+"""MiniMax-English benchmark for the Inflect v2 CoreML pipeline.
+
+Mirrors FluidAudio's `tts-benchmark` metrics (one-shot backend): per-phrase
+warm synth ms (TTFT == synth), p50/p95, aggregate RTFx, cold start, peak RSS.
+Synthesis = espeak G2P + encoder (t512) + host expansion/noise + smallest
+fitting synthesizer bucket. WAVs land as phrase_%03d.wav for WER scoring with
+`fluidaudio tts-asr-verify --score-only`.
+"""
+
+import argparse
+import json
+import resource
+import sys
+import time
+from pathlib import Path
+
+import coremltools as ct
+import numpy as np
+import soundfile as sf
+
+ROOT = Path(__file__).resolve().parent
+HOP = 256
+SAMPLE_RATE = 24000
+
+
+def load_frontend(checkpoint_dir: Path):
+    sys.path.insert(0, str(checkpoint_dir / "runtime"))
+    sys.path.insert(0, str(checkpoint_dir))
+    import commons
+    from inflect_nano_v2_frontend import _configure_espeak, normalize_text
+    from inflect_vits_frontend import _apply_phoneme_overrides
+    from text import cleaned_text_to_sequence
+
+    # The upstream frontend's phonemize() convenience constructs a fresh
+    # espeak backend per call (~450 ms). A persistent backend matches what
+    # an in-process port would do and produces identical phonemes.
+    _configure_espeak()
+    from phonemizer.backend import EspeakBackend
+
+    backend = EspeakBackend("en-us", preserve_punctuation=True, with_stress=True)
+
+    def tokenize(text: str) -> list[int]:
+        phonemes = backend.phonemize([normalize_text(text)], strip=True, njobs=1)[0]
+        phonemes = _apply_phoneme_overrides(phonemes)
+        sequence = cleaned_text_to_sequence(phonemes)
+        return commons.intersperse(sequence, 0)
+
+    return tokenize
+
+
+def edge_fade(waveform: np.ndarray, milliseconds: float = 5.0) -> np.ndarray:
+    frames = min(round(SAMPLE_RATE * milliseconds / 1000.0), waveform.size // 2)
+    if frames <= 0:
+        return waveform
+    ramp = np.linspace(0.0, 1.0, frames, endpoint=True, dtype=np.float32)
+    waveform = waveform.copy()
+    waveform[:frames] *= ramp
+    waveform[-frames:] *= ramp[::-1]
+    return waveform
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--variant", choices=["micro", "nano"], required=True)
+    parser.add_argument("--corpus", type=Path, required=True)
+    parser.add_argument("--audio-dir", type=Path, required=True)
+    parser.add_argument("--output-json", type=Path, required=True)
+    parser.add_argument("--noise-scale", type=float, default=0.667)
+    parser.add_argument(
+        "--buckets", default="256,512,1024,2048",
+        help="comma-separated synthesizer frame buckets (build dirs must exist)")
+    parser.add_argument("--io-fp16", action="store_true", help="use -io16 fp16-I/O bundles")
+    args = parser.parse_args()
+
+    phrases = [
+        l.strip()
+        for l in args.corpus.read_text().splitlines()
+        if l.strip() and not l.strip().startswith("#")
+    ]
+    tokenize = load_frontend(ROOT / "checkpoints" / f"inflect-{args.variant}-v2")
+    args.audio_dir.mkdir(parents=True, exist_ok=True)
+
+    prefix = f"build/inflect-{args.variant}-v2-fp16"
+    suffix = "-io16" if args.io_fp16 else ""
+    io_dtype = np.float16 if args.io_fp16 else np.float32
+    bucket_list = sorted(int(b) for b in args.buckets.split(","))
+    # f1024 predates the t512 conversions; the synthesizer is t_text-independent.
+    t_for = lambda frames: 256 if (frames == 1024 and not args.io_fp16) else 512
+    cold0 = time.perf_counter()
+    encoder = ct.models.MLModel(
+        f"{prefix}-t512-f256/encoder.mlpackage", compute_units=ct.ComputeUnit.ALL)
+    synths = {
+        frames: ct.models.MLModel(
+            f"{prefix}-t{t_for(frames)}-f{frames}{suffix}/synthesizer.mlpackage",
+            compute_units=ct.ComputeUnit.ALL)
+        for frames in bucket_list
+    }
+    t_text = 512
+
+    def synth(text: str, seed: int) -> tuple[np.ndarray, float]:
+        t0 = time.perf_counter()
+        tokens = tokenize(text)
+        n_tok = len(tokens)
+        assert n_tok <= t_text, f"{n_tok} tokens exceed t_text={t_text}: {text}"
+        tokens_pad = np.zeros((1, t_text), dtype=np.int32)
+        tokens_pad[0, :n_tok] = tokens
+        x_mask = np.zeros((1, 1, t_text), dtype=np.float32)
+        x_mask[0, 0, :n_tok] = 1.0
+        enc = encoder.predict({"tokens": tokens_pad, "x_mask": x_mask})
+        w_ceil = np.ceil(np.exp(enc["logw"][0, 0, :n_tok])).astype(np.int64)
+        y_len = int(w_ceil.sum())
+        buckets = sorted(synths)
+        assert y_len <= buckets[-1], f"{y_len} frames exceed largest bucket: {text}"
+        t_frames = next(b for b in buckets if y_len <= b)
+        idx = np.repeat(np.arange(n_tok), w_ceil)
+        m_exp = enc["m_p"][:, :, :n_tok][:, :, idx]
+        logs_exp = enc["logs_p"][:, :, :n_tok][:, :, idx]
+        noise = np.random.default_rng(seed).standard_normal(m_exp.shape, dtype=np.float32)
+        z_p = np.zeros((1, m_exp.shape[1], t_frames), dtype=io_dtype)
+        z_p[:, :, :y_len] = m_exp + noise * np.exp(logs_exp) * args.noise_scale
+        y_mask = np.zeros((1, 1, t_frames), dtype=io_dtype)
+        y_mask[0, 0, :y_len] = 1.0
+        out = synths[t_frames].predict({"z_p": z_p, "y_mask": y_mask})
+        audio = out["audio"][0, 0, : y_len * HOP].astype(np.float32)
+        audio = np.clip(edge_fade(audio), -1.0, 1.0)
+        return audio, (time.perf_counter() - t0) * 1000
+
+    # Cold start = model loads; first synth = first end-to-end call (warm-up).
+    cold_s = time.perf_counter() - cold0
+    first0 = time.perf_counter()
+    synth("Initialization warm-up.", seed=999)
+    first_ms = (time.perf_counter() - first0) * 1000
+    print(f"cold start {cold_s:.2f}s  first synth {first_ms:.0f} ms")
+
+    per_phrase = []
+    total_audio_s = 0.0
+    total_synth_s = 0.0
+    for i, text in enumerate(phrases):
+        audio, ms = synth(text, seed=i)
+        audio_s = len(audio) / SAMPLE_RATE
+        sf.write(args.audio_dir / f"phrase_{i + 1:03d}.wav", audio, SAMPLE_RATE)
+        per_phrase.append({"index": i + 1, "text": text, "synth_ms": ms, "audio_s": audio_s})
+        total_audio_s += audio_s
+        total_synth_s += ms / 1000
+        print(f"[{i + 1:03d}/{len(phrases)}] {ms:6.1f} ms  {audio_s:5.2f}s  {text[:50]}")
+
+    lat = sorted(p["synth_ms"] for p in per_phrase)
+    p50 = lat[len(lat) // 2]
+    p95 = lat[min(len(lat) - 1, round((len(lat) - 1) * 0.95))]
+    peak_rss_mb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1e6
+    summary = {
+        "backend": f"inflect-{args.variant}-v2-coreml",
+        "corpus": args.corpus.name,
+        "phrase_count": len(phrases),
+        "cold_start_s": cold_s,
+        "first_synth_ms": first_ms,
+        "ttft_ms_p50": p50,
+        "ttft_ms_p95": p95,
+        "warm_synth_ms_p50": p50,
+        "warm_synth_ms_p95": p95,
+        "agg_rtfx": total_audio_s / total_synth_s,
+        "peak_rss_mb": peak_rss_mb,
+        "total_audio_s": total_audio_s,
+    }
+    args.output_json.write_text(json.dumps({"summary": summary, "phrases": per_phrase}, indent=1))
+    print(json.dumps(summary, indent=1))
+
+
+if __name__ == "__main__":
+    main()