Commit Graph

2 Commits

Author SHA1 Message Date
kshitij 3277eb8872 refactor(stt): fold review findings into the cloud silence trim
Three-reviewer pass (reuse / quality / efficiency) on the trim diff;
four findings folded:

1. Short-clip input gate (efficiency, HIGH): the trim previously paid
   the full ffmpeg encode before the <10%-saving discard check — every
   dense conversational voice note burned 3 subprocess spawns + a
   complete re-encode on the synchronous response path for nothing.
   New _CLOUD_TRIM_MIN_INPUT_SECONDS=12 gate: below it, savings can't
   matter (a >=10% saving is ~1s of audio, and several providers bill
   a per-request minimum anyway — Groq bills 10s minimum), so the
   whole pipeline is skipped using the duration we already probed.
   Typical 5-10s voice notes now pay 1 ffprobe (~50ms), not 3 spawns +
   encode (~0.3-1s; multi-second on small-VPS gateway hosts).

2. Shared encode profile (reuse, HIGH): the trim's ffmpeg command
   duplicated _transcode_audio_for_stt's encode byte-for-byte (same
   16kHz/mono/AAC-32k/faststart args, same subprocess.run kwargs).
   Extracted _STT_M4A_ENCODE_ARGS + _run_ffmpeg_stt_encode(ffmpeg,
   in, out, audio_filter=None); both call sites now share one owner,
   so codec/bitrate/timeout changes can't drift between the paths.

3. is_truthy_value for the enable flag (quality, MEDIUM): raw
   bool(cfg.get(...)) treated a YAML string "false" as enabled — the
   exact bug class utils.is_truthy_value (already imported, already
   used by is_stt_enabled and the xai/elevenlabs flags) exists for.

4. All-silence guard scales with keep_ms (quality, LOW): the fixed
   0.3s floor equals the default keep window, so an output consisting
   solely of one kept pause could pass as "speech"; now
   max(0.3, 2*keep_seconds).

Also: _probe_audio_duration docstring documents it as the canonical
sync seconds-probe (gateway/run.py and the Telegram adapter carry
local variants of the same ffprobe invocation).

Tests: 24 now — YAML-string-false disables; short clips skip the
encode entirely (encoder mock asserted not-called); E2E fixtures
moved past the input gate. E2E re-verified: 13.2s note -> 6.2s
(-53%), 8s clip skipped with 1 probe.
2026-08-07 19:26:04 +05:30
kshitij a683ef95d2 feat(stt): pre-upload silence trim for cloud providers
Local faster-whisper gets Silero VAD (bf8004e3a) so silence never
reaches the model. Cloud providers got no such protection: the raw
file uploads untouched, so every second of silence in a voice note is
paid for twice — upload time and per-audio-minute billing — and cloud
Whisper hallucinates junk tokens on silent stretches exactly like
local Whisper did before the VAD hardening. A 13s voice note with two
long pauses is billed as 13s of audio to transcribe ~6s of speech.

Close the gap client-side: before uploading to a built-in cloud
provider (groq/openai/mistral/xai/elevenlabs/deepinfra), collapse long
pauses with ffmpeg's silenceremove filter, keeping
stt.cloud_trim_keep_ms (default 300) of every pause so word boundaries
and natural pacing survive. Uses ffmpeg, already a dependency of this
exact path via _transcode_audio_for_stt — no new dependency.

The trim is strictly best-effort — ALL of these upload the original
untouched, transcription never fails because of the trim:
  - stt.cloud_trim_silence: false
  - ffmpeg/ffprobe missing, trim failure, or timeout
  - trimmed result ~empty (mostly-silence clip: the provider, not a
    client-side dB heuristic, decides whether it contains speech)
  - trim saves <10% (re-encoding for nothing)

Command-type and plugin providers are deliberately NOT trimmed: they
may wrap local CLIs that want the original bytes or run their own VAD.

E2E (real ffmpeg + faster-whisper): 13.2s voice note with 7s pause ->
6.2s upload (-53%); transcript of trimmed audio matches the original
on both utterances. Dense-speech and all-silence WAVs correctly fall
back to the original. 22 unit+E2E tests; STT/voice suite failures
identical to upstream/main baseline (all pre-existing).
2026-08-07 19:26:04 +05:30