OpenCut/apps/web/src/media
Andrés Villagrán 7204146467 feat(media): support browsers without WebCodecs via FFmpeg + HTML5 fallbacks
OpenCut's video preview and audio playback both rely on mediabunny, which
in turn requires the WebCodecs API (VideoDecoder / AudioDecoder). On
browsers where WebCodecs is not exposed (e.g. Linux Chromium builds
without proprietary codec support), import and playback both throw
"codec not supported" errors and the editor becomes unusable.

This change makes the editor work on those browsers without sacrificing
the WebCodecs fast-path elsewhere:

- Add a server-side /api/convert-video route that transcodes incoming
  videos to H.264 baseline 3.1 yuv420p via the system ffmpeg binary.
  processMediaAssets uses it automatically when readVideoFile reports
  canDecode=false so the asset stored is always something the browser
  can play.
- Add HTMLVideoElementSink, a CanvasSink-shaped sink backed by a
  hidden <video> element + canvas.drawImage. VideoCache.initializeSink
  selects it only when WebCodecs VideoDecoder is unavailable; otherwise
  it keeps using mediabunny's CanvasSink unchanged.
- Add FallbackAudioBufferSink, an AudioBufferSink-shaped sink backed by
  AudioContext.decodeAudioData (which uses the platform's native audio
  decoders, not WebCodecs). AudioManager and the import-time decode
  path swap to it on canDecode=false. runClipIterator is wrapped in a
  try/catch so a sink failure no longer leaves an unhandled rejection.

Tests cover frame timing, dispose semantics, FPS clamping, the
WebCodecs/AudioDecoder availability probes, AudioBuffer slicing, and
the fallback iterator contract.
2026-05-16 20:23:54 -04:00
..
audio-mastering.ts refactor: restructure files to their domains, new preview overlay system, and dep graph 2026-04-20 11:31:17 +02:00
audio.ts feat(media): support browsers without WebCodecs via FFmpeg + HTML5 fallbacks 2026-05-16 20:23:54 -04:00
ffmpeg-convert.ts feat(media): support browsers without WebCodecs via FFmpeg + HTML5 fallbacks 2026-05-16 20:23:54 -04:00
media-utils.ts refactor: restructure files to their domains, new preview overlay system, and dep graph 2026-04-20 11:31:17 +02:00
mediabunny.ts fix: surface unsupported video codec on import, fix Input leaks 2026-04-25 02:46:31 +02:00
processing.ts feat(media): support browsers without WebCodecs via FFmpeg + HTML5 fallbacks 2026-05-16 20:23:54 -04:00
thumbnail.ts fix: surface unsupported video codec on import, fix Input leaks 2026-04-25 02:46:31 +02:00
types.ts refactor: restructure files to their domains, new preview overlay system, and dep graph 2026-04-20 11:31:17 +02:00
upload-toast.ts refactor: restructure files to their domains, new preview overlay system, and dep graph 2026-04-20 11:31:17 +02:00
use-file-upload.ts refactor: thinner hooks 2026-04-26 17:31:28 +02:00
use-paste-media.ts chore: switch from biome to eslint + prettier; fix ton of lint issues 2026-05-02 03:27:23 +02:00
waveform-summary.ts refactor: restructure files to their domains, new preview overlay system, and dep graph 2026-04-20 11:31:17 +02:00