How far can FFmpeg go in the browser? Single-thread, multi-thread, and SharedArrayBuffer explained

The browser is running a WebAssembly build of FFmpeg, not a built-in FFmpeg command

Chrome or Edge is not exposing the native ffmpeg command directly. FFmpeg source code is compiled to WebAssembly with a toolchain such as Emscripten, and JavaScript manages that runtime. Because a browser cannot freely read arbitrary paths on the user's computer, media is passed in through user-approved browser file APIs instead.

Compiling to WebAssembly also does not automatically include every FFmpeg feature. Decoders, encoders, muxers, filters, and optional libraries are selected at build time, which affects both WASM size and capability. A command that works in a desktop FFmpeg build therefore is not guaranteed to exist in a browser build.

Single-thread FFmpeg can still run away from the UI main thread

The name single-thread can sound as though the entire page, including its UI, must run on one thread. That is not necessary. FFmpeg WASM itself can run inside a Web Worker, separating heavy media work from the main thread that manages the interface and returning progress messages back to the page.

It is more useful to think of single-thread here as an FFmpeg runtime that does not use multiple Pthreads. The ST build also has a concrete advantage: it can avoid SharedArrayBuffer and cross-origin-isolation requirements, which makes distribution much simpler.

SharedArrayBuffer is at the center of the multi-thread build

Multiple FFmpeg threads need more than independent worker copies. The workers need access to the same WebAssembly memory and a way to synchronize. Emscripten's Pthreads implementation therefore uses SharedArrayBuffer.

SharedArrayBuffer lets multiple workers reference shared memory, with Atomics providing synchronization primitives. This makes a browser execution model closer to native multithreading possible, but workers and synchronization have costs of their own, so increasing the thread count does not produce linear speedups.

SharedArrayBuffer changes how the HTML has to be delivered

On current major browsers, the normal way to expose SharedArrayBuffer to a web page requires a secure context and cross-origin isolation. A common configuration uses Cross-Origin-Opener-Policy: same-origin together with Cross-Origin-Embedder-Policy: require-corp (or credentialless) as HTTP response headers. The page can inspect the resulting state through crossOriginIsolated.

This is a major difference from the ST build. An HTML file opened directly with file:// cannot receive those server response headers. Browser Kitty therefore keeps the standard FFmpeg Filter Builder build usable as a portable file:// HTML, while the MT build assumes HTTP(S) plus cross-origin isolation. Browser Kitty also sends Cross-Origin-Resource-Policy: same-origin for that app, but CORP is an additional restriction rather than the direct requirement that makes crossOriginIsolated true.

Multi-thread does not mean automatically or proportionally faster

Video re-encoding and complex filter graphs can be CPU intensive, so an MT build can benefit from multiple cores. ffmpeg.wasm's published performance examples include workloads where the multi-thread build is faster than the single-thread build. Those numbers belong to specific versions, machines, and operations, however; they are not a promise that every workload scales by the same factor.

Some work parallelizes better than others, while worker synchronization, memory management, and thread startup add overhead. MT also tends to use more memory because it adds workers and shared runtime state. For browser video processing, practical limits can therefore come from device memory before raw CPU throughput.

Browser Kitty isolates only the app that needs MT FFmpeg

Cross-origin isolation is not necessarily something to apply to an entire site indiscriminately. COEP changes the conditions under which cross-origin images, iframes, scripts, and other resources can be embedded, which can affect ads and third-party content. Browser Kitty therefore applies COOP / COEP only to /apps/ffmpeg-filter-builder.html through an Azure Static Web Apps route, leaving other tools and portal pages unchanged.

The FFmpeg Filter Builder repository's GitHub Pages MT page has a same-origin coi-serviceworker fallback that loads only when crossOriginIsolated is false, for hosting where the required headers cannot be configured directly. Browser Kitty production does not depend on that fallback because the server response headers make the page cross-origin isolated from the start.

Browser FFmpeg is not a drop-in replacement for native FFmpeg

Browser FFmpeg is useful because it can run without installation, process selected media without the app uploading it to a server, and pair well with GUIs or predefined recipes. For a handful of local files, that can be more convenient than preparing a command-line environment.

Native FFmpeg remains better suited to large batch jobs, very long or high-resolution media, broad codec/library coverage, automation, and maximum throughput. WebAssembly does not turn the browser into the same execution environment as native FFmpeg; it brings a useful subset of FFmpeg into the browser's security and resource constraints.

Tips and limitations

• A single-thread build can still run FFmpeg inside a worker, so ST does not automatically mean the UI must freeze. Separate the browser UI thread from FFmpeg's internal thread model.
• Needing SharedArrayBuffer does not require isolating an entire site. Check the effect on third-party embeds and consider applying COOP / COEP only to the route that needs them.
• Fully local processing is different from offline startup. An MT build may require HTTP(S) delivery while still processing selected video without uploading that video to an external server.
• Watch memory as well as CPU for large video. MT can improve throughput while still hitting device-memory limits first.