How FFmpeg Works — An Explainer for People Who Are Not Developers

A piece of software you have used without knowing it

If you have ever watched a YouTube video, you have used FFmpeg. Not directly — FFmpeg was on the server that turned the original upload into the dozen resolutions YouTube serves you, then quietly stepped aside. The same goes for almost every video you have ever watched online. VLC plays your files using FFmpeg's libraries. Chrome decodes HTML5 video with help from the same code. Twitch, Discord, Zoom, OBS, Plex, Jellyfin — the list of software that depends on FFmpeg is closer to "everyone" than "some." It is one of the most quietly important pieces of free software in existence.

For a piece of code that runs that much of the internet's video pipeline, FFmpeg is small in a way modern software rarely is. It is essentially a command-line program that reads media in one format and writes media in another, with a comprehensive set of knobs in between. There is no graphical interface, no setup wizard, no subscription. You point it at a file, tell it what you want, and it does the work.

What "converting a video" actually means

To understand what FFmpeg is doing when it converts a file, it helps to take apart what a video file actually is. A video file is not a single thing. It is at least three things in a trench coat:

The container. This is the outer wrapper — MP4, MKV, MOV, WebM, AVI. The container's job is to hold the streams inside in a well-defined order, with timestamps, indexes, and metadata, so that a player knows what to play and when.

The video stream. Inside the container is a stream of compressed video frames. The compression algorithm is the codec — H.264, H.265, AV1, VP9. The codec is what determines how big the file is for a given level of visual quality. It is also where almost all the CPU work happens during conversion.

The audio stream (or streams). Same idea, separate compression. AAC is the most common audio codec in modern video files; older containers might have MP3 or AC3.

When you "convert a video," what FFmpeg actually does depends on what you ask for. If you ask for a different container — say, an MP4 to an MKV — FFmpeg can usually just lift the streams out of one wrapper and drop them into the other. The compressed video and audio data are untouched. This is called a *stream copy*, and it is essentially free; the only work is reading the file in and writing it back out with a different wrapper.

If you ask for a different codec, or a different resolution, or a different bitrate, FFmpeg has to actually decode every frame and encode it again. This is *re-encoding*, and it is where the cost lives. Encoding a single second of 1080p video is hundreds of mathematical operations per pixel, repeated thirty times for the frames in that second. Multiplied out, a ten-minute clip is a few minutes of focused CPU work.

The architecture, in two paragraphs

FFmpeg is split into a small command-line front end (ffmpeg) and a much larger collection of libraries underneath. The libraries are where the real knowledge lives: libavcodec knows how to encode and decode codecs, libavformat knows how to read and write containers, libavfilter does scaling and cropping and effects, libavutil handles low-level utilities. When you run a command, the front end is mostly a thin orchestrator: parse the arguments, open the input through libavformat, push frames through whatever filters were requested, hand the results to libavcodec to encode, and write the output back out through libavformat.

That layering is why FFmpeg ends up inside other software so often. A program does not have to invoke the command-line tool; it can link against the libraries directly and use them like any other dependency. This is how VLC plays everything: it does not implement codecs itself, it asks libavcodec.

Why FFmpeg ended up winning

Three things made FFmpeg the de facto standard. First, breadth: it reads and writes more formats than any commercial competitor. If a format exists, there is roughly a ninety-nine percent chance FFmpeg supports it. Second, the license is permissive enough (LGPL by default, GPL for some optional components) that companies can use it inside their products without writing a check. Third, it is genuinely good at what it does — the encoders are competitive with proprietary ones, the API is stable enough to ship against, and the developer community has been actively maintaining it for twenty-five years.

The result is that there is no incentive to build something else. Any company that needs video conversion already has FFmpeg solving the problem.

How FFmpeg ended up in a browser tab

For most of its life, FFmpeg lived on servers and desktops. Running it in a browser was nonsense — browsers ran JavaScript, FFmpeg was written in C, and there was no bridge. Then WebAssembly happened. WebAssembly is a binary format that browsers can execute at near-native speed, designed specifically so that languages like C and C++ and Rust can run inside a web page.

The FFmpeg.wasm project compiles FFmpeg to WebAssembly using a toolchain called Emscripten. The output is a binary your browser can run alongside the page's JavaScript. When you load this site, the browser downloads the FFmpeg.wasm binary once (about thirty megabytes), caches it, and then executes it inside the page. When you drop a file onto the converter, the file is read into the browser's memory and handed to that running FFmpeg instance. The conversion happens entirely inside your browser tab, never touching a server.

The performance cost of running FFmpeg in WebAssembly instead of natively is real but smaller than you would expect — roughly fifty to eighty percent of native speed depending on the codec. For stream copies (no re-encoding), it is essentially identical, because there is barely any work to do. For heavy re-encodes, native FFmpeg with hardware acceleration is still faster, but for most everyday conversions the difference does not matter.

What this site does on top of FFmpeg

The converter at the top of this site is FFmpeg.wasm with a user interface. You drop a file in, you pick a target format, the site builds the appropriate FFmpeg argument string under the hood, and the WebAssembly runtime executes it. If you switch to advanced mode, you can pass arbitrary FFmpeg arguments directly — the same flags you would use on the command line.

If you want to see FFmpeg in action, try a conversion and watch the log panel. The output you see is the same output FFmpeg produces when run from a terminal: a torrent of diagnostic information about what it is reading, what filters it is applying, and how fast it is going. It is the same engine, in a different shell.

For a comparison of FFmpeg with the other common consumer option, see FFmpeg vs HandBrake. For a deeper dive on the privacy implications of running it in a browser, see why client-side conversion is more private.