FFmpeg Notes · he110

Common Commands

Extract mp3: ffmpeg -i input-video.mp4 -q:a 0 -map a output-audio.mp3
- -i input-video.mp4: specifies the input video file.
- -q:a 0: specifies the audio quality; 0 means the highest quality. Adjust this parameter as needed.
- -map a: specifies extracting the audio track; a means audio.
- output-audio.mp3: the output file is saved in MP3 format.
Extract aac: ffmpeg -i input-video.mp4 -vn -acodec copy output-audio.aac
- -vn: means no video track is included
- -acodec copy: copies the input file's audio stream directly to the output file without re-encoding
Extract wav: ffmpeg -i input-video.mp4 -vn -acodec pcm_s16le -ar 44100 -ac 2 output-audio.wav
- -vn: means no video track is included
- -acodec pcm_s16le: specifies the audio codec as PCM, the encoding commonly used for the WAV format.
- -ar 44100: sets the audio sample rate to 44100 Hz.
- -ac 2: sets stereo (two-channel) audio.
- output-audio.wav: the output file is saved in WAV format.
Merging videos cached by the Bilibili mobile app: ffmpeg -i video.m4s -i audio.m4s -c:v copy -c:a copy output.mp4

The FFmpeg Processing Pipeline

Parse the input file into data packets according to its file format
Decode the data packets into raw data
Encode the raw data
Output the encoded data packets according to the file format

graph LR
  input[Input file]
  output[Output file]
  encode[Encoded packets]
  decode[Packets to decode]
  data[Decoded data frames]

  input -->|demuxer| encode
  encode -->|decoder| data
  data -->|encoder| decode
  decode -->|muxer| output

Command Categories

Basic Information Queries

-version: show the version
-demuxers: show all available demuxers, i.e. which formats can be parsed
-muxers: show all available muxers, i.e. which formats can be produced
-devices: show available devices, used for data capture, data input, and so on
-codecs: show all available encoders and decoders
-decoders: show all available decoders
-encoders: show all available encoders
-bsfs: show bitstream filters
-formats: show available formats
-protocols: show available protocols
-filters: show available filters
-pix_fmts: show available pixel formats
-sample_fmts: show available sample formats
-layouts: show channel names
-colors: show recognized color names

Recording

Windows: ffmpeg -f gdigrab -i desktop -f mp4 d:/out.mp4

-f: what format of data to capture
-i: specifies where to capture data from
-s: specifies the video resolution
-r: specifies the frame rate
-ac: specifies the number of audio channels
- 1: mono
- 2: stereo
- 6: 5.1 surround sound
- 8: 7.1 surround sound

Window

Common input formats:

dshow: the DirectShow format, used to record cameras, microphones, screens, windows, and so on on Windows
- Use the command ffmpeg -list_devices true -f dshow -i dummy to list available video devices with their names and index numbers, then choose the corresponding index number or device name as needed
gdigrab: the GDI capture input format, used to record the screen or a serial port on Windows; a specific window can be selected for recording
- No device index is needed; just use parameters like -offset_x, -offset_y, and -video_size to specify the screen area or window title to record
vfwcap: the input source for recording video devices (such as cameras)
- Use the command ffmpeg -list_devices true -f vfwcap -i dummy to list available video devices with their names and index numbers, then choose the corresponding device name or index number as needed

Linux

v4l2: the Video for Linux 2 input format, used to record video devices such as USB cameras
- Use the command v4l2-ctl --list-devices to list available video devices with their names and index numbers, then choose the corresponding device name or index number as needed
alsa: the Advanced Linux Sound Architecture input format, used to record audio devices on Linux, such as microphones and speakers
- No device index is needed; just use the parameter ffmpeg -f alsa -i default to record the default audio device
x11grab: the X11 capture input format, used to record the screen or a window on the X Window system
- No device index is needed; just use parameters like -offset_x, -offset_y, and -video_size to specify the screen area or window title to record

Mac

avfoundation: the AVFoundation input format, used to record the camera, microphone, and screen on a Mac
- Use the command ffmpeg -f avfoundation -list_devices true -i "" to list available video devices with their names and index numbers, then choose the corresponding device name or index number as needed
qtkit: the QuickTime Kit input format, used to record the camera and microphone on a Mac
- Use the command ffmpeg -f qtkit -list_devices true -i "" to list available video devices with their names and index numbers, then choose the corresponding device name or index number as needed

Demuxing and Remuxing

For example: ffmpeg -i xxx.mov -vcodec copy -acodec copy xxx.mp4

-vn: drop the video
-vcodec: what operation to perform on the video
- copy: copy the video data
-an: drop the audio
-acodec: what operation to perform on the audio
- copy: copy the audio data

Processing Raw Data

Raw data is divided into audio data (PCM) and video data (YUV).

Extracting PCM Data

ffmpeg -i input.mp4 -vn -ar 44100 -ac 2 -f s16le out.pcm

When extracting audio, you need to specify the audio sample rate, that is -ar 44100, where a stands for audio and r stands for rate.

-ac2 means using two channels, where a stands for audio and c stands for channel.

-f s16le is the storage format of the extracted audio data; s16le means signed 16 bits little endian.

The extracted audio is also raw data; when playing it you need to supply the sample rate, channel count, and other data, for example: ffplay -ar 44100 -ac 2 -f s16le out.pcm

Extracting YUV Data

ffmpeg -i input.mp4 -an -c:v rawvideo -pix_fmt yuv420p out.yuv

When extracting raw data, you must specify the pixel format, that is -pix_fmt yuv420p.

The extracted video is raw data; when playing it you need to specify the resolution manually, for example: ffplay -s 1024 * 600 out.yuv

Trimming and Merging by Length

Trimming by Length

Example: ffmpeg -i input.mp4 -ss 00:00:00 -t 10 out.ts

-ss: start time, in the format hours:minutes:seconds
-t: video length, in seconds

Merging

Example: ffmpeg -f concat -i inputs.txt out.flv

The format of inputs.txt is:

file '1.ts'
file '2.ts'

Converting Between Images and Video

Video to Images

Example: ffmpeg -i input.flv -r 1 -f image2 image-%3d.jpeg

-r 1: frame rate of 1, taking one image every second
-f image2: specifies conversion to images
image-%3d.jpeg: the output file naming format; %3d means a 3-digit integer

Images to Video

Example: ffmpeg -i image-%3d.jpeg out.mp4

Live Streaming

Pushing a Stream

Example: ffmpeg -re -i out.mp4 -c copy -f flv rtmp://server/live/streamName

-re: slows down the frame rate and speed, syncing them with the stream

Pulling a Stream

Example: ffmpeg -i rtmp://server/live/streamName -c copy dump.flv

Restreaming

Example: ffmpeg -i rtmp://server/live/originalStream -c:a copy -c:v copy -f flv rtmp://server/live/h264Stream

Streaming the Desktop in Real Time

Example: ffmpeg -framerate 15 -f gdigrab -i desktop -s 1280x720 -c:v libx264 -f flv rtmp://localhost:1935/live/room

Filters

Cropping 200px off the video's original width and height: ffmpeg -i input.mov -vf crop=in_w-200:in_h-200 -c:v libx264 -c:a copy out.mp4

-vf: apply a filter to the video crop: the filter name, used for cropping in_w: the video width in_h: the video height

The crop filter format is: crop=out_w:out_h:x:y

out_w: the final video width
out_h: the final video height
x: the x coordinate of the center point kept after cropping
y: the y coordinate of the center point kept after cropping