I've enjoyed using ffmpeg 1000% more since I was able to stop doing manually the tedious task of Googling for Stack Overflow answers and cobbling them into a command and got Chat GPT to write me commands instead.
I use ffmpeg multiple times a week thanks to LLMs. It's my top use-case for my "llm cmd" tool:
uv tool install llm
llm install llm-cmd
llm cmd use ffmpeg to extract audio from myfile.mov and save that as mp3
For the longest time I had ffmpeg in the same bucket as regex: "God I really need to learn this but I'm going to hate it so much." Then ChatGPT came along and solved both problems!
Same here, it's one of these things where AI has taken over completely and I'm just a broker that copy-pastes error traces.
In addition to the many others mentioned, here's a script I just threw together that simplifies a lot of these chained commands - llmpeg: https://github.com/jjcm/llmpeg
If you have ffmpeg installed and an OpenAI env api key set, it should work out of the box.
Demo: https://image.non.io/1c7a92ef-0917-49ef-9460-6298c7a9116c.we...
My experience got even better once I learned how complex filters worked.
Truly, a net positive to my life. Just a few days ago I asked my AI buddy (Claude) to create a zsh script to organize my downloads folder according to the Johnny Decimal system. I’ve since modified it to move the files to a JD setup on my desktop.
The sense of elation I get when I wonder aloud to my digital friend and they generate what I thought was too much to expect. Well worth the subscription.
Basic syntax for re-encoding a video file did take me some time to memorize, but isn't in fact too hard:
ffmpeg <Input file(s)> <Codec(s)> <MAPping of streams> <Video Filters> output_file
ffmpeg -i file1.mp4 -i file2.mkv
- codec: -c:x where x is the type of codec (v: video, a: audio or s:subtitles), followed by its name, like so:
-c:v libx265
- mapping of streams: -map <input_file>:<stream_type>:<order>, like so:
-map 0:v:0 -map 1:a:1 -map 1:a:0 -map 1:s:4
The position of the stream in the output file is determined by the position of the map command in the command line, so for example in the command above we are inverting the position of the audio streams (taken from 'file2.mkv'), as audio stream 1 will be in first position in the output file, and audio stream 0 (the first in the second input file) will be in second position in the output file.
This map thing is for me the most counter-intuitive because it's unusual for a CLI to be order-dependent. But, well, it is.
- video filters: -vf
Video filters can be extremely complex and I don't pretend to know how to use them by heart. But one simple video filter that I use often is 'scale', for resizing a video:
-vf scale=<width>:<height>
-vf scale=320:-1
And that's about it! ffmpeg options are immense, but this gets me through 90% of my video encoding needs, without looking at a manual or ask an LLM. (The only other options I use often are -ss and -t for start time and duration, to time-crop a video.)
I ended up creating my own tool to generate ffmpeg commands https://newbeelearn.com/tools/videoeditor/
I think you're onto something. I've had hit or miss experiences with code from LLMs but it definitely makes the searching part different.
I had a problem I'd been thinking about for some time and I thought "Ill have some LLM give me an answer" and it did - it was wrong and didn't work but it got me to thinking about the problem in a slightly different way and my quacks after that got me an exact solution to this problem.
So I'm willing to give the AI more than partial credit.
I would like to throw in a tool that I built into the ring: gencmd - https://gencmd.com/. There is a web version and also a CLI version.
If the CLI is installed, you can do: gencmd -c ffmpeg extract first 1 minute of video
Or you can just search for the same in the browser page.
I do it the old way: I write down the commands as a shell script, and reuse later.
But really what ffmpeg is missing is an expressive language to describe its operation. Something well-structured, like what jq does for JSON.
For me it was using a container of it, instead of having to install all the things FFmpeg needs on a machine.
ffmpeg and jq are 2 commands I've about given up trying to "use" with any facility and am more than happy to pawn that off to one of the Gippity's; chat, claude, etc.
Why not just use Handbrake? It’s just FFMpeg but with a GUI.
llm - Clippit of 202x, but for the original Pentium was enough.
This reminds me I need to publish my write up on how I've been converting digitized home video tapes into clips using scene detection, but in case anyone is googling for it, here's a gist I landed on that does a good job of it [0] but sometimes it's fooled by e.g. camera flashes or camera shake so I need to give it a start and end file and have ffmpeg concatenate them back together [1]
Weird thing is I got better performance without "-c:v h264_videotoolbox" on latest Mac update, maybe some performance regression in Sequoia? I don't know. The equivalent flag for my windows machine with Nvidia GPU is "-c:v h264_nvenc" . I wonder why ffmpeg doesn't just auto detect this? I get about 8x performance boost from this. Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !
[0] https://gist.githubusercontent.com/nielsbom/c86c504fa5fd61ae...
[1] https://gist.githubusercontent.com/jazzyjackson/bf9282df0a40...
I've gotten pretty good at various bits of ffmpeg over time. Its CLI has a certain logic to it... it's order dependent (not all unix CLIs are).
Lately, I've been playing around with more esoteric functionality. For example, storing raw video straight off a video camera on a fairly slow machine. I built a microscope and it reads frames off the camera at 120FPS in raw video format (YUYV 1280x720) which is voluminous if you save it directly to disk (gigs per minute). Disks are cheap but that seemed wasteful, so I was curious about various close-to-lossless techniques to store the exact images, but compressed quickly. I've noticed that RGB24 conversion in ffmpeg is extremely slow, so instead after playing around with the command line I ended up with:
ffmpeg -f rawvideo -pix_fmt yuyv422 -s 1280x720 -i test.raw -vcodec libx264 -pix_fmt yuv420p movie.mp4 -crf 13 -y
Because I hate command lines, I ended up using ffmpeg-python which composes the command line from this:
self.process = (
ffmpeg.
input(
"pipe:",
format="rawvideo",
pix_fmt="yuyv422",
s="{}x{}".format(1280, 720),
threads=8
)
.output(
fname, pix_fmt="yuv422p", vcodec="libx264", crf=13
)
.overwrite_output()
.global_args("-threads", "8")
.run_async(pipe_stdin=True)
)
I'm still looking for better/faster lossless YUV encoding.
I thought this was going to be a website managed by an experienced user of FFmpeg sharing from their collection of accumulated knowledge, but then was immediately disappointed on the first example I clicked on.
https://www.ffmpegbyexample.com/examples/l1bilxyl/get_the_du...
Don’t call two extra tools to do string processing, that is insane. FFprobe is perfectly capable of giving you just the duration (or whatever) on its own:
ffprobe -loglevel quiet -output_format csv=p=0 -show_entries format=duration video.mp4
Nice! This reminds me of my own ffmpeg cheatsheet; I would imagine that everyone who uses ffmpeg frequently has a similar set of notes
FFmpeg is one of those tools I need to use so infrequently that he exact syntax never seems to stick. I've resorted to using an LLM to give me the command line I need. The only other tool that I ever had trouble with was 1990s-era MegaCLI from LSI Logic, also something I barely used from one year to the next (but one where you really need to get it right under pressure).
Don't forget that Gstreamer exists and its command line and documentation make a little bit more sense than ffmpeg because GStreamer is pipeline based and the composition is a little bit more sane. I stopped using ffmpeg entirely and only use GStreamer for intense video work.
Here is the GitHub repo for a ffmpeg book which may be a nice supplement to this site:
ffmpeg has always felt like a gui application crammed into tui format. I've had the displeasure of using the C api a few times, while it's straight forward in many respects, it makes invalid states extremely easy to represent. I would love a realtime AV1 encoding framework that "just works".
One thing on Linux systems I like to do is build ffmpeg statically.. as distro versions are sometimes too old or don't include modules I prefer.. this containerized version has done wonders for me https://github.com/wader/static-ffmpeg
I love using FFMpeg via Wasm for ... senseless ... mini projects i.e.: https://video-2-sprites.franzai.com/ Video 2 Sprites Converter - totally over-engineered
I love "X by Example" sites! But if you don't work with a tool like ffmpeg imagemagick day in and out, there's no way you'll remember their unintuitive syntax or will want to spend the time to get your one-time job done. I'd still probably not use this site to scan a dozen of examples and try to put together the pieces of the puzzle; instead, I'd probably just use an LLM who already scanned the entire web corpus and can probably get me to a solution faster, right? At that point, I wonder what folks get out of this site?
ah, ffmpeg, the tool that powers the entire online video industry, praised for its stellar code...yet no one still knows how to use it without getting a phd in sherlocking the internet for miniscule and obscure references to common problems.
I was excited to see this as just last night I was using FFmpeg to combine digital (legally owned) movies with multiple parts into a single MKV file. It worked great, the one thing (and this is undoubtedly just a knowledge problem on my part), is I could not get it to copy/include subtitles from both parts. I have a feeling I might need to extract the subtitle files and combine them and then re-include the combined version, but was hoping there would be a solution in this site for doing something like that. Unfortunately, I didn't even see the combine as one of the examples.
For reference:
One-liner:
> ffmpeg -loglevel info -f concat -safe 0 -i <(for f in *.mkv; do echo "file '$(pwd)/$f"; done) -c copy output.mkv
Or the method I ended up using, create a files.txt file with each file listed[0]
> ffmpeg -f concat -safe 0 -i files.txt -c copy output.mkv
files.txt
> file 'file 1.mkv' > file 'file 2.mkv' > # list any additional files
also worthwhile consulting this resource: https://www.youtube.com/watch?v=9kaIXkImCAM
I just use LLMs to help me with ffmpeg (and many other similarly complex tools) commands: https://github.com/dheera/scripts/blob/master/helpme
$ helpme ffmpeg capture video from /dev/video0 every 1 second and write to .jpg files like img00000.jpg, img00001.jpg, ...
$ helpme ffmpeg assemble all the .jpg files into an .mp4 timelapse video at 8fps
$ helpme ffmpeg recompress myvideo.mp4 for HTML5-friendly use and save the result as myvideo_out.webm
I think at this point docs should start to be written not for humans but for LLMs, i.e. package all of it up with the --help into one giant txt file for easy attachment to an LLM when asking the question you'd like. Imo it's a relatively good fit to the current capability.
While it does have a rather unusual/bespoke command syntax (strongly motivating TFA), lately ffmpeg works my webcam more reliably than Google chrome. Too bad my other-side conversations don't have things set up to negotiate a session with it!
"Print a text file to STDOUT using ffmpeg" ffmpeg -v quiet -f data -i input.txt -map 0:0 -c text -f data - I tried this in a directory with input.txt with some random text Nothing.
So changed the verbosity to trace ffmpeg -v trace -f data -i input.txt -map 0:0 -c text -f data -
---snip-- [dost#0:0 @ 0x625775f0ba80] Encoder 'text' specified, but only '-codec copy' supported for data streams [dost#0:0 @ 0x625775f0ba80] Error selecting an encoder Error opening output file -. Error opening output files: Function not implemented [AVIOContext @ 0x625775f09cc0] Statistics: 10 bytes read, 0 seeks
I was expecting text to be written to stdout? What did I miss?
The AMIA has a resource for this as well at https://amiaopensource.github.io/ffmprovisr/
It's not a great name and not very discoverable, but there's a lot of very useful ffmpeg-by-example snippets there with illustrated results, and an explanation of what each option in each example does.
I'm grateful for sites like this and these days LLMs too. ffmpeg is awesome, but I can never rememeber the right incantation. The sites/tools get me the right ffmpeg answer, but I still needed to remember the answer next time (or look things up again). Now I save them as reusable commands: https://lmno.lol/alvaro/how-i-batch-apply-and-save-one-liner...
One thing I have never been able to do reliably is get ffmpeg to split a file based on a fixed size - e.g. break this video into 100MB chunks.
The closest I seem to be able to get is to divide the file size by the file length, add some wiggle room and then split it based on time. Any pointers appreciated.
The great ffmpeg!, Even though you can generate commands using llm its still tedious to adjust things visually on cli so I ended up creating my own tool to generate ffmpeg commands.
Ya know, it's websites like this that make me want to see a "best of HNN" list, so it can be easily found when I'm using ffmpeg and saying "geez, there was some cool ffmpeg site, but where the heck is it?...."
Can we have a best of HNN and put it on there, or vote on it, or whatever?
Somebody please pay people to create sites like this so LLMs can train on them and I never have to spend time thinking about arcane FFMPEG commands ever again. I remember losing whole days in the before times trying to figure out how to make FFMPEG do some highly specific set of things.
ffmpeg is so goated, I used it to merge video/audio from a mic on a camera I made ha
there was one time I didn't use pyaudio correctly so I was using this process where ffmpeg can stitch multiple audio files together into one passed in as an array cli argument, crazy
I was pretty happy because I was able to actually do something in ffmpeg recently. It is this amazingly powerfully tool, but every time I try to use it I get scared off by the inscrutable syntax. But this time as the mental miasma that usually kills my ffmpeg attempts was setting in I noticed something in the filter docs, a single throw away line about including files and the formatting of filters
Anyway long story short, instead of the usual terrifying inline ffmpeg filter tangle. the filter can be structured however you want and you can include it from a dedicated file. It sounds petty, but I really think it was the thing that finally let me "crack" ffmpeg
The secret sauce is the "/", "-/filter_complex file_name" will include the file as the filter.
As I am pretty happy with it I am going to inflect it on everyone here.
In motion_detect.filter
[0:v]
split
[motion]
[original];
[motion]
scale=
w=iw/4:
h=-1,
format=
gbrp,
tmix=
frames=2
[camera];
[1:v]
[camera]
blend=
all_mode=darken,
tblend=
all_mode=difference,
boxblur=
lr=20,
maskfun=
low=3:
high=3,
negate,
blackframe=
amount=1,
nullsink;
[original]
null
ffmpeg -nostats -an -i ip_camera -i zone_mask.png -/filter_complex motion_display.filter -f mpegts udp://127.0.0.1:8888
explanation:
"[]" are named inputs and outputs
"," are pipes
";" ends a pipeline
take input 0 split it into two streams "motion" and "original". the motion stream gets scaled down, converted to gbrp(later blends were not working on yuv data) then temporally mixed with the previous two frames(remove high frequency motion), and sent to stream "camera". Take the zone mask image provided as input 1 and the "camera" stream, mask the camera stream, find the difference with the previous frame to bring out motion, blur to expand the motion pixels and then mask to black/white, invert the image for correct blackframe analyses which will print messages on stdout when too many motion pixels are present. The "original" stream get sent to the output for capture.
One odd thing is the mpegts, I tried a few more modern formats but none "stream" as well as mpegts. I will have to investigate further.
I could, and probably should have, used opencv to do the same. But I wanted to see if ffmpeg could do it.
hey everyone, thanks for the "hunt".
Currently looking for an FFmpeg related job https://gariany.com/about
The first video doesn't seem to work in Safari or Firefox, only Chrome. "Video can't be played because the file is corrupt." On Mac at least.
No one seems to be talking about the website itself.
While as a concept, I absolutely love "X by Example" websites, this one seems to make some strange decisions. First, the top highlighted example is just an overly complicated `cat`. I understand that it's meant to show the versatility of the tool, but it's basically useless.
Then below, there's 3 pages of commands, 10 per page. No ordering whatsoever in terms of usefulness. There looks like there's an upvote but it's actually just a bullet decoration.
There's also a big "try online" button for a feature that's not actually implemented.
All in all, this is a pretty disappointing website that I don't think anyone in this thread will actually use, even though everyone seems to be "praising" it.
Kind of OT: Does anyone know any video editing library in Python that adds fancy/animated text with background?
Are ffmpeg user interested in a cloud base solution?
You push the input files, the command, and fetch the output when done.
if you're the creator might I suggest an ai.txt like a robots.txt of all the content? so we can pass it to an LLM and use it to create custom commands we need based on real needs? that would be awesome!
anyone know how to use ffmpeg to get a video to cross-fade from the end to the start so it makes a nice loop? I cant seem to get the right response from ChatGPT.
Wow, I wish I had seen this literally yesterday!
I love ffmpeg, but yeah, some bits are a little ... obscure.
Right now, I am looking to normalize some audio without using ffmpeg-normalize, a popular Python package. Nothing against it on a personal level, I just ... want to know what is going on, and it's a lot of files and lines of code to do what is basically a two-pass process.
I have a growing interest in metadata and that's also a case which I do not find is often well-addressed.
wish there was a website like this for gstreamer
i believe this is where LLM will excel
Gotta leave this here for those that haven’t seen it :)
Now I know what to read over the next weekend!