An always-on social publishing agent watches what an audience responds to, picks a clip from footage you already have, edits it for the channel and books the post, hourly, unprompted. You can build the loop with ffmpeg, Whisper, a small model and the Instagram API, or let Leo run it. The hard parts are the clip index and the publishing permissions, not the model.
Part of the Always-on agents series, post 31 of 33.
Every other post in this series is about agents I run for my own company, or agents you can build for nothing. This one is about the agent my company makes. I have tried to write it the way I would want a vendor to write it: what the thing actually does, in the words we use on our own pages, then an honest bill of materials for building the same loop yourself, then a straight answer about who should do which.
What Leo does, in its own words
Our product page says it in one sentence: every hour Leo finds a fresh trend, matches a clip from your library, edits it, and books it on the calendar. I would add the rule underneath it that I am proudest of, which is on the same page: no match, no post. Never filler. An hour with nothing worth posting produces nothing, and the calendar stays honest.
Behind that sentence there are four pieces, and our pricing page lists them plainly. A searchable library: footage split into short segments, transcribed and embedded, so a 60 to 90 second moment is found by meaning rather than by filename. Clips that edit themselves: the Composer trims the moment, reframes sideways footage to vertical, burns in captions and speaker tags and picks a cover frame. Signals filtered to an audience profile, which is the listening half, and the always-on hourly trend matching that joins the two. Publishing today goes to Instagram Reels, with YouTube Shorts in beta, through one-click account connection rather than a stored password.
By the three tests from the first post, Leo passes all three. It holds a goal across runs (keep this channel posting video its audience responds to), it starts itself every hour, and it acts, by booking a post, or decides not to and records why. It is the only true agent among the jobs that run my company, and I say so in the closing post of this series.
What you would have to build: the five parts
I get asked, politely, whether this is just ffmpeg and a cron job. Partly, yes. Here is the whole loop as I would build it today with free software, and where each part gets hard.
- 1. Listening. Decide what counts as a signal: a topic rising in your audience's feeds, a competitor's post that is working, a question that keeps coming up. The free version reads public feeds and pages on a timer and asks a model what changed. I built exactly that in the brand-watch post. The expensive version is knowing which audience the signal belongs to.
- 2. A clip index. Split every long recording into moments and make them searchable by what is said. Scene detection finds the cuts; a transcript finds the meaning. This is the part most people underestimate, because the index is only as good as its segmentation, and footage of people talking has very few visual scene changes.
- 3. A decision. Given a signal and the index, pick the one moment that answers it, or decide that nothing does. This is the only step that needs judgement, which by my own rule in the agent versus automation post is the only place a model belongs.
- 4. Editing. Trim to the moment, reframe 16:9 to 9:16, burn in captions, choose a cover frame. ffmpeg does all of it; getting it to look good takes a designer's eye and a few hundred iterations.
- 5. Publishing and a calendar. Post through the official API, on a schedule, with a record of what went where and why. The permissions are the hard part, and I give them their own section below.
The clip index: ffmpeg and Whisper
Two free tools do the heavy lifting, and they are the two I would start with. ffmpeg's select filter has a scene score, a value between 0 and 1 for how likely a frame is to start a new scene; selecting frames above a threshold gives you cut points. Whisper, OpenAI's open-source speech recognition model (MIT licence), gives you timestamped text. Join the two and every recording becomes a list of moments with words attached.
Terminal: find cut points and transcribe# scene changes above a 0.4 score, with their timestamps
ffmpeg -i talk.mp4 -vf "select='gt(scene,0.4)',showinfo" -f null - 2>&1 \
| grep -o 'pts_time:[0-9.]*' > cuts.txt
# word-level timestamps with the small Whisper model (CPU is fine, just slow)
pip install -U openai-whisper
whisper talk.mp4 --model small --output_format json --word_timestamps TrueTwo honest warnings from doing this. First, a person talking to camera for forty minutes produces almost no scene changes, so for the footage brands actually have you segment on the transcript (pauses, topic shifts) and use the scene score only as a tie-breaker. Second, Whisper's small model needs about 2 GB of memory and is a quarter of the speed of its largest model; on a laptop CPU, a one-hour recording can take longer than an hour. Budget for it, or run it overnight, which is rather the point of an always-on agent.
The edit: reframe, caption, cover
Terminal: one moment, vertical, captioned# cut 00:12:40 to 00:13:35, crop the centre to 9:16, burn the captions
ffmpeg -ss 00:12:40 -to 00:13:35 -i talk.mp4 \
-vf "crop=ih*9/16:ih,subtitles=moment.srt:force_style='FontSize=14,Outline=1'" \
-c:a copy moment.mp4
# a cover frame two seconds in
ffmpeg -ss 2 -i moment.mp4 -frames:v 1 cover.jpgA centre crop is the free answer and it is wrong about a third of the time, because the speaker is not in the centre. The next step up is a face detector driving the crop window, and that is where a weekend project becomes a month. Our Composer does that reframing and the speaker tags; I mention it not to sell it but because it is the step where my own estimate was furthest off.
Publishing: the permissions are the product
Here is the part I want every reader to take seriously. There are two ways software can post to Instagram, and they are treated very differently. A tool that logs in with your password breaks the Terms of Use and is one of the ways a business account gets disabled. The official route is the Instagram API, and it has conditions: an Instagram professional account connected to a Page, an app with the content publishing permission (instagram_business_content_publish, or instagram_content_publish with Facebook Login), media hosted on a publicly reachable server at the moment of publishing, and a cap of 100 API-published posts per account in a rolling 24 hours. Publishing is two calls: create a container, then publish it.
Terminal: the two-step publish for a Reel# 1. create the container
curl -X POST "https://graph.instagram.com/v21.0/$IG_ID/media" \
-d media_type=REELS -d video_url="https://your.host/moment.mp4" \
-d caption="..." -d access_token="$TOKEN"
# -> {"id": "<CONTAINER_ID>"}
# 2. publish it (poll the container's status_code until FINISHED first)
curl -X POST "https://graph.instagram.com/v21.0/$IG_ID/media_publish" \
-d creation_id="$CONTAINER_ID" -d access_token="$TOKEN"Getting that app approved is its own project. I wrote up how to get Instagram API access for auto-posting before this series existed, and it is still the post on this blog I would send a developer first. Leo connects accounts with one click through OAuth because we went through that approval once, for everyone; if you build your own, you go through it yourself.
What the loop costs to run
In model fees the loop can be nearly free. The decision step is a handful of calls an hour, and a small local model can make it, with the caveats in the local models post. Transcription is CPU time, not money. Hosting is the machine you already have plus a public bucket for the media the API needs to fetch. The cost that does not show up in a comparison table is the one I keep writing about in this series: the person who checks it. A publishing agent that posts the wrong clip to a brand account is not a bug you fix on Monday.
Where each is the honest choice
| Your situation | What I would do |
|---|---|
| One channel, an engineer with a free week, footage that is mostly screen recordings | Build it. Scene detection works well on screens, the crop is trivial, and you will learn the API. |
| Talking-head footage, a brand account you cannot afford to lose | Be careful with the publishing step above all. Build the listening and the index if you like; use the official API no matter what. |
| A marketing team with no engineer and more than one channel | Use Leo, or something like it. The loop is the easy part to describe and the hard part to keep running. |
| An agency running many client accounts | Use a tool with a workspace per client. That is how our agency plan is built, because per-client permissions are where a homegrown loop breaks. |
I will put it as plainly as I can. If you have an engineer and one channel, build it; you will end up with something you understand and own, and this series gives you most of the parts. If you want the loop run for you, with the publishing permissions, the reframing and the hourly matching already done, that is what Leo is for, and the product page and the pricing page say exactly what each plan includes. Either way, keep the one rule: no match, no post.
Sources
- Leo AI, Always-on: a channel that runs itself (accessed October 20, 2026)
- Leo AI, Pricing and packaging (accessed October 20, 2026)
- Leo AI, Channels: post where it counts (accessed October 20, 2026)
- Meta for Developers, Instagram Platform: content publishing (accessed October 20, 2026)
- OpenAI, Whisper (GitHub README: models, licence, usage) (accessed October 20, 2026)
- FFmpeg, FFmpeg filters documentation: select, crop, subtitles (accessed October 20, 2026)
Written by Lera Leonteva. Leo is my company's product; the product facts above are quoted from our own pages as they read on the access date.