Stabilizing 4K: Detection Cost 4x Realtime. A Downscale Cut It to 0.3x.

Every figure below is measured on VidClean's production worker. Cost is seconds of processing per second of video, so 4.23 means a one-minute 4K clip took four minutes and fourteen seconds of detection alone. Lower is faster.

• Full-resolution detection on 4K ran at a median 4.23x realtime (61 production jobs, August 5 to 10), and 4.02x in the A/B that measured it on August 10 and 11.
• Downscaled detection runs at 0.263x realtime (113 jobs, August 11 to 17; 0.356 in the A/B), a cut of roughly 94% from the median, or 91% in the A/B.
• The cut only counts if the transform file is rescaled correctly. vidstabtransform silently accepts a transforms file measured in the wrong pixel space: exit 0, no warning, visibly unstabilized output, and with the unsharp pass still applied, measurably worse than the untouched source.
• Every uncertain outcome falls back to a full-resolution detect rather than risking a wrong-space trf: unparseable files, zero motion records, field centres outside the frame, scale factors below 1.
• Shake reduction survives the downscale: between 92.9% and 107.5% of the full-resolution result, measured on a 540p-short-side frame.
• Pass 2 cannot be cheated the same way. It is 79% of total job time at the median, because it must touch every real output pixel, and that is the remaining ceiling on 4K work.

Cost figures come from production job logs; the A/B figures from the switchover window of August 10 to 11 (917 jobs across both configurations). See methodology.

Before the change, pass 1 on a 4K file consumed a median 4.23 seconds of compute per second of video. A 30-second 4K clip needed over two minutes of detection before pass 2 had touched a single output frame. Combined with the encode, the arithmetic put real 4K footage on the edge of the processing timeout, and the failure mode when it tipped over was not a slow job: it was a hard rejection telling the user their video was too demanding to stabilize.

4.23x

median full-resolution detection cost on 4K, seconds per second of video

61 production jobs, August 5 to 10, 2026. The A/B measured 4.02x.

The numbers get worse the longer the clip is. Detection cost scales with frames, so a 60fps 4K file costs close to double a 30fps one, and there is no way to skip the work: pass 1 has to see every frame to track the motion through them. That is why the old answer to expensive 4K was simply to not allow much of it. Any video whose short side is 1440 pixels or more is capped at 90 seconds of input, and the cap exists because of this arithmetic, not as a product decision.

The fix had been obvious on paper for a while: camera motion is a low-frequency signal, and tracking it does not need full resolution. Stabilization documentation has long suggested downscaling for analysis. What nobody had measured, at least not in public, was the actual production cost of doing it at 4K, and whether the result still looked stabilized.

On August 10 the worker ran as it always had, with full-resolution detection. On the morning of August 11 it switched to detection on a 540p-short-side stream, with the transform file rescaled back to full resolution before pass 2. Across the two days, 917 production jobs ran through both configurations, and the worker's own logs recorded the pass-1 time of each.

Configuration	Median cost	Jobs timed
Full resolution, A/B (Aug 10-11)	4.02	in 917
Full resolution, production window (Aug 5-10)	4.23	61
Downscaled, A/B (Aug 11)	0.356	in 917
Downscaled, production window (Aug 11-17)	0.263	113

Cost is median seconds of processing per second of 4K video, pass 1 only. The A/B rows come from the switchover window; the production rows from the longer log windows either side of it. Medians differ slightly between the two because they are different samples. The A/B's 4.02 to 0.356 is a 91% cut; the production medians, 4.23 to 0.263, a 94% cut.

The result was a 91% reduction in the A/B, and the production windows agree: 4.02 to 0.356 there, 4.23 to 0.263 over the longer runs. The failure mode that motivated the change, the hard rejection of demanding 4K jobs, disappeared entirely from the logs after the switch, which is the quietest possible confirmation that the cost was real and that it had been blocking real uploads.

Here is the part worth writing down, because it is not documented anywhere else and it cost us a day. The transforms file that vidstabdetect writes lives in the pixel space of the frames it actually analyzed. Detect on a 540p stream and the motion vectors it records are 540p coordinates. Feed that file to vidstabtransform unchanged, and ffmpeg does not notice.

vidstabtransform silently accepts a wrong-space transforms file. It exits with status 0, prints no warning, and warps every frame by motion vectors that are a quarter of the size they need to be. The output is not stabilized at all. And because pass 2 always applies an unsharp filter to counter the softness the warp introduces, the result is worse than the untouched source: the shake stays, and the sharpening makes it crisper. We measured it. A full-resolution pass 2 fed a 540p trf produced output measurably worse than the input, with every status code along the way reporting success.

There is no guardrail in ffmpeg for this. The library has no way to know the analyzed frame size, so the check has to live in the calling code, and it has to be paranoid. The rescale function we shipped does four things before it trusts a downscaled run:

• The file must parse, and must contain at least one motion record. An empty or unparseable trf returns zero records, which is treated as failure.
• The scale factors must be sane. A factor at or below 1 means the scaled dimensions were misread, because this is an upscale back to source space.
• The rescaled field centres must land inside the full-resolution frame. A motion vector pointing outside the source is proof the space is wrong.
• The scale factors come from what ffmpeg reports it actually produced, not from what the filter string asked for. scale=-2:540 forces the height and rounds the width to an even number, so the two axes scale differently, and the factors are derived per axis.

Failing any gate re-runs detection at full resolution, sharing pass 1's existing budget rather than getting a fresh one, because a second full 600-second detect would leave pass 2 no room inside the overall timeout. The cost of a false alarm is one full-resolution detect, which the job could afford once. The cost of a false pass is a user's output with no error at all, which is why the code is built to err loudly in only one direction.

The same discipline applies to the fallback itself. Detection at full resolution after a failed rescale must also succeed, and if it does not, the job fails with a real error instead of proceeding on the bad file. There is no third option: "probably fine" is indistinguishable, in ffmpeg's output, from silently shipping unstabilized video.

The second bug in this story is the kind that hides inside a filter string. The obvious way to downscale for detection is scale=-2:540, which reads as "make it 540p". What it actually does is force the height to 540. On a landscape 3840x2160 file that produces 960x540, which is the frame every retention measurement in this post was made on. On a portrait 2160x3840 phone clip, the same expression produces 304x540.

Source orientation	Analyzed frame	Pixels vs landscape
Landscape 3840x2160	960x540	1x
Portrait 2160x3840	304x540	3.2x fewer

The expression forces the height, so on portrait footage it analyzes 164,160 pixels against the landscape frame's 518,400. The shipped fix scales the short side instead, so portrait 4K analyzes at 540x960.

Why that matters: the retention numbers, 92.9% to 107.5% of the full-resolution result, were measured on a 540-short-side frame, which is 518,400 pixels. A portrait clip analyzed through the naive filter gets 164,160 pixels, 3.2x fewer, which means fewer usable motion fields on exactly the phone footage that dominates this tool's traffic. The fix is one branch: scale the axis that keeps the short side at 540, whichever axis that is. Portrait gets 540:-2, landscape keeps -2:540, and the retention figures hold for both.

The downscale does not make 4K stabilization cheap. It makes it possible. Pass 2, the warp plus encode, still has to touch every real output pixel, and it is 79% of total job time at the median. Detection was the part that could be cheated; the encode is the part that cannot.

The practical ceiling after the change is arithmetic. With detection at 0.3x realtime and pass 2 near 8x realtime at 4K, a one-minute 4K clip still needs roughly nine minutes of compute. That is why the 90-second cap for videos at or above 1440p short side remains: it is what fits inside the processing budget, and it applies to every job, not as a tier choice. A 90-second 4K clip now stabilizes comfortably; a five-minute one still does not fit, and no amount of detection cleverness changes that.

There is a quieter benefit worth naming. The change removed a whole failure mode from the product: the hard rejection of demanding 4K jobs. Before, the expensive detect meant some 4K uploads simply could not be processed at all. After, every 4K job that fits the cap goes through. The 917-job window measured the cost, but the reason to care is that it turned "sometimes this fails" into "this always works".

The numbers come from three sources, and they are not interchangeable. The production medians, 4.23 across 61 jobs and 0.263 across 113 (mean 0.365), come from the same per-job log records the shaky video study used for its cost tables, parsed from the stabilization worker's logs on either side of the change: August 5 to 10 for full-resolution detection, August 11 to 17 for downscaled. The A/B figures, 4.02 and 0.356, were measured on the production worker during the switchover itself, over the 917 jobs of August 10 and 11, when the worker ran both configurations on real traffic.

All cost figures are pass-1 detection only, in median seconds of processing per second of video, on a shared 24-core container running two stabilization jobs at a time, so the absolute numbers include real-world contention. A different machine gives different absolutes; the ratio between the two configurations is the transferable part. The retention range, 92.9% to 107.5%, comes from a controlled comparison of downscaled versus full-resolution detection on the same clips, measured on a 540-short-side frame, before the portrait branch existed. The pass-2 share, 79%, is from the same production logs, where total job time splits roughly 21% detection and 79% everything after.

The full dataset behind this post is small enough to ship as a CSV file, one row per measurement, each carrying its window, sample size and source. The sibling study's per-job cost table, with resolution and frame rate broken out, lives in the shaky video statistics CSV.

This data is licensed CC BY 4.0. Feel free to reuse the numbers with credit to VidClean.

The production medians rest on small windows. 61 jobs for the full-resolution side is enough to trust the direction, and 113 for the downscaled side is solid, but the exact ratios will move with more data, and the 917-job A/B figure is a two-day sample of live traffic, not a controlled benchmark.

The retention range comes from a handful of clips, not from the production stream, and it is a range because some clips measured slightly better downscaled (107.5%) and some slightly worse (92.9%). The retention figures were measured before the portrait branch existed, so they describe landscape frames; portrait retention is assumed to match because the frame is now the same 540-short-side size, but it has not been re-measured separately.

Cost is medians over a noisy shared box. Contention between the two concurrent jobs inflates some records, and the downscaled medians benefit from the fact that the change also removed the expensive tail of jobs that previously got rejected mid-run. Both effects are in the direction of making the after look better, and neither is large enough to change the conclusion that the cut is an order of magnitude.

And this is one tool on one machine in August 2026. If you run vidstab on your own hardware, the ratios are worth borrowing; the absolutes are not.

If you have a shaky 4K clip, the practical advice from this measurement is the same as the shaky video study's, and now you know why. Clips at or above 1440p short side are limited to 90 seconds because of the encode arithmetic above, not because of a product choice. A 4K clip under that limit stabilizes fine now, detection included. A longer one will get through faster and more reliably if you export a 1080p copy first, because pass 2's cost scales with pixels and halving the short side quarters the encode work.

You can stabilize a video here, free in the browser, with no account needed. VidClean is a solo project. Questions about the data, the methodology, or the trf rescale are welcome at hello@vidclean.net.