Skip to content

feat(utils): add prefetch to get_video_frames_generator#2273

Merged
Borda merged 11 commits into
roboflow:developfrom
Ace3Z:feat/get-video-frames-generator-prefetch
Jul 21, 2026
Merged

feat(utils): add prefetch to get_video_frames_generator#2273
Borda merged 11 commits into
roboflow:developfrom
Ace3Z:feat/get-video-frames-generator-prefetch

Conversation

@Ace3Z

@Ace3Z Ace3Z commented May 25, 2026

Copy link
Copy Markdown
Contributor

Closes #1411.

@LinasKo asked for a worked threading example that produces a real FPS improvement for the get_video_frames_generator path. This PR adds an opt-in prefetch: int = 0 argument: when > 0, frames are decoded in a background thread and buffered in a bounded queue, so a CPU-bound consumer can overlap with decode I/O.

for frame in sv.get_video_frames_generator("video.mp4", prefetch=8):
    run_inference(frame)

Default stays 0 (synchronous, behaviour unchanged). The threaded path is a thin wrapper that drives the existing sync generator on one daemon=True thread and pushes frames through a Queue(maxsize=prefetch). No new dependencies, ~30 added lines in video.py. Pattern matches the reader-thread already in process_video further down the same file.

Benchmark

150-frame 1080p h.264 video, fixed CPU consumer simulated with time.sleep:

consumer work per frame sync prefetched=8
10 ms 63.8 fps 78.3 fps (+23%)
20 ms 38.2 fps 39.6 fps (+4%)
40 ms 21.4 fps 22.1 fps (+3%)

Decode alone on this video is ~10 ms/frame, so the speed-up is largest when the consumer cost is roughly decode-bound. Heavier consumers asymptote to no benefit, which is the right behaviour.

Test

test_get_video_frames_generator_prefetch_matches_sync runs the generator twice on the same dummy video with prefetch=0 and prefetch=4 and asserts the two outputs are frame-for-frame identical. Full pytest src/ tests/ is green (1859 passed). Pre-commit clean.

@Ace3Z
Ace3Z requested a review from SkalskiP as a code owner May 25, 2026 11:35
@Borda
Borda requested a review from Copilot May 26, 2026 16:40
@codecov

codecov Bot commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.43478% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 87%. Comparing base (25e879e) to head (c0d942e).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2273   +/-   ##
=======================================
- Coverage       87%     87%   -0%     
=======================================
  Files           85      85           
  Lines        11855   11901   +46     
=======================================
+ Hits         10277   10315   +38     
- Misses        1578    1586    +8     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in prefetch parameter to get_video_frames_generator to overlap frame decoding with CPU-bound consumers by decoding frames on a background thread and buffering them in a bounded Queue. This targets the FPS bottleneck raised in #1411 while keeping the default (prefetch=0) behavior unchanged.

Changes:

  • Extend get_video_frames_generator(..., prefetch: int = 0) with a threaded prefetch path when prefetch > 0.
  • Add internal _prefetched_frames_generator that drives the existing synchronous generator from a daemon thread and yields frames from a queue.
  • Add a regression test ensuring prefetched output matches the synchronous frame sequence exactly.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/supervision/utils/video.py Adds prefetch option and a new internal queued background-reader generator.
tests/utils/test_video.py Adds a test asserting prefetched iteration matches the synchronous generator frame-for-frame.

Comment thread src/supervision/utils/video.py
Comment thread src/supervision/utils/video.py
Adds an opt-in prefetch: int = 0 parameter. When > 0, frames are decoded
in a background thread and buffered in a bounded queue, letting a
CPU-bound consumer overlap with decode I/O.

Default 0 keeps the original synchronous behaviour unchanged. The threaded
path drives the existing sync generator on a daemon thread and pumps
frames through a Queue(maxsize=prefetch). No new dependencies.

Closes roboflow#1411.
@Ace3Z
Ace3Z force-pushed the feat/get-video-frames-generator-prefetch branch from bd3f788 to ba1f44f Compare May 26, 2026 17:12
@Ace3Z

Ace3Z commented May 29, 2026

Copy link
Copy Markdown
Contributor Author

Friendly ping. @Borda, would you have a moment to take a look? Happy to address any feedback.

@Borda Borda self-assigned this Jul 1, 2026
@Borda Borda changed the title feat(utils): add prefetch to get_video_frames_generator feat(utils): add prefetch to get_video_frames_generator Jul 1, 2026
Borda and others added 4 commits July 2, 2026 00:48
Resolved conflict in tests/utils/test_video.py: kept 3 prefetch tests
from PR branch, accepted -> None return type annotation from develop.

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
- Widen reader exception catch from Exception to BaseException; only
  propagate Exception subclasses via sentinel (non-Exception BaseException
  signals clean stop from consumer's perspective)
- Move sentinel push into finally block so it always runs, including on
  abnormal thread exit — prevents consumer deadlock on frame_queue.get()
- Replace blocking get() with get(timeout=0.5) + thread.is_alive() liveness
  check; consumer exits cleanly if reader dies without delivering sentinel
- Add thread.join(timeout=2.0) in generator finally so VideoCapture is
  deterministically released after stop_event; mirrors process_video pattern
- Wrap re-raised reader exception in RuntimeError with chained context so
  consumer traceback includes both reader and consumer call stacks

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
- Add parametrized test verifying stride/start/end parameters are forwarded
  correctly through the prefetch path vs the sync path (stride=2, start/end
  slicing, combined stride+start+end)
- Add prefetch=1 test covering maximum backpressure / minimum queue size
- Fix spurious mid-sentence line break in get_video_frames_generator Returns:
  docstring section

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…strings

- Add ValueError for prefetch < 0 (negative values previously silently fell
  through to synchronous mode with no diagnostic)
- Add memory footprint note to prefetch docstring: each frame = width x height
  x 3 bytes; directs users to VideoInfo.from_video_path() for sizing
- Add prefetch=8 usage example to the Examples block showing threaded mode
- Add Raises section documenting ValueError for negative prefetch
- Add internal docstring to _prefetched_frames_generator explaining sentinel
  protocol (None=EOF, Exception instance=reader error)
- Update 3 new prefetch test docstrings to Scenario/Expected Google style,
  matching the format of the 14 pre-existing tests in the same file
- Add CHANGELOG entry for prefetch parameter under Unreleased > Added

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

Comment thread src/supervision/utils/video.py Outdated
Comment thread src/supervision/utils/video.py
Comment thread tests/utils/test_video.py Outdated
Comment thread tests/utils/test_video.py Outdated
Comment thread tests/utils/test_video.py Outdated
Comment thread tests/utils/test_video.py Outdated
Comment thread docs/changelog.md
Borda and others added 5 commits July 21, 2026 13:38
…mes-generator-prefetch

# Conflicts:
#	docs/changelog.md
#	tests/utils/test_video.py
Narrow reader() to `except Exception` so KeyboardInterrupt/SystemExit no
longer masquerade as normal EOF (item roboflow#3). Document the RuntimeError
raised on decode/open failure, the GIL precondition for the prefetch
speedup, and the deferred error-delivery latency when prefetch>0
(items roboflow#4, roboflow#12).

[resolve roboflow#3,roboflow#4,roboflow#12] PR roboflow#2273 — Copilot (gh) + foundry:sw-engineer,
foundry:doc-scribe (review): reader-thread exception scope, RuntimeError
docs, GIL/benchmark/latency docstring hardening

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
[resolve roboflow#9] PR roboflow#2273 — Copilot (gh): changelog front-matter date was
stale after this PR's edit; site metadata should track last edit date.

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Add `-> None` return annotations to 3 prefetch tests for consistency
with the rest of the file (items roboflow#5, roboflow#6, roboflow#7). Tighten the decode-error
test to assert RuntimeError specifically (not the tautological
`(Exception, RuntimeError)` tuple) plus its `__cause__` chain (item roboflow#8).
Add 4 missing prefetch-path tests: negative prefetch raises ValueError,
stride+early-termination combo, consumer-exception-in-loop thread
cleanup, and reader-outlives-join bounded-close behavior (item roboflow#13).

[resolve roboflow#5,roboflow#6,roboflow#7,roboflow#8,roboflow#13] PR roboflow#2273 — Copilot (gh) + foundry:qa-specialist
(review): annotation nits, exception-type test tightening, missing
prefetch test coverage

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
…ch cases

Step 9 QA gate flagged two untested reachable paths: the documented
"buffered good frames yielded before RuntimeError" ordering guarantee
(only the immediate-open-failure case was covered), and the prefetch
path against a zero-frame video. Add both.

[resolve roboflow#13] PR roboflow#2273 — foundry:qa-specialist (step9 gate): coverage
gaps for changed logic

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
@Borda
Borda merged commit b20d6ea into roboflow:develop Jul 21, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Increasing Video FPS running on CPU Using Threading

3 participants