[dedicatedbroadcastsolutions]

Summary

Give the realtime FFmpeg STREAM consumer modest, configurable buffering to tolerate transient scheduling, encoding and output stalls without immediately discarding combined video/audio input.

• Change STREAM defaults from 1 input frame / 1 encoded packet to 6 input frames / 16 encoded packets.
• Add independently configurable -frame-buffer and -packet-buffer AMCP options, and corresponding XML elements for <ffmpeg> consumers with <realtime>true</realtime>.
• Validate ranges: 1–64 input frames and 1–1024 encoded packets. Invalid or missing values fail consumer creation. Setting both to 1 restores the previous buffering limits.
• Restrict the new options and diagnostics to realtime consumers. FILE and non-realtime XML consumers retain their existing 64/128 queues and behavior; attempts to use the new overrides in these modes fail explicitly.
• Report rejected STREAM input with a warning on the first rejection and at most once every five seconds thereafter. Expose configured capacities and cumulative rejected-frame/audio-sample counts in consumer monitor state.
• Document configuration, latency/memory implications, overload behavior, and standalone regression-test instructions.

Why

An operator reproduced monitoring-audio pops with a 1080i5994 channel feeding clean DeckLink SDI plus a dynamic FFmpeg consumer: 960×540 H.264, stereo Opus, MPEG-TS over multicast UDP. The pops were present in a locally captured multicast stream, before the downstream RTP/WebRTC pipeline.

Removing asynchronous audio timestamp correction did not resolve the problem. Switching only the consumer mode from STREAM to FILE, retaining the same UDP destination and encoding options, produced clean audio and acceptable delay in the operator's test.

That result does not isolate buffering alone: realtime codec-threading behavior also changes between these modes. It does, however, motivate improving STREAM's very small transient headroom without changing codec behavior.

The existing input queue holds combined video/audio frames. When try_push() fails, both video and its audio samples are discarded while video and audio timestamps advance. Packet-output backpressure can stall the encoding worker and fill this input queue. Regular output Opus timestamps therefore do not prove that upstream samples were never discarded.

This PR does not claim that the on-site root cause is conclusively measured or that the new defaults guarantee lossless audio.

Design and scope

Keep the existing bounded FIFO architecture and nonblocking input admission. Do not block channel/SDI playout, split audio/video pipelines, change codec threading, alter GOP/repeated-header settings, or introduce UDP pacing.

Six queued inputs represent approximately 100 ms of media at 59.94 inputs/second, including fields for interlaced channels. This is queue capacity, not an imposed startup/playback delay; queues drain whenever downstream processing permits. Total latency also includes in-flight encoding, muxing and downstream buffering.

The 16-packet queue holds encoded AVPackets, not UDP datagrams, so its capacity is not a fixed duration. Both defaults are initial bounded headroom choices open to discussion.

Under sustained overload, queues still fill and arriving input is still dropped. Warnings and cumulative counters expose that loss; this is not an unlimited-backlog workaround.

Example

ADD 1-603 STREAM "udp://239.1.1.1:5000?pkt_size=1316" -frame-buffer 6 -packet-buffer 16 -format mpegts

<ffmpeg>
    <path>udp://239.1.1.1:5000?pkt_size=1316</path>
    <args>-format mpegts</args>
    <realtime>true</realtime>
    <frame-buffer>6</frame-buffer>
    <packet-buffer>16</packet-buffer>
</ffmpeg>

Prior art informing the approach

• FFmpeg CLI 8.0 thread queues default to 8-frame/8-packet queues; queue admission blocks producers when full. Blocking CasparCG's channel-facing input would not be an appropriate direct copy.
• VLC 3.x UDP output uses a FIFO and dedicated timestamp-scheduled sender with outbound caching. This PR adds headroom, not VLC-style scheduled delay.
• MediaMTX uses configurable bounded output queues and reports slow-reader discards. Queue units differ from CasparCG's raw input frames and encoded packets, so numeric defaults are not directly interchangeable.

Verification

Completed on this upstream-based branch:

• Standalone configuration and TBB queue-contract regression tests, compiled with -Wall -Wextra -Werror.
• The same tests under AddressSanitizer and UndefinedBehaviorSanitizer.
• Production ffmpeg_consumer.cpp translation-unit compilation against the available system dependencies.
• Configuration XML parsing and git diff --check.

Tests cover STREAM/FILE defaults, independent overrides, case-insensitive AMCP names, stripping consumer options while preserving encoder options, invalid/missing/out-of-range values, non-realtime rejection, legacy 1/1 settings, six-input bursts and draining, bounded sustained overload, fractional 48 kHz/59.94 input timestamp/sample accounting, and encoded-packet backpressure/FIFO order.

These are configuration/queue-contract tests, not full consumer/codec integration tests.
This hasn't been tested yet as this should be discussed first.

Warning throttling, real consumer replacement/shutdown, actual audio continuity, A/V sync and latency still require integration/on-site validation. Follow-up acceptance should compare original 1/1 STREAM, buffered FILE-to-UDP and patched STREAM under controlled stalls and sustained load.

Should I remove the AI generated tests and make this just a minimal patch? The rational for this is patch is that a frame only being a fraction of a frame late followed by a slightly early frame can drop an entire frame. It was made configurable so that low latency tuning can be accomplished if desired.

Summary by CodeRabbit

• New Features
  • FFmpeg STREAM consumers now support configurable frame and packet buffer sizes, with documented defaults and limits.
  • Realtime consumers report dropped frames and audio samples when buffering is full.
• Documentation
  • Added configuration guidance covering buffering behavior, queueing, latency, and drop counters.
  • Added instructions for running standalone buffer regression tests.

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

FFmpeg consumers now support configurable frame and packet buffer capacities. Realtime frame drops update counters and produce periodic warnings. Tests cover option parsing, queue limits, audio sample accounting, and packet backpressure.

Changes: FFmpeg consumer buffering

Layer / File(s)	Summary
Configure consumer buffers src/modules/ffmpeg/consumer/buffer_config.h, src/modules/ffmpeg/consumer/ffmpeg_consumer.cpp, src/modules/ffmpeg/CMakeLists.txt, src/shell/casparcg.config, README.md, tools/tests/ffmpeg_consumer_buffers_test.cpp	Adds realtime-dependent buffer defaults, command-line and XML parsing, and configurable frame and packet queue capacities. Configuration tests cover defaults, overrides, and invalid values.
Handle queue saturation and verify queue contracts src/modules/ffmpeg/consumer/ffmpeg_consumer.cpp, tools/tests/ffmpeg_consumer_buffers_test.cpp, README.md	Realtime frame drops update counters and periodic warnings. Tests check frame admission, audio sample accounting, packet backpressure, and FIFO order. The README describes queue behavior and the standalone test command.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Consumer as ffmpeg_consumer
  participant FrameQueue as frame queue
  participant Worker as encoding worker
  participant PacketQueue as packet queue
  Consumer->>FrameQueue: enqueue incoming frame
  FrameQueue-->>Consumer: reject frame when full
  Worker->>FrameQueue: drain queued frame
  Worker->>PacketQueue: enqueue encoded packet
  PacketQueue-->>Worker: apply backpressure when full

Loading

Suggested reviewers: julusian, niklaspandersson

Merge Risk: Minimal · up to 30fa0

The configured buffer limits and realtime drop behavior match the documented paths. No actionable issue remains before merge under normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title names the main change: configurable buffer headroom for the FFmpeg STREAM consumer. The raw summary shows new buffer_config options (-frame-buffer and -packet-buffer, plus XML settings) with…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Title check

Explanation

The title names the main change: configurable buffer headroom for the FFmpeg STREAM consumer. The raw summary shows new buffer_config options (-frame-buffer and -packet-buffer, plus XML settings) with default capacities of 6 frames and 16 packets for realtime STREAM consumers. The title matches this primary change, and the objectives describe the same proposal.

Full details: Docstring Coverage

Explanation

Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 12 functions across 3 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: b89f7c6b-4738-4dbe-a7b9-988866666107

Commits

Reviewing files that changed from the base of the PR and between f4d6970 and 30fa0f5.

Files selected for processing: 6

• README.md
• src/modules/ffmpeg/CMakeLists.txt
• src/modules/ffmpeg/consumer/buffer_config.h
• src/modules/ffmpeg/consumer/ffmpeg_consumer.cpp
• src/shell/casparcg.config
• tools/tests/ffmpeg_consumer_buffers_test.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.