Skip to content

feat(bindings)!: decoders read gaps and ends - #6745

Merged
kixelated merged 11 commits into
mainfrom
quest/m1/api-audit/decode-read-bindings
Oct 26, 2026
Merged

kixelated merged 11 commits into
mainfrom
quest/m1/api-audit/decode-read-bindings

Conversation

@kixelated

@kixelated kixelated commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Completes quest/m1/api-audit/decode-read-bindings.md.

Every binding now reads the same audio and video decode stream that Rust does: frames, gaps, and encoder ends. Before this, moq_audio::decode::Read and moq_video::decode::Read lost their Gap and End variants at the FFI boundary. A player could not tell lost media or a publisher's encoder restart apart from continuous media.

Public API impact

Every item below is breaking. Each package's MIGRATE.md has a "Decoders read gaps and ends" section.

  • moq-ffi:
    • MoqAudioConsumer::next now returns Option<MoqAudioRead> and MoqVideoConsumer::next returns Option<MoqVideoRead>.
    • Each is a tagged enum: Frame { frame }, Gap { timestamp_us, duration }, or End { end_us }. duration is a typed Duration, like every other moq-ffi duration since moq-dev/moq#6720; timestamps stay microseconds.
    • Migrate by matching Frame and taking frame.
  • moq-c:
    • moq_decode_{audio,video}_frame[_free] are now moq_decode_{audio,video}_read[_free]. They fill moq_audio_read / moq_video_read, which carry kind (moq_decode_read_kind), frame, gap (moq_decode_gap), and end_us.
    • Callbacks now receive a read id.
    • Migrate by checking kind == MOQ_DECODE_READ_FRAME before using frame.
  • C++ (generated): next() yields moq::AudioRead / moq::VideoRead. Use std::get_if<moq::AudioRead::kFrame>(&read->get_variant()).
  • Python: consumers yield Read = Frame | Gap | End, using the new moq.audio.Gap / End (and the video equivalents).
  • Swift: the Read typealias. Use case .frame(let frame).
  • Kotlin:
    • Read / ReadFrame / ReadGap / ReadEnd typealiases.
    • Consumer.frames() is now Consumer.reads().
  • Dart: Read / ReadFrame / ReadGap / ReadEnd typedefs. Use if (read is ReadFrame).
  • Go:
    • Next returns the sealed Read interface (ReadFrame, ReadGap, ReadEnd), and All ranges over reads.
    • A video ReadFrame still owns its frame handle.
  • OBS plugin: internal only. Non-frame reads are skipped.

Docs: doc/lib/{c,dart,py,swift,kt,go} are updated.

Wire: none.

Choices made

  • I used a tagged record that mirrors the Rust names, as the quest recommends.
  • moq-c uses a kind discriminant plus flat fields (a C-friendly union shape) behind the existing read-id slab.
  • The interop clients (C, C++, Go, Python) fail on a gap, and on an end that arrives before enough audio.
  • After merging main, the gap's duration follows moq-dev/moq#6720: a typed Duration in moq-ffi and every wrapper (Dart drops its duration extension getter for the generated field). moq-c keeps duration_us, matching its other microsecond fields.

Checks

  • just check passes.
  • just test interop --all passes: 172 checks.
  • New tests:
    • moq-ffi raw_audio_decode_reports_encoder_ends (reanchor and finish).
    • moq-c audio_raw_decode_reads_the_encoder_end.
  • The C fixtures and the C++ probe now assert the read kinds.
  • Not run: Swift was not compiled locally (Linux host). CI covers it.

Decisions

Settled by the maintainer on 2026-10-25.

  1. Confirm the tagged-record shape. The quest asked for confirmation before implementing.
    • ✅ Keep the tagged record that mirrors the Rust names, as implemented (recommended).
    • Frames plus a separate gap and end signal.
  2. Encoded consumers. MoqMediaTrackConsumer (moq-ffi) and moq-c's moq_consume_* also drop container gaps and ends.
    • ✅ A follow-up quest gives them the same read shape (recommended).
    • Extend this PR to cover them.

Suggested follow-ups

  • Encoded consumers read gaps and ends: MoqMediaTrackConsumer in moq-ffi and moq-c's moq_consume_* get the same tagged-record read shape, plus every wrapper over them. Plan it with /quest-plan under the m1 API audit.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 6 commits October 24, 2026 23:46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…for decoder reads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kotlin flows are renamed to reads(), Python's video Read uses Union, and the
C++ probe, OBS tests, and interop clients accept encoder ends without
shadowing their sample counters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Bindings read decoder gaps and ends feat(bindings)!: decoders read gaps and ends Oct 25, 2026
@kixelated
kixelated marked this pull request as ready for review October 25, 2026 16:33
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 6 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ee5e1e5d-3a0c-4388-ae27-20d7d9085c7c
Commits

Reviewing files that changed from the base of the PR and between 8810d73 and 67004b5.

Files selected for processing: 52
  • cpp/moq/test/probe.cpp
  • cpp/obs/src/moq-source.cpp
  • cpp/obs/test/moq-audio-encoder-test.cpp
  • cpp/obs/test/moq-source-test.cpp
  • dart/moq/MIGRATE.md
  • dart/moq/lib/audio.dart
  • dart/moq/lib/video.dart
  • dart/moq/test/moq_test.dart
  • dart/moq_ffi/lib/src/moq.dart
  • doc/lib/c/index.md
  • doc/lib/dart/index.md
  • doc/lib/go/index.md
  • doc/lib/kt/index.md
  • doc/lib/py/index.md
  • doc/lib/swift/index.md
  • go/wrapper/MIGRATE.md
  • go/wrapper/audio/audio.go
  • go/wrapper/internal/bridge/call.go
  • go/wrapper/moq_test.go
  • go/wrapper/video/video.go
  • kt/moq/MIGRATE.md
  • kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Flows.kt
  • kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/audio/Audio.kt
  • kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/video/Video.kt
  • kt/moq/src/jvmAndAndroidTest/kotlin/dev/moq/SmokeTest.kt
  • py/moq-rs/MIGRATE.md
  • py/moq-rs/moq/_read.py
  • py/moq-rs/moq/audio.py
  • py/moq-rs/moq/video.py
  • py/moq-rs/tests/test_local.py
  • quest/m1/api-audit/README.md
  • quest/m1/api-audit/decode-read-bindings.md
  • rs/moq-c/MIGRATE.md
  • rs/moq-c/c-tests/audio-decode-output.c
  • rs/moq-c/c-tests/decode-output.c
  • rs/moq-c/src/audio.rs
  • rs/moq-c/src/decode.rs
  • rs/moq-c/src/lib.rs
  • rs/moq-c/src/test.rs
  • rs/moq-c/src/video.rs
  • rs/moq-ffi/MIGRATE.md
  • rs/moq-ffi/src/audio.rs
  • rs/moq-ffi/src/test.rs
  • rs/moq-ffi/src/video.rs
  • swift/MIGRATE.md
  • swift/Sources/Moq/Audio.swift
  • swift/Sources/Moq/Video.swift
  • swift/Tests/MoqTests/SmokeTests.swift
  • test/interop/clients/c/subscribe.c
  • test/interop/clients/cpp/main.cpp
  • test/interop/clients/go/main.go
  • test/interop/clients/python/interop.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T17:34:06.223495Z 67004b5 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 42c8d03

No actionable bugs found in inspected changes. Exposing frame/gap/end reads consistently across bindings is the right direction; mirroring the Rust enum avoids a separate signaling mechanism. C read-handle lifetime and Go canceled-video-read cleanup preserve frame ownership, and migration notes make breaking API changes explicit.

Verification: static conversions, wrappers, OBS adaptation, generated Dart serialization and tests inspection. No builds, tests or interop checks run; platform-specific binding behavior remains unverified.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Landing summary:

  • Maintainer decisions (2026-10-25) are recorded in the description: keep the tagged-record read shape as implemented. The encoded consumers (MoqMediaTrackConsumer, moq_consume_*) get the same read shape in a follow-up quest, listed under Suggested follow-ups.
  • Merged origin/main. just check and just test interop --all (172 checks) pass locally on 42c8d033ec.
  • Reviews on 42c8d033ec: the OpenAI review found no actionable bugs, and Codex gave a thumbs-up. CodeRabbit was rate-limited.
  • quest check --base origin/main passes. The finished quest and its link are deleted.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 25, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-10-25): the hand-written moq-c ships at this cut with parity, so its decoders read gaps and ends like the other bindings. Please also mark the binding shape as settled in decode-read-bindings.md:16-19.

(Written by Claude Opus 5.5)

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 25, 2026
kixelated and others added 2 commits October 25, 2026 15:45
Gap reads carry a typed duration, following the typed-Duration convention from #6720.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Took over the stale merge hold and resolved the merge_conflict by merging origin/main (no rebase).

  • Conflicts came mostly from moq-dev/moq#6720 (typed durations). Following it, Gap's duration_us is now a typed duration in moq-ffi and every wrapper; timestamps stay microseconds. moq-c keeps duration_us like its other fields. Dart bindings were regenerated, and the Dart wrapper drops its duration extension getter in favor of the generated field.
  • Kept both sides of every MIGRATE.md, and removed the finished quest's link from the api-audit epic.
  • Restored a Go fmt import the auto-merge dropped.

Checks: just check passes (Go re-run after the import fix; the Python tests were cancelled in that run and then passed on their own). just test interop --all ran twice. Run 1 failed only rust-swapped -> js on a browser timeout, and run 2 failed only the gst cells because the plugin didn't load locally. Each cell passed in the other run, and this PR touches neither path. quest check passes against the latest main.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main again after the second merge_conflict (no rebase). The only conflict was rs/moq-ffi/MIGRATE.md, where I kept both sections. The regenerated Dart bindings match the auto-merged ones apart from blank lines. moq-dev/moq#6052 is still open, so the generated C side needs nothing yet.

just check passes, and so does quest check against the latest main. I didn't re-run interop: this merge brought in only main's changes, and the PR's own diff didn't change. Re-queuing d57863907c.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Oct 26, 2026
# Conflicts:
#	quest/m1/api-audit/README.md
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge-queue takeover after the merge_conflict dequeue:

  • Merged main (14 commits, including moq-dev/moq#6740 and moq-dev/moq#6747). The only conflict was quest/m1/api-audit/README.md: this PR deletes the decode-read-bindings link and main deleted the model-private link, so both lines are gone.
  • just check passes. The Dart lane failed only because my shell's uniffi-dart predated the flake's duration-negative patch; regenerating in a fresh nix develop reproduces the checked-in moq.dart byte for byte.
  • quest check --base origin/main passes.
  • just test interop --all: every binding lane passes. In each of two runs, one Rust-to-browser lane failed (rust-swapped -> js, then rust -> js) because the browser's Opus player never left a suspended AudioContext. This PR changes neither side of that lane, so I'm tracking it as a separate flake.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit bc7b089 Oct 26, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/api-audit/decode-read-bindings branch October 26, 2026 22:35
kixelated added a commit that referenced this pull request Oct 26, 2026
Keep both sides' Unreleased MIGRATE entries in moq-ffi, moq-c, and each
wrapper: this branch's Credential and Allocator renames and main's
decoder reads from #6745.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant