Repository navigation
examples/camera_streamer: carry a qualified sensor capture time to LiveKit - #1479
gareth-morgan-nv wants to merge 2 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
…erve a clock responder The AVS1 bundle gains an optional timing extension (flag 0x0008): one 24-byte record per sample with the sensor capture instant on the sender's monotonic clock, the sender's stated uncertainty, the anchor age and a validity bit. The receiver parses and validates it and publishes the fields in the IPC metadata. The mock sender emits it from a synthetic sender monotonic clock with its own offset, drift and steps, writes the true capture instant on this host's clock to a ground-truth file, and serves the sender's stateless UDP clock responder (13-byte Q, 29-byte R) with mute, loss and reorder faults for tests. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
…ground truth Scenarios through the real receiver, bridge and a local LiveKit server with a sender clock offset and drift: accuracy, a clock step, a session restart, a stale clock mapping, lost and reordered replies, invalid timing records, and refusal without a sender clock. The mock's lint fix rides along. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Gareth Morgan <gmorgan@nvidia.com>
83b23e2 to
d950e1f
Compare
57c7ab4 to
eecc167
Compare
|
Review of eecc167: I compiled and ran clock_estimator, capture_qualifier, timestamp_basis, clock_sync, and sender_clock tests successfully (the UDP test requires loopback networking). Rejecting unqualified capture times without substituting frame-ready time is a good recording contract. Two integration caveats: qualified sensor timing currently requires the SIPL sender extension in #1474; Argus does not yet populate it. Also, timestamp qualification depends on the pixels belonging to that metadata: the IPC slot-ownership concern posted on #1478 remains relevant here, but should be fixed once in the shared handoff, not duplicated in the timing code. No new hardware validation was performed in this review. The mock timing accuracy is useful, but should not be presented as physical exposure alignment or photon-to-photon latency. |
|
🤖 Agent-generated (Claude Code · claude-sonnet-5-5) Thanks for the review and for running the bridge's timing tests. This PR is going to be closed, so no further changes will be made here. Your caveats are noted: qualified sensor timing currently needs the SIPL extension in #1474 (Argus does not populate it), the slot-ownership concern belongs in the shared handoff discussed on #1478 and not in the timing code, and the mock timing accuracy should not be presented as physical exposure alignment or photon-to-photon latency. |
Summary
Receiver, mock and bridge side of capture-time alignment. Stacked on the bridge PR.
0x0008: one 24-byte record per sample with the sensor capture instant on the sender's monotonic clock, its stated uncertainty, anchor age and a validity bit. The receiver parses and validates it and publishes the fields in the IPC metadata.capture_monotonic_usonly for frames that qualify: valid record, current mapping with warm-up, mapping plus sensor uncertainty within the budget, possible age. Otherwise no capture time is published and frame-ready time is never substituted. A reply counts only from the sender's address, of the exact size and tag, matching the outstanding request and the current session generation.Limitations
A step of the sender clock shows at the next exchange, so frames in between (at most one exchange interval) carry the old offset; a learned age level cannot tell a step from a stall, so none is used. The sender side (real Orin) is not part of this PR and has not been qualified against it.
Testing
ctest 7/7 (clock-estimator golden vectors from egocap's estimator, qualifier, UDP client against a fake sender, plus the existing targets). A scenario test through the real receiver, bridge and a local LiveKit server (accuracy, step, restart, stale mapping, loss and reorder, invalid records, refusal) passed twice; with a 1.2 s offset and 50 ppm drift the published capture time is within about 100 us of ground truth. A take saved by the unchanged egocap timing recorder matched ground truth within 0.84 ms. The scenario test was reformatted after those runs and not rerun.
🤖 Generated with Claude Code