Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/core/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ add_subdirectory(camera_core)
# Camera interface: shared memory and the socket between processes
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
add_subdirectory(camera_ipc)
# Camera interface: the messages, the plugin's end and the reader's end of the connection
add_subdirectory(camera_protocol)
add_subdirectory(camera_endpoint)
add_subdirectory(camera_reader)
endif()

# Build OXR library (depends on oxr_utils for types)
Expand Down
20 changes: 20 additions & 0 deletions src/core/camera_endpoint/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Agent notes — `camera_endpoint`

**CRITICAL (non-optional):** Before editing this package, complete the mandatory **`AGENTS.md` preflight** in [`../../../AGENTS.md`](../../../AGENTS.md) (read every applicable `AGENTS.md` on your paths, not just this file).

Read [`README.md`](README.md) for what runs on which thread before changing it.

- **Nothing sits between publish and the socket.** A frame event is encoded and sent inside `send_event`, on the publishing thread. Do not add a queue or a thread hop there, and do not log per frame.
- **`send_event` holds its camera group's lock.** It must not take another lock, block or call the publisher. That lock is also what serialises each connection's event encoder.
- **Unsubscribe before a connection is destroyed.** `Publisher::unsubscribe` returning is the guarantee that no `send_event` is running or will start for that sink.
- **A reader is untrusted.** Whatever it sends can at most get it dropped; it must not stop the thread or affect other readers.
- **Reserve, answer, activate.** The handshake response is the first message a reader receives: `subscribe`, send the response with its descriptors, then `activate`. Every way out of that sequence ends in `drop()`, which is the one place a subscription is given up.
- **A test does not assume which thread runs first.** A reader's request and the camera's publish come from two threads and are served in either order: have the reader ask for what comes after a frame set it names, not for "anything", when the test expects a particular one. Run the suite on one CPU (`taskset -c 0`) before pushing; CI's slower machines find these, a workstation does not.
- **A test's helper thread is stopped and joined however the test ends.** A failed `REQUIRE` throws, and a `std::thread` that is still joinable then aborts the process instead of reporting the failure. Use `std::jthread`, or a guard whose destructor joins.
- **A dropped connection returns host-memory leases only because nothing reads them any more.** Do not extend that to a kind of memory whose readers can leave work running on a slot.
- **The constructor takes the path; only `start()` listens.** Readers find a plugin by connecting to every endpoint of their user, so an endpoint that accepts before it can answer costs each of them a wait. Do not listen in the constructor, and call `listen()` before the service thread begins: a listener that does not listen yet polls as hung up.
6 changes: 6 additions & 0 deletions src/core/camera_endpoint/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.20)

add_subdirectory(cpp)
78 changes: 78 additions & 0 deletions src/core/camera_endpoint/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Camera interface: the endpoint

A camera plugin's side of the conversation with readers. It puts a
[`camera_core`](../camera_core) `Publisher` on a socket ([`camera_ipc`](../camera_ipc)) and
speaks the [`camera_protocol`](../camera_protocol).

```cpp
Publisher publisher;
Endpoint endpoint(publisher, socket_path); // takes the path

// ... the plugin opens its cameras, and so knows what each one publishes ...
const GroupId head = publisher.add_group(description);
HostPool left(slots, bytes, "head.left"), right(slots, bytes, "head.right");
endpoint.set_group_memory(head, { left.descriptor(), right.descriptor() });
endpoint.start(); // readers can connect from here

// ... the backend acquires slots, writes pixels and publishes; readers come and go ...
endpoint.stop(); // before the pools go away
```

## When readers can reach it

An endpoint is visible to readers from `start()` to `stop()`, and at no other time.

- **Constructing it takes the path.** The socket file is there and the path is locked, so a
second process that wants the same path fails at once with "another process has taken ...",
before it has opened a camera.
- **Until `start()` a reader that connects is refused at once.** It is not accepted and then
left waiting for an answer nobody sends. A reader that looks through the user's camera
plugins ([`camera_reader`](../camera_reader)) passes over such an endpoint without waiting.
So a plugin constructs its endpoint first, opens its cameras, and calls `start()` when it
can answer.
- **`stop()` gives the path up.** The socket file is removed and a reader is refused again.
A later `start()` takes the path again.

## What runs where

- **One thread of its own** accepts readers and handles what they send: the handshake, the
requests for the newest frame set and the releases. It sleeps in `poll` until a reader
sends something.
- **A frame set is sent from the thread that publishes** to a reader that is waiting for it:
straight from `Publisher::publish`, with a send that does not block. There is no queue and
no hand-over to another thread between a frame set being published and its event being on
the socket. A frame set that already exists when a newest-order reader asks, and the one
that follows a release, are sent from the endpoint's thread.
- **The answer to a handshake comes before any frame set.** The subscription is reserved,
which counts it in the pool's budget, then the response is sent, then delivery starts.
Capture does not wait for any of it. A reader that is gone before it could be answered
leaves no reservation behind.

## When a reader goes away

A closed connection is the signal, whether the reader exited, crashed or was killed. Its
subscription ends and every frame set it held goes back to the pool. A reader that sends
bytes that are not a message, a message only a plugin sends, or an operation this build
does not know, is dropped the same way; other readers are not affected.

Taking the frame sets back at once is right for host memory, which is all there is today: a
reader that is gone reads nothing. A closed connection is not evidence that work a reader
handed to a GPU or a hardware encoder has finished, so a kind of memory where that matters
needs its own rule for these slots before it is served from here (see
[`camera_core`](../camera_core), "What publishing, holding and releasing mean").

## Logging

`isaaccapture.camera_endpoint.Endpoint` logs a reader subscribing, being refused and leaving.
A reader that only asks what is published is logged at debug, because a reader that looks for
its camera asks that of every plugin of its user. Nothing is logged per frame.

## Tests

`tests/cpp/core/camera_endpoint`, run with `ctest -L camera_endpoint`. They are tagged `unit`
and play the reader's part with the socket and the codec directly.
27 changes: 27 additions & 0 deletions src/core/camera_endpoint/cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

cmake_minimum_required(VERSION 3.20)

find_package(Threads REQUIRED)

# The plugin's side of the conversation with readers: serves a Publisher's groups over a socket.
add_library(camera_endpoint STATIC
endpoint.cpp
)

target_include_directories(camera_endpoint INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/inc>
)

target_link_libraries(camera_endpoint
PUBLIC
camera::camera_core
PRIVATE
camera::camera_ipc
camera::camera_protocol
log_bridge::log_bridge_core
Threads::Threads
)

add_library(camera::camera_endpoint ALIAS camera_endpoint)
Loading
Loading