diff --git a/src/core/camera/AGENTS.md b/src/core/camera/AGENTS.md index 45ff6c047c..cb0f2efef9 100644 --- a/src/core/camera/AGENTS.md +++ b/src/core/camera/AGENTS.md @@ -25,6 +25,21 @@ Read [`README.md`](README.md) first: it has the terms (stream, group, window, or - **An empty location looks; a named one does not.** With no endpoint and no instance the reader asks every endpoint `default_endpoint_paths()` lists. With either, it talks to that one plugin alone: the full handshake timeout, and an error if the plugin is not there. Do not fall back from one to the other. - **Looking stays plain.** The endpoints are asked one after another, each with a short fixed timeout. A refused connection is passed over silently. An endpoint that fails to say what it publishes (no answer in time, a refusal, a closed connection, any other failure) is passed over too, named with its reason in `open`'s error and left out by `list`; do not narrow that to the timeout. A group on two endpoints is an error, never a choice. No cache, no index file, no retry, no thread, and no option for any of it. +## Publishing cameras + +These rules hold for the publishing side of this library and for every camera plugin built on it (`src/plugins/camera_*`). + +- **One camera plugin per vendor or SDK.** A plugin links exactly one camera SDK and hands exactly one `Backend` to `run_camera_plugin`. Do not add a second SDK to an existing plugin, a list of backends, or run-time loading of backends. A plugin that needs an SDK gets its own `BUILD_PLUGIN_*` option, off by default, like the repository's other vendor plugins. The steps are in [`README.md`](README.md), "Writing a camera plugin". +- **No SDK types outside a plugin's `core/` directory.** This library, a plugin's `app/` and its `plugin.yaml` see `CameraSource` and this library's own types only, and readers never link a camera SDK. +- **A plugin's `main` calls `run_camera_plugin` and does nothing else.** The arguments, the configuration file, the endpoint and the exit codes are written once, here, so they are the same for every camera plugin. +- **A plugin serves its own cameras.** The configuration file is shared by the camera plugins of a user, and its form is checked whole: an entry that is not an id with a `backend:` and `name: value` settings stops every plugin that reads the file, whichever backend it names. Of a well-formed file a plugin takes the cameras whose `backend:` is its backend's name and leaves the others alone: it never opens or lists them, except to say what the file names when none is its own. +- **A backend refuses what it does not understand.** Read options through `OptionReader` and call `finish()`, so a misspelt option is an error. +- **A camera drops a frame rather than wait.** When `acquire` has no free slot, abandon what was taken for that frame set and carry on. +- **A backend is opened once, with all its cameras.** What an SDK has open belongs to the backend and is shared by the sources it returns; a source is one camera and publishes one group. Do not open an SDK per camera when the SDK serves several from one object. +- **The backend interface shows what the `Publisher` can do, and no more.** `FrameOutput` passes through the publisher's two ways to get a slot, the slot-free callback and the restart, and `CameraSource::backend_reserve()` passes through the reserve. A further need (memory an SDK allocates, a readiness signal for GPU writes) is added with the pool and the backend that have it, not ahead of them. +- **`publish()` means complete.** A source publishes a slot only when everything that writes it has finished; readers look at it from that moment. +- **The usage is terminal output; everything else is logged.** `run_camera_plugin` prints the usage, and the refusal of an argument together with it, as one message on one channel. Every other line goes through `isaaccapture::Logger` under `isaaccapture.camera.`. + ## Slots, windows and leases - **The slot and lease code uses the standard library only.** `types.hpp`, `publisher.hpp` and their sources have no slot memory, sockets, logging or platform headers in them. Memory, connections and platform calls stay in the files of the next section. @@ -80,4 +95,5 @@ Read "What runs where" in the [`README.md`](README.md) before changing it. - **A test does not assume which thread runs first.** A reader's request and the camera's publish come from two threads and are served in either order: have the reader ask for what comes after a frame set it names, not for "anything", when the test expects a particular one. Run the suite on one CPU (`taskset -c 0`) before pushing; CI's slower machines find these, a workstation does not. - **A test waits for what the plugin's thread does; it does not assume it.** `next` returning says nothing about whether the plugin has taken the reader's request, so wait for the effect (`Served::wait_until_sent`) before counting what was sent. - **A test's helper thread is stopped and joined however the test ends.** A failed `REQUIRE` throws, and a `std::thread` that is still joinable then aborts the process instead of reporting the failure. Use `std::jthread`, or a guard whose destructor joins. -- **Tests that look through the user's plugins get a directory of their own.** `EndpointDirectory` (`tests/cpp/core/camera/endpoint_directory.hpp`) points `XDG_RUNTIME_DIR` at a temporary directory and puts it back; a test that uses the real one finds whatever camera plugin the developer has running. +- **Tests that look through the user's plugins, or let a plugin choose its endpoint or its configuration file, get a user's directories of their own.** Point `XDG_RUNTIME_DIR`, and for a plugin's configuration file `XDG_CONFIG_HOME`, at a short temporary directory and put them back, as `EndpointDirectory` (`tests/cpp/core/camera/endpoint_directory.hpp`) and `UserDirectories` (`tests/cpp/core/camera/test_camera_plugin.cpp`) do; a test that uses the real ones finds whatever camera plugin and `cameras.yaml` the developer has. +- **A test that runs `run_camera_plugin` in the test process aims the stop signal at the plugin's thread**, and starts that thread with the signal blocked (`PluginRun` in `tests/cpp/core/camera/test_camera_plugin.cpp`). A signal sent to the process would end the test instead. diff --git a/src/core/camera/README.md b/src/core/camera/README.md index e96e30c853..dec377ac0c 100644 --- a/src/core/camera/README.md +++ b/src/core/camera/README.md @@ -30,6 +30,14 @@ What is in it, included as `` from the target `camera::camera`: - `endpoint.hpp`: `Endpoint`, a camera plugin's end of the connection, which serves a `Publisher`'s groups to readers. - `camera_reader.hpp`: `CameraReader`, the reader's end and what an application uses. +- `camera_source.hpp`: `Backend`, `CameraSource` and `FrameOutput`, what a vendor's capture code + implements and what it publishes through. +- `camera_host.hpp`: `CameraHost`, which gives each camera its pools and puts all of them behind + one endpoint. +- `options.hpp`: `OptionReader`, how a backend reads a camera's options, so that one it does + not know is an error. +- `config.hpp`: `load_config_file`, the cameras a configuration file names for one backend. +- `camera_plugin.hpp`: `run_camera_plugin`, everything a camera plugin's `main` does. ## Terms @@ -150,6 +158,172 @@ Not covered: a process that is stopped while it waits inside `next` is handed, w continues, the frame set that arrived meanwhile. A reader that needs to know how old its frame is has the times of every step in `timeline()`. +## Publishing cameras + +A camera plugin is one vendor's or one SDK's capture code and this library. The capture code +opens the cameras and writes the images. The library gives each camera its pools, publishes the +frame sets, serves them on the plugin's [endpoint](#the-endpoint), and is the plugin's command +line and configuration. + +```cpp +#include + +int main(int argc, char** argv) +{ + return core::camera::run_camera_plugin(argc, argv, my_backend()); +} +``` + +There is one camera plugin per vendor or SDK. Each is a small executable with its own +`plugin.yaml` and install folder, and its own build option when it needs an SDK. Readers find +the plugin that publishes a camera +([How a reader finds its camera](#how-a-reader-finds-its-camera)); none of them links a camera +SDK. [`camera_synthetic`](../../plugins/camera_synthetic) is the plugin that needs no hardware +and the one to copy; see [Writing a camera plugin](#writing-a-camera-plugin). + +`run_camera_plugin` is everything a plugin's `main` does. It reads the arguments, chooses the +cameras, takes the endpoint, opens the cameras through the plugin's backend, serves until +`SIGINT` or `SIGTERM`, and returns the exit code: 0 after a clean stop or `--help`, 2 for a +command line it cannot parse, 1 when the plugin cannot run. An argument that parses and cannot +be honoured, such as an option its backend refuses, returns 1. + +`CameraHost` gives each camera a pool of shared memory per stream ([Host pool](#host-pool)) and +puts all of them behind one endpoint. `run_camera_plugin` uses it; a test can use it without a +process. + +### Running a plugin + +What follows is true of every camera plugin, because all of it is this library. + +**Which cameras.** A camera has an id that readers ask for and the options of the plugin's +backend. The plugin takes its cameras from, in this order: + +1. `--camera=ID[:KEY=VALUE,...]` on the command line, once per camera; +2. `--config=FILE`, or without it `$XDG_CONFIG_HOME/isaaccapture/cameras.yaml` + (`~/.config/isaaccapture/cameras.yaml`) when that file exists; +3. otherwise, or with `--auto`, every camera its backend finds plugged in. + +`--camera`, `--config` and `--auto` are alternatives: a command line that mixes them is +refused. + +**The configuration file** is one for all the camera plugins of a user. A camera takes two +lines, its id and `backend:`, and one more for each option of that backend as `name: value`. +`slots:` at the top level sets the pool size. + +```yaml +# ~/.config/isaaccapture/cameras.yaml +cameras: + head: + backend: synthetic + width: 1920 +slots: 8 +``` + +A plugin serves the cameras whose `backend:` is its own and leaves the others to their +plugins. A file that names cameras, but none for this plugin's backend, stops the plugin with +an error that names the file and lists its cameras with their backends. That is how a +misspelt backend shows. + +**Where it listens.** On `camera-.sock` in the user's endpoint directory +([Where a plugin listens](#where-a-plugin-listens)). A plugin's instance name is the name of +its backend, so the synthetic plugin listens on `camera-synthetic.sock`. A second plugin of +the same backend takes another name with `--instance=NAME`. Give it its own cameras with +`--camera` or `--config` as well: the instance name does not change which cameras a plugin +takes, so without them it serves the same cameras as the first, and a reader that looks for +one of them finds it on two plugins and is refused. `--endpoint=PATH` sets the socket file +outright; a reader is then told the path. Only one plugin serves an endpoint at a time: a +plugin takes its endpoint before it opens its cameras, and accepts readers once they are +open ([When readers can reach it](#when-readers-can-reach-it)). + +**Pool size.** `--slots=N`, 8 by default, is the number of frames kept per stream. A pool +holds the newest frame set, one slot for the camera to write ahead, and the windows of all +readers; a reader that does not fit is refused with the numbers. When readers want every +frame in order, the frame sets waiting for them are kept in place of the newest one: as many +as the largest of their windows ([Pool budget](#pool-budget)). + +**Who starts it.** The application, through the plugin manager, like every other Isaac +Capture plugin: `plugin.yaml` sits next to the executable, and the handle the plugin manager +returns stops the plugin when it goes away. The `--plugin-root-id` the plugin manager adds is +accepted and not used. A plugin can also be run by hand or from a service. A reader never +starts it ([Reading cameras](#reading-cameras)). + +A misspelt argument, option or setting is an error, not something ignored. `SIGINT` and +`SIGTERM` stop the plugin cleanly; readers see their connection close. `--help` prints the +arguments under the name of the executable. + +Every ten seconds the plugin logs, per camera, how many frame sets it published. It warns +when a camera had to drop frame sets because its pool was full, or when a reader did not take +its events; both mean the pool or a reader's window needs another look. The loggers are +`isaaccapture.camera.run_camera_plugin` and `isaaccapture.camera.CameraHost`. The usage, and +the refusal of an argument that comes with it, are for the terminal and are not logged. + +### What a backend implements + +A backend is one camera SDK as its plugin sees it, and a plugin has exactly one. It is one +function that returns a `Backend` (`camera_source.hpp`): + +- a `name`: what a camera's configuration says in `backend:`, and the plugin's instance name; +- a `description`: a short phrase without a full stop, which the plugin's `--help` shows + after the backend's name; +- `open`, which is called once with every camera the plugin is to serve and returns one + `CameraSource` per camera. What the SDK has open is the backend's, not a camera's: an SDK + that serves a whole rig from one object, or keeps state for the process, opens it once in + that call and lets the sources share it; +- optionally `discover`, which returns the cameras that are plugged in, for a plugin that is + not told which to serve. + +Each source publishes one group. It reads its options through `OptionReader` and calls +`finish()`, so a misspelt option is refused. + +A source gets the slot for an image through its `FrameOutput` in one of two ways: + +- **The library picks the slot.** `acquire()` hands out a free slot, the source writes the + image into it and calls `publish()`. When nothing is free the frame is dropped. This is the + way to start with. +- **The SDK picks the buffer.** For an SDK that is given its buffers up front and says which + one it filled: the source hands the SDK the memory of every slot (`slot_count()`, + `slot_memory()`), asks to be told when a slot is free again (`on_slot_free()`), calls + `claim()` for the buffer the SDK filled, then `publish()`. `backend_reserve()` says how + many buffers the SDK needs in hand to keep capturing; readers are never given so many that + it is left with fewer. + +In both, `publish()` says the images are complete. A slot is not handed out or given back to +the SDK while a reader holds it, also across `restart()`, which is how a source says that +its camera restarted. + +### Writing a camera plugin + +[`src/plugins/camera_synthetic`](../../plugins/camera_synthetic) is the template. For a +vendor or SDK called ``: + +1. **Directory.** Copy `src/plugins/camera_synthetic` to `src/plugins/camera_`, without + `tools/`: `camera_bench` reads any camera plugin and stays where it is. +2. **`core/`: the backend.** A static library `camera__core` that links + `camera::camera` and the SDK. It implements `CameraSource` for one camera and has one + function that returns the `Backend`, named ``. No type of the SDK leaves this + directory. +3. **`app/main.cpp`.** A few lines: `return run_camera_plugin(argc, argv, _backend());`. + The executable is `camera__plugin`. There is nothing else to write for the command line, + the configuration file or the endpoint. +4. **`plugin.yaml`.** `name: camera_`, `command: "./camera__plugin"`, a version, a + description and the `devices:` entry of type `camera`. `app/CMakeLists.txt` copies it next + to the built executable, so the plugin manager can start the plugin from the build tree + when it is given the directory above the executable's (`build/src/plugins/camera_`), + as it is given `/plugins` for an install. The plugin is installed to + `plugins/camera_`. +5. **Build option.** A plugin that needs an SDK gets its own `BUILD_PLUGIN_` option, + off by default, next to the other plugin options in the top-level `CMakeLists.txt`, and + `src/plugins/CMakeLists.txt` adds its directory only when the option is on. So nobody + builds or installs an SDK they do not use. `camera_synthetic` needs none and is built + whenever plugins are. +6. **Tests.** In `tests/cpp/plugins/camera_`, wired under the same option, with a + CTest label that starts with `camera_`. Tag what runs without the camera `unit` (the + backend's options and what it describes) and what starts the plugin as a process + `integration`; the cases of `tests/cpp/plugins/camera_synthetic` show both. +7. **Documents.** A `README.md` with the backend's options and what the SDK needs, and an + `AGENTS.md` that points to "Publishing cameras" in [`AGENTS.md`](AGENTS.md) here for the + rules every camera plugin follows. + ## How it works ### Slots, windows and leases @@ -521,9 +695,9 @@ its camera asks that of every plugin of its user. Nothing is logged per frame. ## Tests -`tests/cpp/core/camera`, built as one executable, `camera_tests`. `ctest -L camera` runs them; a -label is a regular expression, so `ctest -L "^camera$"` is the form that selects this library's -tests alone. +`tests/cpp/core/camera`, built as one executable, `camera_tests`. `ctest -L camera` runs them. A +label is a regular expression, so that also runs the tests of the camera plugins, whose labels +start with `camera_`; `ctest -L "^camera$"` is the form that selects this library's tests alone. - **Unit** (`ctest -L unit`): everything that runs in one process. The endpoint's cases play the reader's part with the socket and the codec directly. The reader's cases run it against a @@ -532,6 +706,16 @@ tests alone. directory and put there what the case is about: plugins with different groups or the same one, a plugin that has not started, a socket file nobody is behind, a plugin that does not answer, a plugin that refuses the reader or goes away while it is asked. + + The publishing side's cases use stand-in backends, and what they serve is read with a real + reader in the same process. Camera arguments, options and configuration files: a camera of + the command line is the plugin's backend's; a plugin takes its own cameras from a file it + shares with other plugins; a file with cameras and none for the plugin's backend is refused + with what it names. The host with a camera that publishes on demand, with one whose SDK owns + the buffers, with one that cannot start, and across a camera restart. `run_camera_plugin` on + a thread, stopped with the signal a plugin process is stopped with: the instance name is the + backend's unless it is given; the backend is opened once with all its cameras; what is + served without arguments; what ends a plugin with an error; what is answered with the usage. - **Integration** (`ctest -L integration`): the cases that start a second process. One sends it a pool over the socket: it maps the pool, sees the owner's writes and cannot write itself. The others have the reader in the test process and `camera_test_peer` diff --git a/src/core/camera/cpp/CMakeLists.txt b/src/core/camera/cpp/CMakeLists.txt index 3c75c7b6a9..34d9ea43a9 100644 --- a/src/core/camera/cpp/CMakeLists.txt +++ b/src/core/camera/cpp/CMakeLists.txt @@ -7,20 +7,29 @@ find_package(Threads REQUIRED) # The camera library; this module's README.md says what is in it. Linux only. add_library(camera STATIC + camera_host.cpp + camera_plugin.cpp camera_reader.cpp codec.cpp + config.cpp description.cpp endpoint.cpp endpoint_path.cpp host_pool.cpp message_socket.cpp + options.cpp publisher.cpp + inc/camera/camera_host.hpp + inc/camera/camera_plugin.hpp inc/camera/camera_reader.hpp + inc/camera/camera_source.hpp + inc/camera/config.hpp inc/camera/endpoint.hpp inc/camera/endpoint_path.hpp inc/camera/host_pool.hpp inc/camera/message_socket.hpp inc/camera/messages.hpp + inc/camera/options.hpp inc/camera/publisher.hpp inc/camera/types.hpp inc/camera/unique_fd.hpp @@ -34,6 +43,7 @@ target_link_libraries(camera PRIVATE isaacteleop_schema log_bridge::log_bridge_core + yaml-cpp::yaml-cpp Threads::Threads ) diff --git a/src/core/camera/cpp/camera_host.cpp b/src/core/camera/cpp/camera_host.cpp new file mode 100644 index 0000000000..14bfa08f21 --- /dev/null +++ b/src/core/camera/cpp/camera_host.cpp @@ -0,0 +1,236 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include "inc/camera/camera_host.hpp" + +#include "inc/camera/endpoint.hpp" +#include "inc/camera/host_pool.hpp" + +#include + +#include +#include +#include +#include + +namespace core::camera +{ + +namespace +{ + +// One camera's way into the publisher: slots come from the camera's own pools. +class PoolOutput final : public FrameOutput +{ +public: + PoolOutput(Publisher& publisher, GroupId group, std::vector>& pools) + : m_publisher(publisher), m_group(group), m_pools(pools) + { + } + + std::optional acquire(std::size_t stream) override + { + const std::optional slot = m_publisher.acquire_slot(m_group, stream); + if (!slot) + { + return std::nullopt; + } + return WritableImage{ *slot, m_pools.at(stream)->slot(*slot) }; + } + + std::uint32_t slot_count(std::size_t stream) const override + { + return m_pools.at(stream)->slot_count(); + } + + std::span slot_memory(std::size_t stream, std::uint32_t slot) override + { + return m_pools.at(stream)->slot(slot); + } + + void claim(std::size_t stream, std::uint32_t slot) override + { + m_publisher.claim_slot(m_group, stream, slot); + } + + void on_slot_free(SlotFreeCallback callback) override + { + m_publisher.set_slot_free_callback(m_group, std::move(callback)); + } + + void abandon(std::size_t stream, std::uint32_t slot) override + { + m_publisher.abandon_slot(m_group, stream, slot); + } + + std::uint64_t publish(std::span images, const FrameTimeline& timeline) override + { + return m_publisher.publish(m_group, images, timeline); + } + + void restart() override + { + m_publisher.restart_group(m_group); + } + +private: + Publisher& m_publisher; + GroupId m_group; + std::vector>& m_pools; +}; + +struct Camera +{ + std::unique_ptr source; + std::string id; + GroupId group = 0; + std::vector> pools; // one per stream + std::unique_ptr output; + PublisherStats logged; // the counters at the previous log_activity() +}; + +} // namespace + +class CameraHost::Impl +{ +public: + explicit Impl(const std::string& endpoint_path) + : endpoint(publisher, endpoint_path), logger(isaaccapture::Logger::get("isaaccapture.camera.CameraHost")) + { + } + + Publisher publisher; + Endpoint endpoint; + // Stable addresses: an output points at its camera's pools. + std::vector> cameras; + bool started = false; + std::chrono::steady_clock::time_point activity_logged = std::chrono::steady_clock::now(); + std::shared_ptr logger; +}; + +CameraHost::CameraHost(const std::string& endpoint_path) : m_impl(std::make_unique(endpoint_path)) +{ +} + +CameraHost::~CameraHost() +{ + stop(); +} + +void CameraHost::add_camera(std::unique_ptr source, std::uint32_t slot_count) +{ + if (m_impl->started) + { + throw std::logic_error("cameras are added before the camera host starts"); + } + GroupDescription description = source->describe(); + for (StreamDescription& stream : description.streams) + { + if (stream.slot_count == 0) + { + stream.slot_count = slot_count; + } + } + + auto camera = std::make_unique(); + camera->id = description.id; + // Refuses a description that is not usable before any memory is allocated for it. + const GroupId group = m_impl->publisher.add_group(description, source->backend_reserve()); + camera->group = group; + std::vector descriptors; + for (const StreamDescription& stream : description.streams) + { + camera->pools.push_back(std::make_unique(stream.slot_count, stream.slot_bytes, "camera." + stream.id)); + descriptors.push_back(camera->pools.back()->descriptor()); + } + m_impl->endpoint.set_group_memory(group, std::move(descriptors)); + camera->output = std::make_unique(m_impl->publisher, group, camera->pools); + camera->source = std::move(source); + + const StreamDescription& first = description.streams.front(); + m_impl->logger->info("camera '{}': {} stream(s), {}x{} at {:.2f} fps, {} slots of {} bytes each", description.id, + description.streams.size(), first.width, first.height, first.frame_rate, first.slot_count, + first.slot_bytes); + m_impl->cameras.push_back(std::move(camera)); +} + +void CameraHost::start() +{ + if (m_impl->started) + { + return; + } + m_impl->started = true; + try + { + m_impl->endpoint.start(); + for (const auto& camera : m_impl->cameras) + { + camera->source->start(*camera->output); + } + } + catch (...) + { + // Nothing is left half started: the cameras that did start are stopped and readers + // are not served, so start() can be called again. + stop(); + throw; + } +} + +void CameraHost::stop() +{ + if (!m_impl->started) + { + return; + } + // Cameras first: nothing publishes into a pool or the endpoint once they have stopped. + for (const auto& camera : m_impl->cameras) + { + camera->source->stop(); + } + m_impl->endpoint.stop(); + m_impl->started = false; +} + +std::size_t CameraHost::camera_count() const +{ + return m_impl->cameras.size(); +} + +std::size_t CameraHost::reader_count() const +{ + return m_impl->endpoint.reader_count(); +} + +const Publisher& CameraHost::publisher() const +{ + return m_impl->publisher; +} + +void CameraHost::log_activity() +{ + const auto now = std::chrono::steady_clock::now(); + const double seconds = std::chrono::duration(now - m_impl->activity_logged).count(); + m_impl->activity_logged = now; + for (const auto& camera : m_impl->cameras) + { + const PublisherStats stats = m_impl->publisher.stats(camera->group); + const std::uint64_t published = stats.published - camera->logged.published; + const std::uint64_t dropped = stats.no_free_slot - camera->logged.no_free_slot; + const std::uint64_t unsent = stats.send_failures - camera->logged.send_failures; + camera->logged = stats; + m_impl->logger->info("camera '{}': {} frame sets in {:.1f} s ({:.1f} per second), {} reader(s) connected", + camera->id, published, seconds, seconds > 0.0 ? published / seconds : 0.0, + m_impl->endpoint.reader_count()); + if (dropped != 0 || unsent != 0) + { + m_impl->logger->warn( + "camera '{}': dropped {} frame set(s) for lack of a free slot; {} event(s) " + "were not taken by a reader", + camera->id, dropped, unsent); + } + } +} + +} // namespace core::camera diff --git a/src/core/camera/cpp/camera_plugin.cpp b/src/core/camera/cpp/camera_plugin.cpp new file mode 100644 index 0000000000..e44b6b73d2 --- /dev/null +++ b/src/core/camera/cpp/camera_plugin.cpp @@ -0,0 +1,283 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include "inc/camera/camera_plugin.hpp" + +#include "inc/camera/camera_host.hpp" +#include "inc/camera/config.hpp" +#include "inc/camera/endpoint_path.hpp" +#include "inc/camera/options.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace core::camera +{ + +namespace +{ + +// How often the plugin logs what each camera did. +constexpr time_t ACTIVITY_LOG_SECONDS = 10; + +// The running executable's own name, for the usage and the messages. +std::string program_name(int argc, char** argv, const Backend& backend) +{ + std::string name; + if (argc > 0 && argv[0] != nullptr) + { + name = std::filesystem::path(argv[0]).filename().string(); + } + return name.empty() ? "camera_" + backend.name : name; +} + +std::string usage(const std::string& program, const Backend& backend) +{ + const std::string indent(std::string_view("Usage: ").size() + program.size() + 1, ' '); + return "Usage: " + program + " [--config=FILE | --camera=ID[:KEY=VALUE,...] ... | --auto]\n" + indent + + "[--endpoint=PATH | --instance=NAME] [--slots=N]\n" + "\n" + "The camera plugin of the '" + + backend.name + "' backend: " + backend.description + + ".\n" + "It owns that backend's cameras and serves their frames to readers in other processes.\n" + "Without arguments it serves the cameras its configuration file names for the backend;\n" + "when there is no such file, every camera the backend finds plugged in.\n" + " --config=FILE the cameras to serve (default: $XDG_CONFIG_HOME/isaaccapture/cameras.yaml,\n" + " or ~/.config/isaaccapture/cameras.yaml); the cameras a file names for\n" + " other backends are left to their plugins\n" + " --camera=... one camera: the id readers ask for, then the backend's options for it;\n" + " give it once per camera\n" + " --auto serve every camera the backend finds plugged in, whatever a file says\n" + " --instance=NAME the plugin's instance name (default: " + + backend.name + + "); it listens on\n" + " camera-NAME.sock in this user's endpoint directory\n" + " --endpoint=PATH the socket file readers connect to, instead of one in that directory\n" + " --slots=N frames kept per stream (default 8); a reader with a larger window needs more\n"; +} + +// The value of `--name=value` in `arg`, or nothing when `arg` is another flag. +std::optional flag_value(std::string_view arg, std::string_view name) +{ + if (arg.starts_with(name) && arg.size() > name.size() && arg[name.size()] == '=') + { + return arg.substr(name.size() + 1); + } + return std::nullopt; +} + +struct Arguments +{ + std::string endpoint; + std::string instance; + std::string config_file; + std::vector cameras; + std::optional slots; + bool automatic = false; + bool help = false; +}; + +Arguments parse(int argc, char** argv, const Backend& backend) +{ + Arguments arguments; + for (int i = 1; i < argc; ++i) + { + const std::string_view arg = argv[i]; + if (arg == "--help" || arg == "-h") + { + arguments.help = true; + } + else if (arg == "--auto") + { + arguments.automatic = true; + } + else if (const auto endpoint = flag_value(arg, "--endpoint")) + { + arguments.endpoint = *endpoint; + } + else if (const auto instance = flag_value(arg, "--instance")) + { + arguments.instance = *instance; + } + else if (const auto config = flag_value(arg, "--config")) + { + arguments.config_file = *config; + } + else if (const auto camera = flag_value(arg, "--camera")) + { + arguments.cameras.push_back(parse_camera_argument(*camera, backend.name)); + } + else if (const auto slots = flag_value(arg, "--slots")) + { + std::uint32_t count = 0; + const auto [end, error] = std::from_chars(slots->data(), slots->data() + slots->size(), count); + if (error != std::errc() || end != slots->data() + slots->size()) + { + throw std::invalid_argument("--slots takes a whole number, not '" + std::string(*slots) + "'"); + } + arguments.slots = count; + } + else if (flag_value(arg, "--plugin-root-id")) + { + // The plugin manager gives it to every plugin, as the root under which a plugin + // reports the status of its devices. A camera plugin reports none. + } + else + { + throw std::invalid_argument("unknown argument '" + std::string(arg) + "'"); + } + } + if (!arguments.endpoint.empty() && !arguments.instance.empty()) + { + throw std::invalid_argument("--endpoint and --instance both say where to listen; give one"); + } + const int sources = + (arguments.config_file.empty() ? 0 : 1) + (arguments.cameras.empty() ? 0 : 1) + (arguments.automatic ? 1 : 0); + if (sources > 1) + { + throw std::invalid_argument("--config, --camera and --auto each say which cameras to serve; give one kind"); + } + return arguments; +} + +// The cameras to serve: from the command line, a file, or what is plugged in. +PluginConfig cameras_to_serve(const Arguments& arguments, const Backend& backend, spdlog::logger& logger) +{ + PluginConfig config; + std::string source = "the command line"; + const std::string default_file = default_config_path(); + // A file says which cameras when one is named, or when the user's own is there and --auto + // does not set it aside. + const bool from_file = + arguments.cameras.empty() && + (!arguments.config_file.empty() || (!arguments.automatic && std::filesystem::exists(default_file))); + if (!arguments.cameras.empty()) + { + config.cameras = arguments.cameras; + } + else if (from_file) + { + source = arguments.config_file.empty() ? default_file : arguments.config_file; + config = load_config_file(source, backend.name); + } + else + { + source = "the cameras found plugged in"; + if (backend.discover) + { + config.cameras = backend.discover(); + } + } + if (arguments.slots) + { + config.slot_count = *arguments.slots; + } + if (config.cameras.empty()) + { + // The advice is for where this run looked: a file it read is the one to fill in, --auto + // reads none, and a camera plugged in is found only when no file is read. + throw std::invalid_argument("there is no camera to serve in " + source + ". " + + (arguments.automatic ? "Leave --auto out and name" : "Name") + + " cameras with `backend: " + backend.name + "` in '" + + (from_file ? source : default_file) + "' or with --camera" + + (backend.discover && !from_file ? ", or plug one in" : "")); + } + logger.info("serving {} camera(s) from {}", config.cameras.size(), source); + return config; +} + +} // namespace + +int run_camera_plugin(int argc, char** argv, const Backend& backend) +{ + const std::string program = program_name(argc, argv, backend); + Arguments arguments; + try + { + arguments = parse(argc, argv, backend); + } + catch (const std::exception& error) + { + // The refusal and the usage that follows are one message to the operator, on one channel. + std::cerr << program << ": " << error.what() << "\n\n" << usage(program, backend); + return 2; + } + if (arguments.help) + { + std::cout << usage(program, backend); + return 0; + } + + const auto logger = isaaccapture::Logger::get("isaaccapture.camera.run_camera_plugin"); + try + { + // Every thread started below inherits this mask, so the signals wait for the loop here. + sigset_t stop_signals; + sigemptyset(&stop_signals); + sigaddset(&stop_signals, SIGINT); + sigaddset(&stop_signals, SIGTERM); + pthread_sigmask(SIG_BLOCK, &stop_signals, nullptr); + + const PluginConfig config = cameras_to_serve(arguments, backend, *logger); + std::string endpoint = arguments.endpoint; + if (endpoint.empty()) + { + // A plugin's instance name is its backend's unless it was given another. + endpoint = default_endpoint_path(arguments.instance.empty() ? backend.name : arguments.instance); + prepare_endpoint_directory(endpoint); + } + + CameraHost host(endpoint); + // One call for all the cameras, so that what the SDK has open can be shared among them. + std::vector> sources = backend.open(config.cameras); + if (sources.size() != config.cameras.size()) + { + throw std::logic_error("the '" + backend.name + "' backend was given " + + std::to_string(config.cameras.size()) + " camera(s) and opened " + + std::to_string(sources.size())); + } + for (std::unique_ptr& source : sources) + { + host.add_camera(std::move(source), config.slot_count); + } + host.start(); + logger->info("{} serves {} camera(s) on '{}'", program, host.camera_count(), endpoint); + + // Sleeps until a signal says stop, waking now and then to log what the cameras did. + const timespec activity_interval = { ACTIVITY_LOG_SECONDS, 0 }; + int received = 0; + while ((received = sigtimedwait(&stop_signals, nullptr, &activity_interval)) <= 0) + { + if (errno == EAGAIN) + { + host.log_activity(); + } + } + logger->info("{} stops on signal {}", program, received); + host.stop(); + return 0; + } + catch (const std::exception& error) + { + logger->error("{} cannot run: {}", program, error.what()); + return 1; + } +} + +} // namespace core::camera diff --git a/src/core/camera/cpp/config.cpp b/src/core/camera/cpp/config.cpp new file mode 100644 index 0000000000..5b843ec6b2 --- /dev/null +++ b/src/core/camera/cpp/config.cpp @@ -0,0 +1,142 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include "inc/camera/config.hpp" + +#include +#include +#include +#include + +#include + +namespace core::camera +{ + +namespace +{ + +[[noreturn]] void fail(const std::string& path, const YAML::Mark& mark, const std::string& what) +{ + const std::string line = mark.is_null() ? "" : ":" + std::to_string(mark.line + 1); + throw std::invalid_argument(path + line + ": " + what); +} + +CameraConfig read_camera(const std::string& path, const YAML::Node& id, const YAML::Node& settings) +{ + if (!id.IsScalar() || id.Scalar().empty()) + { + fail(path, id.Mark(), "a camera needs an id"); + } + CameraConfig camera; + camera.id = id.Scalar(); + if (!settings.IsMap()) + { + fail(path, settings.Mark(), "camera '" + camera.id + "' needs a backend, as in `backend: synthetic`"); + } + for (const auto& setting : settings) + { + if (!setting.first.IsScalar() || !setting.second.IsScalar()) + { + fail(path, setting.first.Mark(), "camera '" + camera.id + "' takes settings of the form `name: value`"); + } + if (setting.first.Scalar() == "backend") + { + camera.backend = setting.second.Scalar(); + } + else + { + camera.options[setting.first.Scalar()] = setting.second.Scalar(); + } + } + if (camera.backend.empty()) + { + fail(path, settings.Mark(), "camera '" + camera.id + "' needs a backend, as in `backend: synthetic`"); + } + return camera; +} + +} // namespace + +PluginConfig load_config_file(const std::string& path, std::string_view backend) +{ + YAML::Node root; + try + { + root = YAML::LoadFile(path); + } + catch (const YAML::BadFile&) + { + throw std::invalid_argument(path + ": the configuration file cannot be read"); + } + catch (const YAML::ParserException& error) + { + fail(path, error.mark, "not valid YAML: " + error.msg); + } + if (!root.IsMap()) + { + fail(path, root.Mark(), "a configuration starts with `cameras:`"); + } + + PluginConfig config; + std::vector cameras; // of every backend + for (const auto& entry : root) + { + const std::string key = entry.first.IsScalar() ? entry.first.Scalar() : ""; + if (key == "cameras") + { + if (!entry.second.IsMap()) + { + fail(path, entry.second.Mark(), "`cameras:` lists cameras by id, each with a backend"); + } + for (const auto& camera : entry.second) + { + cameras.push_back(read_camera(path, camera.first, camera.second)); + } + } + else if (key == "slots") + { + try + { + config.slot_count = entry.second.as(); + } + catch (const YAML::Exception&) + { + fail(path, entry.second.Mark(), "`slots:` takes a whole number"); + } + } + else + { + fail(path, entry.first.Mark(), + "there is no setting '" + key + "'; a configuration has `cameras:` and `slots:`"); + } + } + + std::copy_if(cameras.begin(), cameras.end(), std::back_inserter(config.cameras), + [&](const CameraConfig& camera) { return camera.backend == backend; }); + if (config.cameras.empty() && !cameras.empty()) + { + // Most often a misspelt backend, so the message shows what the file says. + std::string named; + for (const CameraConfig& camera : cameras) + { + named += (named.empty() ? "" : ", ") + camera.id + " (backend: " + camera.backend + ")"; + } + throw std::invalid_argument(path + ": none of its cameras has `backend: " + std::string(backend) + + "`, the backend of this plugin. It names: " + named); + } + return config; +} + +std::string default_config_path() +{ + const char* config_home = std::getenv("XDG_CONFIG_HOME"); + if (config_home != nullptr && config_home[0] != '\0') + { + return std::string(config_home) + "/isaaccapture/cameras.yaml"; + } + const char* home = std::getenv("HOME"); + return std::string(home != nullptr ? home : "") + "/.config/isaaccapture/cameras.yaml"; +} + +} // namespace core::camera diff --git a/src/core/camera/cpp/inc/camera/camera_host.hpp b/src/core/camera/cpp/inc/camera/camera_host.hpp new file mode 100644 index 0000000000..97c4d13797 --- /dev/null +++ b/src/core/camera/cpp/inc/camera/camera_host.hpp @@ -0,0 +1,72 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include "camera_source.hpp" +#include "publisher.hpp" + +#include +#include +#include +#include + +namespace core::camera +{ + +// Slots per stream unless a camera says otherwise: one for the newest frame set, one for the +// camera to write ahead, and six for readers' windows. +inline constexpr std::uint32_t DEFAULT_SLOT_COUNT = 8; + +/** + * @brief The cameras of one plugin process: gives each its pools and serves them all to + * readers on one endpoint. + */ +class CameraHost +{ +public: + /** + * @throws std::system_error if another process serves the endpoint or it cannot be bound. + */ + explicit CameraHost(const std::string& endpoint_path); + + // Stops the cameras and the endpoint. + ~CameraHost(); + + CameraHost(const CameraHost&) = delete; + CameraHost& operator=(const CameraHost&) = delete; + + /** + * @brief Add a camera and allocate its pools. Call before start(). + * @param slot_count For streams the camera described without one. + * @throws std::invalid_argument if the camera's description is not usable or its id is taken. + */ + void add_camera(std::unique_ptr source, std::uint32_t slot_count = DEFAULT_SLOT_COUNT); + + /** + * @brief Start serving readers, then the cameras. When a camera cannot start, everything + * that did is stopped again before its error is passed on, and start() can be retried. + */ + void start(); + + // Stops the cameras, then closes every reader. + void stop(); + + std::size_t camera_count() const; + std::size_t reader_count() const; + + /** + * @brief Log what each camera did since the previous call: frame sets published, and as a + * warning, frame sets it had to drop for lack of a free slot and events a reader did not take. + */ + void log_activity(); + + // For counters and descriptions; publishing goes through each camera's FrameOutput. + const Publisher& publisher() const; + +private: + class Impl; + std::unique_ptr m_impl; +}; + +} // namespace core::camera diff --git a/src/core/camera/cpp/inc/camera/camera_plugin.hpp b/src/core/camera/cpp/inc/camera/camera_plugin.hpp new file mode 100644 index 0000000000..bccd9c5524 --- /dev/null +++ b/src/core/camera/cpp/inc/camera/camera_plugin.hpp @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include "camera_source.hpp" + +namespace core::camera +{ + +/** + * @brief Everything a camera plugin's main() does: reads the arguments, chooses the cameras, + * takes the plugin's endpoint, opens the cameras through `backend` and serves them to readers + * until SIGINT or SIGTERM. `--help` prints the arguments; which cameras and which endpoint + * they give is in this library's README.md, "Running a plugin". + * + * Call it from main(), before another thread exists: it blocks the two signals for the + * calling thread, so that the threads it and the backend start inherit that, and takes them + * itself. + * + * @return The process's exit code: 0 after a clean stop or `--help`, 2 for a command line it + * cannot parse, which is answered with the usage, 1 when the plugin cannot run, which is + * logged. An argument that parses and cannot be honoured, such as an option the backend + * refuses, returns 1. + */ +int run_camera_plugin(int argc, char** argv, const Backend& backend); + +} // namespace core::camera diff --git a/src/core/camera/cpp/inc/camera/camera_source.hpp b/src/core/camera/cpp/inc/camera/camera_source.hpp new file mode 100644 index 0000000000..5cd0f844aa --- /dev/null +++ b/src/core/camera/cpp/inc/camera/camera_source.hpp @@ -0,0 +1,156 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include "types.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace core::camera +{ + +// One camera as the configuration names it. +struct CameraConfig +{ + std::string id; // the group readers ask for + std::string backend; // the backend, and so the plugin, that opens it + std::map options; // the backend's own settings +}; + +// A free slot of one stream, for the next image. +struct WritableImage +{ + std::uint32_t slot = 0; + std::span data; +}; + +/** + * @brief Where a camera's frames go: the pools of its streams and the way to publish into + * them. Safe to call from the camera's capture thread while readers come and go. + * + * A camera gets the slot for an image in one of two ways, and uses one of them throughout: + * - acquire(): the library hands out a free slot and the camera writes the image into it. + * - claim(): an SDK that is given its buffers up front says which one it filled. The camera + * hands the slots' memory to the SDK before capturing (slot_count(), slot_memory()), asks + * to be told when a slot is free again (on_slot_free()), and names slots with claim(). + */ +class FrameOutput +{ +public: + using SlotFreeCallback = std::function; + + virtual ~FrameOutput() = default; + + /** + * @brief A free slot of one stream to write the next image into. + * @return Nothing when every slot is in use: drop the frame, and abandon() what was already + * acquired for it. + * @throws std::logic_error after on_slot_free(): that camera names its slots with claim(). + */ + virtual std::optional acquire(std::size_t stream) = 0; + + // The slots of one stream's pool, and the memory of one of them. The memory stays where it + // is for as long as the output exists, so an SDK can be given it once. + virtual std::uint32_t slot_count(std::size_t stream) const = 0; + virtual std::span slot_memory(std::size_t stream, std::uint32_t slot) = 0; + + /** + * @brief Say that the SDK filled this slot. + * @throws std::invalid_argument if a reader or the newest frame set still holds the slot: + * the SDK was handed a buffer that was not free. + */ + virtual void claim(std::size_t stream, std::uint32_t slot) = 0; + + /** + * @brief Be told when a published slot is held by no reader and is no longer the newest + * frame set's, so that its buffer can go back to the SDK. Called with no lock of the + * library held, on the thread that let go of the slot last: the library's own when a + * reader gave a frame set back or went away, the camera's from inside publish() when a + * newer frame set took its place. Set it before the first claim(). + */ + virtual void on_slot_free(SlotFreeCallback callback) = 0; + + // Gives back a slot from acquire() or claim() that will not be published. + virtual void abandon(std::size_t stream, std::uint32_t slot) = 0; + + /** + * @brief Publish one frame set: images[i] is stream i's, in a slot from acquire() or + * claim(). Publishing says the images are complete, and readers may look at them from + * this moment on: call it once everything that writes them has finished. + * @param timeline capture_ns and sdk_ready_ns, on the host's monotonic clock. + * @return The frame set's sequence number. + */ + virtual std::uint64_t publish(std::span images, const FrameTimeline& timeline) = 0; + + /** + * @brief Say that the camera restarted. Readers see a new generation, and nothing from + * before the restart is handed out any more. Frame sets readers still hold stay theirs, + * and their slots out of use, until they are given back. + */ + virtual void restart() = 0; +}; + +/** + * @brief One configured camera: one or more streams captured together. A plugin's backend + * implements this for its SDK. + */ +class CameraSource +{ +public: + virtual ~CameraSource() = default; + + /** + * @brief What the camera publishes: the group, its streams and the size of a slot. A + * stream's slot_count may be left 0 to take the plugin's pool size. + */ + virtual GroupDescription describe() const = 0; + + /** + * @brief Slots the camera needs for itself, on top of the readers' and the newest frame + * set's: one to write the next image into, more for an SDK that keeps several captures in + * flight. Readers are never given so many slots that the camera is left with fewer. + */ + virtual std::uint32_t backend_reserve() const + { + return 1; + } + + // Start capturing into `output`, on a thread of the source's own. + virtual void start(FrameOutput& output) = 0; + + // Stop capturing. The output is not called once this has returned. Also called for a + // source that did not start, when another camera of the plugin failed to; it then does nothing. + virtual void stop() = 0; +}; + +// One camera SDK as its plugin sees it. A camera plugin has exactly one: the vendor's capture +// code fills this in and hands it to run_camera_plugin(). +struct Backend +{ + // What a camera's configuration names in `backend:`, and the plugin's instance name unless + // it is given another: letters, digits, '-' and '_'. + std::string name; + std::string description; // a phrase without a full stop; --help shows it after the backend's name + + // Opens the cameras the configuration names for this backend, all in one call, and returns + // one source per camera in the same order. What the SDK has open belongs to the backend, + // not to a camera: an SDK that serves several cameras from one object, or keeps state for + // the whole process, opens it once here and lets the sources it returns share it. Throws + // std::invalid_argument for options the backend does not accept. + std::function>(std::span)> open; + + // The cameras of this backend that are plugged in, each ready to be opened. Left empty by + // a backend that has nothing to look for. + std::function()> discover = {}; +}; + +} // namespace core::camera diff --git a/src/core/camera/cpp/inc/camera/config.hpp b/src/core/camera/cpp/inc/camera/config.hpp new file mode 100644 index 0000000000..563a93e1ce --- /dev/null +++ b/src/core/camera/cpp/inc/camera/config.hpp @@ -0,0 +1,49 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include "camera_host.hpp" +#include "camera_source.hpp" + +#include +#include +#include +#include + +namespace core::camera +{ + +// The cameras a plugin is to serve and the size of their pools. +struct PluginConfig +{ + std::vector cameras; + std::uint32_t slot_count = DEFAULT_SLOT_COUNT; +}; + +/** + * @brief Read the cameras a configuration file names for one backend. The camera plugins of + * a user share the file, and each camera in it names its backend. A camera takes two lines + * and one per option: + * + * @code{.yaml} + * cameras: + * head: + * backend: synthetic + * width: 1920 # anything but `backend` is an option of that backend + * slots: 8 # optional + * @endcode + * + * The cameras of other backends are left out: they belong to other plugins. + * + * @throws std::invalid_argument naming the file and the line that cannot be used; or, when + * the file names cameras and none of them for `backend`, the file and its cameras with their + * backends, which is how a misspelt backend shows. + */ +PluginConfig load_config_file(const std::string& path, std::string_view backend); + +// Where a plugin looks for its configuration when it is not told: +// $XDG_CONFIG_HOME/isaaccapture/cameras.yaml, or ~/.config/isaaccapture/cameras.yaml. +std::string default_config_path(); + +} // namespace core::camera diff --git a/src/core/camera/cpp/inc/camera/options.hpp b/src/core/camera/cpp/inc/camera/options.hpp new file mode 100644 index 0000000000..c9733b9997 --- /dev/null +++ b/src/core/camera/cpp/inc/camera/options.hpp @@ -0,0 +1,56 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include "camera_source.hpp" + +#include +#include +#include +#include +#include + +namespace core::camera +{ + +/** + * @brief Reads a camera's options for a backend and refuses what the backend does not know, + * so a misspelt option is an error and not a silently ignored setting. + */ +class OptionReader +{ +public: + explicit OptionReader(const CameraConfig& config); + + /** + * @brief The option's value, or `fallback` when it is not set. + * @throws std::invalid_argument if the value is not a number between `minimum` and `maximum`. + */ + std::uint32_t number(std::string_view key, std::uint32_t fallback, std::uint32_t minimum, std::uint32_t maximum); + + // The option's text, or `fallback` when it is not set. + std::string text(std::string_view key, std::string_view fallback); + + /** + * @brief Call once every option was read. + * @throws std::invalid_argument naming an option that was set but never read. + */ + void finish() const; + + // For a backend's own complaints: "camera 'head' (synthetic backend): ". + [[noreturn]] void fail(const std::string& what) const; + +private: + const CameraConfig& m_config; + std::set> m_read; +}; + +/** + * @brief Parse one `--camera` argument of a plugin: ID[:KEY=VALUE[,KEY=VALUE]...]. The camera + * is the plugin's own, so it gets `backend`, the name of the plugin's backend. + * @throws std::invalid_argument if it does not have that form. + */ +CameraConfig parse_camera_argument(std::string_view argument, std::string_view backend); + +} // namespace core::camera diff --git a/src/core/camera/cpp/options.cpp b/src/core/camera/cpp/options.cpp new file mode 100644 index 0000000000..817fe21dbc --- /dev/null +++ b/src/core/camera/cpp/options.cpp @@ -0,0 +1,107 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include "inc/camera/options.hpp" + +#include +#include +#include + +namespace core::camera +{ + +namespace +{ + +constexpr std::string_view CAMERA_ARGUMENT_FORM = "ID[:KEY=VALUE[,KEY=VALUE]...]"; + +} + +OptionReader::OptionReader(const CameraConfig& config) : m_config(config) +{ +} + +std::uint32_t OptionReader::number(std::string_view key, std::uint32_t fallback, std::uint32_t minimum, std::uint32_t maximum) +{ + m_read.emplace(key); + const auto found = m_config.options.find(std::string(key)); + if (found == m_config.options.end()) + { + return fallback; + } + const std::string& value = found->second; + std::uint32_t result = 0; + const auto [end, error] = std::from_chars(value.data(), value.data() + value.size(), result); + if (error != std::errc() || end != value.data() + value.size() || result < minimum || result > maximum) + { + fail("option '" + std::string(key) + "' is '" + value + "'; it takes a whole number from " + + std::to_string(minimum) + " to " + std::to_string(maximum)); + } + return result; +} + +std::string OptionReader::text(std::string_view key, std::string_view fallback) +{ + m_read.emplace(key); + const auto found = m_config.options.find(std::string(key)); + return found == m_config.options.end() ? std::string(fallback) : found->second; +} + +void OptionReader::finish() const +{ + for (const auto& [key, value] : m_config.options) + { + if (!m_read.contains(key)) + { + fail("there is no option '" + key + "'"); + } + } +} + +void OptionReader::fail(const std::string& what) const +{ + throw std::invalid_argument("camera '" + m_config.id + "' (" + m_config.backend + " backend): " + what); +} + +CameraConfig parse_camera_argument(std::string_view argument, std::string_view backend) +{ + const auto fail = [&](const std::string& what) -> void + { + throw std::invalid_argument("--camera=" + std::string(argument) + ": " + what + "; the form is " + + std::string(CAMERA_ARGUMENT_FORM)); + }; + + CameraConfig config; + const std::size_t colon = argument.find(':'); + config.id = argument.substr(0, colon); + config.backend = backend; + if (config.id.empty()) + { + fail("a camera needs an id"); + } + if (config.id.find('=') != std::string::npos) + { + // Options whose id was forgotten would otherwise be served as a camera of that name. + fail("a camera needs an id before its options"); + } + + std::string_view rest = colon == std::string_view::npos ? std::string_view() : argument.substr(colon + 1); + while (!rest.empty()) + { + const std::size_t comma = rest.find(','); + const std::string_view pair = rest.substr(0, comma); + const std::size_t equals = pair.find('='); + if (equals == std::string_view::npos || equals == 0) + { + fail("'" + std::string(pair) + "' is not KEY=VALUE"); + } + if (!config.options.emplace(pair.substr(0, equals), pair.substr(equals + 1)).second) + { + fail("option '" + std::string(pair.substr(0, equals)) + "' is given twice"); + } + rest = comma == std::string_view::npos ? std::string_view() : rest.substr(comma + 1); + } + return config; +} + +} // namespace core::camera diff --git a/src/plugins/CMakeLists.txt b/src/plugins/CMakeLists.txt index c26d9384c2..a716a11046 100644 --- a/src/plugins/CMakeLists.txt +++ b/src/plugins/CMakeLists.txt @@ -26,6 +26,7 @@ add_subdirectory(rebot_devarm_leader) add_subdirectory(manus) add_subdirectory(manus_calibration) add_subdirectory(haptikos) +add_subdirectory(camera_synthetic) if(BUILD_PLUGIN_NOITOM_MOCAP) add_subdirectory(noitom_mocap) diff --git a/src/plugins/camera_synthetic/AGENTS.md b/src/plugins/camera_synthetic/AGENTS.md new file mode 100644 index 0000000000..7e1ec4b81c --- /dev/null +++ b/src/plugins/camera_synthetic/AGENTS.md @@ -0,0 +1,15 @@ + + +# Agent notes — `camera_synthetic` plugin + +**CRITICAL (non-optional):** Before editing this package, complete the mandatory **`AGENTS.md` preflight** in [`../../../AGENTS.md`](../../../AGENTS.md) (read every applicable `AGENTS.md` on your paths, not just this file). + +The rules every camera plugin follows are under "Publishing cameras" in [`../../core/camera/AGENTS.md`](../../core/camera/AGENTS.md). Read them first; what follows is this plugin's own. + +- **This plugin is the template.** A vendor's camera plugin is made by copying it ([Writing a camera plugin](../../core/camera/README.md#writing-a-camera-plugin)), so keep it the smallest complete camera plugin: a backend in `core/`, a `main` of a few lines in `app/`, a `plugin.yaml`. What every camera plugin needs goes into the camera library, not here. +- **No SDK and no library beyond the repository's own, ever.** It is the one camera plugin that is built whenever plugins are (`BUILD_PLUGINS`), the tests of the camera interface run it, and it is the camera plugin for the Python wheel to bundle, where a plugin's binary must not need a shared library the wheel does not ship ([`src/core/AGENTS.md`](../../core/AGENTS.md)). It has no build option of its own for the same reason. +- **The pattern is an interface.** `synthetic_fill` and `synthetic_frame_number` are what `camera_bench --check-synthetic` and the tests check frames against; change the pattern and all of them in one pass. +- **`camera_bench` reads any camera plugin.** It lives here because it checks the synthetic pattern; it links the camera library and this plugin's `core/`, never a camera SDK. diff --git a/src/plugins/camera_synthetic/CMakeLists.txt b/src/plugins/camera_synthetic/CMakeLists.txt new file mode 100644 index 0000000000..49b1bf7799 --- /dev/null +++ b/src/plugins/camera_synthetic/CMakeLists.txt @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# The synthetic camera plugin: cameras that need no hardware, served to readers like any +# vendor's. It needs no SDK, so it has no build option of its own and is built with the plugins. +if(NOT CMAKE_SYSTEM_NAME STREQUAL "Linux") + message(STATUS "Skipping camera_synthetic plugin (Linux only)") + return() +endif() + +# app/ copies the manifest next to the executable, which makes that directory one the plugin +# manager can start the plugin from. +set(CAMERA_SYNTHETIC_PLUGIN_MANIFEST "${CMAKE_CURRENT_SOURCE_DIR}/plugin.yaml") + +add_subdirectory(core) +add_subdirectory(app) +add_subdirectory(tools) + +install(FILES plugin.yaml README.md DESTINATION plugins/camera_synthetic) diff --git a/src/plugins/camera_synthetic/README.md b/src/plugins/camera_synthetic/README.md new file mode 100644 index 0000000000..2b395394f4 --- /dev/null +++ b/src/plugins/camera_synthetic/README.md @@ -0,0 +1,154 @@ + + +# Synthetic camera plugin + +A camera plugin that needs no hardware. Its cameras write frames whose every byte is known +from the frame number, so a reader can check that it received the whole of the frame it was +told about. It is what the tests and the latency bench read, a way to try the camera +interface on any Linux machine, and the plugin a vendor's camera plugin is copied from +([Writing a camera plugin](../../core/camera/README.md#writing-a-camera-plugin)). + +Like every camera plugin it is one process that owns its cameras and serves their frames to +any number of readers in other processes, which read its buffers in place through a +[`CameraReader`](../../core/camera/README.md#reading-cameras). + +```yaml +# ~/.config/isaaccapture/cameras.yaml +cameras: + head: + backend: synthetic +``` + +```bash +camera_synthetic_plugin +``` + +```cpp +CameraReader reader = CameraReader::open("head"); +while (auto frames = reader.next(std::chrono::seconds(1))) +{ + const ImageView left = frames->image(0); +} +``` + +Neither side is told where the other is: the plugin listens on `camera-synthetic.sock` in +this user's endpoint directory, and the reader looks through that directory for the plugin +that publishes `head`. + +## Running it + +**By hand**, from the build tree or from `plugins/camera_synthetic` of an install: + +```bash +./build/src/plugins/camera_synthetic/app/camera_synthetic_plugin --camera=head:width=1920,height=1080 +``` + +**Through the plugin manager**, as an application starts any Isaac Capture plugin. The +plugin's name is `camera_synthetic`; `plugin.yaml` sits next to the executable, in the build +tree and in the install. The handle stops the plugin when it goes away. + +```cpp +core::PluginManager manager({ "/plugins" }); +std::unique_ptr plugin = manager.start("camera_synthetic", "camera", { "--camera=head" }); +``` + +The arguments are those of every camera plugin; +[Running a plugin](../../core/camera/README.md#running-a-plugin) has the whole of it. + +| Argument | | +|---|---| +| `--camera=ID[:KEY=VALUE,...]` | One camera: the id readers ask for and its options. Give it once per camera. | +| `--config=FILE` | The cameras to serve, instead of `~/.config/isaaccapture/cameras.yaml`. The plugin serves the cameras with `backend: synthetic` and leaves the others to their plugins. | +| `--auto` | Serve what is found plugged in, whatever a file says. A synthetic camera is never found, so this plugin then has nothing to serve and stops. | +| `--instance=NAME` | Listen on `camera-NAME.sock` instead of `camera-synthetic.sock`, for a second synthetic plugin beside the first. Give the second its own cameras with `--camera` or `--config`; from the user's file it would serve the same cameras as the first, and a reader that looks for one of them is refused because two plugins publish it. | +| `--endpoint=PATH` | Listen on this socket file. A reader is then told the path. | +| `--slots=N` | Frames kept per stream, 8 by default. | +| `--help` | Print the arguments. | + +Without `--camera`, `--config` and a file at the default path, the plugin has no camera and +stops with an error that says so. + +## Options of a synthetic camera + +| Option | Default | Takes | +|---|---|---| +| `width` | 1280 | 16 to 8192 | +| `height` | 720 | 16 to 8192 | +| `fps` | 60 | 1 to 1000 | +| `streams` | 2 | 1 to 8, captured together as one frame set | +| `format` | `bgra8` | `bgra8`, `rgba8`, `gray8`, `nv12` | + +The streams of a camera `head` are `head/left` and `head/right` when there are two, and +`head/0`, `head/1`, ... otherwise. Frame n of stream s starts with n as eight little-endian +bytes; every other byte of the image is `(n + 17 * s) & 0xff` +(`core/inc/camera_synthetic/synthetic_backend.hpp`). An option the plugin does not know is an +error. + +## Measuring + +Every frame set carries the times it passed each step, on the host's monotonic clock. +`camera_bench` reads a group like any other reader, from whichever camera plugin publishes +it, and prints them: + +```text +$ camera_bench --group=head --frames=600 +head: 2 stream(s) + head/left 1280x720 bgra8, 60.00 fps, 8 slots of 3686400 bytes + head/right 1280x720 bgra8, 60.00 fps, 8 slots of 3686400 bytes +read 600 frame sets in 10.00 s (60.0 per second), skipped 0 + p50 p90 p99 max (microseconds) + capture -> sdk ready ... + sdk ready -> published ... + published -> sent ... + sent -> reader ... + capture -> reader ... +``` + +`published -> sent` and `sent -> reader` are what the plugin and the process boundary add; the +steps before them belong to the camera and its SDK. + +Without `--group` the bench lists what this user's camera plugins publish, and the endpoint +of the plugin that publishes each group: + +```text +$ camera_bench +head: 2 stream(s) on /run/user/1000/isaaccapture/camera-synthetic.sock + head/left 1280x720 bgra8, 60.00 fps, 8 slots of 3686400 bytes + head/right 1280x720 bgra8, 60.00 fps, 8 slots of 3686400 bytes +``` + +The bench finds the plugin as a reader does. `--instance=NAME` or `--endpoint=PATH` reads +from that one plugin instead. `--window` and `--order` read as a recorder would, and +`--check-synthetic` checks every byte of every image against the synthetic pattern. + +## Layout + +- `core/`: the synthetic capture code, the plugin's backend. A static library, so that the + bench and the tests check frames against the same pattern. +- `app/`: the `camera_synthetic_plugin` executable: a `main` that hands the backend to + `run_camera_plugin`. +- `tools/`: the `camera_bench` executable. +- `plugin.yaml`: the manifest the plugin manager reads. + +Linux only, and built whenever plugins are (`BUILD_PLUGINS`, on by default): it needs no +SDK, so it has no build option of its own. It is installed to `plugins/camera_synthetic` +with `camera_bench`, `plugin.yaml` and this file. + +## Tests + +`tests/cpp/plugins/camera_synthetic`, run with `ctest -L camera_synthetic`. + +- **Unit** (`ctest -L unit`): the synthetic backend's options and layouts; two synthetic + cameras read through one host, every byte checked. +- **Integration** (`ctest -L integration`): `camera_synthetic_plugin` and `camera_bench` as + processes of their own. A reader in the test process reads from the plugin and checks every + byte; a second plugin cannot take a running one's endpoint; `SIGTERM` is a clean stop; a + plugin that was killed is replaced by the next; a plugin without arguments serves its + cameras of the user's configuration file on `camera-synthetic.sock`, where a reader that + was told nothing finds it; a second instance runs beside it and both are found by group; a + file whose cameras are all another backend's stops the plugin; the plugin manager starts + and stops the plugin; the bench finds a camera, reports each step and lists what is + published and where. diff --git a/src/plugins/camera_synthetic/app/CMakeLists.txt b/src/plugins/camera_synthetic/app/CMakeLists.txt new file mode 100644 index 0000000000..851bfaccfc --- /dev/null +++ b/src/plugins/camera_synthetic/app/CMakeLists.txt @@ -0,0 +1,17 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +add_executable(camera_synthetic_plugin + main.cpp +) + +target_link_libraries(camera_synthetic_plugin PRIVATE + camera_synthetic::camera_synthetic_core + camera::camera +) + +add_custom_command(TARGET camera_synthetic_plugin POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different "${CAMERA_SYNTHETIC_PLUGIN_MANIFEST}" "$/plugin.yaml" + COMMENT "Putting the camera_synthetic plugin's manifest next to its executable") + +install(TARGETS camera_synthetic_plugin RUNTIME DESTINATION plugins/camera_synthetic) diff --git a/src/plugins/camera_synthetic/app/main.cpp b/src/plugins/camera_synthetic/app/main.cpp new file mode 100644 index 0000000000..9e601c9c36 --- /dev/null +++ b/src/plugins/camera_synthetic/app/main.cpp @@ -0,0 +1,12 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include + +// A camera plugin is its backend and nothing else: the arguments, the configuration, the +// endpoint and the serving are the camera library's. +int main(int argc, char** argv) +{ + return core::camera::run_camera_plugin(argc, argv, plugins::camera_synthetic::synthetic_backend()); +} diff --git a/src/plugins/camera_synthetic/core/CMakeLists.txt b/src/plugins/camera_synthetic/core/CMakeLists.txt new file mode 100644 index 0000000000..0133f63ecd --- /dev/null +++ b/src/plugins/camera_synthetic/core/CMakeLists.txt @@ -0,0 +1,25 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +find_package(Threads REQUIRED) + +# The plugin's capture code: a camera that needs no hardware, frames with known content at a +# set rate. Kept apart from the executable so that the bench and the tests can check a frame +# against the same pattern. +add_library(camera_synthetic_core STATIC + synthetic_backend.cpp + inc/camera_synthetic/synthetic_backend.hpp +) + +target_include_directories(camera_synthetic_core INTERFACE + $ +) + +target_link_libraries(camera_synthetic_core + PUBLIC + camera::camera + PRIVATE + Threads::Threads +) + +add_library(camera_synthetic::camera_synthetic_core ALIAS camera_synthetic_core) diff --git a/src/plugins/camera_synthetic/core/inc/camera_synthetic/synthetic_backend.hpp b/src/plugins/camera_synthetic/core/inc/camera_synthetic/synthetic_backend.hpp new file mode 100644 index 0000000000..74b8219d4b --- /dev/null +++ b/src/plugins/camera_synthetic/core/inc/camera_synthetic/synthetic_backend.hpp @@ -0,0 +1,34 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#pragma once + +#include + +#include +#include +#include + +namespace plugins::camera_synthetic +{ + +/** + * @brief A camera that needs no hardware. Options: `width`, `height`, `fps`, `streams` and + * `format` (bgra8, rgba8, gray8 or nv12). + * + * Frame n of stream s starts with n as eight little-endian bytes; every other byte of the + * image is synthetic_fill(n, s). A reader can therefore check that it got the whole of the + * frame it was told about. + */ +core::camera::Backend synthetic_backend(); + +// The value of every byte of a synthetic image after its first eight. +constexpr std::byte synthetic_fill(std::uint64_t frame, std::size_t stream) +{ + return static_cast((frame + 17 * stream) & 0xff); +} + +// The frame number a synthetic image starts with. +std::uint64_t synthetic_frame_number(std::span image); + +} // namespace plugins::camera_synthetic diff --git a/src/plugins/camera_synthetic/core/synthetic_backend.cpp b/src/plugins/camera_synthetic/core/synthetic_backend.cpp new file mode 100644 index 0000000000..17be107d5f --- /dev/null +++ b/src/plugins/camera_synthetic/core/synthetic_backend.cpp @@ -0,0 +1,219 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include "inc/camera_synthetic/synthetic_backend.hpp" + +#include + +#include +#include +#include +#include +#include +#include + +namespace plugins::camera_synthetic +{ + +using core::camera::Backend; +using core::camera::CameraConfig; +using core::camera::CameraSource; +using core::camera::FrameOutput; +using core::camera::FrameTimeline; +using core::camera::GroupDescription; +using core::camera::ImageInfo; +using core::camera::OptionReader; +using core::camera::PixelFormat; +using core::camera::PlaneSize; +using core::camera::StreamDescription; +using core::camera::WritableImage; + +namespace +{ + +std::int64_t now_ns() +{ + return std::chrono::duration_cast(std::chrono::steady_clock::now().time_since_epoch()).count(); +} + +class SyntheticSource final : public CameraSource +{ +public: + explicit SyntheticSource(const CameraConfig& config) + { + OptionReader options(config); + const std::uint32_t width = options.number("width", 1280, 16, 8192); + const std::uint32_t height = options.number("height", 720, 16, 8192); + const std::uint32_t fps = options.number("fps", 60, 1, 1000); + const std::uint32_t streams = options.number("streams", 2, 1, core::camera::MAX_STREAMS_PER_GROUP); + const std::string format_name = options.text("format", "bgra8"); + options.finish(); + + PixelFormat format = PixelFormat::BGRA8; + if (format_name == "rgba8") + { + format = PixelFormat::RGBA8; + } + else if (format_name == "gray8") + { + format = PixelFormat::GRAY8; + } + else if (format_name == "nv12") + { + format = PixelFormat::NV12; + } + else if (format_name != "bgra8") + { + options.fail("option 'format' is '" + format_name + "'; it takes bgra8, rgba8, gray8 or nv12"); + } + + m_period = std::chrono::nanoseconds(1'000'000'000 / fps); + m_group.id = config.id; + for (std::uint32_t i = 0; i < streams; ++i) + { + StreamDescription stream; + // A pair reads better as left and right than as 0 and 1. + stream.id = config.id + "/" + (streams == 2 ? (i == 0 ? "left" : "right") : std::to_string(i)); + stream.pixel_format = format; + stream.width = width; + stream.height = height; + stream.plane_count = core::camera::plane_count_of(format); + std::uint64_t offset = 0; + for (std::uint32_t plane = 0; plane < stream.plane_count; ++plane) + { + const PlaneSize size = core::camera::plane_size_of(format, width, height, plane); + stream.planes[plane] = { static_cast(offset), static_cast(size.row_bytes) }; + offset += size.row_bytes * size.rows; + } + stream.slot_bytes = offset; + stream.frame_rate = fps; + m_group.streams.push_back(stream); + } + } + + ~SyntheticSource() override + { + stop(); + } + + GroupDescription describe() const override + { + return m_group; + } + + void start(FrameOutput& output) override + { + if (m_thread.joinable()) + { + return; + } + m_stopping = false; + m_thread = std::thread([this, &output] { run(output); }); + } + + void stop() override + { + if (!m_thread.joinable()) + { + return; + } + m_stopping = true; + m_thread.join(); + } + +private: + void run(FrameOutput& output) + { + std::vector images(m_group.streams.size()); + auto tick = std::chrono::steady_clock::now(); + for (std::uint64_t frame = 0;; ++frame) + { + if (!sleep_until(tick)) + { + return; + } + FrameTimeline timeline; + timeline.capture_ns = now_ns(); + if (write(output, frame, images)) + { + timeline.sdk_ready_ns = now_ns(); + output.publish(images, timeline); + } + tick += m_period; + // After a stall, carry on from now instead of bursting to catch up. + tick = std::max(tick, std::chrono::steady_clock::now() - m_period); + } + } + + // Sleeps until `tick` in steps short enough for stop() to be prompt at any frame rate. + // False when the camera was stopped meanwhile. A flag and short sleeps, not a condition + // variable: GCC 11's ThreadSanitizer misreads a wait on the steady clock as a double lock. + bool sleep_until(std::chrono::steady_clock::time_point tick) const + { + constexpr std::chrono::milliseconds LONGEST_STEP(50); + while (!m_stopping) + { + const auto now = std::chrono::steady_clock::now(); + if (now >= tick) + { + return true; + } + std::this_thread::sleep_until(std::min(tick, now + LONGEST_STEP)); + } + return false; + } + + // Writes one frame set. False when a stream has no free slot: the frame set is dropped. + bool write(FrameOutput& output, std::uint64_t frame, std::vector& images) + { + for (std::size_t stream = 0; stream < images.size(); ++stream) + { + const std::optional image = output.acquire(stream); + if (!image) + { + for (std::size_t taken = 0; taken < stream; ++taken) + { + output.abandon(taken, images[taken].slot); + } + return false; + } + std::memset(image->data.data(), std::to_integer(synthetic_fill(frame, stream)), image->data.size()); + std::memcpy(image->data.data(), &frame, sizeof(frame)); + images[stream] = {}; + images[stream].slot = image->slot; + images[stream].device_frame_id = frame; + } + return true; + } + + GroupDescription m_group; + std::chrono::nanoseconds m_period{}; + std::thread m_thread; + std::atomic m_stopping{ false }; +}; + +} // namespace + +Backend synthetic_backend() +{ + return { "synthetic", "frames with known content at a set rate; needs no hardware", + [](std::span cameras) + { + // Synthetic cameras have nothing in common, so there is nothing to open once. + std::vector> sources; + for (const CameraConfig& camera : cameras) + { + sources.push_back(std::make_unique(camera)); + } + return sources; + } }; +} + +std::uint64_t synthetic_frame_number(std::span image) +{ + std::uint64_t frame = 0; + std::memcpy(&frame, image.data(), std::min(image.size(), sizeof(frame))); + return frame; +} + +} // namespace plugins::camera_synthetic diff --git a/src/plugins/camera_synthetic/plugin.yaml b/src/plugins/camera_synthetic/plugin.yaml new file mode 100644 index 0000000000..5939bbaf76 --- /dev/null +++ b/src/plugins/camera_synthetic/plugin.yaml @@ -0,0 +1,11 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: camera_synthetic +description: "Synthetic cameras, frames with known content, served to readers in other processes" +command: "./camera_synthetic_plugin" +version: "0.1.0" +devices: + - path: "/camera" + type: "camera" + description: "Synthetic camera frames" diff --git a/src/plugins/camera_synthetic/tools/CMakeLists.txt b/src/plugins/camera_synthetic/tools/CMakeLists.txt new file mode 100644 index 0000000000..03b25fb68e --- /dev/null +++ b/src/plugins/camera_synthetic/tools/CMakeLists.txt @@ -0,0 +1,14 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Reads one group from a running camera plugin and prints where each frame set spent its time. +add_executable(camera_bench + main.cpp +) + +target_link_libraries(camera_bench PRIVATE + camera::camera + camera_synthetic::camera_synthetic_core +) + +install(TARGETS camera_bench RUNTIME DESTINATION plugins/camera_synthetic) diff --git a/src/plugins/camera_synthetic/tools/main.cpp b/src/plugins/camera_synthetic/tools/main.cpp new file mode 100644 index 0000000000..e7a7a0e630 --- /dev/null +++ b/src/plugins/camera_synthetic/tools/main.cpp @@ -0,0 +1,315 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; + +namespace +{ + +constexpr std::string_view USAGE = + "Usage: camera_bench [--group=ID] [--frames=N] [--window=N] [--order=newest|in_order] [--check-synthetic]\n" + " [--endpoint=PATH | --instance=NAME]\n" + "\n" + "Reads one group from a running camera plugin and prints where each frame set spent its time.\n" + "It finds the plugin that publishes the group among this user's camera plugins. Without\n" + "--group it lists what they publish, and on which endpoint.\n" + " --endpoint=PATH read from the plugin on this socket file, instead of looking for one\n" + " --instance=NAME read from the plugin with this instance name, instead of looking for one\n" + " --frames=N frame sets to read (default 300)\n" + " --window=N frame sets held for this reader at a time (default 1)\n" + " --order=... newest skips to the latest frame set, in_order reads every one\n" + " --check-synthetic check every byte of every image against the synthetic backend's pattern\n"; + +// How long this reader waits for a frame set before it gives up on the camera. +constexpr std::chrono::seconds FRAME_TIMEOUT(5); + +std::optional flag_value(std::string_view arg, std::string_view name) +{ + if (arg.starts_with(name) && arg.size() > name.size() && arg[name.size()] == '=') + { + return arg.substr(name.size() + 1); + } + return std::nullopt; +} + +std::uint32_t number_of(std::string_view flag, std::string_view text) +{ + std::uint32_t value = 0; + const auto [end, error] = std::from_chars(text.data(), text.data() + text.size(), value); + if (error != std::errc() || end != text.data() + text.size() || value == 0) + { + throw std::invalid_argument(std::string(flag) + " takes a whole number above zero, not '" + std::string(text) + + "'"); + } + return value; +} + +struct Arguments +{ + PluginLocation plugin; + std::string group; + std::uint32_t frames = 300; + ReaderOptions options; + bool check_synthetic = false; + bool help = false; +}; + +Arguments parse(int argc, char** argv) +{ + Arguments arguments; + for (int i = 1; i < argc; ++i) + { + const std::string_view arg = argv[i]; + if (arg == "--help" || arg == "-h") + { + arguments.help = true; + } + else if (arg == "--check-synthetic") + { + arguments.check_synthetic = true; + } + else if (const auto endpoint = flag_value(arg, "--endpoint")) + { + arguments.plugin.endpoint = *endpoint; + } + else if (const auto instance = flag_value(arg, "--instance")) + { + arguments.plugin.instance = *instance; + } + else if (const auto group = flag_value(arg, "--group")) + { + arguments.group = *group; + } + else if (const auto frames = flag_value(arg, "--frames")) + { + arguments.frames = number_of("--frames", *frames); + } + else if (const auto window = flag_value(arg, "--window")) + { + arguments.options.window = number_of("--window", *window); + } + else if (const auto order = flag_value(arg, "--order")) + { + if (*order != "newest" && *order != "in_order") + { + throw std::invalid_argument("--order takes newest or in_order, not '" + std::string(*order) + "'"); + } + arguments.options.order = *order == "newest" ? DeliveryOrder::NEWEST : DeliveryOrder::IN_ORDER; + } + else + { + throw std::invalid_argument("unknown argument '" + std::string(arg) + "'"); + } + } + if (!arguments.plugin.endpoint.empty() && !arguments.plugin.instance.empty()) + { + throw std::invalid_argument("--endpoint and --instance both say which plugin to read from; give one"); + } + return arguments; +} + +const char* name_of(const StreamDescription& stream) +{ + if (stream.codec == Codec::H264) + { + return "h264"; + } + if (stream.codec == Codec::H265) + { + return "h265"; + } + switch (stream.pixel_format) + { + case PixelFormat::BGRA8: + return "bgra8"; + case PixelFormat::RGBA8: + return "rgba8"; + case PixelFormat::NV12: + return "nv12"; + case PixelFormat::GRAY8: + return "gray8"; + } + return "unknown"; +} + +// `endpoint` is the socket path of the plugin that publishes the group, when it is to be shown. +void print_group(const GroupDescription& group, const std::string& endpoint = {}) +{ + std::printf("%s: %zu stream(s)%s%s\n", group.id.c_str(), group.streams.size(), endpoint.empty() ? "" : " on ", + endpoint.c_str()); + for (const StreamDescription& stream : group.streams) + { + std::printf(" %-24s %ux%u %s, %.2f fps, %u slots of %llu bytes\n", stream.id.c_str(), stream.width, + stream.height, name_of(stream), stream.frame_rate, stream.slot_count, + static_cast(stream.slot_bytes)); + } +} + +std::int64_t now_ns() +{ + return std::chrono::duration_cast(std::chrono::steady_clock::now().time_since_epoch()).count(); +} + +// One step of a frame set's way from the sensor to this reader. +struct Stage +{ + const char* name; + std::vector nanoseconds; + + // Skips a step whose start or end the camera did not report. + void add(std::int64_t from, std::int64_t to) + { + if (from != TIME_UNKNOWN && to != TIME_UNKNOWN) + { + nanoseconds.push_back(to - from); + } + } + + void print() + { + if (nanoseconds.empty()) + { + std::printf(" %-22s not reported by this camera\n", name); + return; + } + std::sort(nanoseconds.begin(), nanoseconds.end()); + const auto at = [&](double fraction) { + return nanoseconds[static_cast(fraction * static_cast(nanoseconds.size() - 1))] / 1000.0; + }; + std::printf(" %-22s %10.1f %10.1f %10.1f %10.1f\n", name, at(0.5), at(0.9), at(0.99), at(1.0)); + } +}; + +// Whether every byte of the image is what the synthetic backend wrote for its frame number. +bool is_synthetic(const ImageView& image, std::size_t stream) +{ + const std::uint64_t frame = plugins::camera_synthetic::synthetic_frame_number(image.data); + const std::byte fill = plugins::camera_synthetic::synthetic_fill(frame, stream); + const auto rest = image.data.subspan(std::min(image.data.size(), sizeof(frame))); + return image.info.device_frame_id == frame && + std::all_of(rest.begin(), rest.end(), [fill](std::byte value) { return value == fill; }); +} + +int run(const Arguments& arguments) +{ + if (arguments.group.empty()) + { + const std::vector listings = CameraReader::list(arguments.plugin); + for (const GroupListing& listing : listings) + { + if (listing.unsupported.empty()) + { + print_group(listing.group, listing.endpoint); + } + else + { + std::printf("%s: on %s, cannot be read by this build: %s\n", listing.group.id.c_str(), + listing.endpoint.c_str(), listing.unsupported.c_str()); + } + } + if (listings.empty()) + { + std::printf("no camera plugin of this user answered with a group\n"); + } + return 0; + } + + CameraReader reader = CameraReader::open(arguments.group, arguments.options, arguments.plugin); + print_group(reader.group()); + + std::array stages = { { { "capture -> sdk ready", {} }, + { "sdk ready -> published", {} }, + { "published -> sent", {} }, + { "sent -> reader", {} }, + { "capture -> reader", {} } } }; + std::uint64_t skipped = 0; + std::uint64_t corrupt = 0; + const std::int64_t started = now_ns(); + for (std::uint32_t i = 0; i < arguments.frames; ++i) + { + const std::optional frames = reader.next(FRAME_TIMEOUT); + const std::int64_t received = now_ns(); + if (!frames) + { + std::fprintf(stderr, "camera_bench: no frame set from '%s' for %lld seconds\n", arguments.group.c_str(), + static_cast(FRAME_TIMEOUT.count())); + return 1; + } + const FrameTimeline& timeline = frames->timeline(); + stages[0].add(timeline.capture_ns, timeline.sdk_ready_ns); + stages[1].add(timeline.sdk_ready_ns, timeline.published_ns); + stages[2].add(timeline.published_ns, timeline.sent_ns); + stages[3].add(timeline.sent_ns, received); + stages[4].add(timeline.capture_ns, received); + skipped += frames->skipped(); + if (arguments.check_synthetic) + { + for (std::size_t stream = 0; stream < frames->image_count(); ++stream) + { + corrupt += is_synthetic(frames->image(stream), stream) ? 0 : 1; + } + } + } + const double seconds = static_cast(now_ns() - started) / 1e9; + + std::printf("read %u frame sets in %.2f s (%.1f per second), skipped %llu\n", arguments.frames, seconds, + arguments.frames / seconds, static_cast(skipped)); + std::printf(" %-22s %10s %10s %10s %10s (microseconds)\n", "", "p50", "p90", "p99", "max"); + for (Stage& stage : stages) + { + stage.print(); + } + if (arguments.check_synthetic) + { + std::printf("synthetic pattern: %llu image(s) did not match\n", static_cast(corrupt)); + } + return corrupt == 0 ? 0 : 1; +} + +} // namespace + +// Everything this tool prints is its result or its usage, for a terminal; it logs nothing. +int main(int argc, char** argv) +{ + Arguments arguments; + try + { + arguments = parse(argc, argv); + } + catch (const std::exception& error) + { + std::cerr << "camera_bench: " << error.what() << "\n\n" << USAGE; + return 2; + } + if (arguments.help) + { + std::cout << USAGE; + return 0; + } + try + { + return run(arguments); + } + catch (const std::exception& error) + { + std::cerr << "camera_bench: " << error.what() << "\n"; + return 1; + } +} diff --git a/tests/cpp/core/camera/CMakeLists.txt b/tests/cpp/core/camera/CMakeLists.txt index 4b7ae03686..a92b486bfe 100644 --- a/tests/cpp/core/camera/CMakeLists.txt +++ b/tests/cpp/core/camera/CMakeLists.txt @@ -6,6 +6,9 @@ cmake_minimum_required(VERSION 3.20) find_package(Threads REQUIRED) add_executable(camera_tests + test_camera_config.cpp + test_camera_host.cpp + test_camera_plugin.cpp test_camera_reader.cpp test_codec.cpp test_cross_process.cpp diff --git a/tests/cpp/core/camera/test_camera_config.cpp b/tests/cpp/core/camera/test_camera_config.cpp new file mode 100644 index 0000000000..aed6ae0ad5 --- /dev/null +++ b/tests/cpp/core/camera/test_camera_config.cpp @@ -0,0 +1,209 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; +using Catch::Matchers::ContainsSubstring; + +namespace +{ + +class TemporaryDirectory +{ +public: + TemporaryDirectory() + { + std::string pattern = (std::filesystem::temp_directory_path() / "camera_config_XXXXXX").string(); + REQUIRE(::mkdtemp(pattern.data()) != nullptr); + m_path = pattern; + } + + ~TemporaryDirectory() + { + std::error_code ignored; + std::filesystem::remove_all(m_path, ignored); + } + + // Writes a configuration file and returns its path. + std::string write(const std::string& text) const + { + const std::string path = (m_path / "cameras.yaml").string(); + std::ofstream(path) << text; + return path; + } + +private: + std::filesystem::path m_path; +}; + +// Sets an environment variable for one test and puts back what was there. +class ScopedEnvironment +{ +public: + ScopedEnvironment(const char* name, const char* value) : m_name(name) + { + if (const char* previous = std::getenv(name)) + { + m_previous = previous; + } + if (value != nullptr) + { + ::setenv(name, value, 1); + } + else + { + ::unsetenv(name); + } + } + + ~ScopedEnvironment() + { + if (m_previous) + { + ::setenv(m_name, m_previous->c_str(), 1); + } + else + { + ::unsetenv(m_name); + } + } + +private: + const char* m_name; + std::optional m_previous; +}; + +} // namespace + +TEST_CASE("a configuration file names cameras, their backends and options", "[unit][camera][config]") +{ + TemporaryDirectory directory; + const PluginConfig config = load_config_file(directory.write("cameras:\n" + " head:\n" + " backend: synthetic\n" + " wrist:\n" + " backend: synthetic\n" + " width: 640\n" + " format: gray8\n" + "slots: 12\n"), + "synthetic"); + REQUIRE(config.slot_count == 12); + REQUIRE(config.cameras.size() == 2); + REQUIRE(config.cameras[0].id == "head"); // in the order of the file + REQUIRE(config.cameras[0].backend == "synthetic"); + REQUIRE(config.cameras[0].options.empty()); + REQUIRE(config.cameras[1].id == "wrist"); + REQUIRE(config.cameras[1].options.size() == 2); + REQUIRE(config.cameras[1].options.at("width") == "640"); + REQUIRE(config.cameras[1].options.at("format") == "gray8"); + + // Without `slots:` the pools have the default size; a file may also name no camera yet. + REQUIRE(load_config_file(directory.write("cameras:\n head:\n backend: synthetic\n"), "synthetic").slot_count == + DEFAULT_SLOT_COUNT); + REQUIRE(load_config_file(directory.write("cameras: {}\n"), "synthetic").cameras.empty()); +} + +TEST_CASE("a plugin takes its backend's cameras from a file it shares with other plugins", "[unit][camera][config]") +{ + TemporaryDirectory directory; + const std::string file = directory.write( + "cameras:\n" + " head:\n" + " backend: rig\n" + " serial: 1234\n" + " left_wrist:\n" + " backend: single\n" + " chest:\n" + " backend: rig\n" + " right_wrist:\n" + " backend: single\n" + "slots: 12\n"); + + const PluginConfig rig = load_config_file(file, "rig"); + REQUIRE(rig.cameras.size() == 2); + REQUIRE(rig.cameras[0].id == "head"); // its own, in the order of the file + REQUIRE(rig.cameras[0].backend == "rig"); + REQUIRE(rig.cameras[0].options.at("serial") == "1234"); + REQUIRE(rig.cameras[1].id == "chest"); + REQUIRE(rig.slot_count == 12); // `slots:` is for every plugin + + const PluginConfig single = load_config_file(file, "single"); + REQUIRE(single.cameras.size() == 2); + REQUIRE(single.cameras[0].id == "left_wrist"); + REQUIRE(single.cameras[1].id == "right_wrist"); +} + +TEST_CASE("a file with cameras and none for the plugin's backend is refused with what it names", "[unit][camera][config]") +{ + TemporaryDirectory directory; + // A misspelt backend is a camera of a plugin that does not exist. Nothing else would say so. + const std::string file = directory.write( + "cameras:\n" + " head:\n" + " backend: sythetic\n" + " wrist:\n" + " backend: rig\n"); + REQUIRE_THROWS_WITH(load_config_file(file, "synthetic"), + ContainsSubstring(file + ": none of its cameras has `backend: synthetic`") && + ContainsSubstring("head (backend: sythetic), wrist (backend: rig)")); + REQUIRE_THROWS_AS(load_config_file(file, "synthetic"), std::invalid_argument); + + // The plugin whose backend the file does name is not troubled by the other camera. + REQUIRE(load_config_file(file, "rig").cameras.size() == 1); +} + +TEST_CASE("a configuration file that cannot be used says where and why", "[unit][camera][config]") +{ + TemporaryDirectory directory; + const auto error_of = [&](const std::string& text) + { + try + { + (void)load_config_file(directory.write(text), "synthetic"); + } + catch (const std::invalid_argument& error) + { + return std::string(error.what()); + } + return std::string("no error"); + }; + + REQUIRE_THAT(error_of("cameras:\n head:\n width: 640\n"), + ContainsSubstring("cameras.yaml:3: camera 'head' needs a backend")); + REQUIRE_THAT(error_of("cameras:\n head: synthetic\n"), ContainsSubstring("camera 'head' needs a backend")); + REQUIRE_THAT(error_of("cameras:\n head:\n backend: synthetic\n modes: [a, b]\n"), + ContainsSubstring("cameras.yaml:4: camera 'head' takes settings of the form `name: value`")); + REQUIRE_THAT(error_of("camera:\n head:\n backend: synthetic\n"), + ContainsSubstring("cameras.yaml:1: there is no setting 'camera'")); + REQUIRE_THAT(error_of("cameras:\n - head\n"), ContainsSubstring("lists cameras by id")); + REQUIRE_THAT( + error_of("cameras: {}\nslots: many\n"), ContainsSubstring("cameras.yaml:2: `slots:` takes a whole number")); + REQUIRE_THAT(error_of("- just\n- a list\n"), ContainsSubstring("starts with `cameras:`")); + REQUIRE_THAT(error_of("cameras:\n head: [unclosed\n"), ContainsSubstring("not valid YAML")); + + REQUIRE_THROWS_WITH(load_config_file("/nonexistent/cameras.yaml", "synthetic"), ContainsSubstring("cannot be read")); +} + +TEST_CASE("the default configuration file is in the user's configuration directory", "[unit][camera][config]") +{ + { + const ScopedEnvironment config_home("XDG_CONFIG_HOME", "/etc/xdg-test"); + REQUIRE(default_config_path() == "/etc/xdg-test/isaaccapture/cameras.yaml"); + } + const ScopedEnvironment config_home("XDG_CONFIG_HOME", nullptr); + const ScopedEnvironment home("HOME", "/home/somebody"); + REQUIRE(default_config_path() == "/home/somebody/.config/isaaccapture/cameras.yaml"); +} diff --git a/tests/cpp/core/camera/test_camera_host.cpp b/tests/cpp/core/camera/test_camera_host.cpp new file mode 100644 index 0000000000..eb0ad672cb --- /dev/null +++ b/tests/cpp/core/camera/test_camera_host.cpp @@ -0,0 +1,498 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; +using namespace std::chrono_literals; +using Catch::Matchers::ContainsSubstring; + +namespace +{ + +constexpr std::uint32_t WIDTH = 64; +constexpr std::uint32_t HEIGHT = 48; + +class TemporaryDirectory +{ +public: + TemporaryDirectory() + { + std::string pattern = (std::filesystem::temp_directory_path() / "camera_host_XXXXXX").string(); + REQUIRE(::mkdtemp(pattern.data()) != nullptr); + m_path = pattern; + } + + ~TemporaryDirectory() + { + std::error_code ignored; + std::filesystem::remove_all(m_path, ignored); + } + + std::string socket_path() const + { + return (m_path / "cameras.sock").string(); + } + +private: + std::filesystem::path m_path; +}; + +// A camera that publishes when the test says so. `control` outlives the source, which the +// host owns. +struct ManualControl +{ + FrameOutput* output = nullptr; // set while the camera runs + std::size_t streams = 0; + + // Publishes one frame set whose every pixel is `value`. False when it had to be dropped. + bool publish(std::uint8_t value) + { + std::vector images(streams); + for (std::size_t stream = 0; stream < streams; ++stream) + { + const std::optional image = output->acquire(stream); + if (!image) + { + for (std::size_t taken = 0; taken < stream; ++taken) + { + output->abandon(taken, images[taken].slot); + } + return false; + } + std::fill(image->data.begin(), image->data.end(), std::byte{ value }); + images[stream].slot = image->slot; + } + output->publish(images, {}); + return true; + } +}; + +class ManualSource final : public CameraSource +{ +public: + ManualSource(std::string id, std::size_t streams, ManualControl& control, std::uint32_t slot_count = 0) + : m_id(std::move(id)), m_slot_count(slot_count), m_control(control) + { + m_control.streams = streams; + } + + GroupDescription describe() const override + { + GroupDescription group; + group.id = m_id; + for (std::size_t i = 0; i < m_control.streams; ++i) + { + StreamDescription stream; + stream.id = m_id + "/" + std::to_string(i); + stream.pixel_format = PixelFormat::GRAY8; + stream.width = WIDTH; + stream.height = HEIGHT; + stream.plane_count = 1; + stream.planes[0].stride_bytes = WIDTH; + stream.slot_bytes = WIDTH * HEIGHT; + stream.slot_count = m_slot_count; + stream.frame_rate = 30.0; + group.streams.push_back(stream); + } + return group; + } + + void start(FrameOutput& output) override + { + m_control.output = &output; + } + + void stop() override + { + m_control.output = nullptr; + } + +private: + std::string m_id; + std::uint32_t m_slot_count; + ManualControl& m_control; +}; + +// A camera whose SDK is given its buffers up front, fills the one it chooses and stops capturing +// when it runs out. `control` plays the SDK and outlives the source, which the host owns. +struct SdkControl +{ + static constexpr std::uint32_t BUFFERS = 6; + static constexpr std::uint32_t RESERVE = 3; // buffers the SDK needs in hand to keep capturing + + FrameOutput* output = nullptr; // set while the camera runs + std::mutex mutex; // a buffer comes back on whichever thread let go of it + std::deque with_sdk; // the buffers the SDK has to fill + std::vector> buffers; // the memory it was given, by slot + + std::size_t in_hand() + { + const std::lock_guard lock(mutex); + return with_sdk.size(); + } + + // The SDK fills the next of its buffers with `value` and the camera publishes it. + void capture(std::uint8_t value) + { + std::uint32_t slot = 0; + { + const std::lock_guard lock(mutex); + slot = with_sdk.front(); + with_sdk.pop_front(); + } + std::fill(buffers[slot].begin(), buffers[slot].end(), std::byte{ value }); + output->claim(0, slot); + ImageInfo image; + image.slot = slot; + output->publish({ &image, 1 }, {}); + } +}; + +class SdkSource final : public CameraSource +{ +public: + explicit SdkSource(SdkControl& control) : m_control(control) + { + } + + GroupDescription describe() const override + { + StreamDescription stream; + stream.id = "sdk/0"; + stream.pixel_format = PixelFormat::GRAY8; + stream.width = WIDTH; + stream.height = HEIGHT; + stream.plane_count = 1; + stream.planes[0].stride_bytes = WIDTH; + stream.slot_bytes = WIDTH * HEIGHT; + stream.slot_count = SdkControl::BUFFERS; + stream.frame_rate = 30.0; + return { .id = "sdk", .streams = { stream } }; + } + + std::uint32_t backend_reserve() const override + { + return SdkControl::RESERVE; + } + + void start(FrameOutput& output) override + { + m_control.output = &output; + output.on_slot_free( + [this](std::size_t, std::uint32_t slot) + { + const std::lock_guard lock(m_control.mutex); + m_control.with_sdk.push_back(slot); + }); + // The SDK is given every slot of the pool as one of its buffers. + for (std::uint32_t slot = 0; slot < output.slot_count(0); ++slot) + { + m_control.buffers.push_back(output.slot_memory(0, slot)); + m_control.with_sdk.push_back(slot); + } + } + + void stop() override + { + m_control.output = nullptr; + } + +private: + SdkControl& m_control; +}; + +bool wait_until(const std::function& condition) +{ + const auto deadline = std::chrono::steady_clock::now() + 2s; + while (!condition()) + { + if (std::chrono::steady_clock::now() > deadline) + { + return false; + } + std::this_thread::sleep_for(1ms); + } + return true; +} + +bool all_pixels_are(const FrameSet& frames, std::uint8_t value) +{ + const auto pixels = frames.image(0).data; + return std::all_of(pixels.begin(), pixels.end(), [value](std::byte pixel) { return pixel == std::byte{ value }; }); +} + +} // namespace + +TEST_CASE("a camera argument names an id and options, and the camera is the plugin's backend's", "[unit][camera]") +{ + const CameraConfig bare = parse_camera_argument("head", "synthetic"); + REQUIRE(bare.id == "head"); + REQUIRE(bare.backend == "synthetic"); + REQUIRE(bare.options.empty()); + + const CameraConfig full = parse_camera_argument("wrist:width=640,height=480,format=gray8", "rig"); + REQUIRE(full.id == "wrist"); + REQUIRE(full.backend == "rig"); + REQUIRE(full.options.size() == 3); + REQUIRE(full.options.at("width") == "640"); + REQUIRE(full.options.at("format") == "gray8"); + + // What follows the id is options, each KEY=VALUE. Options with no id before them are not a + // camera of that name. + for (const char* malformed : { "", ":width=640", "width=640", "width=640,height=480", "head:synthetic", + "head:width", "head:=3", "head:width=1,width=2" }) + { + REQUIRE_THROWS_WITH(parse_camera_argument(malformed, "synthetic"), ContainsSubstring("ID[:KEY=VALUE")); + } +} + +TEST_CASE("options a backend does not read are refused", "[unit][camera]") +{ + const CameraConfig config = parse_camera_argument("head:width=640,serial=1234,widht=7", "rig"); + OptionReader options(config); + REQUIRE(options.number("width", 1280, 16, 8192) == 640); + REQUIRE(options.number("height", 720, 16, 8192) == 720); // not set: the fallback + REQUIRE(options.text("serial", "") == "1234"); + REQUIRE(options.text("mode", "hd720") == "hd720"); + REQUIRE_THROWS_WITH(options.finish(), ContainsSubstring("camera 'head' (rig backend): there is no option 'widht'")); + + REQUIRE_THROWS_WITH(options.number("serial", 0, 0, 100), ContainsSubstring("whole number from 0 to 100")); + const CameraConfig words = parse_camera_argument("head:width=wide,height=-3,fps=60.5", "rig"); + OptionReader strict(words); + REQUIRE_THROWS_AS(strict.number("width", 1, 0, 100), std::invalid_argument); + REQUIRE_THROWS_AS(strict.number("height", 1, 0, 100), std::invalid_argument); + REQUIRE_THROWS_AS(strict.number("fps", 1, 0, 100), std::invalid_argument); +} + +TEST_CASE("a host gives each camera its pools and serves them all on one endpoint", "[unit][camera]") +{ + TemporaryDirectory directory; + ManualControl head; + ManualControl wrist; + CameraHost host(directory.socket_path()); + host.add_camera(std::make_unique("head", 2, head)); + host.add_camera(std::make_unique("wrist", 1, wrist, 4), 16); // the camera's own count wins + REQUIRE(host.camera_count() == 2); + + ManualControl unused; + REQUIRE_THROWS_AS(host.add_camera(std::make_unique("head", 1, unused)), std::invalid_argument); + REQUIRE_THROWS_AS(host.add_camera(std::make_unique("empty", 0, unused)), std::invalid_argument); + + host.start(); + REQUIRE(head.output != nullptr); + REQUIRE_THROWS_AS(host.add_camera(std::make_unique("late", 1, unused)), std::logic_error); + + const std::vector groups = CameraReader::list({ .endpoint = directory.socket_path() }); + REQUIRE(groups.size() == 2); + REQUIRE(groups[0].group.id == "head"); + REQUIRE(groups[0].group.streams[0].slot_count == DEFAULT_SLOT_COUNT); + REQUIRE(groups[1].group.id == "wrist"); + REQUIRE(groups[1].group.streams[0].slot_count == 4); + + CameraReader head_reader = CameraReader::open("head", {}, { .endpoint = directory.socket_path() }); + CameraReader wrist_reader = CameraReader::open("wrist", {}, { .endpoint = directory.socket_path() }); + REQUIRE(host.reader_count() == 2); + + REQUIRE(head.publish(0x21)); + REQUIRE(wrist.publish(0x42)); + const std::optional pair = head_reader.next(2s); + REQUIRE(pair.has_value()); + REQUIRE(pair->image_count() == 2); + REQUIRE(pair->image(0).data.front() == std::byte{ 0x21 }); + REQUIRE(pair->image(1).data.back() == std::byte{ 0x21 }); + const std::optional single = wrist_reader.next(2s); + REQUIRE(single.has_value()); + REQUIRE(single->image(0).data.front() == std::byte{ 0x42 }); + REQUIRE(host.publisher().stats(0).published == 1); + + host.stop(); + REQUIRE(head.output == nullptr); + REQUIRE(host.reader_count() == 0); +} + +TEST_CASE("a host whose camera cannot start is left stopped, and can be started again", "[unit][camera]") +{ + // A camera that fails to start the first time, as one does whose device is not ready yet. + class FailingOnce final : public CameraSource + { + public: + explicit FailingOnce(ManualControl& control) : m_inner("wrist", 1, control) + { + } + + GroupDescription describe() const override + { + return m_inner.describe(); + } + + void start(FrameOutput& output) override + { + if (!m_failed) + { + m_failed = true; + throw std::runtime_error("the camera is not ready"); + } + m_inner.start(output); + } + + void stop() override + { + m_inner.stop(); + } + + private: + ManualSource m_inner; + bool m_failed = false; + }; + + TemporaryDirectory directory; + ManualControl head; + ManualControl wrist; + CameraHost host(directory.socket_path()); + host.add_camera(std::make_unique("head", 1, head)); + host.add_camera(std::make_unique(wrist)); + + // The first camera had started when the second failed. It is stopped again, and no reader + // is served by a host that is only half there. + REQUIRE_THROWS_WITH(host.start(), ContainsSubstring("the camera is not ready")); + REQUIRE(head.output == nullptr); + REQUIRE(wrist.output == nullptr); + REQUIRE_THROWS_AS(CameraReader::list({ .endpoint = directory.socket_path() }), CameraReaderError); + + // Started again, everything runs. + host.start(); + REQUIRE(head.output != nullptr); + REQUIRE(wrist.output != nullptr); + REQUIRE(CameraReader::list({ .endpoint = directory.socket_path() }).size() == 2); + CameraReader reader = CameraReader::open("wrist", {}, { .endpoint = directory.socket_path() }); + REQUIRE(wrist.publish(7)); + REQUIRE(reader.next(2s)->image(0).data.front() == std::byte{ 7 }); +} + +TEST_CASE("a camera drops a frame set when its pool is full, and never overwrites a held one", "[unit][camera]") +{ + TemporaryDirectory directory; + ManualControl camera; + CameraHost host(directory.socket_path()); + host.add_camera(std::make_unique("head", 2, camera), 3); + host.start(); + + // A window of one in a pool of three: the newest, the reader's, and one to write ahead. + CameraReader reader = CameraReader::open("head", {}, { .endpoint = directory.socket_path() }); + REQUIRE(camera.publish(1)); + const std::optional held = reader.next(2s); + REQUIRE(held.has_value()); + for (std::uint8_t value = 2; value < 20; ++value) + { + REQUIRE(camera.publish(value)); // the ring lets go of the frame set before, so there is room + } + REQUIRE(held->image(0).data.front() == std::byte{ 1 }); + REQUIRE(host.publisher().stats(0).no_free_slot == 0); + + // A backend that takes a slot and neither publishes nor abandons it is what fills a pool. + const std::optional forgotten = camera.output->acquire(0); + REQUIRE(forgotten.has_value()); + REQUIRE_FALSE(camera.publish(99)); // stream 0 has no free slot left; the frame set is dropped + REQUIRE(host.publisher().stats(0).no_free_slot == 1); + REQUIRE(held->image(0).data.front() == std::byte{ 1 }); + + camera.output->abandon(0, forgotten->slot); + REQUIRE(camera.publish(100)); +} + +TEST_CASE("a camera whose SDK owns the buffers is never left short of them", "[unit][camera]") +{ + TemporaryDirectory directory; + SdkControl sdk; + CameraHost host(directory.socket_path()); + host.add_camera(std::make_unique(sdk)); + host.start(); + REQUIRE(sdk.in_hand() == SdkControl::BUFFERS); + REQUIRE_THROWS_AS(sdk.output->acquire(0), std::logic_error); // this camera names its slots itself + + // Of six buffers the SDK keeps three. That leaves the newest frame set and two for readers. + REQUIRE_THROWS_WITH(CameraReader::open("sdk", { .window = 3, .order = DeliveryOrder::NEWEST }, + { .endpoint = directory.socket_path() }), + ContainsSubstring("3 for the backend")); + CameraReader reader = CameraReader::open( + "sdk", { .window = 2, .order = DeliveryOrder::NEWEST }, { .endpoint = directory.socket_path() }); + + sdk.capture(1); + std::optional first = reader.next(2s); + REQUIRE(first.has_value()); + sdk.capture(2); + std::optional second = reader.next(2s); + REQUIRE(second.has_value()); + + // The camera runs on while the reader holds both. The SDK finds its reserve in hand before + // every capture, and is never handed a buffer a reader still looks at. + for (std::uint8_t value = 3; value < 40; ++value) + { + REQUIRE(sdk.in_hand() >= SdkControl::RESERVE); + sdk.capture(value); + } + REQUIRE(all_pixels_are(*first, 1)); + REQUIRE(all_pixels_are(*second, 2)); + REQUIRE_THROWS_AS(sdk.output->claim(0, first->image(0).info.slot), std::invalid_argument); + + // A buffer goes back to the SDK when its last holder lets go of it. + const std::size_t before = sdk.in_hand(); + first.reset(); + REQUIRE(wait_until([&] { return sdk.in_hand() == before + 1; })); + + // And when the readers are gone, every buffer but the newest frame set's is the SDK's again. + host.stop(); + REQUIRE(sdk.in_hand() == SdkControl::BUFFERS - 1); + REQUIRE(all_pixels_are(*second, 2)); // still mapped, and as it was +} + +TEST_CASE("a camera that restarts says so, and what a reader holds stays as it was", "[unit][camera]") +{ + TemporaryDirectory directory; + ManualControl camera; + CameraHost host(directory.socket_path()); + host.add_camera(std::make_unique("head", 1, camera), 4); + host.start(); + CameraReader reader = CameraReader::open( + "head", { .window = 2, .order = DeliveryOrder::NEWEST }, { .endpoint = directory.socket_path() }); + + REQUIRE(camera.publish(1)); + const std::optional held = reader.next(2s); + REQUIRE(held.has_value()); + REQUIRE(held->generation() == 0); + + camera.output->restart(); + for (std::uint8_t value = 2; value <= 12; ++value) + { + REQUIRE(camera.publish(value)); + } + REQUIRE(all_pixels_are(*held, 1)); + + const std::optional after = reader.next(2s); + REQUIRE(after.has_value()); + REQUIRE(after->generation() == 1); + REQUIRE(all_pixels_are(*after, 12)); +} diff --git a/tests/cpp/core/camera/test_camera_plugin.cpp b/tests/cpp/core/camera/test_camera_plugin.cpp new file mode 100644 index 0000000000..2a4bbd7d26 --- /dev/null +++ b/tests/cpp/core/camera/test_camera_plugin.cpp @@ -0,0 +1,512 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; +using namespace std::chrono_literals; +using Catch::Matchers::ContainsSubstring; + +namespace +{ + +// Sets an environment variable for one test and puts back what was there. +class ScopedEnvironment +{ +public: + ScopedEnvironment(const char* name, const std::string& value) : m_name(name) + { + if (const char* previous = std::getenv(name)) + { + m_previous = previous; + } + ::setenv(name, value.c_str(), 1); + } + + ~ScopedEnvironment() + { + if (m_previous) + { + ::setenv(m_name, m_previous->c_str(), 1); + } + else + { + ::unsetenv(m_name); + } + } + +private: + const char* m_name; + std::optional m_previous; +}; + +// A user's runtime and configuration directories for one test. A plugin that is not told where +// to listen or which cameras to serve looks there, and so does a reader that looks for the +// plugin; without them a test would meet the developer's own plugins and cameras.yaml. +class UserDirectories +{ +public: + UserDirectories() + { + std::string pattern = (std::filesystem::temp_directory_path() / "camera_run_XXXXXX").string(); + REQUIRE(::mkdtemp(pattern.data()) != nullptr); + m_path = pattern; + std::filesystem::create_directories(m_path / "run"); + std::filesystem::create_directories(m_path / "config" / "isaaccapture"); + m_runtime.emplace("XDG_RUNTIME_DIR", (m_path / "run").string()); + m_config_home.emplace("XDG_CONFIG_HOME", (m_path / "config").string()); + } + + ~UserDirectories() + { + m_config_home.reset(); + m_runtime.reset(); + std::error_code ignored; + std::filesystem::remove_all(m_path, ignored); + } + + UserDirectories(const UserDirectories&) = delete; + UserDirectories& operator=(const UserDirectories&) = delete; + + // The socket path of an endpoint in the user's endpoint directory. + std::string endpoint(const std::string& socket_name) const + { + return (m_path / "run" / "isaaccapture" / socket_name).string(); + } + + // The path of a file of this test. "config/isaaccapture/cameras.yaml" is the user's own. + std::string file(const std::string& name) const + { + return (m_path / name).string(); + } + + // Writes a file and returns its path. + std::string write(const std::string& name, const std::string& text) const + { + std::ofstream(file(name)) << text; + return file(name); + } + +private: + std::filesystem::path m_path; + std::optional m_runtime; + std::optional m_config_home; +}; + +// A camera that describes itself and publishes nothing. These cases are about which cameras a +// plugin serves and where; test_camera_host.cpp has the frames. +class IdleSource final : public CameraSource +{ +public: + explicit IdleSource(std::string id) : m_id(std::move(id)) + { + } + + GroupDescription describe() const override + { + StreamDescription stream; + stream.id = m_id + "/0"; + stream.pixel_format = PixelFormat::GRAY8; + stream.width = 64; + stream.height = 48; + stream.plane_count = 1; + stream.planes[0].stride_bytes = 64; + stream.slot_bytes = 64 * 48; + stream.frame_rate = 30.0; + return { .id = m_id, .streams = { stream } }; + } + + void start(FrameOutput&) override + { + } + + void stop() override + { + } + +private: + std::string m_id; +}; + +// What a stand-in backend was given, one entry per call of its open(). The plugin calls it on +// its thread and the test looks from its own. +class Opened +{ +public: + void add(std::span cameras) + { + const std::lock_guard lock(m_mutex); + m_calls.emplace_back(cameras.begin(), cameras.end()); + } + + std::vector> calls() + { + const std::lock_guard lock(m_mutex); + return m_calls; + } + +private: + std::mutex m_mutex; + std::vector> m_calls; +}; + +// Stands for a vendor's capture code: opens an idle camera for each one it is given. +Backend standin_backend(const std::string& name, Opened& opened) +{ + Backend backend; + backend.name = name; + backend.description = "a stand-in for a vendor's capture code"; + backend.open = [&opened](std::span cameras) + { + opened.add(cameras); + std::vector> sources; + for (const CameraConfig& camera : cameras) + { + sources.push_back(std::make_unique(camera.id)); + } + return sources; + }; + return backend; +} + +// run_camera_plugin() as the main() of a plugin, on a thread of this process. +class PluginRun +{ +public: + PluginRun(Backend backend, const std::vector& arguments, const std::string& program = "camera_rig") + : m_backend(std::move(backend)) + { + m_arguments.push_back(program); + m_arguments.insert(m_arguments.end(), arguments.begin(), arguments.end()); + + // A plugin is stopped with a signal, and here the signal is aimed at the plugin's + // thread. That thread is born with it blocked, so the plugin's own wait is the only + // place that can take it, however early it is sent. + sigset_t stop_signals; + sigemptyset(&stop_signals); + sigaddset(&stop_signals, SIGINT); + sigaddset(&stop_signals, SIGTERM); + sigset_t previous; + pthread_sigmask(SIG_BLOCK, &stop_signals, &previous); + m_thread = std::thread( + [this] + { + std::vector argv; + for (std::string& argument : m_arguments) + { + argv.push_back(argument.data()); + } + argv.push_back(nullptr); + m_exit_code = run_camera_plugin(static_cast(m_arguments.size()), argv.data(), m_backend); + m_ended = true; + }); + pthread_sigmask(SIG_SETMASK, &previous, nullptr); + } + + ~PluginRun() + { + if (m_thread.joinable()) + { + stop(); + } + } + + PluginRun(const PluginRun&) = delete; + PluginRun& operator=(const PluginRun&) = delete; + + // Waits for the plugin to return by itself and gives its exit code: -2 if it still runs + // after `timeout`. + int wait(std::chrono::seconds timeout = 30s) + { + const auto deadline = std::chrono::steady_clock::now() + timeout; + while (!m_ended) + { + if (std::chrono::steady_clock::now() > deadline) + { + return -2; + } + std::this_thread::sleep_for(5ms); + } + m_thread.join(); + return m_exit_code; + } + + // Stops the plugin as the plugin manager stops a process, and gives its exit code. + int stop() + { + if (!m_ended) + { + pthread_kill(m_thread.native_handle(), SIGINT); + } + m_thread.join(); + return m_exit_code; + } + +private: + Backend m_backend; + std::vector m_arguments; + std::thread m_thread; + std::atomic m_ended{ false }; + int m_exit_code = -1; // written by the thread before m_ended, read after it +}; + +// What a reader that looks for cameras finds, once a plugin serves. A plugin that was just +// started needs a moment; one that never serves gives an empty list. +std::vector list_when_served(const PluginLocation& plugin = {}) +{ + const auto deadline = std::chrono::steady_clock::now() + 10s; + while (std::chrono::steady_clock::now() < deadline) + { + try + { + std::vector groups = CameraReader::list(plugin); + if (!groups.empty()) + { + return groups; + } + } + catch (const CameraReaderError&) + { + // The plugin that was named does not listen yet. + } + std::this_thread::sleep_for(10ms); + } + return {}; +} + +struct Ended +{ + int exit_code = -1; + std::string terminal; // what the plugin wrote for the operator: its usage, a refused argument +}; + +// Runs a plugin that is expected to return by itself, and keeps what it wrote to the terminal. +Ended run_to_end(const Backend& backend, + const std::vector& arguments, + const std::string& program = "camera_rig") +{ + std::ostringstream terminal; + std::streambuf* const output = std::cout.rdbuf(terminal.rdbuf()); + std::streambuf* const errors = std::cerr.rdbuf(terminal.rdbuf()); + Ended ended; + { + PluginRun plugin(backend, arguments, program); + ended.exit_code = plugin.wait(); + } + std::cout.rdbuf(output); + std::cerr.rdbuf(errors); + ended.terminal = terminal.str(); + return ended; +} + +} // namespace + +TEST_CASE("a plugin's instance name is its backend's, unless it is given another", "[unit][camera]") +{ + const UserDirectories user; + Opened opened; + { + // Neither the plugin nor the reader is told where the other is. + PluginRun plugin(standin_backend("rig", opened), { "--camera=head", "--plugin-root-id=ignored" }); + const std::vector groups = list_when_served(); + REQUIRE(groups.size() == 1); + REQUIRE(groups[0].group.id == "head"); + REQUIRE(groups[0].endpoint == user.endpoint("camera-rig.sock")); + REQUIRE(CameraReader::list({ .instance = "rig" }).size() == 1); + REQUIRE(plugin.stop() == 0); + } + REQUIRE_FALSE(std::filesystem::exists(user.endpoint("camera-rig.sock"))); + { + // A second plugin of the same backend takes another name. + PluginRun plugin(standin_backend("rig", opened), { "--camera=head", "--instance=bench" }); + const std::vector groups = list_when_served(); + REQUIRE(groups.size() == 1); + REQUIRE(groups[0].endpoint == user.endpoint("camera-bench.sock")); + REQUIRE(plugin.stop() == 0); + } + { + // A socket file that is named outright is the endpoint, wherever it is. + const std::string elsewhere = user.file("elsewhere.sock"); + PluginRun plugin(standin_backend("rig", opened), { "--camera=head", "--endpoint=" + elsewhere }); + const std::vector groups = list_when_served({ .endpoint = elsewhere }); + REQUIRE(groups.size() == 1); + REQUIRE(groups[0].endpoint == elsewhere); + REQUIRE(CameraReader::list().empty()); + REQUIRE(plugin.stop() == 0); + } + + // A camera of the command line is the plugin's own: it carries the backend's name. + const std::vector> calls = opened.calls(); + REQUIRE(calls.size() == 3); + REQUIRE(calls[0].size() == 1); + REQUIRE(calls[0][0].id == "head"); + REQUIRE(calls[0][0].backend == "rig"); +} + +TEST_CASE("a plugin opens its backend once, with the cameras a shared file names for it", "[unit][camera]") +{ + // "rig" stands for an SDK that serves all its cameras from one object, which it opens once. + // The file is also the one of the plugin for "single", whose camera is not this plugin's. + const UserDirectories user; + const std::string file = user.write("rig.yaml", + "cameras:\n" + " left:\n" + " backend: rig\n" + " serial: 17\n" + " top:\n" + " backend: single\n" + " right:\n" + " backend: rig\n" + "slots: 5\n"); + Opened opened; + PluginRun plugin(standin_backend("rig", opened), { "--config=" + file }); + const std::vector groups = list_when_served(); + REQUIRE(groups.size() == 2); + REQUIRE(groups[0].group.id == "left"); + REQUIRE(groups[1].group.id == "right"); + REQUIRE(groups[0].group.streams[0].slot_count == 5); + + const std::vector> calls = opened.calls(); + REQUIRE(calls.size() == 1); + REQUIRE(calls[0].size() == 2); + REQUIRE(calls[0][0].id == "left"); + REQUIRE(calls[0][0].options.at("serial") == "17"); + REQUIRE(calls[0][1].id == "right"); + REQUIRE(plugin.stop() == 0); +} + +TEST_CASE("a plugin that is not told which cameras to serve reads the user's file, or looks for cameras", "[unit][camera]") +{ + const UserDirectories user; + Opened opened; + Backend backend = standin_backend("rig", opened); + backend.discover = [] { + return std::vector{ { "rig-17", "rig", { { "serial", "17" } } }, { "rig-18", "rig", {} } }; + }; + + { + // No file: the cameras the backend finds plugged in, as it described them. + PluginRun plugin(backend, {}); + const std::vector groups = list_when_served(); + REQUIRE(groups.size() == 2); + REQUIRE(groups[0].group.id == "rig-17"); + REQUIRE(groups[1].group.id == "rig-18"); + REQUIRE(opened.calls().back().at(0).options.at("serial") == "17"); + REQUIRE(plugin.stop() == 0); + } + + user.write("config/isaaccapture/cameras.yaml", + "cameras:\n" + " head:\n" + " backend: rig\n" + " depth:\n" + " backend: other\n"); + { + // The user's file says which cameras, and this plugin's are the ones of its backend. + PluginRun plugin(backend, {}); + const std::vector groups = list_when_served(); + REQUIRE(groups.size() == 1); + REQUIRE(groups[0].group.id == "head"); + REQUIRE(plugin.stop() == 0); + } + { + // --auto sets the file aside. + PluginRun plugin(backend, { "--auto" }); + REQUIRE(list_when_served().size() == 2); + REQUIRE(plugin.stop() == 0); + } + + // A backend with nothing to look for has no camera to serve then. + REQUIRE(run_to_end(standin_backend("rig", opened), { "--auto" }).exit_code == 1); +} + +TEST_CASE("a plugin that cannot run ends with an error and serves nothing", "[unit][camera]") +{ + const UserDirectories user; + Opened opened; + const Backend backend = standin_backend("rig", opened); + + // A file whose cameras are all other plugins', and a file that is not there. + const std::string others = user.write("others.yaml", "cameras:\n head:\n backend: rgi\n"); + REQUIRE(run_to_end(backend, { "--config=" + others }).exit_code == 1); + REQUIRE(run_to_end(backend, { "--config=" + user.file("missing.yaml") }).exit_code == 1); + REQUIRE(opened.calls().empty()); + + // A backend that does not return a camera for each one it was given. + Backend losing = backend; + losing.open = [](std::span) { return std::vector>(); }; + REQUIRE(run_to_end(losing, { "--camera=head" }).exit_code == 1); + + // A backend that refuses a camera's options, and a pool too small for any reader. + Backend refusing = backend; + refusing.open = [](std::span cameras) -> std::vector> + { throw std::invalid_argument("camera '" + cameras.front().id + "' (rig backend): there is no option 'colour'"); }; + REQUIRE(run_to_end(refusing, { "--camera=head:colour=red" }).exit_code == 1); + REQUIRE(run_to_end(backend, { "--camera=head", "--slots=1" }).exit_code == 1); + + // An endpoint another plugin has taken: the second plugin finds out before it opens a camera. + PluginRun first(backend, { "--camera=head" }); + REQUIRE(list_when_served().size() == 1); + const std::size_t opens = opened.calls().size(); + REQUIRE(run_to_end(backend, { "--camera=wrist" }).exit_code == 1); + REQUIRE(opened.calls().size() == opens); + const std::vector groups = CameraReader::list(); + REQUIRE(groups.size() == 1); + REQUIRE(groups[0].group.id == "head"); + REQUIRE(first.stop() == 0); +} + +TEST_CASE("a plugin answers arguments it cannot use with its usage, under its own name", "[unit][camera]") +{ + const UserDirectories user; + Opened opened; + const Backend backend = standin_backend("rig", opened); + const auto refused = [&](const std::vector& arguments) + { + const Ended ended = run_to_end(backend, arguments, "/opt/plugins/camera_rig"); + REQUIRE(ended.exit_code == 2); + REQUIRE_THAT(ended.terminal, ContainsSubstring("Usage: camera_rig [")); + return ended.terminal; + }; + + REQUIRE_THAT( + refused({ "--camera=head", "--frobnicate" }), ContainsSubstring("camera_rig: unknown argument '--frobnicate'")); + // What follows a camera's id is options, each KEY=VALUE. + REQUIRE_THAT(refused({ "--camera=head:rig" }), ContainsSubstring("'rig' is not KEY=VALUE")); + REQUIRE_THAT(refused({ "--camera=head", "--endpoint=" + user.endpoint("x.sock"), "--instance=other" }), + ContainsSubstring("both say where to listen")); + REQUIRE_THAT(refused({ "--auto", "--camera=head" }), ContainsSubstring("give one kind")); + REQUIRE_THAT(refused({ "--camera=head", "--slots=many" }), ContainsSubstring("--slots takes a whole number")); + REQUIRE(opened.calls().empty()); + + const Ended help = run_to_end(backend, { "--help" }, "/opt/plugins/camera_rig"); + REQUIRE(help.exit_code == 0); + REQUIRE_THAT(help.terminal, ContainsSubstring("Usage: camera_rig [") && ContainsSubstring("the 'rig' backend") && + ContainsSubstring("a stand-in for a vendor's capture code")); +} diff --git a/tests/cpp/plugins/CMakeLists.txt b/tests/cpp/plugins/CMakeLists.txt index fd7e2d2e68..80ffa13a2c 100644 --- a/tests/cpp/plugins/CMakeLists.txt +++ b/tests/cpp/plugins/CMakeLists.txt @@ -11,6 +11,7 @@ endif() if(CMAKE_SYSTEM_NAME STREQUAL "Linux") add_subdirectory(spacemouse) add_subdirectory(gamepad) + add_subdirectory(camera_synthetic) endif() if(BUILD_PLUGIN_OGLO) diff --git a/tests/cpp/plugins/camera_synthetic/CMakeLists.txt b/tests/cpp/plugins/camera_synthetic/CMakeLists.txt new file mode 100644 index 0000000000..728091baa0 --- /dev/null +++ b/tests/cpp/plugins/camera_synthetic/CMakeLists.txt @@ -0,0 +1,33 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +cmake_minimum_required(VERSION 3.20) + +find_package(Threads REQUIRED) + +add_executable(camera_synthetic_tests + test_camera_bench.cpp + test_camera_synthetic.cpp +) + +target_link_libraries(camera_synthetic_tests PRIVATE + camera_synthetic::camera_synthetic_core + camera::camera + teleop_plugin_manager + Catch2::Catch2WithMain + Threads::Threads +) + +# Some cases start the plugin or the bench as a process of its own. +target_compile_definitions(camera_synthetic_tests PRIVATE + CAMERA_SYNTHETIC_EXECUTABLE="$" + CAMERA_SYNTHETIC_DIRECTORY="$" + CAMERA_BENCH_EXECUTABLE="$" +) +add_dependencies(camera_synthetic_tests camera_synthetic_plugin camera_bench) +# Those paths are into the build tree, so this binary only runs where it was built: keep it +# off the venv PATH of a pip build. +set_target_properties(camera_synthetic_tests PROPERTIES ISAACTELEOP_NEEDS_SOURCE_TREE TRUE) + +message(STATUS "camera_synthetic_tests target enabled with Catch2") +catch_discover_tests(camera_synthetic_tests ADD_TAGS_AS_LABELS) diff --git a/tests/cpp/plugins/camera_synthetic/test_camera_bench.cpp b/tests/cpp/plugins/camera_synthetic/test_camera_bench.cpp new file mode 100644 index 0000000000..ad140de556 --- /dev/null +++ b/tests/cpp/plugins/camera_synthetic/test_camera_bench.cpp @@ -0,0 +1,183 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; +using Catch::Matchers::ContainsSubstring; + +namespace +{ + +class TemporaryDirectory +{ +public: + TemporaryDirectory() + { + std::string pattern = (std::filesystem::temp_directory_path() / "camera_bench_XXXXXX").string(); + REQUIRE(::mkdtemp(pattern.data()) != nullptr); + m_path = pattern; + } + + ~TemporaryDirectory() + { + std::error_code ignored; + std::filesystem::remove_all(m_path, ignored); + } + + std::string file(const std::string& name) const + { + return (m_path / name).string(); + } + +private: + std::filesystem::path m_path; +}; + +// Sets an environment variable for one test, and for the processes it starts, and puts back +// what was there. +class ScopedEnvironment +{ +public: + ScopedEnvironment(const char* name, const std::string& value) : m_name(name) + { + if (const char* previous = std::getenv(name)) + { + m_previous = previous; + } + ::setenv(name, value.c_str(), 1); + } + + ~ScopedEnvironment() + { + if (m_previous) + { + ::setenv(m_name, m_previous->c_str(), 1); + } + else + { + ::unsetenv(m_name); + } + } + +private: + const char* m_name; + std::optional m_previous; +}; + +struct BenchRun +{ + int exit_code = -1; + std::string output; // what the bench printed, results and errors together +}; + +// Runs camera_bench to its end with its output going to a file. +BenchRun run_bench(const TemporaryDirectory& directory, const std::vector& arguments) +{ + const std::string capture = directory.file("bench_output.txt"); + std::vector owned = { CAMERA_BENCH_EXECUTABLE }; + owned.insert(owned.end(), arguments.begin(), arguments.end()); + std::vector argv; + for (std::string& argument : owned) + { + argv.push_back(argument.data()); + } + argv.push_back(nullptr); + + const pid_t pid = ::fork(); + REQUIRE(pid >= 0); + if (pid == 0) + { + const int file = ::open(capture.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0600); + ::dup2(file, STDOUT_FILENO); + ::dup2(file, STDERR_FILENO); + ::execv(argv[0], argv.data()); + ::_exit(127); + } + int status = 0; + REQUIRE(::waitpid(pid, &status, 0) == pid); + + BenchRun result; + result.exit_code = WIFEXITED(status) ? WEXITSTATUS(status) : -1; + std::ifstream stream(capture); + result.output.assign(std::istreambuf_iterator(stream), std::istreambuf_iterator()); + return result; +} + +} // namespace + +TEST_CASE("the bench finds a camera in another process and reports each step", "[integration][camera_synthetic]") +{ + // The camera is served where a synthetic plugin serves it by default, in a user's endpoint + // directory that is this test's own. The bench is not told where that is. + TemporaryDirectory directory; + std::filesystem::create_directory(directory.file("run")); + const ScopedEnvironment runtime("XDG_RUNTIME_DIR", directory.file("run")); + const Backend backend = plugins::camera_synthetic::synthetic_backend(); + const std::string endpoint = default_endpoint_path(backend.name); + prepare_endpoint_directory(endpoint); + const CameraConfig config = parse_camera_argument("head:width=320,height=240,fps=200", backend.name); + CameraHost host(endpoint); + host.add_camera(std::move(backend.open({ &config, 1 }).front())); + host.start(); + + const BenchRun listing = run_bench(directory, {}); + REQUIRE(listing.exit_code == 0); + REQUIRE_THAT(listing.output, ContainsSubstring("head: 2 stream(s) on " + endpoint) && + ContainsSubstring("head/left") && ContainsSubstring("320x240 bgra8, 200.00 fps")); + + const BenchRun run = run_bench(directory, { "--group=head", "--frames=50", "--check-synthetic" }); + REQUIRE(run.exit_code == 0); + REQUIRE_THAT(run.output, ContainsSubstring("read 50 frame sets")); + REQUIRE_THAT(run.output, ContainsSubstring("capture -> sdk ready") && ContainsSubstring("sent -> reader") && + ContainsSubstring("capture -> reader")); + REQUIRE_THAT(run.output, ContainsSubstring("synthetic pattern: 0 image(s) did not match")); + + // The plugin can still be named, by its instance or by its socket file. + const BenchRun in_order = run_bench( + directory, { "--instance=synthetic", "--group=head", "--frames=20", "--window=3", "--order=in_order" }); + REQUIRE(in_order.exit_code == 0); + REQUIRE_THAT(in_order.output, ContainsSubstring("read 20 frame sets")); + REQUIRE(run_bench(directory, { "--endpoint=" + endpoint, "--group=head", "--frames=5" }).exit_code == 0); + + // The plugin's own counters saw the bench's readers come, read and go. + REQUIRE(host.publisher().stats(0).events_sent >= 75); + host.log_activity(); + + const BenchRun unknown = run_bench(directory, { "--group=chest" }); + REQUIRE(unknown.exit_code == 1); + REQUIRE_THAT(unknown.output, ContainsSubstring("publishes group 'chest'")); + const BenchRun not_there = run_bench(directory, { "--endpoint=" + endpoint, "--group=chest" }); + REQUIRE(not_there.exit_code == 1); + REQUIRE_THAT(not_there.output, ContainsSubstring("no group 'chest'")); + + REQUIRE(run_bench(directory, { "--endpoint=" + endpoint, "--instance=other" }).exit_code == 2); + REQUIRE(run_bench(directory, { "--order=backwards" }).exit_code == 2); + REQUIRE(run_bench(directory, { "--help" }).exit_code == 0); + + // With no plugin running there is nothing to list, and a plugin that is named does not answer. + host.stop(); + const BenchRun nothing = run_bench(directory, {}); + REQUIRE(nothing.exit_code == 0); + REQUIRE_THAT(nothing.output, ContainsSubstring("no camera plugin of this user answered with a group")); + const BenchRun nobody = run_bench(directory, { "--endpoint=" + endpoint, "--group=head" }); + REQUIRE(nobody.exit_code == 1); + REQUIRE_THAT(nobody.output, ContainsSubstring("no camera plugin answers on")); +} diff --git a/tests/cpp/plugins/camera_synthetic/test_camera_synthetic.cpp b/tests/cpp/plugins/camera_synthetic/test_camera_synthetic.cpp new file mode 100644 index 0000000000..93a15b8f3e --- /dev/null +++ b/tests/cpp/plugins/camera_synthetic/test_camera_synthetic.cpp @@ -0,0 +1,488 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace core::camera; +using namespace plugins::camera_synthetic; +using namespace std::chrono_literals; +using Catch::Matchers::ContainsSubstring; + +namespace +{ + +class TemporaryDirectory +{ +public: + TemporaryDirectory() + { + std::string pattern = (std::filesystem::temp_directory_path() / "camera_synth_XXXXXX").string(); + REQUIRE(::mkdtemp(pattern.data()) != nullptr); + m_path = pattern; + } + + ~TemporaryDirectory() + { + std::error_code ignored; + std::filesystem::remove_all(m_path, ignored); + } + + std::string socket_path() const + { + return (m_path / "cameras.sock").string(); + } + + const std::filesystem::path& path() const + { + return m_path; + } + +private: + std::filesystem::path m_path; +}; + +// Sets an environment variable for one test, and for the processes it starts, and puts back +// what was there. +class ScopedEnvironment +{ +public: + ScopedEnvironment(const char* name, const std::string& value) : m_name(name) + { + if (const char* previous = std::getenv(name)) + { + m_previous = previous; + } + ::setenv(name, value.c_str(), 1); + } + + ~ScopedEnvironment() + { + if (m_previous) + { + ::setenv(m_name, m_previous->c_str(), 1); + } + else + { + ::unsetenv(m_name); + } + } + +private: + const char* m_name; + std::optional m_previous; +}; + +// A user's runtime and configuration directories for one test and the plugins it starts. A +// plugin that is not told where to listen or which cameras to serve looks there, and so does a +// reader that looks for the plugin; without them a test would meet the developer's own plugins +// and cameras.yaml. +class UserDirectories +{ +public: + UserDirectories() + : m_runtime("XDG_RUNTIME_DIR", (m_directory.path() / "run").string()), + m_config_home("XDG_CONFIG_HOME", (m_directory.path() / "config").string()) + { + std::filesystem::create_directories(m_directory.path() / "run"); + std::filesystem::create_directories(m_directory.path() / "config" / "isaaccapture"); + } + + // The socket path of an endpoint in the user's endpoint directory. + std::string endpoint(const std::string& socket_name) const + { + return (m_directory.path() / "run" / "isaaccapture" / socket_name).string(); + } + + std::string file(const std::string& name) const + { + return (m_directory.path() / name).string(); + } + + // Writes a file and returns its path. + std::string write(const std::string& name, const std::string& text) const + { + std::ofstream(file(name)) << text; + return file(name); + } + + // Writes the user's own configuration file, the one a plugin reads when it is not given one. + void write_config(const std::string& text) const + { + write("config/isaaccapture/cameras.yaml", text); + } + +private: + TemporaryDirectory m_directory; + ScopedEnvironment m_runtime; + ScopedEnvironment m_config_home; +}; + +// One synthetic camera, opened from what follows `--camera=` on the plugin's command line. +std::unique_ptr open_synthetic(const std::string& argument) +{ + const Backend backend = synthetic_backend(); + const CameraConfig config = parse_camera_argument(argument, backend.name); + return std::move(backend.open({ &config, 1 }).front()); +} + +// Every byte of a synthetic image is known from its frame number and its stream. +void require_synthetic(const ImageView& image, std::size_t stream) +{ + const std::uint64_t frame = synthetic_frame_number(image.data); + REQUIRE(image.info.device_frame_id == frame); + const std::byte fill = synthetic_fill(frame, stream); + const auto rest = image.data.subspan(sizeof(frame)); + REQUIRE(std::all_of(rest.begin(), rest.end(), [fill](std::byte value) { return value == fill; })); +} + +// The plugin as a process of its own. +class PluginProcess +{ +public: + explicit PluginProcess(const std::vector& arguments) + { + std::vector owned = { CAMERA_SYNTHETIC_EXECUTABLE }; + owned.insert(owned.end(), arguments.begin(), arguments.end()); + std::vector argv; + for (std::string& argument : owned) + { + argv.push_back(argument.data()); + } + argv.push_back(nullptr); + m_pid = ::fork(); + REQUIRE(m_pid >= 0); + if (m_pid == 0) + { + ::execv(argv[0], argv.data()); + ::_exit(127); + } + } + + ~PluginProcess() + { + if (m_pid > 0) + { + ::kill(m_pid, SIGKILL); + ::waitpid(m_pid, nullptr, 0); + } + } + + // Waits for the process to end and returns its exit code: -1 if a signal ended it, -2 if it + // was still running after `timeout`. + int wait(std::chrono::seconds timeout = 30s) + { + const auto deadline = std::chrono::steady_clock::now() + timeout; + int status = 0; + while (::waitpid(m_pid, &status, WNOHANG) == 0) + { + if (std::chrono::steady_clock::now() > deadline) + { + return -2; + } + std::this_thread::sleep_for(10ms); + } + m_pid = -1; + return WIFEXITED(status) ? WEXITSTATUS(status) : -1; + } + + int stop(int signal) + { + REQUIRE(::kill(m_pid, signal) == 0); + return wait(); + } + +private: + pid_t m_pid = -1; +}; + +// A plugin that was just started needs a moment before it listens. +std::optional open_when_ready(const PluginLocation& plugin, + const std::string& group, + const ReaderOptions& options = {}) +{ + const auto deadline = std::chrono::steady_clock::now() + 10s; + while (std::chrono::steady_clock::now() < deadline) + { + try + { + return CameraReader::open(group, options, plugin); + } + catch (const CameraReaderError&) + { + std::this_thread::sleep_for(10ms); + } + } + return std::nullopt; +} + +// Reads whatever is still on its way; ends by throwing when the plugin's side has closed. +void read_until_closed(CameraReader& reader) +{ + for (int i = 0; i < 1000; ++i) + { + (void)reader.next(2s); + } +} + +} // namespace + +TEST_CASE("the synthetic backend refuses options it does not know", "[unit][camera_synthetic]") +{ + // The name is what a camera's configuration says in `backend:`, and the plugin's instance name. + REQUIRE(synthetic_backend().name == "synthetic"); + + REQUIRE_THROWS_WITH(open_synthetic("head:widht=640"), ContainsSubstring("no option 'widht'")); + REQUIRE_THROWS_WITH(open_synthetic("head:width=wide"), ContainsSubstring("whole number from 16 to 8192")); + REQUIRE_THROWS_WITH(open_synthetic("head:fps=0"), ContainsSubstring("camera 'head' (synthetic backend)")); + REQUIRE_THROWS_WITH(open_synthetic("head:streams=9"), ContainsSubstring("option 'streams'")); + REQUIRE_THROWS_WITH(open_synthetic("head:format=jpeg"), ContainsSubstring("bgra8, rgba8, gray8 or nv12")); + + const GroupDescription pair = open_synthetic("head")->describe(); + REQUIRE(pair.id == "head"); + REQUIRE(pair.streams.size() == 2); + REQUIRE(pair.streams[0].id == "head/left"); + REQUIRE(pair.streams[1].id == "head/right"); + REQUIRE(pair.streams[0].slot_bytes == 1280u * 720u * 4u); + + const GroupDescription nv12 = open_synthetic("rig:streams=3,format=nv12,width=64,height=48")->describe(); + REQUIRE(nv12.streams[2].id == "rig/2"); + REQUIRE(nv12.streams[2].pixel_format == PixelFormat::NV12); + REQUIRE(nv12.streams[2].plane_count == 2); + REQUIRE(nv12.streams[2].planes[1].offset_bytes == 64u * 48u); + REQUIRE(nv12.streams[2].slot_bytes == 64u * 48u * 3u / 2u); +} + +TEST_CASE("synthetic cameras reach readers whole and in step", "[unit][camera_synthetic]") +{ + TemporaryDirectory directory; + CameraHost host(directory.socket_path()); + host.add_camera(open_synthetic("head:width=160,height=120,fps=200")); + host.add_camera(open_synthetic("wrist:width=64,height=48,fps=200,streams=1,format=gray8"), 6); + REQUIRE_THROWS_AS(host.add_camera(open_synthetic("head")), std::invalid_argument); // id taken + REQUIRE(host.camera_count() == 2); + host.start(); + REQUIRE_THROWS_AS(host.add_camera(open_synthetic("late")), std::logic_error); + + const auto groups = CameraReader::list({ .endpoint = directory.socket_path() }); + REQUIRE(groups.size() == 2); + REQUIRE(groups[0].group.streams[0].slot_count == DEFAULT_SLOT_COUNT); + REQUIRE(groups[1].group.streams[0].slot_count == 6); + + // A pair arrives as one frame set: the same frame number in both eyes, each with its own fill. + CameraReader head = CameraReader::open("head", {}, { .endpoint = directory.socket_path() }); + CameraReader wrist = CameraReader::open("wrist", { .window = 2 }, { .endpoint = directory.socket_path() }); + std::uint64_t previous = 0; + for (int i = 0; i < 20; ++i) + { + const std::optional frames = head.next(2s); + REQUIRE(frames.has_value()); + REQUIRE(frames->image_count() == 2); + require_synthetic(frames->image(0), 0); + require_synthetic(frames->image(1), 1); + REQUIRE(frames->image(0).info.device_frame_id == frames->image(1).info.device_frame_id); + REQUIRE(frames->sequence() > previous); + REQUIRE(frames->timeline().capture_ns <= frames->timeline().sdk_ready_ns); + REQUIRE(frames->timeline().sdk_ready_ns <= frames->timeline().published_ns); + previous = frames->sequence(); + } + + // Whatever a reader was not given is accounted for: no frame set goes missing unreported, + // however far the reader fell behind while the loop above ran. + std::optional first = wrist.next(2s); + REQUIRE(first.has_value()); + std::uint64_t last = first->sequence(); + first.reset(); + for (int i = 0; i < 20; ++i) + { + const std::optional frames = wrist.next(2s); + REQUIRE(frames.has_value()); + REQUIRE(frames->sequence() == last + 1 + frames->skipped()); + require_synthetic(frames->image(0), 0); + last = frames->sequence(); + } + REQUIRE(host.reader_count() == 2); + + host.stop(); + // What was already on its way is still delivered; then the reader is told. + REQUIRE_THROWS_AS(read_until_closed(head), CameraDisconnected); +} + +TEST_CASE("a reader in another process gets frames from the plugin", "[integration][camera_synthetic]") +{ + TemporaryDirectory directory; + const std::string endpoint = directory.socket_path(); + PluginProcess plugin( + { "--endpoint=" + endpoint, "--camera=head:width=320,height=240,fps=120", "--plugin-root-id=ignored" }); + + std::optional reader = open_when_ready({ .endpoint = endpoint }, "head"); + REQUIRE(reader.has_value()); + std::uint64_t previous = 0; + for (int i = 0; i < 30; ++i) + { + const std::optional frames = reader->next(2s); + REQUIRE(frames.has_value()); + require_synthetic(frames->image(0), 0); + require_synthetic(frames->image(1), 1); + REQUIRE(frames->sequence() > previous); + previous = frames->sequence(); + } + + // A second plugin cannot take the endpoint of a running one. + PluginProcess second({ "--endpoint=" + endpoint, "--camera=head" }); + REQUIRE(second.wait() == 1); + REQUIRE(reader->next(2s).has_value()); + + // SIGTERM is a clean stop; the reader is told. + REQUIRE(plugin.stop(SIGTERM) == 0); + REQUIRE_THROWS_AS(read_until_closed(*reader), CameraDisconnected); + + // A plugin that was killed leaves a socket file behind; the next one replaces it. + { + PluginProcess killed({ "--endpoint=" + endpoint, "--camera=head:width=64,height=48" }); + REQUIRE(open_when_ready({ .endpoint = endpoint }, "head").has_value()); + REQUIRE(killed.stop(SIGKILL) == -1); + } + REQUIRE(std::filesystem::exists(endpoint)); + PluginProcess next({ "--endpoint=" + endpoint, "--camera=head:width=64,height=48" }); + std::optional again = open_when_ready({ .endpoint = endpoint }, "head"); + REQUIRE(again.has_value()); + REQUIRE(again->next(2s).has_value()); +} + +TEST_CASE("a reader finds the plugin that serves its camera, and a second instance beside it", + "[integration][camera_synthetic]") +{ + // Neither side is told where the other is: the plugin has no --endpoint and the reader no + // location. Both look in this user's directories, which are this test's own. + const UserDirectories user; + user.write_config( + "cameras:\n" + " head:\n" + " backend: synthetic\n" + " width: 160\n" + " height: 120\n" + " fps: 120\n" + " depth:\n" // another plugin's camera, in the file all of them share + " backend: other\n"); + + PluginProcess plugin({}); + std::optional reader = open_when_ready({}, "head"); + REQUIRE(reader.has_value()); + const std::optional frames = reader->next(2s); + REQUIRE(frames.has_value()); + REQUIRE(frames->image(0).stream->width == 160); + require_synthetic(frames->image(0), 0); + // The plugin's instance name is its backend's, and it serves that backend's cameras only. + REQUIRE(std::filesystem::is_socket(user.endpoint("camera-synthetic.sock"))); + REQUIRE(CameraReader::list().size() == 1); + + // A second synthetic plugin runs beside it under another instance name, with cameras of its + // own. A reader finds each camera by its group, whichever plugin has it. + PluginProcess second({ "--instance=bench", "--camera=wrist:width=64,height=48,streams=1" }); + std::optional wrist = open_when_ready({}, "wrist"); + REQUIRE(wrist.has_value()); + REQUIRE(wrist->next(2s).has_value()); + REQUIRE(CameraReader::open("head").next(2s).has_value()); + const std::vector groups = CameraReader::list(); + REQUIRE(groups.size() == 2); + REQUIRE(groups[0].group.id == "wrist"); // in the order of the socket paths + REQUIRE(groups[0].endpoint == user.endpoint("camera-bench.sock")); + REQUIRE(groups[1].group.id == "head"); + REQUIRE(groups[1].endpoint == user.endpoint("camera-synthetic.sock")); + + // A reader that names an instance reads from that plugin and no other. + REQUIRE(CameraReader::open("wrist", {}, { .instance = "bench" }).next(2s).has_value()); + REQUIRE_THROWS_WITH( + CameraReader::open("wrist", {}, { .instance = "synthetic" }), ContainsSubstring("no group 'wrist'")); + REQUIRE_THROWS_WITH( + CameraReader::open("head", {}, { .instance = "nobody" }), ContainsSubstring("camera-nobody.sock")); +} + +TEST_CASE("the plugin takes its cameras from a file that is named, or says why it cannot", + "[integration][camera_synthetic]") +{ + const UserDirectories user; + + // Another file than the user's can be named. + const std::string other = + user.write("other.yaml", "cameras:\n chest:\n backend: synthetic\n width: 64\n height: 48\n"); + PluginProcess plugin({ "--config=" + other }); + REQUIRE(open_when_ready({}, "chest").has_value()); + + // A file whose cameras are all another backend's has nothing for this plugin: the camera + // is misspelt, or it is another plugin's and this one was started by mistake. + const std::string theirs = user.write("theirs.yaml", "cameras:\n head:\n backend: other\n"); + REQUIRE(PluginProcess({ "--instance=second", "--config=" + theirs }).wait() == 1); + REQUIRE(PluginProcess({ "--instance=third", "--config=" + user.file("missing.yaml") }).wait() == 1); + + // Without a file the plugin looks for cameras, and the synthetic backend has none to find. + REQUIRE(PluginProcess({ "--instance=fourth" }).wait() == 1); + // --auto sets the user's file aside, so it is the same with a camera in that file. + user.write_config("cameras:\n head:\n backend: synthetic\n"); + REQUIRE(PluginProcess({ "--instance=fifth", "--auto" }).wait() == 1); +} + +TEST_CASE("the plugin manager can start the camera_synthetic plugin", "[integration][camera_synthetic]") +{ + TemporaryDirectory directory; + const std::string endpoint = directory.socket_path(); + + // A plugin directory is a manifest next to its executable; the build tree has one. + core::PluginManager manager({ std::filesystem::path(CAMERA_SYNTHETIC_DIRECTORY).parent_path().string() }); + const std::vector names = manager.get_plugin_names(); + REQUIRE(std::find(names.begin(), names.end(), "camera_synthetic") != names.end()); + { + const std::unique_ptr plugin = manager.start( + "camera_synthetic", "camera_test", { "--endpoint=" + endpoint, "--camera=head:width=64,height=48" }); + std::optional reader = open_when_ready({ .endpoint = endpoint }, "head"); + REQUIRE(reader.has_value()); + REQUIRE(reader->next(2s).has_value()); + REQUIRE_NOTHROW(plugin->check_health()); + } + // The handle going away stops the plugin, which removes its socket file on the way out. + REQUIRE_FALSE(std::filesystem::exists(endpoint)); +} + +TEST_CASE("the plugin refuses arguments it cannot use", "[integration][camera_synthetic]") +{ + // The --help plugin is given no endpoint. Should it start after all, it does so in this + // test's directories and not in the developer's. + const UserDirectories user; + const std::string endpoint = "--endpoint=" + user.file("cameras.sock"); + REQUIRE(PluginProcess({ "--help" }).wait() == 0); + REQUIRE(PluginProcess({ endpoint, "--instance=other" }).wait() == 2); + REQUIRE(PluginProcess({ endpoint, "--auto", "--camera=head" }).wait() == 2); + REQUIRE(PluginProcess({ endpoint, "--camera=head", "--frobnicate" }).wait() == 2); + // What follows a camera's id is options, each KEY=VALUE. + REQUIRE(PluginProcess({ endpoint, "--camera=head:synthetic" }).wait() == 2); + REQUIRE(PluginProcess({ endpoint, "--camera=" }).wait() == 2); + // Arguments that parse but cannot be honoured end the plugin with an error in its log. + REQUIRE(PluginProcess({ endpoint, "--camera=head:colour=red" }).wait() == 1); + REQUIRE(PluginProcess({ endpoint, "--camera=head", "--slots=1" }).wait() == 1); +}