Skip to content

feat(arty): add worker initialization hooks and Tokio compatibility - #984

Open
martintmk wants to merge 3 commits into
mainfrom
martintmk-worker-initialization-hook
Open

martintmk wants to merge 3 commits into
mainfrom
martintmk-worker-initialization-hook

Conversation

@martintmk

@martintmk martintmk commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Arty needs a worker initialization hook so integrations can configure each worker before tasks run. This change integrates martintmk/oxidizer@c1d1154af2c59357e92edebb7646068e747e4cb2 and adapts it to the current runtime hook API.

Worker initialization API

The new arty::runtime::hook::WorkerInit event exposes thread(&self) -> &Thread. Register it with the existing HookFn and .on(...) APIs:

use arty::runtime::Runtime;
use arty::runtime::hook::{HookFn, WorkerInit};

let runtime = Runtime::builder()
    .on(HookFn::new((), |event: &mut WorkerInit, ()| {
        let thread = event.thread();
        // Initialize this worker.
    }))
    .build()?;
runtime.stop()?;

Each callback runs once per async worker. Builder callbacks finish before startup returns. Runtime control can register callbacks after startup through cx.control().on(...).

The dispatcher stores retained WorkerInit registrations. It drops the retained data when shutdown starts.

Tokio compatibility API

TokioCompatExt replaces arty_tokio::init with these methods:

pub trait TokioCompatExt {
    type Output;

    fn tokio_compat(self) -> Self::Output;
    fn tokio_compat_with(self, tokio: impl Into<TokioHandle>) -> Self::Output;
}

The trait is sealed. A builder method returns RuntimeBuilder. A method on &RuntimeControl returns ().

Use a default owned Tokio runtime:

use arty_tokio::TokioCompatExt as _;

cx.control().tokio_compat();

Supply an explicit Tokio runtime or handle:

use arty_tokio::{TokioCompatExt as _, TokioHandle};

cx.control().tokio_compat_with(TokioHandle::default());
cx.control().tokio_compat_with(tokio.handle());

The same API configures a runtime before startup:

let default_runtime = Runtime::builder()
    .tokio_compat()
    .build()?;

let explicit_runtime = Runtime::builder()
    .tokio_compat_with(tokio.handle())
    .build()?;

TokioHandle accepts an owned Tokio Runtime, an owned Handle, or &Handle. Arty drops an owned Tokio runtime when Arty shuts down. The application owns a runtime supplied through a handle and must keep it alive.

The first initialization wins on each worker. Later calls do nothing on that worker, even with a different Tokio runtime. For a current-thread Tokio runtime, the application must run Runtime::block_on on another thread.

Timers require the Tokio time feature and an enabled time driver. Networking and I/O require the applicable Tokio features and an enabled I/O driver. An explicit Tokio runtime must configure the drivers that its tasks use.

Coverage

Integration tests cover initialization, first-initialization behavior, runtime ownership, nonblocking shutdown, and external runtime destruction. The shutdown regression test checks the typed Arty shutdown error when a Tokio blocking task waits for an Arty task.

Copilot AI balanced review requested due to automatic review settings November 13, 2026 12:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical issues remain: hook ownership cycles leak dispatcher resources, cloning application data under the registry mutex can deadlock, and nested registration can violate callback order across workers.

3 open issues
What changed in this PR

Adds worker initialization hooks to Arty and uses them to configure Tokio support through runtime builders and runtime controls.

Changes:

  • Introduces WorkerInit callbacks and retained registrations for workers that start later.
  • Replaces arty_tokio::init with TokioSupportExt and expands TokioHandle to support owned runtimes.
  • Updates tests, documentation, and examples for initialization, runtime ownership, and shutdown.
File Description
crates/​arty/​tests/​runtime_hooks.rs Adds builder and runtime-control initialization tests.
crates/​arty/​tests/​public_paths.rs Checks public WorkerInit trait implementations.
crates/​arty/​src/​runtime/​worker/​asynchronous/​worker.rs Separates scheduler setup from hook installation.
crates/​arty/​src/​runtime/​worker/​asynchronous/​thread.rs Installs hooks before reporting worker readiness.
crates/​arty/​src/​runtime/​worker/​asynchronous/​local_worker.rs Relocates and executes initialization callbacks.
crates/​arty/​src/​runtime/​worker/​asynchronous/​command.rs Clarifies the hook command’s documentation.
crates/​arty/​src/​runtime/​mod.rs Documents initialization hooks.
crates/​arty/​src/​runtime/​hook/​worker_init.rs Defines the new initialization event.
crates/​arty/​src/​runtime/​hook/​registry.rs Adds initialization-hook registration support.
crates/​arty/​src/​runtime/​hook/​mod.rs Exports WorkerInit and documents callback behavior.
crates/​arty/​src/​runtime/​dispatch/​core.rs Retains initialization hooks for later workers.
crates/​arty/​src/​runtime/​dispatch/​client.rs Exposes retained hooks internally.
crates/​arty/​src/​runtime/​control.rs Documents live-worker initialization registration.
crates/​arty/​src/​runtime/​bootstrap/​startup.rs Retains builder hooks after initial activation.
crates/​arty_tokio/​tests/​support/​mod.rs Removes the obsolete runtime-construction helper.
crates/​arty_tokio/​tests/​lifecycle.rs Updates ownership and shutdown tests.
crates/​arty_tokio/​tests/​compatibility.rs Tests the extension API and first-initialization behavior.
crates/​arty_tokio/​src/​lib.rs Updates public APIs, runtime ownership, and documentation.
crates/​arty_tokio/​src/​initialization.rs Removes transactional initialization machinery.
crates/​arty_tokio/​src/​extensions.rs Implements Tokio support through worker hooks.
crates/​arty_tokio/​src/​error.rs Removes obsolete initialization errors.
crates/​arty_tokio/​README.md Regenerates documentation for the new API.
crates/​arty_tokio/​examples/​tokio_library.rs Removes the old library-integration example.
crates/​arty_tokio/​examples/​tokio_drivers.rs Removes the old timer and networking example.
crates/​arty_tokio/​examples/​external_runtime.rs Removes the old external-runtime example.
crates/​arty_tokio/​examples/​arty_tokio_basic.rs Adds a default-runtime extension example.
crates/​arty_tokio/​Cargo.toml Updates dependencies, API metadata, and example registration.
Cargo.lock Removes unused arty_tokio dependency entries.

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 12:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Retained hooks can leak dispatcher resources through ownership cycles, cloning callback data under the registry mutex can deadlock, and nested registrations can violate callback ordering across workers.

4 open issues

Review effort: Balanced

Comment thread crates/arty_tokio/src/lib.rs Fixed
Copilot AI balanced review requested due to automatic review settings November 13, 2026 12:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

request_stop() can deadlock on a single-worker runtime because dropping the owned Tokio runtime may block the stopping worker before Arty task cancellation; runtime release must be nonblocking and covered by a regression test.

1 open issue
4 resolved since last review

Review effort: Balanced

Comment thread crates/arty_tokio/src/extensions.rs Resolved
@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (2be413c) to head (2bd9402).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##            main     #984      +/-   ##
=========================================
  Coverage   99.9%   100.0%              
=========================================
  Files        808      120     -688     
  Lines      68565     7245   -61320     
=========================================
- Hits       68556     7245   -61311     
+ Misses         9        0       -9     
Flag Coverage Δ
linux 100.0% <100.0%> (ø)
linux-arm 100.0% <100.0%> (ø)
scheduled ?
windows 100.0% <100.0%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI balanced review requested due to automatic review settings November 13, 2026 13:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because an unpublished finding flags the new public Tokio Runtime conversion as missing from the required external-type allowlist.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 13:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because an unpublished finding indicates the regression test can pass on unrelated join failures without confirming shutdown cancellation.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 13:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because an unpublished test concern leaves requested-shutdown cancellation indistinguishable from other join failures.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 13:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The shutdown regression test does not verify the typed shutdown cause, and the compatibility documentation omits required Tokio features and enabled runtime drivers.

2 open issues

Review effort: Balanced

Comment thread crates/arty_tokio/tests/lifecycle.rs Resolved
Comment thread crates/arty_tokio/src/lib.rs Resolved
Copilot AI balanced review requested due to automatic review settings November 13, 2026 14:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Persisting Tokio context via a forgotten EnterGuard and introducing nonblocking cross-runtime shutdown require human assessment and runtime validation, which was not performed in this review.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Comment thread crates/arty_tokio/Cargo.toml Resolved
Copilot AI balanced review requested due to automatic review settings November 13, 2026 15:07
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@martintmk
martintmk force-pushed the martintmk-worker-initialization-hook branch from 6758602 to a1ab14b Compare November 13, 2026 15:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the worker-initialization hooks and Tokio compatibility API changes.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 13, 2026 15:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Nested WorkerInit registration still reverses callback order on remote workers, violating the promised registration order and requiring ordered publication plus a two-worker regression test.

1 open issue

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 13, 2026 15:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Retaining Tokio context until thread exit and shutting down owned runtimes in the background require human review of lifecycle test results unavailable in this dependency-free snapshot.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Comment thread crates/arty_tokio/src/extensions.rs Resolved
@psandana

Pato Sandaña (psandana) commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Pato's Pull Request Agent: The Fast Checks are failing on all four targets because the Anvil Clippy group failed. The workflow is still in progress, so its failure logs are not yet available. Please inspect the Clippy diagnostics, fix any PR-introduced warnings, and rerun the checks before merge.

Comment thread crates/arty_tokio/src/extensions.rs Resolved
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 13, 2026 15:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Retaining Tokio context via a forgotten EnterGuard and thread-local teardown, alongside background runtime shutdown, requires human review of lifetime assumptions and runtime validation that was unavailable.

No open issues 🎉

Review effort: Balanced

data.0.fetch_add(1, Ordering::SeqCst);
}));
for worker in workers {
scheduler.spawn_at(&worker, (), |()| async {}).join().unwrap();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI assistant here! Automated review comment from an AI agent -- not a human reviewer. Please verify before acting.

AI reviewer: GPT-6 Astra

Hook registrations use the system command queue, but these tasks use a separate spawn queue. A worker can finish its system scan, receive both commands, and complete the task before it processes the hook. Then join returns before initialization, so the assertion can see fewer than two calls. Wait for a signal from each callback, and fix the same assumption in arty_tokio/tests/compatibility.rs at lines 118-120 and 171-172.


impl Default for TokioHandle {
fn default() -> Self {
let runtime = Runtime::new().unwrap_or_else(|error| panic!("failed to create the default Tokio runtime: {error}"));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI assistant here! Automated review comment from an AI agent -- not a human reviewer. Please verify before acting.

cargo-evaluate guideline: M-CANONICAL-DOCS

TokioHandle::default panics if Runtime::new fails, for example when a thread cannot start. tokio_compat calls this path, but neither API documents the panic. Document the panic and show how to handle the io::Result from Builder::build before calling TokioHandle::from(runtime). Alternatively, add a fallible constructor such as TokioHandle::new() -> io::Result<Self>.

Reason: TokioHandle has a summary and an example. However, Default::default can panic at line 105 when Tokio runtime creation fails. The docs only say that Default creates and owns a new runtime, and they do not mention the panic. Their # Panics coverage is missing for that applicable case. The crate docs also lack a # Panics section.
Recommendation: Document the panic in a # Panics section, for example on the type docs: # Panics followed by 'Calling TokioHandle::default panics if the Tokio runtime cannot be created.'

cargo-evaluate guideline: M-DESIGN-FOR-AI

Reason: TokioHandle::default creates a Tokio runtime via Runtime::new() and panics on failure (line 105). Failure is signalled by a panic instead of a Result, and callers cannot supply a runtime configuration through that entry point. The docs for Default do not mention the panic. Callers can use From<Runtime> with their own builder, which mitigates the I/O concern.
Recommendation: Add a # Panics section to the Default impl or the type docs, and point to TokioHandle::from(runtime) as the fallible, configurable alternative. Alternatively, add a TokioHandle::new() -> io::Result<Self> constructor.

cargo-evaluate guideline: M-MOCKABLE-SYSCALLS

Reason: TokioHandle::default creates its own runtime through Runtime::new() (line 105). This is the MyIoLibrary::default() pattern the rule discourages. Creating a runtime starts threads and I/O and time drivers. The result depends on the environment and cannot be mocked. Callers can supply their own runtime through From<Runtime> and From<Handle>, which mitigates the problem, but the Default constructor still creates its own core.
Recommendation: Remove the Default impl, or make creation explicit and fallible with TokioHandle::new() -> io::Result<Self>. Document From<Runtime> and From<Handle> as the way to supply a runtime.

cargo-evaluate guideline: M-PANIC-IS-STOP

Reason: TokioHandle::default panics when Runtime::new() fails (line 105). Runtime creation can fail for environmental reasons, such as OS resource exhaustion or thread spawn failure. That is not a programming error, so the panic is used to handle an external error condition. Default cannot return a Result, but a fallible constructor could.
Recommendation: Add a fallible constructor such as TokioHandle::new() -> io::Result<Self>, or have callers build a runtime with Builder and use TokioHandle::from(runtime). Avoid panicking on environmental failures.

@sgalkin Sergey Galkin (sgalkin) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Two nonblocking documentation questions below.

🤖 Authored by Copilot.

where
Self: Sized,
{
self.tokio_compat_with(TokioHandle::default())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should callers coordinate tokio_compat() once per Arty runtime? A second default call leaves initialized workers unchanged but still creates and retains another owned Tokio runtime until shutdown; could we document that setup responsibility explicitly?

Nonblocking.

🤖 Authored by Copilot.

/// For a current-thread runtime, also run
/// [`Runtime::block_on`](tokio::runtime::Runtime::block_on) on another thread
/// while Arty tasks use Tokio. Entering its handle does not drive Tokio
/// tasks, timers, or I/O.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we explicitly state that current-thread runtimes must remain application-owned and be supplied through a handle? Moving one into opaque TokioHandle removes access to the Runtime::block_on operation required to drive it. The driving requirement is already documented; this would make the ownership restriction explicit too.

Nonblocking.

🤖 Authored by Copilot.

@sandersaares

Copy link
Copy Markdown
Member

[Copilot speaking]

Pull request overview (auto-generated).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants