Schema-derived agent tools: generate a bounded tool roster from an ActiveRecord model

[TonsOfFun]

The gap

Agents running in the dashboard only get generic tools — fetch_url, browse_page, web_search, calculate, render_ui, save_memory, browser control, call_agent. None of them models the host application's domain.

So an agent asked "which clients have overdue tickets?" has nothing to call. It answers from the prompt and fabricates the rest.

ActiveAgent::SchemaGenerator already turns a model into JSON Schema — that's the parameter-schema half. This PR adds the missing half: a layer that turns a model plus a declared boundary into callable, enumerable tools.

The DSL

class TicketTools < ActiveAgent::SchemaTools
  model Ticket
  filterable :client, :assignee, :status    # the ONLY columns usable as filters
  returns :id, :subject, :status, :due_date # the ONLY columns returned
  scope { |actor| TicketPolicy::Scope.new(actor, Ticket).resolve }  # optional
end

TicketTools.tool_definitions.map { |d| d[:name] }
# => ["find_tickets", "count_tickets", "get_ticket"]

TicketTools.call("find_tickets", actor: current_user, status: "open")
# => { results: [{ id: 1, subject: "...", status: "open", due_date: "..." }],
#      count: 1, truncated: false }

Definitions come out in the same flat {name:, description:, parameters:} function-calling shape AgentToolbox::DEFINITIONS already uses, so wiring is a drop-in later.

Design properties

1. Allowlist-gated, never open-ended

Only filterable columns may be filtered on; only returns columns are ever read back.

An undeclared column is rejected, not silently dropped. That distinction matters: silently ignoring an unknown filter answers a broader question than the model asked while still looking like a success — which is exactly how a model ends up confidently reporting the unfiltered set as if it were filtered. Rejecting is also what keeps the boundary real; without it a model could filter on users.password_digest one character at a time and read a secret out of the row counts.

The projection is enforced twice — in SQL and again in Ruby. The Ruby side is the one that actually guarantees it: a scope block ending in includes or a raw select can hand back a record carrying more columns than were asked for.

2. Fixed roster via define_method, NOT method_missing

Deliberate, and load-bearing. Every consumer needs to enumerate the roster before any call happens:

• the dashboard UI lists available tools,
• MCP tools/list must answer without being told a name first,
• eval | tools: expectations assert against a known set.

method_missing can answer "do you respond to this?" but cannot answer "what is there?". Tools are built at declaration time and exposed via .tool_definitions / .tool_names / .tool?.

3. Authorization is the host's seam — not implemented here

scope takes a block receiving the caller's actor and returning a relation; generated tools query through it. SchemaTools does not know what an actor is and deliberately does not try to authorize — hosts have Pundit, CanCan, or nothing, and a framework guess would be either wrong or in the way.

Omitting scope runs unscoped. That is a legitimate host choice for a single-tenant or already-trusted context, not a default to fight.

One related decision: get_* uses find_by through the scoped relation, so a record the actor can't see reads as "not found" rather than as a 404-vs-403 signal the model could use to probe for existence.

4. Results are bounded

A find_* with no filters would otherwise select the whole table into a prompt. Default limit 25, hard cap 100, and a truncated flag so the model can tell "these are all of them" from "these are the first 25". An oversized limit is clamped rather than rejected — a model asking for 1000 rows wants as many as it can get, and an error would just make it ask again.

5. Parameter schemas come from SchemaGenerator

Not a local type map. This is why role arrives with its enum populated from User's inclusion validator — hand-rolled type mapping wouldn't know that.

Scope discipline

• Lives in lib/active_agent/ — framework-level and host-agnostic, not in actionagent/.
• Read-only. No create/update/delete tools are generated; a test asserts no tool name contains a write verb.
• No changes to AgentToolbox or any dashboard wiring. Registration is a deliberate follow-up; this PR is self-contained.
• Tested against the dummy app's existing User / Post models — no invented domain models.

Tests

36 new tests, 110 assertions, in test/schema_tools_test.rb, covering the security boundary explicitly: undeclared filter rejected, undeclared return column not leaked, limit enforced, scope block invoked with the actor, absent scope runs unscoped.

Verified the tests actually fail without the implementation. Removing the file is only a LoadError, which proves the file is required but not that each assertion is load-bearing — so each boundary was sabotaged independently in the restored file:

Sabotage	Result
filter allowlist silently ignores undeclared columns	3 failures
projection returns all attributes	3 failures
limit bounding removed	2 failures
scope block ignored (always unscoped)	4 failures
restored	0 failures

Full suite: 1845 → 1881 runs (+36, exactly the new tests), 0 failures. The 253 errors are pre-existing and unchanged — all are Missing credentials ... OPENAI_API_KEY from this local environment having no .env.test, unrelated to this change. actionagent suite: 340 runs, 1780 assertions, 0 failures. RuboCop clean on all three changed files.

For a reviewer to scrutinize

• belongs_to names resolve to foreign keys — filterable :client becomes client_id in the tool signature. Convenient, but it means the model filters by raw id, which it can only know from a prior tool result. Filtering by a human-readable association attribute would be friendlier and is a plausible follow-up.
• Errors are returned as { error: ... } rather than raised, matching the AgentToolbox.call contract, so a model that guesses a column name gets a correction it can act on instead of killing the run. Genuine programming errors still raise.
• inherited copies declarations by value so a subclass can't silently widen its parent's allowlist — worth confirming that's the semantics you want versus forbidding subclassing outright.
• No ordering support on find_*. Combined with a limit, "first 25" is currently whatever the DB returns. Adding a declared-allowlist order is the obvious next increment.

🤖 Generated with Claude Code

https://claude.ai/code/session_01He34kWksjpqPPDpCCqJq2C

[TonsOfFun]

Update — this PR now carries the registration seam too (e12d964c)

The original diff was the DSL only. Wiring it into a real host surfaced that nothing consumed it, so the seam that PROPOSAL_schema_tools.md §"Engine integration" specified is now in this PR as well. This closes #438.

The gap it closes

An agent could list a generated tool name and get nothing:

AgentToolbox.definitions_for(agent.tools)  # => []
AgentToolbox.function?("find_tickets")     # => false

The model then receives no schemas and invents tool names in prose. Verbatim from a real eval run before the fix:

<function_calls> <invoke name="mcp-obsidian-server_search">
<function_calls> <invoke name="search_tickets">

Every scenario scored 0.0 on expected_tools and the report recommended prompt changes. The cause was wiring. Same shape as #425 and #433. After registering, the same suite went 0/8 → 2/8 with real tool_calls recorded and a genuine "expected tool not called" fault.

What was added

Following the mcp_catalog precedent:

ActionAgent.configure do |config|
  config.schema_tools = [TicketTools, TaskTools, MilestoneTools]
end

• Agent.available_tools offers each generated tool beside the built-ins, so they render as individually selectable cards in the editor. AVAILABLE_TOOLS keeps its meaning as the built-in set. Computed per call — a host's tool classes reload in development, so a memoised list is empty at boot or stale after.
• AgentToolbox resolves and dispatches them, passing actor: through untouched including nil. The host's scope decides what an unattributed run may read; the engine never widens it.
• A new agent named after a model starts with that model's tools selected (Reservation → ReservationTools → ReservationAgent). A default, never a restriction: any agent may enable any tool, and an explicit selection — including a deliberate empty one — is never overwritten.

Class names may be strings, resolved lazily, because a host declares these in an initializer that runs before its own autoloading.

One latent bug worth knowing about

The convention deliberately does not reuse telemetry_agent_class. That method runs parameterize.camelize, which turns an already-camelised "TicketAgent" into "Ticketagent" — matching nothing — while "Milestone Agent" happens to survive. The convention normalises both sides to letters instead, so TicketAgent, Ticket Agent and ticket_agent all match.

That bug still exists in telemetry_agent_class itself and is not fixed here, since it is the trace-correlation key and changing it would need its own change and a look at existing rows.

Verified

• 10 new tests in actionagent/test/schema_tools_registration_test.rb
• In a host app: all nine tools render as cards; deselecting one drops SELECTED TOOLS 9 → 8 and marks the agent unsaved
• Host suite green (2495 runs, 0 failures) against this branch

Caveat: I could not run this repo's own suite in my worktree — bundle exec rake test fails on cannot load such file -- active_storage/engine, which is pre-existing and unrelated to this diff. CI is green on all 8 checks, which is the real signal.

[TonsOfFun]

Follow-up — 46a04e21 removes the two pieces of host boilerplate (refs #440)

Direct response to the review feedback that the config restates what the framework already knows.

Discovery is the default

config.schema_tools now defaults to nil, meaning discover, and schema_tools_path ("app/agent_tools") is scanned for SchemaTools subclasses. Adding a tool is adding a file. An explicit array still wins; a nil path disables discovery entirely.

Verified in a host app with no config.schema_tools at all — all nine generated tools appear in the editor payload and the naming convention still attaches them.

Files are loaded before reading descendants, because in development nothing has referenced those constants yet: the list would otherwise be empty at boot and fill in only once something happened to touch them.

Anonymous classes are excluded from discovery on purpose. They work when declared explicitly, but a runtime-built class cannot supersede itself, so discovery would accumulate one per reload. Measured before excluding them: two classes built across two reload cycles, both retained. This is also the constraint on the DB-backed idea in #441.

scope_by_policy

Resolves Reservation → ReservationPolicy::Scope and calls .new(actor, model).resolve:

class ReservationTools < ActiveAgent::SchemaTools
  model Reservation
  filterable :status
  returns :id, :status
  scope_by_policy
end

Verified the boundary is identical to the hand-written block: owner sees all 6 tickets, a partner sees only their own client (client_ids == [13]), a nil actor sees 0.

Opt-in rather than automatic, deliberately — silently scoping a class that declared no scope would change what an existing tool returns, and a host may authorize somewhere other than a Pundit-shaped policy. A missing or mistyped policy raises at declaration rather than quietly reading the whole table.

One honest limitation

scope_by_policy gives you exactly the policy scope. A tool that also wants archived_at: nil still writes the block out — the host app in question kept two of its three classes explicit for that reason. Composing a convention with an extra condition is the obvious next thing, but I did not want to invent a DSL for it without a second real use case.

Verification

• 3 new tests for scope_by_policy, 3 for discovery
• Host suite green: 2495 runs, 0 failures, against this branch
• Same caveat as before: I could not run this repo's suite locally (cannot load such file -- active_storage/engine, pre-existing); CI is the signal.