Skip to content

Settings: define each model provider once, and let every feature pick providers by protocol #147

Description

@harenome

Priority

extreme. The change reaches the relay's key headers and the settings that hold keys.

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • the standard (ports, kits, relay, runs)

Today

Each feature keeps its own list of model providers, and the lists disagree:

Where Providers
app/nullius/providers.py (PROVIDERS, PROVIDER_BASE, PROVIDER_KEY) openrouter, openai, custom
app/static/builder/command-builder-core.js (PROVIDERS) a copy of the above
app/nullius/providers.py (TESTED) the above and anthropic
app/nullius/standard/run.py (PROVIDERS), the CLI's --provider openai, openrouter, anthropic, custom
app/nullius/judge/client.py (LLM_PROVIDERS) openrouter, openai, local, local:<ID>
app/nullius/judge/client.py (SYSTEM_ONE_PROVIDERS) typesafe, openrouter
app/nullius/onboarding.py (PROVIDER_KEYS) OpenRouter, OpenAI, MODEL_API_URL

The effects:

  • Anthropic is in Settings, the provider test, the CLI and the relay, but not in the builder, the LLM judge or onboarding. A user with only an Anthropic key is told that no provider is set up.
  • The OpenAI-compatible endpoint is custom in recipes and the builder, and local or local:<ID> in Settings and the judge.
  • OpenRouter's base URL is written eight times, in five files.
  • System One speaks only TypeSafe's wire format (/systemone). It cannot ask OpenRouter Decisions (/api/alpha/decisions, models of many makers) or OpenAI Decisions (/v1/decisions). Its OpenRouter model list is fixed in code. It ignores the usage.cost that OpenRouter returns, and it drops a refusal answer without a word.
  • A custom provider has a URL and a key only. It cannot say that it serves another protocol at another path.

Wanted

One definition of each model provider, which every feature reads. A feature asks for the protocol that it speaks and gets the providers that can serve it.

  • Protocols. A protocol is a wire format, defined once: its default path and how the key is sent. chat, responses and messages (the names agent definitions use), systemone, decisions (OpenRouter) and openai-decisions.
  • One record shape. A provider is {name, base, key, endpoints}. key names the secret setting that holds the key, never the key. endpoints maps a protocol to a path (appended to base) or a full URL. A preset (OpenRouter, OpenAI, Anthropic, TypeSafe) is the same record with its name, base, key setting and known endpoints fixed. A custom provider is a record that the user writes whole.
  • Storage in layers. A record is one JSON setting: MODEL_<ID> for a custom provider, OPENROUTER, OPENAI, ANTHROPIC or TYPESAFE for the additions to a preset. The preset in code, settings.json and the environment merge field by field. An environment value is a partial JSON object, and null removes a field (RFC 7396). Settings shows read-only only the fields that the environment sets.
  • Keys stay separate secret settings, with the names they have today, so masking, redaction and "a run gets only its provider's key" do not change.
  • Migration. MODEL_<ID>_NAME and MODEL_<ID>_API_URL are still read as name and base. Saving a card moves them into the record. nullius migrate --providers previews the same rewrite, and --apply writes it. Recipes do not change. LLM_JUDGE_PROVIDER still reads local and local:<ID>.
  • Consumers filter by protocol. The builder, standard runs and the CLI ask for what the agent speaks. The LLM judge asks for chat or messages. System One asks for a decision protocol. Onboarding counts any run provider. The duplicate lists go.
  • Settings cards. A card shows Base and, in a collapsed Endpoints section, one row for each protocol. A preset's known endpoints are fixed. An endpoint added to a preset is a path, or a full URL on the preset's own host, because the preset's key goes there.
  • System One in each decision protocol, with one adapter for the TypeSafe shape (systemone, decisions) and one for openai-decisions. It keeps usage.cost and records refusal as no answer.
  • Judge cards. Each judge has Provider, then Protocol when the provider serves two that the judge speaks (else one quiet line), then Model. New settings LLM_JUDGE_PROTOCOL and SYSTEM_ONE_PROTOCOL.
  • A live decision-model list. On OpenRouter, System One offers OpenRouter's public list of decision models, asked without a key, kept 6 hours, with the fixed list as a fallback.

Acceptance

  • app/nullius/providers.py is the only place that names a provider's base URL and key setting. The builder, the provider test, standard runs, the CLI, both judges and onboarding read it.
  • A provider record never holds a key. Settings refuses a record whose key names another setting than its own.
  • A record merges preset, file and environment field by field. A partial environment object changes only its fields, and null removes a field. Tests cover both.
  • Old MODEL_<ID>_NAME and MODEL_<ID>_API_URL settings still work. nullius migrate --providers previews the rewrite and --apply writes it. Saved recipes open unchanged.
  • The builder offers Anthropic only to an agent of the standard that speaks messages. Review and Run refuse a provider that does not serve what the recipe's agents and judge speak.
  • The relay sends x-api-key and anthropic-version only on a messages route, and a bearer header elsewhere.
  • An endpoint added to a preset is refused unless it is a path or a URL on the preset's host.
  • System One asks systemone, decisions and openai-decisions, keeps usage.cost, and records a refusal as no answer. Tests use fixtures, with no live calls.
  • The judge cards show Protocol only when the chosen provider serves two protocols of that judge.
  • docs/settings/model-providers.md, system-one.md, llm-judge.md, the settings reference and the AGENTS.md map describe the registry.

Dependencies and related

Activity

  1. added
    enhancementNew feature or request
    P2Planned: limited in scope, has a way around it, or a missing feature
    on Nov 16, 2026
  2. added theissue type on Nov 16, 2026
  3. added a commit that references this issue on Nov 17, 2026
    ce8b39d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Planned: limited in scope, has a way around it, or a missing featureenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions