Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ and its resources. The source layout stays under `app/`.
a Harbor benchmark added from the Benchmarks page's form; `citation`, an entry's
BibTeX read into the paper its page shows; `storage`, whole-folder user overlays and restoring bundled entries; `sources`, where each dataset's data comes
from for each way it runs), `recipes/`
(`launch`, the builder's vocabulary), `providers` (where a run's model calls go, and the builder's steps),
(`launch`, the builder's vocabulary), `providers` (the model provider registry: protocols, presets and each provider's record, read by every consumer; and the builder's steps),
`harbor/` (`source`: the Harbor a recipe pins - built, kept, scanned for its agents;
`feed`: the live feed any Harbor writes, from its own trial hooks),
`importers/` (`atif` ATIF, `otel` OpenTelemetry, and the `cli`), `judge/` (`questions`, `state`, `client`, `store`, `batch`, `reconcile`, `lanes`),
Expand Down
28 changes: 20 additions & 8 deletions app/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -24,26 +24,38 @@
# The key of the OpenAI-compatible endpoint at MODEL_API_URL: a recipe on that endpoint is sent there with this key. Optional, for a server that checks none: without it the run is sent the word 'local' as its key. A run gets it as API_TOKEN, OPENAI_API_KEY and LLM_API_KEY in its environment - over stdin to a host - and never in a recipe or a log.
# e.g. sk-…
MODEL_API_KEY=
# The URL of an OpenAI-compatible endpoint, API path included, required for a recipe on it: there is no default to fall back on, and the builder offers the endpoint only once it is set. It is shown on the page, so it holds no credential: no user or password, no key in its query. More endpoints, each with a name, are MODEL_<ID>_NAME, MODEL_<ID>_API_URL and MODEL_<ID>_API_KEY. A run gets it as API_URL, OPENAI_BASE_URL and LLM_BASE_URL.
# The base URL of an OpenAI-compatible endpoint, API path included, required for a recipe on it: there is no default to fall back on, and the builder offers the endpoint only once it is set. It is shown on the page, so it holds no credential: no user or password, no key in its query. More custom providers, each with a name, are a record MODEL_<ID> and a key MODEL_<ID>_API_KEY. A run gets it as API_URL, OPENAI_BASE_URL and LLM_BASE_URL.
# e.g. https://…/v1
MODEL_API_URL=
# An OpenAI key, for a recipe whose provider is OpenAI: that run is sent to OpenAI's own endpoint, https://api.openai.com/v1, with this key - never with the model API key or its endpoint. It reaches the run as the model key does, and never a recipe or a log.
# An OpenAI key, for a recipe whose provider is OpenAI: that run is sent to OpenAI's own base with this key, never with the model API key or its endpoint. It reaches the run as the model key does, and never a recipe or a log.
OPENAI_API_KEY=
# An OpenRouter key, for a recipe whose provider is OpenRouter: that run is sent to OpenRouter's own endpoint, https://openrouter.ai/api/v1, with this key - never with the model API key or its endpoint. It reaches the run as the model key does, and never a recipe or a log.
# An OpenRouter key, for a recipe whose provider is OpenRouter: that run is sent to OpenRouter's own base with this key, never with the model API key or its endpoint. It reaches the run as the model key does, and never a recipe or a log.
OPENROUTER_API_KEY=
# An Anthropic key, for a run of the Nullius standard whose provider is Anthropic: its relay sends the calls to Anthropic's own endpoint, https://api.anthropic.com/v1, with this key. Only the relay holds it - never an agent, a recipe or a log.
# An Anthropic key, for a run of the Nullius standard whose agent speaks Anthropic's Messages, and for the LLM judge: the calls go to Anthropic's own base with this key. Only the relay and the judge hold it, never an agent, a recipe or a log.
ANTHROPIC_API_KEY=
# TypeSafe.ai key for the System One judgement on TypeSafe's own endpoint, POST https://api.typesafe.ai/v1/systemone — sent as a bearer header there and nowhere else.
# A TypeSafe key, for the System One judgement on TypeSafe's own base: sent as a bearer header there and nowhere else.
# e.g. ts-…
TYPESAFE_API_KEY=
# Where the System One judgement is asked: typesafe (its own endpoint, with TYPESAFE_API_KEY) or openrouter (https://openrouter.ai/api/v1/systemone, the same contract, with OPENROUTER_API_KEY). Unset: TypeSafe when its key is set, else OpenRouter when its key is.
# The provider the System One judgement is asked on: one that serves a decision protocol, such as typesafe or openrouter, with that provider's own key. Unset: TypeSafe when its key is set, else OpenRouter when its key is.
SYSTEM_ONE_PROVIDER=
# The System One model, as the provider names it: on TypeSafe jev-latest, on OpenRouter ~typesafe/jev-latest, typesafe/jev-router, jaredpalmer/kev-4b... Unset: the provider's Jev.
# The decision protocol System One is asked in, on a provider that serves more than one: systemone, decisions or openai-decisions. Unset, or one the provider does not serve: on OpenRouter systemone for TypeSafe's models and decisions for another maker's, else the first the provider serves.
SYSTEM_ONE_PROTOCOL=
# The System One model, as the provider names it: on TypeSafe jev-latest, on OpenRouter ~typesafe/jev-latest or typesafe/jev-1.14. Unset: the provider's default.
SYSTEM_ONE_MODEL=
# The provider of the LLM judgement, the second judgement beside System One: openrouter, openai, local (the endpoint at MODEL_API_URL) or local:<ID> (the endpoint MODEL_<ID>_API_URL), asked with that provider's own key and endpoint from the model providers above. Unset: the first of them that is set.
# The provider of the LLM judgement, the second judgement beside System One: openrouter, openai, anthropic, custom (the endpoint at MODEL_API_URL) or custom:<ID> (a custom provider), asked with that provider's own key and base from the model providers above. local and local:<ID> are read as custom. Unset: the first of them that is set, Anthropic last.
LLM_JUDGE_PROVIDER=
# The protocol the LLM judge is asked in, on a provider that serves both: chat (OpenAI Chat Completions) or messages (Anthropic Messages). Unset, or one the provider does not serve: the first the provider serves, chat before messages.
LLM_JUDGE_PROTOCOL=
# The model of the LLM judgement, as its provider names it (on OpenRouter, vendor/model). It is asked the same questions as System One and answers them as JSON; its numbers are what it states, so a choice or a score carries no confidence.
LLM_JUDGE_MODEL=
# Additions to the OpenRouter provider, as a JSON object without the key: endpoints of protocols the preset does not list, {"endpoints": {"<protocol>": "/path" or "https://full/url"}}. In the environment, a partial object merged over this one.
OPENROUTER=
# Additions to the OpenAI provider, as a JSON object without the key: endpoints of protocols the preset does not list, {"endpoints": {"<protocol>": "/path" or "https://full/url"}}. In the environment, a partial object merged over this one.
OPENAI=
# Additions to the Anthropic provider, as a JSON object without the key: endpoints of protocols the preset does not list, {"endpoints": {"<protocol>": "/path" or "https://full/url"}}. In the environment, a partial object merged over this one.
ANTHROPIC=
# Additions to the TypeSafe provider, as a JSON object without the key: endpoints of protocols the preset does not list, {"endpoints": {"<protocol>": "/path" or "https://full/url"}}. In the environment, a partial object merged over this one.
TYPESAFE=

# --- Run hosts ---
# Machines a recipe can run on, over ssh: an alias from ~/.ssh/config, or user@host[:port][:/path/to/key]. Only the name is stored; keys stay where they are.
Expand Down
6 changes: 3 additions & 3 deletions app/nullius/execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@
from datetime import datetime, timezone
from typing import Callable, Dict, List, Optional

from nullius import hosts, paths, prices as price_list, progress, redact, secrets, settings
from nullius import hosts, paths, prices as price_list, progress, providers, redact, secrets, settings
from nullius.harbor import source as harbor_source
from nullius.standard import container, payload as standard_payload

Expand Down Expand Up @@ -339,7 +339,7 @@ def withheld() -> List[str]:
Custom keys - and GitHub tokens from saved settings or the
shell Nullius was started in. Every named endpoint's settings too, so a
run gets its own endpoint's key alone. A data provider's token name is not one."""
secret = {v.name for v in settings.VARIABLES if v.secret or v.kind == "endpoint"} | set(settings.endpoint_names())
secret = {v.name for v in settings.VARIABLES if v.secret or v.kind in ("endpoint", "provider")} | set(settings.endpoint_names())
return sorted(secret | {old for old, new in settings.INSTEAD.items() if new in secret}
| set(KEY_NAMES + URL_NAMES) | {"GITHUB_TOKEN", "GH_TOKEN"} | set(settings.agent_key_names())
| set(_licence_names()))
Expand Down Expand Up @@ -390,7 +390,7 @@ def __init__(self, plan: Plan, root: Optional[str] = None, run: Optional[Callabl
self._run = run or (lambda argv, timeout=600.0, stdin=None: subprocess_run(argv, timeout, stdin, stop=self._stop))
# an endpoint of the run's own (url_var: an OpenAI-compatible one) may check no key: it is sent the placeholder
self._key = (key if key is not None else env.get(plan.key_var, "")) or (settings.MODEL_KEY_PLACEHOLDER if plan.url_var else "")
self._url = url if url is not None else (env.get(plan.url_var, "") if plan.url_var else plan.base_url)
self._url = url if url is not None else (providers.base_of(plan.url_var, env) if plan.url_var else plan.base_url)
if plan.keyless: # a run that calls no model is given nothing it could leak
self._key, self._url = "", ""
# the agent keys Settings holds (CURSOR_API_KEY, ...): a Harbor run's agent may need its own vendor's; the standard's never gets one
Expand Down
Loading
Loading