Skip to content

Standard: no --retries for an attempt that fails on the infrastructure or the agent, and --repeat is not named --attempts #152

Description

@aitianshi

Priority

extreme

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • catalogue (benchmarks, agents, capabilities)
  • the standard (ports, kits, relay, runs)
  • CI, workflows and skills

Today

An attempt is one run of a task, counted whatever its result; a retry runs an attempt again after the infrastructure or the agent failed, never because the task was failed. Harbor has both: n_attempts, 1 by default, and max_retries, 0 by default, which retries any exception but those its RetryConfig excludes (timeouts, refusals, a wrong model or key, a usage limit, the verifier's own reward errors).

The standard has --repeat K for attempts and one fixed retry: a unit that ended rate-limited runs once more after the others (#90). An agent crash, a provider error or a setup error is never retried, and once a run ends no page says an attempt was retried. The builder calls Harbor's attempts "Repeats per task" and its retries "Retries per trial", 3 by default.

Wanted

  • cli run --retries N, 0 by default, by Harbor's retry rule, and --attempts K as the name of --repeat K.
  • The builder names both settings alike for both runners: Attempts per task (or unit) and Retries per attempt.
  • A retried attempt says so on the pages.

Acceptance

  • cli run --retries N (0 to 9, default 0) runs an attempt again, up to N times, only when it ended in a retryable failure: a rate limit, a provider or network error, a setup error, an agent crash, an output-length error, a verifier error. Never an agent or verifier timeout, a refusal, a wrong model or key, a missing secret, a usage limit, the spending cap (the run's or the unit's), a stopped run, or a task the agent failed.
  • Retries run after the other units, one at a time and paced; nothing is retried once the run is stopped or halted, and the spending cap counts every try.
  • Each earlier try is kept whole under the job's retried/, numbered, where no reader takes it for a trial; the kept trial's result.json says how many retries it took, and config.json records retries.
  • cli run --attempts K runs each unit K times; --repeat K still works.
  • A standard recipe carries retries; a Harbor recipe's retries default to 0; the builder labels the settings "Attempts per task" (or unit) and "Retries per attempt" for both runners; a saved recipe opens as it was saved.
  • A retried attempt says so on the run page, the trial page, All attempts and the live board.
  • docs/standard.md, docs/cli/run.md, docs/builder.md and the field guide describe both settings.
  • Tests cover the retry rule, the rounds of retries, the kept tries, a stop, the recipe keys and the pages.

Dependencies and related

Activity

  1. self-assigned this
    on Nov 16, 2026
  2. added
    enhancementNew feature or request
    P2Planned: limited in scope, has a way around it, or a missing feature
    on Nov 16, 2026
  3. changed the title [-]Standard: no --attempts beside --repeat, and "attempt" names a repeat in the builder, recipes and driver protocol[/-] [+]Standard: no --retries for an attempt that fails on the infrastructure or the agent, and --repeat is not named --attempts[/+] on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P2Planned: limited in scope, has a way around it, or a missing featureenhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions