Skip to content

Configuration schema: report, validate, and build forms from it - #33

Open
vitramir wants to merge 11 commits into
mainfrom
app-configuration-schema
Open

vitramir wants to merge 11 commits into
mainfrom
app-configuration-schema

Conversation

@vitramir

Copy link
Copy Markdown
Contributor

Implements the Apps Service half of app configuration schema.

Depends on agynio/api#197 — CI will not build until that module publishes.

What changes

  • internal/schema — parser for the honored subset, backward-compatibility checker, configuration validator. Hand-rolled rather than a JSON Schema library: the subset is deliberately narrower than the standard, so a library would accept oneOf / $ref / nesting that the Console cannot render a form from, and the rejection has to happen at report time where the app author can act on it.
  • ReportConfigurationSchema — authorized to the app's own identity, replaces the stored schema wholesale.
  • Validation on InstallApp / UpdateInstallation, returning one error per property.
  • Reference resolution — x-agyn-ref routed to the service that owns the kind, with the organization travelling in the same call so the check and the scoping cannot disagree. No resolver configured means the write fails rather than storing a reference nothing checked.
  • Migration 0004 — configuration_schema, schema_reported_at, and a GIN index on installation configuration for the drained-property lookup.

Compatibility

A reported schema must accept everything the stored one accepted. Removing a property, changing its type, adding to required, or narrowing a constraint is refused. Deprecation is the removal path, and a deprecated property becomes removable once no installation of the app carries it — a question only this service can answer, since it spans organizations the reporting app cannot see.

The author is the one who cannot see the breakage: an app is installed into organizations it has no access to, holding configuration it cannot read or migrate. The rule costs a two-step migration for a type change; the alternative costs every installing organization a broken form they did not cause.

One thing worth reviewing closely

A stored schema that no longer parses is treated as absent, not as an error. Reserving x-agyn-secret made every already-stored schema unparseable, and returning an error for that stranded the app completely — every report rejected because of the state of the schema it was trying to replace. The compatibility guarantee lapses for that one transition, which is the price of ever narrowing the subset.

Verified end to end on a local VM: report at startup, per-property validation, cross-service reference checks, compatibility refusal, deprecation, and drained-property removal.

Apps report a JSON Schema for their installation configuration. The honored
subset is validated by hand rather than with a JSON Schema library: the subset
is deliberately narrower than the standard, so a library would accept
constructs the Console cannot render a form from.

- Parse enforces the honored subset and rejects at report time, where the app
  author can fix it
- CheckCompatible refuses a schema that would stop accepting what the previous
  one accepted; a deprecated property is removable once nothing carries it
- StripSecrets omits x-agyn-secret values rather than masking them, so a
  read-modify-write round-trip cannot submit a placeholder as the credential
ReportConfigurationSchema is authorized to the app's own identity and replaces
the stored schema wholesale, rejecting a document outside the honored subset or
one that would stop accepting what the previous schema accepted. A rejected
report leaves the previous schema whole, including the compatible changes in
the same document.

Redaction is a property of toProtoInstallation rather than of a list of
methods, so a read path cannot be added without deciding: it takes the app's
schema and cannot build the message otherwise. Secret properties are omitted
and named in secret_keys_set; UpdateInstallation carries stored values back in
when a write omits them, which is what makes an unchanged round-trip safe.

References are resolved through a ReferenceResolver. With none configured an
installation carrying one is refused rather than stored unvalidated, since the
app would otherwise discover it wrong at runtime with no form to correct it.
Routes each reference kind to the service that owns it: agent and environment
to Agents, model to LLM. The organization travels with the question so the
existence check and the scoping cannot disagree.

An unknown kind errors rather than resolving leniently — the vocabulary is
closed, so reaching this branch is a programming error, and failing the write
is better than storing a reference nothing ever checked.
x-agyn-secret marked a property and the service then omitted it from every read
but one. The value was still stored in plain text, still submitted in plain
text, and still returned to the app — so the hiding was a promise the platform
could not keep, and it bought a redaction path, a merge-on-write rule, and an
app lookup on every installation read to keep up the appearance.

The marker stays as what it honestly is: a hint that the Console should mask the
field. Configuration round-trips whole again, and reads no longer resolve the
app's schema. Clearing the marker is an ordinary cosmetic change, so the
compatibility rule that rejected it is gone too.

Secrets belong in the Secrets service, referenced rather than copied. That is
its own change.
The keyword is the right name for a property that names a Secret, which does
not exist yet. Keeping it as a display hint would mean migrating every app that
adopted it once it comes to mean something real, so it is rejected at report
time with an error naming the alternative.

format: password is standard JSON Schema and says only how to draw the field.
It is the one honored format that validates nothing, so gaining it on an
existing property is a compatible change where gaining uuid or email is not.
Reserving x-agyn-secret made every schema already stored with it unparseable,
and appSchema returned an error for that — which stranded the app completely:
every report was rejected for the state of the schema it was trying to replace,
and every installation write failed against a schema nobody could evaluate.

A stored schema that no longer parses was valid when written, so the subset has
tightened underneath it. It is treated as absent and logged. The compatibility
guarantee lapses for that one transition, which is the price of ever narrowing
the subset — and far cheaper than an app that can never report again.
Dockerfile.localtest builds against protos generated on the host, because the
api module carrying them is not published yet. It is a stopgap for testing on
the local VM, not something the repo should ship.
The generate step lists proto paths explicitly, and resolving an x-agyn-ref
means calling Agents and LLM — so the build found no package for either. It
passed locally only because those protos had been generated by hand while the
api module was unpublished.
The paths were spelled out in buf.gen.yaml, the Dockerfile, and devspace.yaml.
Adding agents and llm to the first two left e2e building from the third, which
still had the old list — so the service failed to compile in-cluster and the
run timed out waiting for it to become healthy.

The Dockerfile and devspace now run plain `buf generate`, which takes the
module and paths from buf.gen.yaml, the way CI already did.
The run provisioned with agynio/bootstrap, which no longer leaves a Gateway
token behind, so every run died at "AGYN_API_TOKEN is not set" before a single
test ran. Bootstrap is the deprecated install path; the platform installs from
the umbrella chart onto the prebuilt VM now, and agents, gateway and console-app
already provision that way — console-app's e2e is green on it.

Also picks up what those workflows carry and this one lacked: a concurrency
group so a new push cancels the previous run, a 60 minute timeout, and
packages: read.
The deploy died on 'namespaces "platform" not found'. That name came from
bootstrap, which built the cluster itself; the platform VM installs the
umbrella chart into agyn-platform, which is what agents already targets.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant