Skip to content

fix(ctl): render schema load errors reported at the schema level - #1340

Closed
polmichel wants to merge 10 commits into
stablefrom
pmi-schema-load-error-rendering-1-11
Closed

polmichel wants to merge 10 commits into
stablefrom
pmi-schema-load-error-rendering-1-11

Conversation

@polmichel

@polmichel polmichel commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Infrahub 1.11.0+ reports every write-contract violation of a schema as a single value_error located on the schema entry (body/schemas/<i>), with the violations joined in the message as <field path>: <message> (received: <value>). The renderer only understood errors located on the failing field, so display_schema_load_errors printed Unable to load the schema: with nothing under it when given a 1.11 response. This is what the integration tests exercise, and why they fail on #1330.

Server payload, before and since 1.11.0

The same request, as infrahubctl schema load sends it (the check endpoint validates the same model), against both server generations. Two violations: namespace is not settable on an extension, and made_up is not a field of an attribute.

{
  "schemas": [
    {
      "version": "1.0",
      "extensions": {
        "nodes": [
          {
            "kind": "BuiltinTag",
            "namespace": "Forbidden",
            "attributes": [{"name": "speed", "kind": "Number", "made_up": true}]
          }
        ]
      }
    }
  ]
}

Both responses below were captured by posting this body through FastAPI's TestClient to a route declared with the endpoint's own request model, SchemasLoadAPI from backend/infrahub/api/schema.py. FastAPI validates the body before the endpoint runs, so no database is involved and the 422 is the one the real endpoint returns.

Up to 1.10: one entry per failing field

Infrahub 1.10.8 (registry.opsmill.io/opsmill/infrahub:1.10.8, test run inside the image with docker run --rm -i --entrypoint python <image> - < test.py). Each entry is located on the field and carries the offending value:

{
  "detail": [
    {
      "type": "extra_forbidden",
      "loc": ["body", "schemas", 0, "extensions", "nodes", 0, "attributes", 0, "made_up"],
      "msg": "Extra inputs are not permitted",
      "input": true
    },
    {
      "type": "extra_forbidden",
      "loc": ["body", "schemas", 0, "extensions", "nodes", 0, "namespace"],
      "msg": "Extra inputs are not permitted",
      "input": "Forbidden"
    }
  ]
}

Since 1.11.0: one entry per schema

Since opsmill/infrahub#9814 (shipped in 1.11.0b1 and every 1.11.x), the endpoints validate the payload with the SDK's own validate_schema and re-raise the joined messages as one ValueError. Infrahub stable at afa9dcf75 (1.11.3.dev), confirmed against a live 1.11.2 container. One detail entry for two violations, loc stopping at the schema index, and input carrying the whole submitted schema rather than the offending value:

{
  "detail": [
    {
      "type": "value_error",
      "loc": ["body", "schemas", 0],
      "msg": "Value error, extensions.nodes[0].namespace: Unknown field, it is not part of the schema (received: 'Forbidden'); extensions.nodes[0].attributes[0].made_up: Unknown field, it is not part of the schema (received: True)",
      "input": {"version": "1.0", "extensions": {"nodes": [{"kind": "BuiltinTag", "namespace": "Forbidden", "attributes": [{"name": "speed", "kind": "Number", "made_up": true}]}]}},
      "ctx": {"error": {}}
    }
  ]
}

The renderer handles both shapes and produces the same line for the same failing field.

Which call paths reach the collapsed response

Not every caller sees this response. Checked against the live 1.11.2 with the SDK at v1.23.2 and at this branch:

Call path Reaches the collapsed 422? Why
POST /api/schema/load or /check directly (curl, HTTP client) Yes Nothing validates the payload before the server.
client.schema.load() / client.schema.check() in Python, then display_schema_load_errors Yes The SDK client sends the payload as is. This is what the integration tests in TestInfrahubSchemaLoadErrorRendering do, and why they fail on #1330.
infrahubctl schema load / schema check, SDK at the same level as the server No The command validates the files offline first with validate_schema, the same function the server runs, and exits with the per-field messages before any request is sent.
infrahubctl schema load with a schema the offline validator accepts but the server rejects on internal rules (e.g. a Dropdown attribute without choices) No, different error These rules raise from the field itself, so the 422 keeps a full loc down to the attribute and renders correctly with and without this PR.

The collapse therefore affects API callers and Python users of the SDK client, not infrahubctl users whose SDK is at the level of the server.

What changed

  • Not changed: the server response and what client.schema.load() / check() return. Both still carry the joined string; only the rendering of that string is fixed.
  • Schema-level errors are rendered. The joined message is split into violations, each one is located back in the submitted schema files and rendered through the existing node-level formatting. A violation that does not resolve to a node is printed verbatim with its schema file instead of being dropped.
  • Splitting is safe. A separator is only honoured when the text before it is a complete violation, so a received value containing ; <word>: (a description, for example) is not cut in two.
  • Missing fields show no value instead of a misleading (None).
  • Nested paths are kept (separate commit, pre-existing): an error below an attribute or relationship now names the failing field, Attribute: serial | parameters.regex ([) rather than Attribute: serial ([). An error on an item of a scalar list field (display_labels[0]) no longer crashes the renderer.

Before this PR / after this PR

Same 1.11 server response, rendered through display_schema_load_errors. An infrahubctl user with an SDK at the level of the server never reaches this path: the offline validator rejects the file first.

Before:

Unable to load the schema:

After:

Unable to load the schema:
  Node: InfraDevice | namespace (infra) | String should match pattern '^[A-Z][a-z0-9]+$' (value_error)
  Node: InfraDevice | Attribute: serial | parameters.regex ([) | String should be a valid regex (value_error)
  Node: BuiltinTag (extensions/nodes) | Attribute: speed | made_up (True) | Unknown field, it is not part of the schema (value_error)

Against an Infrahub 1.10 server the output is unchanged, except that an error below an attribute now names the failing field (Attribute: serial | parameters.regex) instead of stopping at the attribute.

Tests

Unit coverage of the renderer, split by component. Shared fixtures live in tests/helpers/schema_load_errors.py.

Component Module
Message parsing: splitting, whole-value detection, field paths, received values tests/unit/ctl/schema/test_load_errors_message_parsing.py
Locating an error in the submitted files: loc validation, node lookup, element labels tests/unit/ctl/schema/test_load_errors_location.py
Field-level rendering, every server version tests/unit/ctl/schema/test_load_errors_field_level.py
Schema-level rendering, 1.11.0+, including contract tests that build the message with the validate_schema().raise_for_status() the server relies on tests/unit/ctl/schema/test_load_errors_schema_level.py
Offline validation exit path and check warnings tests/unit/ctl/test_schema_app.py

The integration tests in TestInfrahubSchemaLoadErrorRendering are unchanged and exercise this path against the 1.11.2 container pulled in by #1330.

Next steps

  • Infrahub: pin the format. Add a server-side test that loads an invalid schema and asserts the shape the SDK now parses: loc of body/schemas/<i>, type value_error, message segments <field path>: <message> (received: <repr>) joined by ; . Today a change there only surfaces as failing integration tests.
  • Infrahub: return structured violations. SchemaValidationResult.errors already carries field and message per violation. Emitting one detail entry per violation, with loc built from field and input carried separately, removes the need to parse a joined string at all.
  • SDK: retire the parser once the structured response ships, keeping only the field-level path. The contract tests here then become plain rendering checks.

Context

Surfaced by #1330.

🤖 Generated with Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 9, 2026

Copy link
Copy Markdown

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 775f8cb
Status: ✅  Deploy successful!
Preview URL: https://d4601d6a.infrahub-sdk-python.pages.dev
Branch Preview URL: https://pmi-schema-load-error-render.infrahub-sdk-python.pages.dev

View logs

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.82353% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/ctl/schema.py 98.82% 0 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##           stable    #1340      +/-   ##
==========================================
+ Coverage   85.01%   85.31%   +0.29%     
==========================================
  Files         148      148              
  Lines       13304    13355      +51     
  Branches     1963     1965       +2     
==========================================
+ Hits        11311    11394      +83     
+ Misses       1426     1405      -21     
+ Partials      567      556      -11     
Flag Coverage Δ
integration-tests 38.74% <41.17%> (-0.11%) ⬇️
python-3.10 58.45% <98.82%> (+0.55%) ⬆️
python-3.11 58.45% <98.82%> (+0.55%) ⬆️
python-3.12 58.45% <98.82%> (+0.55%) ⬆️
python-3.13 58.47% <98.82%> (+0.57%) ⬆️
python-3.14 58.47% <98.82%> (+0.57%) ⬆️
python-filler-3.12 23.66% <0.00%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/ctl/schema.py 85.02% <98.82%> (+12.51%) ⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread infrahub_sdk/ctl/schema.py
Comment thread infrahub_sdk/ctl/schema.py Outdated
Comment thread infrahub_sdk/ctl/schema.py
pol-opsmill and others added 5 commits September 10, 2026 10:25
Infrahub 1.11.0+ reports every write-contract violation of a schema as one
value error located on the schema entry, with one `<field path>: <message>
(received: <value>)` segment per violation in the message. The renderer now
splits that message, rebuilds the field location from each path and renders
each violation through the existing node-level formatting, so the output
names the node, the field and the submitted value as it does for errors
located on the field itself. Violations that do not resolve to a node are
printed verbatim with their schema file instead of being dropped.

A separator is only honoured when the text before it is a complete
violation, so a received value that itself contains `; <word>: ` (for
example a description) is not split in two. A violation without a received
value, such as a missing required field, is rendered without a value
instead of a misleading `(None)`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tribute

The renderer stopped at the attribute or relationship name for any error
located deeper (e.g. `attributes[0].parameters.regex`,
`attributes[0].choices[1].label`), so the output did not say which field
failed. The path below the element is now rendered after the element
label, with the CapitalCase union tag pydantic inserts for the attribute
kind (`attributes[0].Text.name`) dropped.

An error on an item of a scalar list field such as `display_labels[0]`
was treated as an element of a collection of dicts and crashed the
renderer with an AttributeError. It is now rendered as a field path with
its index.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add a dedicated module for the rendering of `schema load` / `schema check`
errors: message splitting (separators inside received values, missing
fields, non-literal values), field path parsing and formatting, schema-level
error detection, location validation, node lookup, element labels, every
field-level and schema-level location shape on nodes, generics and
extensions, verbatim fallback, multi-schema payloads, warnings and the
offline validation exit path. Contract tests build the message with the
same `validate_schema(...).raise_for_status()` the server relies on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The single test module covering the rendering of `schema load` errors is
split into one file per concern under tests/unit/ctl/schema/: parsing of
the schema-level message, locating an error in the submitted schema,
rendering of field-level errors, and rendering of schema-level errors
together with the server contract checks. The offline validation and
warning tests join test_schema_app.py, which already covers that path.
Builders and schema payloads shared across those files move to
tests/helpers/schema_load_errors.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@polmichel
polmichel force-pushed the pmi-schema-load-error-rendering-1-11 branch from 0ee315d to 954c022 Compare September 10, 2026 09:56

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread infrahub_sdk/ctl/schema.py Outdated
Comment thread tests/unit/ctl/schema/test_load_errors_schema_level.py Outdated
Comment thread tests/helpers/schema_load_errors.py
Comment thread changelog/+schema-load-error-rendering.fixed.md Outdated
polmichel and others added 5 commits September 10, 2026 12:57
Every CapitalCase segment below an attribute was dropped as if it were the
union tag pydantic inserts (`attributes[0].Text.name`). An unknown key
that happens to be capitalised, such as `parameters.Regex`, was therefore
hidden from the rendered path, which is the very key the user has to fix.

Only a segment that is a known attribute kind and sits right after the
attribute index is dropped now. Relationships never carry a union tag, so
nothing is dropped there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…est exercises it

The description under test was valid, so it never reached the error
message and the test only checked the namespace error next to it. The
description is now too long as well, which puts the separator-bearing
value in the message; the test asserts it is rendered whole on one line.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ctl/schema

The schema-level rendering tests and their two local helpers in
tests/unit/sdk/test_schema.py were superseded by the dedicated modules
under tests/unit/ctl/schema, which cover the same cases through the shared
fixtures in tests/helpers/schema_load_errors.py. The tests that predate
this change stay where they were.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ot change

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants