Torque blueprint schema alignment + tests - #184
Open
dannykQuali wants to merge 24 commits into
Open
dannykQuali wants to merge 24 commits into
dannykQuali wants to merge 24 commits into
Conversation
…onal inputs
Sync with cs2018 (39042894f2) and cs2018-ui:
- resources: "reference" long syntax ({ids: ...}) alongside the string form
- outputs: "sensitive" flag
- grains: shell file "tag", OpenTofu script hooks + opentofuvars-files,
stack-name-prefix, env_configuration, target/deployment-engine short syntax,
source name/chart-version/path-in-archive, host docker permissions,
remote/cloud backend fields
- new top-level sections: api_access, family, template
- environment: description/tags/env_references_values/labels; workflow: enabled
- inputs: "dictionary" type added, legacy "execution-host"/"env" removed
(server rejects them), object defaults, vCenter + target_name overrides
- grain kinds: "arm" removed (not in GrainKind)
- customization: launch-form image/actions/params, steps general/inputs,
category/section image+actions, input display_name/allowed-values-details/
display-as/columns, legacy array form, customization.inputs/params/layout/
grains-map
Consolidate duplicated definitions:
- one GrainSpecSourceObject for all source objects (ScriptSource and
TfVarsFileSourceObject deleted; script sources now require "path"),
SourceFileObject wrapper for values/workspace/tfvars/opentofuvars files
- StoreFileSourceObject (instructions+layout), EnvironmentLabelsArray,
KeyValuePairs, StringMap, TemplateParamsObject
- dead-weight cleanup (always-true oneOf, $ref siblings, stray keywords)
Document optional-input semantics in descriptions: an input is required iff
it has no pattern; blank-allowed needs an empty-matching pattern (^$|...);
default: "" does not make an input optional; sensitive inputs pre-fill empty;
all-hidden launch-form sections still render.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documentation pass over every schema field, aimed at humans and AI agents
authoring blueprints (surfaced by VS Code as hover/completion docs):
- 126 new descriptions: a Liquid-templating primer on the root, per-section
explanations, per-input-type and per-grain-kind semantics, source pinning,
resource requirement shapes (quantity: single vs list), launch-form
customization, grain-map authoring rules
- wording aligned with the customer-facing docs (docs.qtorque.io) where they
cover the field; source-derived wording kept for features the public docs
do not cover yet (resources, env_references, target/resource/dictionary
inputs, api_access)
- optional-input semantics documented where authors make mistakes: required
iff no pattern, '^$|' convention for blank-allowed, default: "" gotcha,
sensitive inputs are write-only after launch (no copy / no Relaunch
carry-over / re-enter on edit)
- every non-obvious claim audited against cs2018 source; corrected findings:
auto-approve/auto-retry default true, auto-approve: false requires runner
storage, mode is per-kind (terraform: managed/no-termination, argocd: data),
spec.region is CloudFormation-only, space-scoped workflows are manual-only,
env references target published environments
Removed dead fields the server ignores, so the extension flags them instead
of silently accepting them:
- grains.<name>.spec.host (GrainSpecYaml has no such member)
- agent.image (parsed but consumed nowhere; runner images come from
RunnerMgmt image mapping)
Also fixed two title typos ("Envrironment Variable", "LaunchForm Object").
Validation behavior is otherwise unchanged: with description/title stripped,
the schema is structurally identical to the previous commit except the two
removed fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Torque server parses neither grains.<name>.spec.host (GrainSpecYaml has no such member) nor agent.image (parsed but consumed nowhere). The JSON schema already rejects both since 4852bf2; removing them from the language server's tree model keeps the two models of the language consistent. No references to either field existed in server code. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s, robustness The Python language server's spec2 support lagged years behind the schema: its tree model knew only spec_version/description/inputs/outputs/grains, so every modern blueprint got false "Parent node does not have child" errors, and its expression validator rejected current Liquid. Tree model (server/ats/trees/blueprint_v2.py, rewritten): - mirrors the audited JSON schema: all top-level sections (metadata, environment, workflow, instructions, layout, labels, env_references, resources, api_access, customization, family, template) and the full grain/spec/agent/scripts/source/input/output field set - scalar short forms (target, deployment-engine, reference, blueprint-labels, self-service) via Union annotations; new FreeFormNode/FreeFormProperty primitives for opaque content (customization, inventory-file, provider attributes, string maps) - dead fields stay rejected (spec.host, agent.image, input display-style); fixed the bogus display-style -> style input field - framework (common.py): Union-aware get_child and PropertyNode.__getattr__, guarded _get_seq_nodes, per-class field-mapping cache (parser hot path) Expression validation (server/validation/bp_v2_validator.py): - prefixes: + .resources, .env_references (cs2018's tests use both) - reserved variables: + envId, environmentName, blueprintName, ownerEmail, accountName, spaceName; filters: + key_access, strip, with argument handling; bare .grains.<g>.outputs accepted (valid all-outputs form) - free-form subtrees are skipped: customization uses the UI's template dialect and inventories may hold Jinja - not grain Liquid New semantic validations, each mirroring a cs2018 server rule: - resource requirement: exactly one of selector/reference - grain: agent and target are mutually exclusive - mode per kind (terraform: managed/no-termination; argocd: data, mandatory) - auto-approve: false requires runner storage (use-storage) - workflow: scope enum, space scope is manual-triggers-only, per-trigger event/cron shape, timeout integer >= 5 minutes Robustness (a validator exception wipes ALL diagnostics for the file): - fixed crashes on empty spec:, variable-like keys in free-form sections, .grains.<g>.scripts refs to grains without scripts, Union-typed property access; loop aborts no longer suppress later grains' diagnostics; orphaned short-form nodes (commands, blueprint-labels) now surface errors; verified by a 5000+-prefix typing simulation with zero exceptions Schema parity fixes discovered by this work: - aws-cdk script hooks are ScriptOutputsObject (match ScriptsYaml) - provider-overrides items closed to name/source/version/attributes - version-like fields (spec version, tf-version, estimated-ccu, chart-version, provider version) accept unquoted YAML numbers, with a lossy-float authoring warning on the Terraform version fields Tests: 48 new tests (tests/test_spec2.py, tests/test_spec2_semantics.py) covering tree coverage, expression rules, semantic rules, and robustness; fixed tests/test_ast.py posixpath.dirname import that broke the suite on Windows. Full suite: 71 tests, green on the pinned Python 3.7 stack. Docs: docs/spec2-language-support.md records the two-layer design, the dead-field policy, the rule tables with their cs2018 provenance, and the test setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Validated both layers (JSON schema + Python language server) against every
spec_version: 2 file in six production blueprint repos, then fixed each
divergence from actual Torque server behavior (all verified in cs2018
source). Result: fully-clean files 61 -> 349 of 452, crashes 1 -> 0,
language-server errors 3301 -> 375, warnings 1058 -> 16 - and everything
still flagged was individually confirmed to be a genuine blueprint defect
(dead keys the server drops, undeclared outputs, misspelled context members).
Parser (server/ats/parser.py):
- flow-style YAML support: [a, b] sequences and {k: v} mappings are
translated to the equivalent block token stream (162 real files
previously failed wholesale with "Scalar cannot be accepted here")
- unprintable characters (mojibake C1 bytes) are replaced 1:1 instead of
crashing pyyaml; server.py's raw yaml.load path sanitizes the same way
- fixed two unmasked pre-existing bugs: a block sequence at the key's
indentation corrupted the node stack, and the branch never closed the
map element so the next key silently erased the previous one
Expression validation (server/validation/bp_v2_validator.py):
- proper path tokenizer: bracket access (.inputs["Name With Spaces"],
.inputs.["X"]), dotless forms ({{ inputs.X }}), and leading-dot reserved
variables ({{ .envId }}) - DotLiquid's path scanner drops the dot
- .grains.<g>.activities.<activity>.commands.<cmd>.outputs.<out> validated
as a first-class path (cs2018 CreateActivitiesContext), with output-name
checking against the command
- depends-on closure is transitive (cs2018 GetAllDependentGrains recurses)
- grain members: outputs/scripts/activities/is_active; prefixes gained
management_server, and bindings for workflow-scoped blueprints
- filters: the full DotLiquid standard set plus Torque customs
(key_access, json, json_escape, resource_property, resource_object),
chained pipes, arguments after ':'
- unused-input warning matches all real usage spellings
Tree model (server/ats/trees/blueprint_v2.py):
- authentication entries and source store/path/files paths take Liquid
(the server resolves them); grain spec input values may be YAML lists
or mappings (the server JSON-ifies them)
Schema (client/schemas/blueprint-spec2-schema.json):
- grain input values accept arrays; blueprint-label values accept
numbers/booleans; spec_version accepts 2, "2" and "2-preview"
(all mirror the cs2018 parser)
Tests: tests/test_spec2_real_world.py (20 tests) pins every rule above;
full suite 91 tests green. docs/spec2-language-support.md updated to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OpenShift target support adds no blueprint YAML fields - targets are referenced by name - so only value lists change: - allowed-credential-providers: add 'kubernetes' and 'sourceControl' (pre-existing gaps vs CredentialCloudTypes.Supported). Deliberately NOT 'openshift': openshift targets authenticate with kubernetes credentials (TargetCapabilityValidatorBase mapping), and the description now says so. - resource selector provider-type: 'openshift' added to the soft enum (new __openshift__ built-in inventory provider type). - target-filters cloud-providers: items upgraded to the same soft-enum pattern with the six server-validated target types (aws, azure, vcenter, intersight, kubernetes, openshift). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ExpressionValidationVisitor computed a diagnostic position as (node line, node column + offset-in-value), which is only correct for a single-line scalar. In a block scalar (command: |) the offset indexes the joined text, so errors landed on the block header's line with an impossible column - a real corpus file reported 236:154 where line 236 is 'command: |' (24 chars) and the expression was on line 239. Shell grains are the most common case, so most expression squiggles pointed nowhere.
The visitor now receives the document and, for multi-line values, locates each match by scanning the document lines within the node's line span, consuming occurrences in order so repeated expressions map to successive lines. Single-line positions are unchanged, and the old arithmetic remains the fallback when no document is available or a match cannot be located (a validator exception would wipe every diagnostic for the file).
Found by validating 454 real blueprints; that finding now reports 239:20, the exact column of the '{{'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two human-operated scripts for testing both validation layers against
customer blueprint corpora, which carry proprietary data and must never be
read by an AI.
tools/blueprint-corpus/sanitize_blueprints.py (stdlib only, no install):
- text-based redaction, so comments, flow style, indentation quirks and
even mojibake survive - a load-and-dump would normalize away the very
defects the validators must be tested against, and would fail on files
that do not parse
- two mechanisms: secret-ish key names redact the whole value (including
default:/value: under such a key, the most common real leak), and value
patterns redact substrings, including inside shell command bodies
- keeps the corpus faithful: keys are never rewritten, pure-Liquid values
and Liquid spans are never touched, pattern:/validation-description: are
exempt from both mechanisms (a pattern decides Torque's required-vs-
optional semantics), and content-addressed keys (commit, image, digest,
tag, version, path, store, ...) are exempt from the entropy heuristics
- never modifies the input: sources open read-only, output-inside-input is
refused, and every source file's sha256 is asserted unchanged; non-YAML
files are never copied, so a sanitized tree cannot hide a raw file
- reports what was redacted by location, category, length and a hash
prefix - never the value - plus a residual scan of the output and a
review list of near misses; --self-test (94 checks) lets the operator
verify the tool on their own machine
tools/blueprint-corpus/validate_blueprints.py (pyyaml + jsonschema):
- runs the JSON schema layer (with oneOf drill-down so messages are
readable) and the language-server layer (parser, tree errors, semantic
diagnostics, crashes) over a corpus
- report levels: full reports are confidential; safe-summary.txt is
designed to be shareable and always uses anonymous file ids resolved
locally via path-map.txt, so it cannot leak identifying file names
- triage against known_findings.json (43 verified entries): findings are
labeled KNOWN-BLUEPRINT-DEFECT / KNOWN-TOOL-LIMITATION, and NEW clusters
are surfaced first because those are the likely tool bugs
- runs on any modern Python: falls back to a minimal pygls stub, verified
to produce byte-identical reports to real pygls 0.11.3 on Python 3.7
over the 454-file reference corpus
Verified on the six internal ZeroTouch repos: 454 spec2 files, 350 fully
clean, 0 crashes, 790 findings all KNOWN. Sanitization is validation-
neutral - all 454 files produce identical finding sets before and after,
line numbers included - and git reports zero modified files in all six
source repos.
Also documents six newly verified blueprint-defect classes found this way,
including env-vars under activities.destroy (silently dropped, so those
variables never reach the commands) and {{ .outputs[...] }} in workflows
(the correct form is {{ .bindings.outputs[...] }}).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both workflows have been failing on every push, including on master, since well before the current work. ci.yml: the test job targeted runs-on ubuntu-20.04, an image GitHub retired, so both matrix legs were cancelled within seconds and the Python suite has not actually run in CI for months. Now ubuntu-22.04, the newest image still offering a Python 3.7 build (the server pins pygls==0.11.3, which predates the lsprotocol API). The 3.6 leg is dropped: no 3.6 build exists on 22.04 and 3.6 has been EOL since 2021. vscode_build.yml / release.yml: 'vsce package' died with 'ReferenceError: ReadableStream is not defined' - the workflows pinned Node 16 while an unpinned 'npm install -g vsce' pulled transitive deps (undici 7, cheerio 1.2.0, whatwg-*) that require Node >= 18, where ReadableStream became a global. Now Node 20, the renamed and maintained @vscode/vsce pinned to ^3, and the redundant 'npm install' after 'npm ci' removed - that re-resolution is what let the drift in. release.yml had no npm ci at all, so its npm install became npm ci. Action versions bumped off the deprecated Node runtimes (checkout@v2 -> v4, setup-node@v1 -> v4, setup-python@v2 -> v5). Verified locally on Node 22 before changing anything: npm ci, npm run compile and npm run webpack all succeed (webpack 5.64 has no OpenSSL regression on modern Node), and the failing step packages a 235 KB vsix of 54 files with both vsce@2.15.0 and @vscode/vsce 3.9.2. Also confirmed super-linter v4 disables trailing-spaces and treats document-start and line-length as warnings, so touching these files cannot turn the green lint job red. Still broken and deliberately untouched: release.yml uses ::set-output (disabled by GitHub in 2023) and the archived create-release@v1 / upload-release-asset@v1. Whether setup-python@v5 can provision 3.7 on ubuntu-22.04 can only be confirmed by a real run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds docs/schema-removals.md: what was removed from the spec2 schema, the evidence each removed item is dead in the Torque server, the measured blast radius, options, and rollback. Written for a manager to approve before the change reaches production. The investigation behind it corrected a wrong assumption of mine: the schema is NOT authoring-only-until-released. The Torque web UI fetches this exact file live from raw.githubusercontent master and feeds it to monaco-yaml in six editor surfaces (both designers, the setup wizard blueprint viewer, the grain inventory and reserved-resources tabs, and the custom-workflow YAML view), so the change reaches every Torque user within minutes of the merge, with no release and no version pin. Conversely the extension itself stopped registering the schema in 0.3.3, so the UI is now its primary consumer. What Torque accepts is unchanged: POST /validations/blueprints runs the C# V2BlueprintValidation validators, and cs2018 has no JSON-schema-based blueprint validation in any .cs source. No deployment behavior changes and nothing needs un-deploying. Measured over 527 real spec2 blueprints: 12 files (2.3%) will show a new editor error, and in all 12 the error is correct - it points at a key Torque already ignores. Also records the one place the schema is deliberately stricter than the server (script source requires path), the one place the consolidation loosened (tfvars source gained keys), and a separate recommendation to pin the UI to a tag or SHA instead of master. known_findings.json: corpus figure corrected to the 454-file calibration set plus the 73-file out-of-sample check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
It was written for a manager's one-time approval decision and is delivered as a shared page, not as repository documentation. The corpus-count fix to known_findings.json from the same commit stays. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…open schema maps Re-verified against cs2018 origin/main c0f49bd04d (2026-09-14) and cs2018-ui 9fe9e4b01b. The schema had been synced to a source snapshot from 2026-08-24; three backend changes landed after it and are now mirrored in both layers (JSON schema and the Python language server): - inputs.<name>.optional (cs2018 7571846285, 2026-08-30): bool?, only meaningful for string and dictionary inputs. 'optional: true' with a pattern that rejects the empty string is a blueprint validation error, which the language server now reports too, evaluating the pattern the way the server's PatternEmptyValueEvaluator does (regex-literal unwrapping, no added anchors, an unevaluable pattern never fails). The launch form's required-ness rule changed the same day - a default, even "", now makes an unset-optional input optional unless its pattern rejects "" - so the default/sensitive/pattern/ validation-description descriptions that stated the old rule were corrected. - Server-side pattern enforcement at launch (cs2018 8d62b1be5d, 2026-09-09), with Liquid allowed in pattern and validation-description. - inputs.<name>.target-filters.labels[].values (cs2018 eec4e2859a): a list alternative to value, any-of match; value and values together is an error in the schema (not/required) and a language-server diagnostic. Name maps: the server constrains grain names to ^[a-zA-Z0-9 \-_]+$ with no length limit and does not constrain input or output names at all. The schema demanded 3-45 characters and, with no additionalProperties on the inputs, grains, outputs, env_references and resources maps, silently skipped every off-pattern entry instead of validating it - real blueprints name inputs OS and 'Ethernet1/1 Port Group', grains db, outputs id. Inputs/outputs/ env_references/resources now accept any name; grains mirror the server regex and reject the rest. Script hook objects (ScriptObject, ScriptOutputsObject) are now closed: the server's ScriptYaml reads only source/arguments(/outputs), so 'files' under a hook - present in 13 local blueprints - was accepted while the listed files were never delivered. Catalog: the two entries that called 'optional' a dead key are removed (true until 2026-08-30), and the schema-layer twin of the files-under-hook finding is added. Corpus re-run over 528 local blueprints from ten sources: 412 fully clean, 0 crashes, 727 findings, all KNOWN, 0 NEW. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four spelling and hyphenation defects in the schema's hover text reached a colleague's PR because nothing in the pipeline reads prose: the tests are behavioral, super-linter checks JSON syntax, and a misspelled identifier such as EnvironementVariable is internally consistent so every $ref resolves. codespell 2.4.3 is configured in .codespellrc so a bare `codespell` from the repo root behaves identically locally and in CI. Skipped on purpose: the two CHANGELOGs (shipped release history, quoted verbatim in the docs) and the gitignored client/ packaging copies. No ignore-words-list: a one-off false positive gets an inline `codespell:ignore` on its own line instead, so a real misspelling can never be whitelisted repo-wide. Fixed: README 'opeation'; a parser comment 'doesnt'; 'unparseable' in the sanitizer's docstring and self-test labels; a test grain named 'loner'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… real ones tests/schema_fixtures holds 28 valid and 26 invalid spec2 blueprints written from scratch with generic content - no real hosts, repositories, products or credentials - each exercising one construct family. The test contract: - every valid fixture validates with zero errors; - every invalid fixture produces the errors named in its `# expect-error:` header lines, matched against all nested oneOf/anyOf messages, and an invalid fixture that is silently accepted is a failure - that is exactly the defect class (dead keys, off-pattern names, open hook objects) this corpus exists to catch; - the union of key paths across the valid fixtures covers every one of the 232 schema-accepted paths that 528 local blueprints actually use (required-paths.txt), so the corpus stays representative of real usage in a checkable way. tools/blueprint-corpus/classify_corpus_paths.py is the read-only classifier that produced required-paths.txt by walking documents alongside the schema; the test reuses its normalization, and the README documents how to regenerate the list when the corpus or the schema changes. Runs on Python 3.7 and current Pythons with only pyyaml and jsonschema; 100 consecutive runs, no flakes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… name rules and fixture corpus ci.yml gains a `spellcheck` job (codespell 2.4.3 on Python 3.12) and a `schema-fixtures` job that runs tests/test_schema_fixtures.py on Python 3.12 with nothing installed but pyyaml and jsonschema, proving the fixture corpus needs none of the legacy language-server stack. The Python 3.7 `test` job now installs jsonschema, which the schema-layer test modules import; pip resolves 4.17.3 there, the last release supporting 3.7. docs/spec2-language-support.md: a "Source freshness" rule (fetch and compare to origin before trusting a local cs2018/cs2018-ui clone, record the SHA, treat a negative grep as evidence only with a positive control), the September changes to inputs (optional, the launch-form required-ness rule, server-side pattern enforcement, target-filter label values), what a name may be, the spell-check gate, and the fixture corpus contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o draft-07 Every rule below was read from cs2018 origin/main c0f49bd04d and is pinned by tests/test_spec2_server_parity.py and tests/test_spec2_required_parity.py. Draft upgrade. The schema declared draft-06 but used if/then/else in five places (approval channels, tolerations) - keywords that do not exist before draft-07, so compliant validators ignored them and those rules were dead: an approval channel with no approvers and a toleration 'Equal' with no key/value produced zero errors. $schema is now draft-07 and every validator in tests and tools is Draft7Validator. Both consumers (yaml-language-server behind redhat.vscode-yaml, and cs2018-ui's monaco-yaml) support draft-07. Closed objects the deserializer would silently prune: Backend (and its workspaces items: name, prefix, project, tags), TemplateStorage, GrainTag, SourceFileObject. Backend 'type' is the server's BackendType enum. Mandatory-field parity: a source needs store or path (family member: both); agent needs name; instructions path must end .md and layout path .yaml, case- insensitively; template placeholders need a non-blank path; approval channel approver lists need at least one entry; a parameter input needs parameter-name; a cloudformation grain needs region and one of authentication/agent/target; backend needs type and, per type, s3 bucket+region, azurerm storage-account- name+container-name, gcs bucket, http base-address, remote organization plus a non-empty workspaces list whose entries carry name or prefix (hostname and token are optional on the server), cloud nothing further. workflow.scope is the EntityType enum (space, env, env_resource); workflow.timeout is an integer of at least 5 minutes, a numeric string, or a Liquid expression; the trigger event list gains 'Tag Updates Detected'. Audits that changed nothing: the two-way key audit found only dead server constants with no YamlMember (compute-service, cloud-account, role-arn, external-id); all 32 input-source override keys match provider constants; no unreachable definitions; every other enum already exact. GrainKind's internal mock-terraform/mock-helm are deliberately not added. Evidence: 528 sanitized local blueprints re-validated after every change - 412 fully clean, 0 crashes, 727 findings all KNOWN, 0 NEW. None of the new rules rejects a working blueprint. README: the supported Python floor is 3.7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The coverage contract is raised from "the 232 key paths the real blueprints use" to everything the schema declares: every property of every definition (352), every enum value (115) and every definition (58), plus the 232 real-world paths. The sets are computed from the schema itself at test time, so a property added without a fixture fails CI; the exclusion lists exist and are empty. Fixtures: 46 valid (was 28) and 50 invalid (was 26), still minimal and generic. New valid fixtures cover the environment section, full source objects at every position, all eleven script hooks, on-destroy, agent docker/kubernetes and tolerations, runner-configuration-override, all six backend types, template storage, approval channels, every input-source override, every provider enum, all workflow trigger events and timeout forms, and the whole customization tree. New invalid fixtures pin each server rule from the parity pass. The classifier now carries the owning definition through $ref/allOf/oneOf so a walk records visited properties, enum values and definitions, and a SchemaEnumerator computes the full sets by the same rules. Invalid fixtures pin offending values rather than library wording, because jsonschema 4.17 (Python 3.7) and 4.18+ phrase minItems/minLength failures differently. Verified on Python 3.7 and on fresh 3.13/3.14 venvs with only pyyaml and jsonschema; 30 consecutive runs without a flake; no real content in any fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docs/spec2-language-support.md gains section 10: the draft-07 upgrade with the proof that the if/then rules were dead under draft-06, the required-field parity table with its server sources, the Terraform backend rules per type, the closed objects and their server classes, the audits that found nothing to change, and the 528-blueprint evidence that no new rule rejects a working blueprint. Stale statements fixed: the "scope has no enum" example (it does now), the workflow rules the schema can now express itself, and the testing section's list of schema-layer modules and the coverage surface (352 properties, 115 enum values, 58 definitions). The fixtures README states the raised coverage contract and the jsonschema wording caveat; one fixture header comment no longer overstates what it pins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ent every property All 177 existing descriptions were checked claim by claim against cs2018 origin/main c0f49bd04d and the cs2018-ui source. Thirteen were corrected: - the input object's summary still described the pre-2026-08-30 required-ness rule; it now defers to optional / default / pattern - input depends-on also supports string inputs (InputTypesWithDependsOnSupport) - allowed-values: a single value is auto-selected unless the style is multi-select - resource-selector is optional on resource inputs; the server never requires it - workflow label-selector is a legacy alias of resource-types (AutoMapper reads ResourceTypes = LabelSelector ?? ResourceTypes), not of labels-selector - target has no scalar short form: the property carries only [YamlMember], there is no converter, and 607 of 607 real usages write the object - the schema's string branch is removed - spec.version is the same setting as tf-version (both set is an error) and is exclusive with binary; when skips the grain and every dependent; auto-tag defaults to true; namespace vs target-namespace; the launch-form input types are spelled out Every property now has a description (136 had none): the Environment-as-Code section, all backend and template-storage fields, kubernetes and docker runner permissions, runner overrides, approval conditions, shell file entries, every input-source override, the resource-selector stubs, and the customization UI contract. The UI's layout.exclude.resources (resources_layout_section.tsx) is added to the schema with a fixture. Corpus unchanged: 528 blueprints, 412 fully clean, 0 crashes, 727 findings all KNOWN, 0 NEW. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o longer aborts the parse
The schema dropped its scalar branch for grain spec target (the server declares
GrainSpecYaml.Target with a plain [YamlMember] - no short syntax, no converter),
so the language-server tree model follows: target is GrainSpecTargetObject only,
declared like agent. tests/test_spec2_server_parity.py pins both layers.
Making that change exposed a pre-existing parser defect: a scalar given to any
object-only property (target, agent, backend, scripts, ...) raised a
ParserError, which server.py turned into a single diagnostic while discarding
the whole tree - every other diagnostic in the file vanished. The parser now
records a node error ("Scalar cannot be accepted here. Object expected"), keeps
the property on the stack the way the accepted-scalar path does, and continues
with the next key, so the rest of the file is still validated.
docs/spec2-language-support.md: the shorthand examples no longer cite target
(only [YamlShortSyntax] properties have one); a note on why the parser records
rather than raises; and section 11 recording the descriptions pass - the 13
corrected claims, the 136 newly described properties, and the rule that any
description stating a default, an "ignored"/"required"/"only" claim or a number
must cite its server source or be re-verified at the next sync.
Corpus unchanged: 528 blueprints, 412 fully clean, 0 crashes, 727 findings all
KNOWN, 0 NEW.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The schema is public: the Torque web UI fetches it live from this repository's master branch and shows descriptions on hover, and anyone can read the raw file. Its text must read as product documentation for blueprint authors, not as notes between engineers. Until now the $comments deliberately cited server validator classes, error codes, source files and a corpus count as provenance. Of the 26 comments, 13 are deleted (the description or the shape already said it) and 10 are rewritten in plain language stating the rule and why; the three that only describe JSON Schema mechanics stay. Eight descriptions are reworded - "the server" becomes "Torque", an instruction aimed at a UI operator becomes a description of the field, two ungrammatical launch-form texts are fixed. Nothing is lost: docs/spec2-language-support.md gains section 12, a 26-row table mapping each schema rule to the server class, method or error code that defines it, and section 11's rule now points there. tests/test_schema_public_voice.py enforces the policy - no server class or method names, error codes, source paths, internal tooling or repository names, corpus figures, or R&D voice anywhere in the schema's titles, descriptions and comments, and no JSON Schema jargon or "the server" in descriptions. The schema-fixtures CI job runs it alongside the fixture corpus on modern Python. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Backend definition carried two "$comment" keys in one object - one about the closed key set, one about the per-type required fields. JSON parsers accept that and keep the last one silently, so json.load, Draft7Validator.check_schema, every test and the corpus run all passed while any consumer that keeps the first occurrence saw different text. The two comments are merged into one. tests/test_schema_syntax.py makes this class of defect visible: it parses the schema with an object_pairs_hook and fails on any duplicate key in any object, and also pins UTF-8 without BOM, LF endings, the declared draft-07 with a meta-validation pass, and that every $ref resolves to a definition. The schema-fixtures CI job runs it on modern Python alongside the fixture corpus and the customer-facing wording gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e secrets, release.yml quoting On a pull request super-linter lints every file changed against master, so PR #184's "lint and format" job failed across nine linters that the per-push runs never exercised. Everything was reproduced locally at the tool versions super-linter v4 pins, with positive controls, and is now clean. The one finding that mattered: the corpus sanitizer's self-test planted its synthetic secrets as literals (an AWS key, a GitHub token, a Slack token, a Google key, a JWT, a PEM header), and gitleaks flagged the token twice - any scanner or GitHub push protection would. The fakes are now assembled at runtime by concatenation, so no literal in the public repository matches a secret pattern, and the self-test still exercises every detector (94/94). pylint reported a "syntax error" in the same file because it parses with type comments enabled and an example comment line read "type: password"; the comment is reworded. Three pre-existing pylint false positives in old server code are disabled narrowly, inline, with the reason stated. Formatting: black and isort (profile black) over the 20 Python files in the change, with .github/linters configs so CI and local runs agree; flake8 fixes by hand where black does not reach (unused imports and locals, a raw-string regex, ambiguous names, wrapped long lines; the sanitizer's typing imports are kept - its PEP 484 type comments use them). Two jscpd clones factored into helpers. Markdown: fence languages added, emphasis style and table-length settings configured, terminology aligned with textlint. release.yml: shell expansions quoted and the disabled ::set-output replaced by $GITHUB_OUTPUT. Behaviour unchanged: 173 tests green three times; 528 sanitized blueprints still 412 fully clean, 0 crashes, 727 findings all KNOWN, 0 NEW. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.