Skip to content

feat(site): unify website features with the X-Mech catalogue - #2021

Draft
realmarcin wants to merge 8 commits into
mainfrom
feat/unify-site-features-20261025
Draft

realmarcin wants to merge 8 commits into
mainfrom
feat/unify-site-features-20261025

Conversation

@realmarcin

@realmarcin realmarcin commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Brings the five website features that the 2026-10-25 X-Mech site review left partial on TraitMech up to the fleet's wave-1 unified standard. Every change is in the owning renderer or template (scripts/render_trait_pages.py, scripts/render_discussions_shell.py, src/traitmech/templates/). pages/ and the discussions shell were then regenerated with just gen-pages and render_discussions_shell.py. No curated record, schema, embedding or vendored governance file is touched.

Most of the 1,130 record-page diffs are the footer licence and skip-link markup, which every page carries. main was merged in three times (after #2013, #2020 and #2026), and each time pages/ was regenerated rather than resolved by side, as the curation playbook says. After #2026, regeneration changed seven record pages and schema.html relative to git's textual auto-merge. If another curation PR lands first, run just gen-pages again in the same way.

Key Before (2026-10-25 review) Change Test Browser evidence (local, 390 and 1280 px)
evidence partial: 870 of 1,130 record pages showed PMID/DOI/URL tokens as plain text (synonym sources, curation-history changes, notes, rationale, graph descriptions, 66 :ref{doi=...} directives) One autolink filter escapes the text and then links only PMID, PMCID, DOI and http(s) tokens whose target reference_url() accepts. Trailing punctuation and unbalanced closing brackets are trimmed, and the visible text is unchanged. Because research previews are Markdown, a token also ends at ] and drops trailing * and backtick, so link syntax, bold and inline code never reach an href (review finding F1). It is applied to notes, snippets, curation changes, session events, discussion text, graph/edge descriptions, canonical-example notes, dataset descriptions, pathway basis and research previews. Synonym sources use the existing citation() macro. test_autolink_* (25 cases, including unsafe schemes, attribute break-out, and Markdown link, bold and inline-code shapes), test_research_preview_links_carry_no_markdown_debris (every rendered research preview), test_free_text_references_link_in_every_record_context, test_real_synonym_source_from_the_review_is_linked A corpus audit of all 1,130 rendered record pages finds 0 pages with unanchored PMID/DOI/URL tokens (baseline render of main: 870 pages, 3,474 tokens). There are 0 doi.org hrefs containing %5D, %2A or %60 (4 before the F1 fix, on syntrophy, halophilic, axially filamented and flask shaped). A DOM walk finds 0 loose tokens on 6 record pages. The biofilm formation synonym source DOI:10.1038/nrmicro.2016.94 is now a link.
term_links partial: traitmech: node groundings unlinked; kg-microbe node CURIEs unlabeled and unlinked; MICRO xrefs and rdfs predicates plain text New term_link filter. A CURIE naming a TraitMech record (traitmech: or METPO:) opens that record's page. The page's own id resolves like its Identifier card (METPO to w3id, a local id to its YAML) instead of linking to itself. kg-microbe nodes show their label and link: parent-proxy nodes go to the parent record, direct matches to w3id. MICRO and rdfs were added to the resolver map. Other well-formed prefixes fall back to https://bioregistry.io/<CURIE>. A traitmech: id with no record page stays visible but unlinked, so it never becomes a 404 or #. test_curie_resolver_map_with_registry_fallback (8), test_own_record_groundings_open_their_record_page, test_kg_microbe_nodes_show_the_label_and_link_the_record, test_direct_metpo_node_resolves_to_the_authority_not_a_self_link, test_cross_references_use_the_resolver_map Unlinked CURIE <code> on record pages went from 2,188 traitmech + 1,525 METPO + 507 rdfs + 185 NCBITaxon + 4 MICRO to only the header's own identifier, which is linked from the Identifier card on the same page. There are 0 href="#" anchors. With the keyboard, Tab+Enter on the kg-microbe node of biofilm formation opens phenotype. Clicking an own-record grounding on psychrotolerant opens cold_shock_response, traitmech:000206.
schema_docs partial: only the raw LinkML YAML was linked docs/SCHEMA.md is not current with the schema: it lacks causal_graphs, canonical_examples, discussions and datasets, and was last touched 2026-09-09 while the schema changed 2026-10-07. So it is not linked. Instead, pages/schema.html is generated by the renderer from LinkML SchemaView: all 15 classes with induced slots (range, cardinality, description, pattern) and all 15 enums with permissible values. Range names link to their class or enum anchors, and the source YAMLs and schema licence are linked. Schema prose is shown unlinked because its identifiers are format examples such as PMID:12345678. "Schema" in the nav, the landing Data card and every footer now point to it, and the footer keeps a YAML link. The page carries no clock, and the pages/ staleness gate checks it. test_schema_reference_documents_every_class_slot_and_enum, test_schema_page_matches_the_committed_render_and_anchors_resolve (compares the render with the committed pages/schema.html, which a separate gen-pages process wrote, above the stamped footer; F3), test_schema_link_is_visible_from_every_page_type (6) Reached from the landing by keyboard (Tab, Enter): 15 class sections, 15 enums, 24 TraitRecord rows, 0 dangling in-page anchors. At 390 px the slot table scrolls inside its named region ("TraitRecord slots"), with a visible focus outline and arrow-key scrolling. A theme chosen on this page carries to the landing.
license partial: template footers linked external deeds; the discussions index (linked from 640 record pages) stated no licence Footers now use the fleet wording taken from LICENSE: "Project data: CC BY 4.0 (LICENSE-DATA). Code: BSD-3-Clause (LICENSE-CODE). Third-party source material keeps its own terms (licence scope, LICENSE)". This is in base.html and the landing. The discussions shell injects the same footer between its markers, guarded by a new </body> anchor check. test_every_page_type_footer_states_the_repository_licences (9 page types), test_footer_licences_match_the_licence_files (3 footers: base, landing, discussions shell; the expected words are derived from LICENSE, LICENSE-DATA and LICENSE-CODE, then read from the rendered footer; F2), test_discussions_index_carries_the_licence_footer_once, test_committed_discussions_index_is_current_with_the_shell All three licence-file links are in the footer of 15 checked pages, including the discussions index
responsive partial: the Text map and Sequence map drew a fixed 520 px SVG: 553 px page width at 390 px, 793 px after narrowing from 1,280 Both now use the viewBox plus width:100% pattern that umap.html and graph.html already use. Related issue #1927 names only umap/graph, which already fit on main, so this PR leaves its status to the owner. All four map data tables scroll in a role="region" with tabindex="0" and an aria-label, with a focus outline. Also found and fixed: below 860 px the facet sidebar stacks above the content but stayed position: sticky. While scrolling it covered the catalogue table on Browse (704 of 844 px of the viewport) and on category pages, and the plot on the maps. It now scrolls away with the page, and it stays sticky on wide screens. test_map_plot_scales_to_its_container (4), test_map_data_table_scrolls_in_a_named_focusable_region (4), test_facet_sidebar_is_not_pinned_over_content_on_narrow_screens 0 horizontal overflow on 15 pages at both widths (landing, browse, category, schema, 4 records, discussions, 4 maps). After narrowing each map from 1,280 to 390 px, the SVG is 324 px wide with no overflow. At 390 px, the point 150 px from the top while scrolled into the table or plot is no longer under the sidebar on Browse, the map or a category page (it was before). The keyboard opens the map table disclosure and focuses its region, which holds 1,130 rows.

Catalogue candidate features (fleet candidate review, 2026-10-25)

Key Before (candidate rating) Change Test Browser evidence (local, 390 and 1280 px)
skip_link missing: the first Tab stop was a logo or nav link on the landing, Browse, records, category pages and all four maps base.html and the landing start with "Skip to main content", shown only while focused, targeting <main id="main-content" tabindex="-1">. The discussions index already had one from the shared claw template. test_first_tab_stop_skips_to_main (10 page types: landing, Browse, category, record, 4 maps, schema, 404) On 10 pages at both widths, the first Tab focuses the visible skip link and Enter moves focus into <main>.
not_found_page missing: GitHub's generic 404 at /TraitMech/, /pages/ and /pages/traits/ecology/ just gen-pages now also writes a themed 404.html at the repository root, which is this branch-built site's publishing root. It is rendered from templates/404.html (base shell, search into Browse, links to home, Browse, schema and All Mech projects). Every link, the stylesheet and the script are site-root-absolute (/TraitMech/pages/...), because GitHub serves the file at whatever missing path was asked for. It carries no corpus stamp, so curation PRs never rewrite it. test_not_found_page_is_current_and_every_site_link_is_root_absolute: the committed file equals a fresh render, every site link is /TraitMech/-absolute and names a tracked file, and there is no corpus stamp. A local server that answers missing paths the way Pages does (404 status, root 404.html) was hit at /TraitMech/no-such-page, /TraitMech/pages/no-such-page-q7.html and /TraitMech/pages/traits/ecology/nope.html. Each returns 404 with the styled page and theme toggle, all 15 site links answer 200, and there is no overflow. Searching "pathogen" from the deep 404 lands on Browse showing "9 of 1130 records shown".
curation_notice partial: the landing says "seeded from METPO and curated incrementally", but nothing explained the four statuses The landing gains a "Curation status" section listing each status with its record count. Each record page puts a "What this status means" disclosure beside its status. Every definition is quoted from the schema's MappingStatusEnum at render time and never paraphrased, with a link to the schema page's status definitions. test_record_page_explains_its_status_in_the_schemas_words (4 statuses, compared with a live SchemaView), test_landing_states_what_each_status_means_with_its_count, test_committed_pages_carry_the_curation_notice The landing lists SEEDED 38, PROPOSED 615, REVIEWED 427 and DEPRECATED 50 records with their schema definitions, with no overflow. On animal pathogen, the keyboard reaches and opens the disclosure, which shows "Curator has signed off on label / definition / parents."

Candidate features left out or partial: whether REVIEWED means a human review is not stated on the site. The curation playbook's checklist says "REVIEWED after human curator signoff". However, every recorded curation_history event in the corpus (6,255) names claude, codex or a script, including the 192 changes that read as promotions to REVIEWED. The repository does not settle the question, so the site quotes only the schema ("Curator has signed off on label / definition / parents."). It also says each record page lists every curation event with its curator. The repository owner should decide the wording; the question is tracked in issue #2027.

Sibling-Mech link guard (the standard's cross_mech_links repair rule):

  • test_sibling_links_in_renderer_sources_use_verified_routes and test_every_sibling_link_in_the_built_site_uses_a_verified_route allow only the standard's verified search and record routes, PathwayMech's pages/records/ among them, plus the exact site roots used by the footer's sibling links. The second test also covers the /404.html page and the discussions index.
  • The current site passes. Its only deep sibling links are 15 distinct PathwayMech records, all of which answered 200 on 2026-10-25, as did the four sibling site roots and the suite directory.
  • Mutation-checked three ways, each of which goes red:
    • a footer link moved off a site root;
    • the PathwayMech route mistyped in the graph adapter;
    • an unverified sibling search injected into a built page.

Mutation check: 17 targeted reverts of individual changes, each run in a fresh pytest process, each turned at least one of the new tests red. Two tests were first green under a mutation and were then tightened: the synonym DOI is also cited elsewhere on the page, and the footer also links the schema page. The review fixes add six more mutations:

  • tokens no longer stop at ];
  • * and backtick no longer trimmed;
  • LICENSE and LICENSE-CODE consistently changed to MIT while the footer still says BSD (only the new licence test goes red);
  • footer drifts to "CC BY-SA 4.0";
  • schema template edited without regenerating pages/;
  • schema classes reordered.

The candidate features add nine more:

  • skip link removed from base.html;
  • skip link removed from the landing;
  • <main> id removed;
  • 404 links made relative;
  • 404 given the corpus footer;
  • status disclosure removed;
  • status meanings paraphrased by the renderer;
  • landing notice removed;
  • the renderer stops passing status meanings, with pages/ regenerated.

All of them turn the new tests red.

Independent review follow-ups

Checks run (final, at 5ba96634: everything above, the #2033 autolink fix, and main at bc1dfa79)

Main has since gained #2029 and #2032. Between them they touch only reviews, workflow records, INFRA_TASKS.md, history records and scripts/audit_evidence_snippets.py with its test. They change no renderer source, data or generated output, so they were not merged in.

  • autolink merges two parenthesised URLs joined without whitespace into one wrong link #2033 fix (5ba96634): a URL or DOI token now ends at the first ) that closes no ( opened inside it, and scanning resumes after the token rather than the regex match, the approach of the PathwayMech port.
    • Covered by test_autolink_ends_a_token_at_a_parenthesis_it_never_opened, with 4 cases: two parenthesised URLs joined by -, see (https://a/x), (https://b/y), https://en.wikipedia.org/wiki/Foo_(bar) kept whole, and (see 10.1016/0005-2744(70)90044-4).
    • Both halves are mutation-checked: dropping the cut or the resume each turns the joined-URL case red.
    • just gen-pages after the fix changed no committed page.
  • just qc: exit 0 (8.7 min). This includes ruff, validate-strict, audit-discussions-data with render_discussions_shell.py --check, and audit-derived-reports (OK pages/, all reports current).
  • Focused tests: tests/test_unified_site_features.py plus the existing render and discussions tests: 199 passed. The unified-features file alone: 94 passed.
  • uv run pytest tests/ -q after uv sync --extra dev, the CI install step, at d3879590, before the autolink merges two parenthesised URLs joined without whitespace into one wrong link #2033 change to autolink() and its tests: 4,966 passed, 1 failed. The failure is test_record_review_contract.py::test_every_saved_structured_bundle_is_valid, which needs review-base commit cd2380c02edc in the local object store. The commit exists on GitHub, and CI fetches it with fetch_record_review_bases.py --apply before pytest. This run skipped that network step. CI passed this test at an earlier head of this PR.
  • Corpus audits of all 1,130 rendered record pages:
    • 0 pages with an unanchored PMID/DOI/URL (main: 870).
    • doi.org hrefs containing %5D, %2A or %60: 0 (4 before the F1 fix).
    • Of all rel="nofollow noreferrer" hrefs, 0 carry Markdown debris.
    • 0 href="#" anchors.
  • Browser, at 390 and 1,280 px with keyboard and Tab order, at d3879590 (site output is unchanged since):
    • The general harness passed 58 of 58 checks; the only console error is the browser's automatic /favicon.ico request (404).
    • The candidate-feature harness passed 31 of 31.
    • The four previously broken research-preview pages pass.
  • kg-microbe-site check --mech traitmech, run from claw with TRAITMECH_ROOT set to this worktree: clean, 1,147 pages.

Left out, and why

  • docs/SCHEMA.md is stale but was not edited here. It is a hand-written Markdown page outside this change's tooling. A follow-up should replace its table with a pointer to pages/schema.html.
  • The record header's own identifier stays plain <code>. The same identifier is linked in the Identifier card beside it, and linking both would add a duplicate tab stop.
  • The discussions index still has no build stamp or URL state. build_provenance and shareable_state are present on the main surfaces and are not in this wave's scope for that page.
  • just qc rewrites the tracked reports/proposal_citation_audit.tsv. The committed copy is already stale against main's data (11 rows, for example gao_her_system after the Cycle 4 Stage 5 record curation). This is pre-existing, not part of this change, and is excluded from the commit.

Review

Two independent read-only reviews: the first (head 22ee3d47) found four issues, filed as #2022 to #2025; the delta review (head d3879590) verified the fixes with 14 targeted reverts and found three nits, filed as #2030 and #2031 (the third is the owner decision already in #2027).

Fixes #2022
Fixes #2023
Fixes #2024
Fixes #2031
Fixes #2033

Filed, not addressed by this PR: #2025 (TaxonMech links for canonical-example taxa, a new cross-Mech relation), #2027 (whether REVIEWED means human review), #2028 (stale tracked citation-audit report), #2030 (guard test coverage of dashboard pages).

🤖 Generated with Claude Code

realmarcin and others added 2 commits October 25, 2026 14:24
The 2026-10-25 X-Mech site review left five catalogue features partial on
TraitMech. This brings each to the fleet's wave-1 standard in the owning
renderer and templates, then regenerates pages/ and the discussions shell.

- evidence: 870 of 1,130 record pages showed PMID, DOI or URL tokens as plain
  text (synonym sources, curation-history changes, notes, rationale, graph
  descriptions, ':ref{doi=...}' directives in snippets). A single autolink
  filter escapes record text and links only tokens whose target
  reference_url() accepts; synonym sources use the existing citation macro.
- term_links: traitmech: and METPO: groundings that name a TraitMech record
  now open that record's page (the page's own id resolves like its Identifier
  card), kg-microbe nodes show their label and link their record, MICRO and
  rdfs join the resolver map, and other prefixes fall back to bioregistry.io.
  A local id with no record page stays unlinked rather than becoming a 404.
- schema_docs: docs/SCHEMA.md has fallen behind the schema (no causal_graphs,
  canonical_examples, discussions or datasets), so the site no longer points
  at raw YAML only: pages/schema.html is generated from LinkML's SchemaView
  (every class with its induced slots, every enum) and linked from the nav,
  the landing Data card and every footer.
- license: footers now state the licences exactly as LICENSE, LICENSE-DATA
  and LICENSE-CODE do and link those files; the discussions index, which
  republishes record text, gets the same footer through its shell.
- responsive: the Text and Sequence maps drew a fixed-width SVG that
  overflowed a 390 px viewport; they now scale like the PaCMAP and graph
  maps, and each map's data table scrolls in a named, focusable region.
  Below 860 px the facet sidebar, stacked above the content, stayed
  sticky and covered the catalogue table or map plot while scrolling (up to
  704 of 844 px on Browse); it now scrolls away with the page.

Output stays deterministic (no clock), so the pages/ staleness gate holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained a curation PR (#2013) after this branch was cut. Per the
curation playbook, the three conflicted record pages were cleared and all of
pages/ was regenerated with just gen-pages rather than resolved by side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
realmarcin and others added 3 commits October 25, 2026 15:12
…at they claim

Review of this PR found three gaps.

The deep-research previews are Markdown shown verbatim, and the autolink
token ran on through Markdown syntax: "[10.1038/x](https://doi.org/10.1038/x)"
has balanced parentheses, so the DOI swallowed "](https://...)" into its href,
and "**10.x/y**" or "`10.x/y`" kept the closing marks. Four committed pages
(syntrophy, halophilic, axially filamented, flask shaped) linked percent-encoded
debris such as .../nrmicro2166%5D(... . Tokens now end at "]" and drop
trailing "*" and "`" (no DOI or URL in the corpus contains a bracket), with
cases for link syntax, bold and inline code, plus a check over every rendered
research preview. pages/ regenerated: 0 doi.org hrefs with %5D, %2A or %60
(was 4).

The licence test read only the licence files. It now derives the footer words
from LICENSE, LICENSE-DATA and LICENSE-CODE and checks them against the
rendered base, landing and discussions footers, so a change to the files or
to a footer that the other does not follow turns it red.

The schema-page test called itself a determinism check but compared two
renders in one process. It now compares the render with the committed
pages/schema.html above the footer, which a separate gen-pages process wrote.

Fixes #2022, fixes #2023, fixes #2024

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the Cycle 5 Stage 4 review files (#2020). It touches no generated
output or rendering source, so nothing needed regenerating for the merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…notice

The fleet's catalogue-candidate review rated three features on TraitMech
below present.

- skip_link (missing): the first Tab stop on every page type was a logo or
  nav link. base.html and the landing now start with "Skip to main content",
  which is shown only while focused and moves focus into <main id="main-content">.
- not_found_page (missing): a mistyped address at any depth got GitHub's
  generic 404. GitHub Pages serves the publishing root's 404.html for every
  missing URL, and this branch-built site publishes the repository root, so
  `just gen-pages` now also writes /404.html from a template. Its links,
  stylesheet and script are site-root-absolute (/TraitMech/pages/...) because a
  relative link would resolve against the missing path. It carries no corpus
  stamp, so curation PRs never rewrite it, and a test checks that the
  committed copy is current.
- curation_notice (partial): nothing explained SEEDED, PROPOSED, REVIEWED or
  DEPRECATED. The landing now lists each status with its record count, and
  each record page puts a "What this status means" disclosure beside its
  status. Every definition is quoted from the schema's MappingStatusEnum at
  render time rather than paraphrased. Whether REVIEWED implies human review is
  left unstated: the curation playbook says "REVIEWED after human curator
  signoff", but every recorded curation event names claude, codex or a
  maintenance script, so the repository does not settle it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
realmarcin and others added 2 commits October 25, 2026 16:04
The fleet's unified standard asks every Mech for a cheap guard: each link to
a sibling Mech must use a route the fleet verified, because tests cannot call
the network and a guessed path on another site fails silently.

The new tests allow only the standard's verified search and record routes
(PathwayMech's pages/records/ among them) and the exact site roots that the
footer's sibling links use. They check every renderer template and script,
and every page of the built site, including /404.html and the discussions
index. The current corpus passes: its only deep sibling links are 15
PathwayMech records, and they answered 200 when checked on 2026-10-25.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main gained a curation PR (#2026) that edits seven trait records and their
pages. Git merged the generated pages textually, which mixes main's data with
this branch's templates. Per the curation playbook, pages/ was regenerated
with just gen-pages rather than trusted: seven record pages and schema.html
differed from the auto-merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
autolink() trimmed only an unbalanced closer at the very end of a token, and
resumed scanning after the whole regex match. In
"(https://a.example/x)-(https://b.example/y)" the parentheses balance across
both URLs, so the first link's href became
"https://a.example/x)-(https://b.example/y" and the second URL got no link of
its own.

A token now ends at the first ")" that closes no "(" opened inside it, and
scanning resumes after the token rather than the match, so what was cut off
is scanned again. This is the approach of the PathwayMech port. Balanced
parentheses inside a URL or DOI are kept, as in
https://en.wikipedia.org/wiki/Foo_(bar) and 10.1016/0005-2744(70)90044-4.
Regenerating pages/ changes nothing: no committed page has this shape.

Fixes #2033

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant