Skip to content

chantier: audit remediation — confirmed findings #142

Closed

Description

@LeadcodeDev

An end-to-end audit of the six crates (71,627 lines) produced 180 findings across 15 subsystem auditors. Each finding cites an exact file:line, a minimal verbatim snippet, and a concrete trigger scenario; most carry a reproduction.

Findings rated Critical/High/Medium were then put through independent adversarial verifiers — one reading the code for a guard that would invalidate the claim, one attempting the reproduction, one checking the repository's own documented intent — all briefed to refute by default.

Scope of this chantier

In scope — the 98 findings that survived verification. Every item below is backed by a verifier majority and, in most cases, a reproduction.

Out of scope, deliberately:

  • 75 findings that no verifier challenged (the verification wave was cut short). They are credible leads, not established facts. Acting on them risks "fixing" correct code — 4 findings were refuted outright, so the false-positive rate is not zero. The Architecture, Code quality, Security and Concurrency domains have no verified finding at all and are therefore untouched here.
  • Supply-chain analysis (dependencies, CVEs, CI/CD, crates.io publication), excluded from the audit brief.

The dominant failure mode

Not crashes — silent divergence. rustmotion validate, which CLAUDE.md makes mandatory before delivery, answers "Valid scenario" on captions that run off the device, on CJK text rendered as empty boxes, on a world view whose positions it computes 860px away from what gets painted, on padding: {"horizontal":100} that evaluates to zero, and on five scenarios that panic or hang the renderer. A wrong video produced without any error is worse than a crash.

A second, related motif: the skill rules — the contract a generating LLM reads — prescribe knobs the engine ignores (amplitude on float_3d, delay/duration on pulse/float/shake/spin, loop on keyframe effects, style.wrap, margin-left).

Workstreams

Each lands as its own PR into chantier/audit-remediation, which merges to main as a single final PR.

  • Encoding — ffmpeg argument order, diagnosability, remaining findings (11)
  • Geometry validation — leaks and false negatives in the overflow guard (11)
  • Text & fonts — measure/paint divergence, glyph fallback (10)
  • Schema & serde — the contract with generating LLMs (10)
  • Components — painters must never panic on degenerate input (9)
  • Animation — presets must honour their configuration (8)
  • Transitions & world — frame accounting and camera continuity (8)
  • Layout & taffy — unit resolution and intrinsic measurement (8)
  • Dataviz — degenerate scales, labels outside the measured box (8)
  • HTML & CLI — commands must not lie about their exit code nor destroy source files (8)
  • Paint pass — decoration order, opacity composition (7)

Activity

  1. self-assigned this
    on Aug 8, 2026
  2. LeadcodeDev commented on Aug 8, 2026

    @LeadcodeDev
    OwnerAuthor

    Workstream state — round 2 (parallel)

    Three workstreams dispatched concurrently. Recorded here so the orchestration survives a session boundary.

    Frozen contracts — read-only for every workstream

    • Component enum and its dispatch arms — crates/rustmotion-components/src/lib.rs
    • RustmotionError — crates/rustmotion-core/src/error.rs
    • The Painter trait and PaintCtx — crates/rustmotion-core/src/traits/painter.rs
    • Every public schema type under crates/rustmotion-core/src/schema/

    A workstream needing a line in any of these reports the exact line instead of writing it; the orchestrator applies them all in one pass at integration.

    File partition — writes are disjoint, reads may overlap freely

    Workstream Owns Findings
    batch crates/rustmotion-cli/src/commands/batch.rs 4
    html crates/rustmotion-html/src/{element,lib,style}.rs 4
    dataviz crates/rustmotion-components/src/{chart/bar,chart/mod,heatmap,progress,sparkline,table,treemap}.rs 8

    Orchestrator-owned convergence points, written by nobody else: every lib.rs used for registration, mod.rs, Cargo.toml, Cargo.lock (regenerated, never hand-edited).

    rustmotion-html/src/lib.rs is owned by the html workstream rather than treated as a convergence point: it carries the transpiler itself, and no sibling touches it.

    Execution regime

    Concurrent workstreams in a shared checkout. Sub-agents perform no git operations and run tests scoped to their own crate only — a full-suite run mid-round would test a tree carrying siblings' unfinished work. The orchestrator runs fmt + clippy + the full suite once, after integrating all three.

    Each agent gets its own CARGO_TARGET_DIR: cargo locks the target directory, so a shared one would serialize exactly the parallelism the partition just bought.

    Decision log

    • Only the 98 verified findings are in scope. The 75 that no verifier challenged are excluded: acting on them risks "fixing" correct code, and 4 findings were refuted outright, so the false-positive rate is not zero. Rejected alternative: implement everything and rely on tests to catch mistakes — tests written from a wrong premise confirm the wrong premise.
    • Supply-chain analysis excluded from the audit brief; 3 findings dropped accordingly.
  3. LeadcodeDev commented on Aug 8, 2026

    @LeadcodeDev
    OwnerAuthor

    Round 2 integrated — 6 workstreams landed, ~35 confirmed findings closed

    chantier/audit-remediation now carries six squashed workstreams. Full suite on the integrated tree: 752 passed, 0 failed; fmt and clippy --workspace -- -D warnings clean.

    PR Workstream Findings
    #143 Encoding — ffmpeg argument order, diagnosability 2
    #144 Components — five painters aborting on valid input 5
    #145 CLI — helper commands destroying user files 4
    #146 batch — exit code, geometry bypass, path traversal 4
    #149 HTML dialect — silent losses become named errors 4
    #150 Dataviz — start_at, layout box, degenerate scales 8 + 5

    Same defect, other files

    The dataviz workstream owned seven files; three of its findings existed identically in files it did not own. Rather than leave a known bug behind a green suite, they were applied at integration: progress_at ignoring start_at in gauge.rs and dot_map.rs, the filter_map label drift in chart/line.rs (×2) and chart/waterfall.rs, and the flat-series divisor in stat.rs.

    This is what the report-don't-write rule bought: a sub-agent that widened its own scope would have produced a merge conflict; one that stayed silent would have shipped a partial fix.

    Follow-up recorded, not done

    progress_at now exists in seven near-identical copies and the flat-series scale in three. The rule of three is well past — a crate-level ramp_progress(start_at, duration, time) would stop the next component reintroducing the defect. Deliberately deferred: an eight-file refactor would have buried sixteen bug fixes underneath it.

    Process note for whoever picks this up

    Merging #146 with --delete-branch removed the base branch of a PR stack, which closed the dependent PRs (#147, #148) irrecoverably — a closed PR whose base is gone cannot be reopened or retargeted. They were recreated as #149 and #150 from the same commits. Do not use --delete-branch on a PR that is the base of another.

    Remaining

    Validation géométrique (11) · Texte & polices (10) · Schéma & serde (10) · Animation (8) · Transitions & world (8) · Layout & taffy (8) · Paint pass (7) · Encoding remainder (9).

  4. LeadcodeDev commented on Aug 8, 2026

    @LeadcodeDev
    OwnerAuthor

    Workstream state — round 3 (parallel, 4 workstreams, 36 findings)

    Four mutually disjoint domains. The remaining four (Geometry validation, Schema & serde, Animation, Layout & taffy) all collide with each other through intrinsic.rs, scene.rs, animator.rs and paint_pass.rs, so they are serialized into a later round rather than forced parallel.

    File partition — writes disjoint, reads free

    Workstream Owns Findings
    encodage rustmotion/src/encode/{audio,video_audio,video/{ffmpeg,formats,h264}}.rs, rustmotion-cli/src/commands/{still,batch}.rs 11
    texte rustmotion-components/src/{caption,intrinsic,codeblock/{highlight,render}}.rs, rustmotion-core/src/engine/{renderer/{fonts,text},text/cosmic}.rs 10
    transitions rustmotion/src/engine/{world,render/scene}.rs, rustmotion/src/encode/video/tasks.rs 8
    paint rustmotion-core/src/engine/paint_pass.rs, rustmotion-components/src/legacy_dispatch.rs, rustmotion/src/engine/render/background.rs 7

    Frozen contracts unchanged from round 2: Component enum + dispatch, RustmotionError, Painter/PaintCtx, public schema/ types. Orchestrator-owned: registration lib.rs, mod.rs, Cargo.toml; Cargo.lock regenerated, never hand-edited.

    intrinsic.rs is assigned to texte for this round. Layout & taffy and Geometry validation also have findings in it — deliberately deferred rather than split, since two workstreams editing the same measurement code is the definition of a wrong decomposition.

    Execution regime

    Unchanged: shared checkout, no git operations by sub-agents, per-agent CARGO_TARGET_DIR, tests scoped to the owning crate. Orchestrator runs fmt + clippy --workspace + full suite once after integrating all four.

  5. LeadcodeDev commented on Aug 10, 2026

    @LeadcodeDev
    OwnerAuthor

    Clos : les 98 constats du périmètre sont corrigés et mergés dans main le 2026-08-10 via la PR #163 (rebase merge — les 14 commits de lots sont préservés individuellement, la bisection reste utilisable).

    CI sur main : verte, les trois jobs.

    Les 14 lots

    PR Lot
    #143 Ordre des entrées ffmpeg
    #144 Painters qui abandonnaient sur entrée valide au schéma
    #145 Commandes utilitaires qui détruisaient des fichiers
    #146 batch qui ne validait pas comme render
    #149 Pertes silencieuses du dialecte HTML
    #150 Dataviz : start_at, boîte de layout, échelles dégénérées
    #151 Encodage : chaque format rend les frames qu'il annonce
    #152 Vue world : téléportation caméra et budget de frames
    #153 Texte : fallback de glyphe, unités réelles, polices demandées
    #154 Paint : ombres, couche d'opacité bornée, banding réel
    #156 Layout : unités de viewport et cascade CSS
    #158 Animation : les boutons atteignent enfin le solveur
    #160 Le validateur appelle le moteur au lieu de le reforker
    #161 Puits silencieux de désérialisation

    Vérification

    cargo test --workspace : 21 cibles, 0 échec. fmt --check et clippy --workspace -D warnings propres. 6 nouveaux fichiers de test. 7 des 8 exemples passent rustmotion validate — le huitième est #157, dont l'échec est prouvé préexistant à main.

    Ce qui reste, suivi ailleurs

    Quatre défauts trouvés pendant la remédiation, tous hors du périmètre de cette issue :

    Et les 78 constats non vérifiés restent non traités, comme annoncé dans le périmètre d'origine. 12 relèvent de la chaîne d'approvisionnement, explicitement exclue. Les 66 autres — surtout concurrence/performance, qualité de code et architecture — demandent une nouvelle passe de vérification adversariale avant tout correctif : le taux de faux positifs n'est pas nul, 4 constats ayant été réfutés d'emblée.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions