diff --git a/.agents/skills/puppetcontrol/SKILL.md b/.agents/skills/puppetcontrol/SKILL.md index 70c9ad6..dc9b76e 100644 --- a/.agents/skills/puppetcontrol/SKILL.md +++ b/.agents/skills/puppetcontrol/SKILL.md @@ -14,7 +14,7 @@ Read repo-root AGENTS.md, docs/TECHNICAL_CONCEPT.md, and docs/CHARACTER_IMPORT.m 5. Stage the proposal using `node dist/cli/index.js character landmarks stage --proposal `. Rev2D validates the isolated candidate and returns a work/overlay.png. **Actually inspect the overlay** before `character landmarks accept --candidate --reviewed`. Accepting replaces avatar.json with a backup and revokes all existing pose approvals. Never auto-pass --reviewed solely because a validator passed. 6. Build using node dist/cli/index.js character build (use --force if prior output exists). Inspect any geometryWarnings, particularly BODY_PIVOT_TOO_LOW and IRIS_LANDMARK_MISSING; these do not replace pixel inspection. Source/landmark/model/texture hashes are recorded in avatar-build.json and scene preflight rejects stale assets. Inspect every returned review/poses.png, turn.png, eyes.png and mouth.png. If incorrect, repair avatar.json and rebuild using --force; this resets the generated manifest and any edited scaffold. 7. To test optional drawn eye/mouth variants on a built actor, request edit masks, locally supply reviewed `variants/.png`, and explicitly import. With reviewed closed+half eye drawings, `character variants import eyes_closed eyes_half --experimental-short-fade --reviewed-art` shortens fade windows but makes poses change faster and may still ghost; it is a visual experiment, never a default or automatic quality fix. The import revokes prior approval and refreshes provenance only after re-rendering review sheets. -8. For a compatible imported avatar whose extracted lash warps look unnatural, optionally run `character eyes vectorize --experimental`. This creates a **new provenance-tracked vector eyelid model**, preserves source drawings, revokes existing approval, and renders real 30 FPS blink evidence. It uses the eye-mask vertices for a continuously deforming line, not alpha-switching. Derived rigs require fresh SHA-256-bound eyes/full/video evidence before `character approve`; if missing or stale, run `character eyes review ` and inspect the returned real frames. Neither evidence nor software can prove artistic acceptance or user consent. See `docs/VECTOR_EYELIDS.md`. +8. For a compatible imported avatar whose extracted lash warps look unnatural, optionally run `character eyes vectorize --experimental`. This creates a **new provenance-tracked vector eyelid model**, preserves source drawings, revokes existing approval, and renders real 30 FPS blink evidence. It uses the eye-mask vertices for a continuously deforming line, not alpha-switching. Derived rigs require fresh SHA-256-bound eyes/full/video plus 16-case head/gaze/expression evidence before `character approve`; if missing or stale, run `character eyes review ` and inspect the returned real frames. Neither evidence nor software can prove artistic acceptance or user consent. Also run `character qa eye-stress ` to render 16 actual combined blink/gaze/head/expression states and inspect both contact sheets. The current-model proof includes that sheet, but a validator cannot confirm its artistic quality; Satoko's independent MIT trial still shows closed-eye residue. See `docs/VECTOR_EYELIDS.md` and `docs/SECOND_STYLE_EYE_QA.md`. 9. For a deliberately non-blended eye-art experiment, `character variants import eyes_closed eyes_half --experimental-stepped-eyes --reviewed-art` selects only one complete eye appearance outside tiny threshold windows. It can remove ghosted doubled lashes but creates visible pose pops; do not call it seamless or automatically approve. See `docs/EYE_TRANSITIONS.md` and compare actual frames. 10. Before approving a verified imported rig, run `node dist/cli/index.js character qa blink ` to generate an ordered 25-frame actual eyelid transition contact sheet (`blink-eyes.png` and `blink-full.png`); inspect EyeOpen=0.19 and 0.69 and the real eye artwork seams. The QA report flags motion stalls but never passes visual approval. Also run `character qa blink --timed` for a 36-frame synthetic 30 FPS video and review the actual moving result. Different fade policies may produce identical frames when their transition windows fall between sampled timestamps; see `docs/EYE_TIMING.md`. 11. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA may an agent/user explicitly run `node dist/cli/index.js character approve --reviewed` (the flag is an acknowledgement, not evidence of review) to record neutral-pose approval. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction. diff --git a/README.md b/README.md index e4df2eb..f5159d1 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ The long-term goal: **generate artwork when necessary, animate and reuse it inde - Local PNG pixel and ordered-frame QA (full frame or explicit region) reports RGB activity, unexpected transparency, candidate motion jumps and changed-control stalls; these metrics never auto-approve a rig. - Import review can render a 25-frame parameter-sweep contact sheet and a separate **36-frame, 30 FPS synthetic timed-blink MP4** with motion warnings; see [Character Review](docs/CHARACTER_REVIEW.md) and [Time-domain Blink QA](docs/EYE_TIMING.md). The opt-in [stepped-eye experiment](docs/EYE_TRANSITIONS.md) removes sampled opacity ghosting but does not provide a seamless morph. The two modes even produced byte-identical frames in one real-time test. - Imported flat-image actors start with narrower editable head/gaze motion limits; see [Look Motion](docs/LOOK_MOTION.md). -- Optional **experimental continuous vector eyelids** replace warped raster lash outlines with thin mask-following Rev2D mesh strips while keeping the iris clipped. This improves one real Hana sample without automatic approval. A **current-model SHA-256-bound eye/full/video proof** is required before explicit approval; use `character eyes review ` to refresh it. See [Vector Eyelids](docs/VECTOR_EYELIDS.md). +- Optional **experimental continuous vector eyelids** use mask-following Rev2D strips instead of warped raster lashes. **Two independent MIT samples have been tested**: Hana improved, while Satoko still has visible eyelid/iris residue and remains unapproved. [See matched before/after frames](docs/media/satoko-vector-eye-comparison.png) and [second-style findings](docs/SECOND_STYLE_EYE_QA.md). `character qa eye-stress ` now renders 16 eye/gaze/head/expression cases; derived-rig approval requires these plus the current-model SHA-256-bound blink/MP4 sheets. Refresh via `character eyes review `. [Vector eyelid details](docs/VECTOR_EYELIDS.md). - Optional SHA-256 Rhubarb cue bindings can detect when audio or mouth-cue bytes are replaced after alignment; a bound scene fails validation/render instead of silently using stale lip sync. - Agent-readable CLI output, tests and CI. diff --git a/docs/CHARACTER_REVIEW.md b/docs/CHARACTER_REVIEW.md index 2f11fce..56a942c 100644 --- a/docs/CHARACTER_REVIEW.md +++ b/docs/CHARACTER_REVIEW.md @@ -51,7 +51,7 @@ For a genuinely deforming eyelid outline instead of alpha-switching whole-eye pi node dist/cli/index.js character eyes vectorize avatars/portrait --experimental ~~~ -It uses the existing Rev2D eye-white mask to deform upper/lower lashes continuously, keeps the iris clipped, preserves the previous model, writes new fingerprints, and revokes approval. Unlike the sprite transitions it simplifies the painted lashes; visually inspect the resulting video. **The generated model requires a model-hash-bound eye/full/MP4 review proof before approval.** If evidence is missing, run `character eyes review ` and examine all fresh frames. See [VECTOR_EYELIDS.md](VECTOR_EYELIDS.md). +It uses Rev2D's existing eye-white mask to deform upper/lower lashes continuously, preserves the previous model, writes new fingerprints and revokes approval. **Approval now requires a model-hash-bound blink MP4 plus full/eyes sheets and a 16-case head/gaze/expression eye sheet.** If evidence is missing or old, run `character eyes review `; visually inspect every case. The separate `character qa eye-stress ` command works on any provenance-tracked imported rig. The [second MIT character study](SECOND_STYLE_EYE_QA.md) still shows a greenish closed-eye trace and remains unapproved. See [VECTOR_EYELIDS.md](VECTOR_EYELIDS.md). ## 3. Inspect exact blink transition frames diff --git a/docs/IMPLEMENTATION_PLAN.md b/docs/IMPLEMENTATION_PLAN.md index 47891bf..475b615 100644 --- a/docs/IMPLEMENTATION_PLAN.md +++ b/docs/IMPLEMENTATION_PLAN.md @@ -87,6 +87,8 @@ - [x] Add a 36-frame, 30 FPS time-domain blink QA export with deterministic synthetic timing, unique Rev2D state caching, and JSON motion warnings. The actual short-fade and stepped 30 FPS frames were SHA-256 identical for this one schedule, showing threshold tuning alone is insufficient. Four mesh/overlay variants failed visual QA and were not integrated. - [x] Build an opt-in, provenance-verified **continuous vector eyelid** rig: replace distorted raster lash warps with curved vertex strips driven by Rev2D's existing eye mask, preserving iris clipping and source art. Inspect real 9-state, 30 FPS blink and head/gaze sweeps; 90-frame end-to-end H.264 export verified. The Hana experiment remains unapproved and artists may prefer their own lash styling. - [x] Make derived vector rig approvals require a SHA-256-bound current-model full/eye/30 FPS MP4 review package. Reject stale/missing/tampered files and renamed vector geometry; add `character eyes review` recovery without changing approval. Real licensed Hana test regenerated evidence successfully and remained unapproved. +- [x] Import independent MIT Satoko chibi artwork, annotate/build under the pinned Rev2D backend, apply and verify continuous vector eyelids; record actual good and bad frames versus original. The second style is compatible but green/gray lash leakage remains and it is **not approved**. +- [x] Add 16-case combined eye/gaze/head/expression visual QA with model-range validation and real engine renders, and require its hashed contact sheet alongside 30 FPS blink evidence in version-2 vector review proof. Regression tests and local stress renders on Hana + Satoko. - [ ] Produce and visually approve high-quality closed/half eye and mouth artwork; demonstrate seamless real rendered transitions across full blink playback and multiple character styles. The local proof drawing stays unapproved. ## First video milestone — reproducible usable slice (2026-11-07) diff --git a/docs/SECOND_STYLE_EYE_QA.md b/docs/SECOND_STYLE_EYE_QA.md new file mode 100644 index 0000000..ccec09a --- /dev/null +++ b/docs/SECOND_STYLE_EYE_QA.md @@ -0,0 +1,48 @@ +# Second illustration and combined-expression eye QA (2026-11-08) + +**Status:** second-style integration **PASS**; subjective character-quality approval **FAIL / NOT APPROVED**. Do not turn this into a generic one-click importer claim. + +## Independent licensed artwork + +The second illustration is **Satoko**, an original code-drawn chibi character from Henrik Hanssen's [anime-character-creator](https://github.com/rhedak/anime-character-creator), MIT-licensed, at pinned commit `8a52cd146445b1aee4d890df5e44bcf9102cceef`. This differs substantially from RevStudio's Hana: simple vector strokes, flat-colored hair, large green irises, tiny nose and mouth, full body and a short stylized face. It is not a recolor or re-render of Hana. The original PNG checkout and all generated model files remain under ignored `.cache/`; this repository includes only a small attributed **derived comparison crop**. + +The source `ref-out/satoko.png` (800×1000) was locally resized to 512×640 and imported without network processing. Coordinates were placed manually from the source-pixel grid and inspected on a real landmark overlay. Rev2D's initial landmark validator reported **zero warnings**, then the full build reported **two nonblocking PuppetControl triage warnings**: body pivot too low and body fade starting below the chest. Neither was suppressed; they need a proper body rig correction if we ever use Satoko for a full performance. + +## Actual imported vector rig + +- `character import` → source/spec created, unapproved. +- `character build` → model, textures, review PNGs and SHA-256 record, unapproved. +- `character eyes vectorize --experimental` → four continuous upper/lower eyelid strips, a separate model, fresh hash record and current-model blink review package, **approved:false**. +- `character qa eye-stress` → 16 real pinned Rev2D snapshots combining eye closure, asymmetric wink, gaze left/right/up/down, head yaw/pitch, happy/confused mouth, speaking and combined head+gaze+expression. Contact sheets and a JSON manifest written to a unique ignored QA directory. +- `character eyes review` → now regenerates model-bound `blink-eyes.png`, `blink-full.png`, 36-frame 30 FPS H.264 blink and the **16-case combined-control eye sheet**. The review proof is version 2, binding all four artifacts and exact model SHA-256. Old version-1 proofs are rejected; existing vector avatars can regenerate evidence without changing their approval. + +Sample contact sheets were checked visually in both Satoko and Hana. The combination cases are bounded diagnostics, not human-quality certification; static pictures do not prove motion continuity. + +## Actual image comparison + +![Satoko original raster eye deformation (top) versus experimental vector eyelids (bottom), six matched Rev2D states](media/satoko-vector-eye-comparison.png) + +The top row exhibits pointed lash tips, dark overlaps and iris trace during quarter closure. The vector bottom row reduces those artifacts and closes more cleanly. **Remaining defects:** green/gray iris residue beneath almost-closed lids, simplified or doubled fine lines at the lash boundary, and body/head deformation risks from the first landmark pass. The vector eye technique *runs* on a second independent art style, but the current output is **not good enough to automatically approve or describe as polished Live2D quality**. + +The original Hana sample has cleaner near-closed geometry under vector eyelids, but its facial art also remains experimental. The difference is why independent styles matter. + +## Reproducing locally + +Requires a pinned Rev2D checkout and FFmpeg in PATH. Reuse an existing imported, provenance-verified, **unapproved** character rig: + +~~~powershell +node dist/cli/index.js character qa eye-stress .cache/avatar-onboarding/satoko-second-20261108 +node dist/cli/index.js character qa eye-stress .cache/avatar-onboarding/vector-review-proof-20261108 +~~~ + +Each invocation creates `outputs/qa/eye-stress//` with `frames/frame-000.png` through `frame-015.png`, `expression-full.png`, `expression-eyes.png`, and `eye-stress.json`. No input model or approval bit is changed, and missing/limited controls are rejected. For a vector rig, `character eyes review ` regenerates the model-bound proof with both a neutral timed blink and expression stress case. + +## Full-scene integration check + +For renderer integration only, a separate **disposable copy** of Satoko was marked `approved:true` by a QA script. The original imported source project and its model-bound review package remain `approved:false`. On the QA copy, the normal `scene new` → `scene validate` → `scene render` workflow produced a real **3.000-second silent H.264 MP4 at 512×640, 30 FPS, 90 frames**, passing the project's FFprobe acceptance checker. Frames at 0, 1 and 2 seconds were also visually inspected, with the same cartoon character identity preserved. This tests software integration **only**, not approval or animation quality. + +## Still blocked + +- A human/artist must repair Satoko's closed-lid color/iris leakage and review gaze/face/neck. **It is not approved.** +- Both artworks still lack broad pose and full-speaking quality certification. We need a third, independently prepared character style before enabling any universal default. +- The proof verifies artifact bytes and model identity; it does **not** prove a human inspected a sheet or that it is aesthetically acceptable. Never automatically run `character approve --reviewed`. diff --git a/docs/THIRD_PARTY_NOTICES.md b/docs/THIRD_PARTY_NOTICES.md index 7cb3f46..259df1b 100644 --- a/docs/THIRD_PARTY_NOTICES.md +++ b/docs/THIRD_PARTY_NOTICES.md @@ -32,4 +32,36 @@ SOFTWARE. --- +## Anime Character Creator — Satoko illustration + +- Source: https://github.com/rhedak/anime-character-creator +- Pinned commit: 8a52cd146445b1aee4d890df5e44bcf9102cceef +- Original source file: `ref-out/satoko.png` (800×1000). +- Use: a locally resized, manually landmarked, **unapproved** second-style import for vector eyelid QA. Only an attributed derived eye comparison crop is included in `docs/media/satoko-vector-eye-comparison.png`; the original full picture and models are not bundled. +- Original license: MIT License, Copyright (c) 2026 Henrik Hanssen. + +### MIT License (Henrik Hanssen) + +Copyright (c) 2026 Henrik Hanssen + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +--- + Rhubarb, FFmpeg, and optional image models are not bundled into the PuppetControl project. Their binaries and weights have their own licenses; see project documentation and upstream sources. diff --git a/docs/VECTOR_EYELIDS.md b/docs/VECTOR_EYELIDS.md index 1fbd184..2aa9eae 100644 --- a/docs/VECTOR_EYELIDS.md +++ b/docs/VECTOR_EYELIDS.md @@ -31,7 +31,7 @@ It deliberately **revokes previous approval before publishing** the new model. I Old `review/poses.png`, `review/turn.png`, `review/eyes.png` and `review/mouth.png` may have been generated **before** vectorization. Their presence alone is insufficient evidence for the new eyelid rig. -The vectorize operation now writes `review/vector-eyelids-proof.json` only **after** a real timed Rev2D blink and full/eyes contact sheets have rendered. It copies those three artifacts to a unique `review/vector-eyes-/` directory. The proof binds the **exact model SHA-256** from `avatar-build.json` to the SHA-256 hashes of all three review artifacts. `character approve --reviewed` refuses vector geometry without a matching proof, even if its model filename was manually changed. +The vectorize operation now writes `review/vector-eyelids-proof.json` only **after** a real timed Rev2D blink, full/eyes contact sheets, and 16 actual combined head/gaze/expression renders have completed. It copies those four artifacts to a unique `review/vector-eyes-/` directory. The proof binds the **exact model SHA-256** from `avatar-build.json` to the SHA-256 hashes of all **four** review artifacts (full-frame blink sheet, eye blink sheet, timed MP4, and combined head/gaze/expression eye-stress sheet). This is proof format version 2; older version 1 proofs must be regenerated. `character approve --reviewed` refuses vector geometry without a matching proof, even if its model filename was manually changed. If FFmpeg fails, review files are lost or a generated package becomes stale, rerun: @@ -59,6 +59,12 @@ In this single test, the vector strips eliminate the sharp outward triangular la A disposable, unapproved imported Hana project successfully generated the model and hash record with `character eyes vectorize`. An **additional separate QA-only copy** was temporarily marked renderable to exercise the standard first-video path; its 3-second 512×640, 30 FPS, 90-frame H.264 output passed FFprobe acceptance. The real imported test source remains `approved:false`. +## Second art-style trial and combined-expression review + +The same vector rig process was applied to independently imported MIT-licensed **Satoko** artwork (Henrik Hanssen, pinned source commit in [third-party notices](THIRD_PARTY_NOTICES.md)). It generated a valid separate model, a 36-frame timed blink, 16 combined head/gaze/mouth/eye renders and a 90-frame full-scene integration video on a disposable QA-only renderable copy. **Visual approval still failed:** Satoko has a thin green/gray near-closed iris trace and simplified lash lines. Compare [matched states](media/satoko-vector-eye-comparison.png) and read [the full second-style report](SECOND_STYLE_EYE_QA.md). + +The optional `character qa eye-stress ` command exports a **4×4 / 16-case** full-face and eye-crop sheet and a JSON report of the exact parameter combinations. The model-bound review proof now requires the stress-eye sheet along with the two neutral blink sheets and timed MP4. This is additional evidence, not an automated beauty/quality threshold. + ## Remaining gates 1. Inspect frame-by-frame eyelid curves at 0, 0.1, 0.25, 0.5, 0.75, 1, plus head turns and iris extremes. No metric certifies the art. diff --git a/docs/media/satoko-vector-eye-comparison.png b/docs/media/satoko-vector-eye-comparison.png new file mode 100644 index 0000000..bf1b915 Binary files /dev/null and b/docs/media/satoko-vector-eye-comparison.png differ diff --git a/src/character/eye-stress.ts b/src/character/eye-stress.ts new file mode 100644 index 0000000..6cc7752 --- /dev/null +++ b/src/character/eye-stress.ts @@ -0,0 +1,153 @@ +/** + * Deterministic opt-in 4x4 eye/expression/head/gaze stress review. + * + * Unlike a neutral blink, these exact 16 combinations expose whether an + * imported rig retains eyelid/iris occlusion when head or facial controls + * move. This command produces QA evidence, NOT an artistic approval gate. + */ +import { randomUUID, createHash } from "node:crypto"; +import { execFile } from "node:child_process"; +import { mkdir, readFile, stat, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { promisify } from "node:util"; +import { characterSchema } from "../scene/schemas.js"; +import { AvatarWorkflowError, checkedPath, pinnedRev2D, runRev2D } from "./avatar.js"; +import { verifyAvatarBuildRecord } from "./provenance.js"; +import { eyeRegionFromAvatarSpec } from "./blink-review.js"; + +const execFileAsync = promisify(execFile); +const CONTROLS = [ + "EyeLOpen", "EyeROpen", "EyeBallX", "EyeBallY", + "AngleX", "AngleY", "MouthOpen", "MouthForm", +] as const; +type Control = typeof CONTROLS[number]; +export type EyeStressCase = { id: string; params: Record }; +export type ParameterLimit = { id: string; min: number; max: number; default?: number }; +const BASE: Record = { + EyeLOpen: 1, EyeROpen: 1, EyeBallX: 0, EyeBallY: 0, + AngleX: 0, AngleY: 0, MouthOpen: 0, MouthForm: 0, +}; +const CASES: Array<[string, Partial>]> = [ + ["open-neutral", {}], + ["half-neutral", { EyeLOpen: 0.45, EyeROpen: 0.45 }], + ["nearly-closed", { EyeLOpen: 0.08, EyeROpen: 0.08 }], + ["closed-neutral", { EyeLOpen: 0, EyeROpen: 0 }], + ["gaze-left-half", { EyeLOpen: 0.35, EyeROpen: 0.35, EyeBallX: -0.2 }], + ["gaze-right-half", { EyeLOpen: 0.35, EyeROpen: 0.35, EyeBallX: 0.2 }], + ["gaze-up-half", { EyeLOpen: 0.35, EyeROpen: 0.35, EyeBallY: 0.2 }], + ["gaze-down-half", { EyeLOpen: 0.35, EyeROpen: 0.35, EyeBallY: -0.2 }], + ["yaw-left-quarter", { EyeLOpen: 0.25, EyeROpen: 0.25, AngleX: -8 }], + ["yaw-right-quarter", { EyeLOpen: 0.25, EyeROpen: 0.25, AngleX: 8 }], + ["pitch-up-half", { EyeLOpen: 0.4, EyeROpen: 0.4, AngleY: 5 }], + ["pitch-down-half", { EyeLOpen: 0.4, EyeROpen: 0.4, AngleY: -5 }], + ["happy-speaking", { MouthForm: 0.6, MouthOpen: 0.65 }], + ["happy-look-right", { EyeLOpen: 0.4, EyeROpen: 0.4, + AngleX: 8, EyeBallX: 0.2, MouthForm: 0.6, MouthOpen: 0.4 }], + ["confused-look-left", { EyeLOpen: 0.35, EyeROpen: 0.35, + AngleX: -8, EyeBallX: -0.2, MouthForm: -0.3 }], + ["wink-and-talk", { EyeLOpen: 0, EyeROpen: 1, + MouthOpen: 0.7, MouthForm: 0.2, AngleY: 5 }], +]; + +export function eyeStressCases(parameters: readonly ParameterLimit[]): EyeStressCase[] { + if (!Array.isArray(parameters)) { + throw new AvatarWorkflowError("EYE_STRESS_CONTROLS_UNSUPPORTED", "Rev2D parameter metadata is missing"); + } + const known = new Map(parameters.map(p => [p.id, p])); + if (known.size !== parameters.length) { + throw new AvatarWorkflowError("EYE_STRESS_CONTROLS_UNSUPPORTED", "Duplicate Rev2D parameter IDs"); + } + return CASES.map(([id, values]) => { + const params = { ...BASE, ...values }; + for (const name of CONTROLS) { + const limit = known.get(name); + const value = params[name]; + if (!limit || !Number.isFinite(limit.min) || !Number.isFinite(limit.max) || + limit.min > value || limit.max < value) { + throw new AvatarWorkflowError("EYE_STRESS_CONTROLS_UNSUPPORTED", + "Missing/incompatible Rev2D control " + name + " for sample " + id); + } + } + return { id, params }; + }); +} + +async function sheet(frames: string, output: string, crop?: string) { + const filters = [ + ...(crop ? [crop] : []), + crop ? "scale=400:-2:flags=lanczos" : "scale=256:-2:flags=lanczos", + "tile=4x4", + ].join(","); + await execFileAsync("ffmpeg", [ + "-nostdin", "-hide_banner", "-loglevel", "error", "-n", + "-framerate", "1", "-i", frames, "-vf", filters, + "-frames:v", "1", output, + ], { shell: false, windowsHide: true, timeout: 90_000, maxBuffer: 4_000_000 }); +} + +export async function createEyeStressReview(projectDir: string, root = process.cwd()) { + const rootPath = resolve(root); + const project = await checkedPath(rootPath, resolve(rootPath, projectDir)); + const characterPath = await checkedPath(project, join(project, "character.json")); + const manifest = characterSchema.parse(JSON.parse(await readFile(characterPath, "utf8")) as unknown); + if (!manifest.provenance || !manifest.render || manifest.render.backend !== "rev2d" || + manifest.poseFamilies.length !== 1) { + throw new AvatarWorkflowError("EYE_STRESS_IMPORTED_RIG_REQUIRED", + "Stress review requires one provenance-verified imported Rev2D character"); + } + const integrity = await verifyAvatarBuildRecord(characterPath, manifest.render.modelPath); + if (!integrity.valid) throw new AvatarWorkflowError(integrity.code, integrity.message); + const modelPath = await checkedPath(project, join(project, manifest.render.modelPath)); + const info = await stat(modelPath); + if (!info.isFile() || info.size <= 1 || info.size > 20_000_000) { + throw new AvatarWorkflowError("EYE_STRESS_MODEL_INVALID", "Rev2D model absent or oversized"); + } + const modelBytes = await readFile(modelPath); + const model = JSON.parse(modelBytes.toString("utf8")) as { + meta?: { width: number; height: number }; + parameters?: ParameterLimit[]; + }; + const cases = eyeStressCases(model.parameters ?? []); + const width = model.meta?.width, height = model.meta?.height; + if (!Number.isSafeInteger(width) || !Number.isSafeInteger(height) || + width! < 256 || height! < 256 || width! > 4096 || height! > 4096) { + throw new AvatarWorkflowError("EYE_STRESS_MODEL_INVALID", "Unsupported model dimensions"); + } + const specPath = await checkedPath(project, join(project, "avatar.json")); + const spec = JSON.parse(await readFile(specPath, "utf8")) as unknown; + const region = eyeRegionFromAvatarSpec(spec, width!, height!); + const backend = await pinnedRev2D(rootPath); + const destination = await checkedPath(rootPath, + join(rootPath, "outputs", "qa", "eye-stress", randomUUID())); + const framesDirectory = join(destination, "frames"); + await mkdir(framesDirectory, { recursive: true }); + for (const [i, item] of cases.entries()) { + const file = join(framesDirectory, "frame-" + String(i).padStart(3, "0") + ".png"); + const params = CONTROLS.flatMap(name => ["--param", name + "=" + item.params[name]]); + await runRev2D(backend, [ + "render", modelPath, "-o", file, ...params, "--bg", "#f0eaf4", "--ss", "1", + ], 60_000); + } + const input = join(framesDirectory, "frame-%03d.png"); + const fullSheet = join(destination, "expression-full.png"); + const eyeSheet = join(destination, "expression-eyes.png"); + await sheet(input, fullSheet); + await sheet(input, eyeSheet, + `crop=${region.width}:${region.height}:${region.x}:${region.y}`); + const reportPath = join(destination, "eye-stress.json"); + const result = { + valid: true, approvedByThisCommand: false, status: "visual-review-required", + profile: "combined-eye-gaze-head-expression", + characterId: manifest.id, + modelPath: manifest.render.modelPath, + modelSha256: createHash("sha256").update(modelBytes).digest("hex"), + cases, region, framesDirectory, fullSheet, eyeSheet, reportPath, + notes: [ + "All 16 frames come from actual pinned Rev2D renders; inspect both full and eye crops.", + "This checks combined controls at bounded points. It does not prove quality across continuous motion or other illustration styles.", + "Character approval is never changed or implied.", + ], + }; + await writeFile(reportPath, JSON.stringify(result, null, 2) + "\n", { flag: "wx" }); + return result; +} diff --git a/src/character/eyelid-vector-workflow.ts b/src/character/eyelid-vector-workflow.ts index ce5a2d0..653cd3b 100644 --- a/src/character/eyelid-vector-workflow.ts +++ b/src/character/eyelid-vector-workflow.ts @@ -17,6 +17,7 @@ import { verifyAvatarBuildRecord, writeAvatarBuildRecord, } from "./provenance.js"; import { createBlinkReview } from "./blink-review.js"; +import { createEyeStressReview } from "./eye-stress.js"; import { publishVectorReviewProof } from "./vector-review-evidence.js"; import { vectorizeImportedEyelids } from "./eyelid-vector.js"; @@ -119,13 +120,17 @@ export async function createVectorEyelidRig( let review: Record; try { const rendered = await createBlinkReview(project, { root, timed: true }); - const proof = await publishVectorReviewProof(project, relModel, rendered, root); + const stress = await createEyeStressReview(project, root); + const proof = await publishVectorReviewProof(project, relModel, + { ...rendered, stressSheet: stress.eyeSheet }, root); review = { video: proof.video, eyes: proof.eyes, full: proof.full, proofFile: proof.proofFile, + stress: proof.stress, report: rendered.reportPath, + stressReport: stress.reportPath, }; } catch (cause) { review = { diff --git a/src/character/vector-review-evidence.ts b/src/character/vector-review-evidence.ts index 520a98b..3e9edce 100644 --- a/src/character/vector-review-evidence.ts +++ b/src/character/vector-review-evidence.ts @@ -18,6 +18,7 @@ import { AVATAR_BUILD_RECORD, avatarBuildRecordSchema, verifyAvatarBuildRecord, } from "./provenance.js"; import { createBlinkReview } from "./blink-review.js"; +import { createEyeStressReview } from "./eye-stress.js"; import { characterSchema } from "../scene/schemas.js"; const HASH = z.string().regex(/^[0-9a-f]{64}$/); @@ -26,13 +27,14 @@ const evidenceFileSchema = z.strictObject({ sha256: HASH, }); const proofSchema = z.strictObject({ - version: z.literal(1), + version: z.literal(2), kind: z.literal("vector-eyelids"), model: evidenceFileSchema, artifacts: z.strictObject({ eyes: evidenceFileSchema, full: evidenceFileSchema, video: evidenceFileSchema, + stress: evidenceFileSchema, }), }); type Proof = z.infer; @@ -109,9 +111,9 @@ export async function requiresVectorReviewProof(projectDir: string, modelPath: s export async function publishVectorReviewProof( projectDir: string, modelRelative: string, - rendered: { eyeSheet: string; fullSheet: string; timedVideo?: string | undefined }, + rendered: { eyeSheet: string; fullSheet: string; timedVideo?: string | undefined; stressSheet: string }, root = process.cwd(), -): Promise<{ proofFile: string; eyes: string; full: string; video: string }> { +): Promise<{ proofFile: string; eyes: string; full: string; video: string; stress: string }> { const project = await checkedPath(root, resolve(root, projectDir)); if (!isVectorEyelidModelPath(modelRelative)) { error("Only an explicitly derived vector eyelid model can publish this review proof"); @@ -127,9 +129,10 @@ export async function publishVectorReviewProof( eyes: ["blink-eyes.png", rendered.eyeSheet, "png"], full: ["blink-full.png", rendered.fullSheet, "png"], video: ["blink-timed-30fps.mp4", rendered.timedVideo, "mp4"], + stress: ["expression-eyes.png", rendered.stressSheet, "png"], } as const; const artifacts = {} as Proof["artifacts"]; - for (const key of ["eyes", "full", "video"] as const) { + for (const key of ["eyes", "full", "video", "stress"] as const) { const [name, original, kind] = names[key]; const source = await checkedPath(root, resolve(original)); const input = await checkedBytes(source, kind === "mp4" ? 50_000_000 : 25_000_000, kind); @@ -141,7 +144,7 @@ export async function publishVectorReviewProof( } const proof: Proof = proofSchema.parse({ - version: 1, kind: "vector-eyelids", + version: 2, kind: "vector-eyelids", model, artifacts, }); const proofFile = await checkedPath(project, join(project, VECTOR_REVIEW_PROOF)); @@ -159,6 +162,7 @@ export async function publishVectorReviewProof( proofFile, eyes: join(project, artifacts.eyes.path), full: join(project, artifacts.full.path), video: join(project, artifacts.video.path), + stress: join(project, artifacts.stress.path), }; } @@ -193,7 +197,7 @@ export async function verifyVectorReviewProof( const base = "review/vector-eyes-" + match[1] + "/"; for (const [kind, filename] of [ ["eyes", "blink-eyes.png"], ["full", "blink-full.png"], - ["video", "blink-timed-30fps.mp4"], + ["video", "blink-timed-30fps.mp4"], ["stress", "expression-eyes.png"], ] as const) { const reference = proof.artifacts[kind]; requireRelativeEvidencePath(reference.path, base); @@ -220,7 +224,9 @@ export async function refreshVectorReviewProof( } await currentModel(project, model); const rendered = await createBlinkReview(project, { root, timed: true }); - const proof = await publishVectorReviewProof(project, model, rendered, root); + const stress = await createEyeStressReview(project, root); + const proof = await publishVectorReviewProof(project, model, + { ...rendered, stressSheet: stress.eyeSheet }, root); return { status: "review-evidence-refreshed", approved: manifest.poseFamilies.some(pose => pose.approved), diff --git a/src/cli/index.ts b/src/cli/index.ts index c6c433f..3aa68bc 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -24,6 +24,7 @@ import { comparePngSequence } from "../character/frame-sequence-qa.js"; import { createBlinkReview } from "../character/blink-review.js"; import { createVectorEyelidRig } from "../character/eyelid-vector-workflow.js"; import { refreshVectorReviewProof } from "../character/vector-review-evidence.js"; +import { createEyeStressReview } from "../character/eye-stress.js"; import { DEFAULT_LOOK_LIMITS, IMPORTED_INITIAL_LOOK_LIMITS } from "../scene/look-limits.js"; import { createCueBinding, verifyCueBinding, CueBindingError } from "../audio/cue-provenance.js"; @@ -47,6 +48,7 @@ function usage(): never { " puppetctl character variants import [--experimental-short-fade|--experimental-stepped-eyes] --reviewed-art\n" + " puppetctl character qa compare [--region x,y,w,h]\n" + " puppetctl character qa blink [--timed] [--region x,y,w,h]\n" + + " puppetctl character qa eye-stress \n" + " puppetctl character qa sequence ... [--region x,y,w,h] [--cut-after i,j] [--control-values v1,v2,...]\n" + " puppetctl character landmarks assist \n" + " puppetctl character landmarks propose --provider codex\n" + @@ -210,6 +212,13 @@ async function run(): Promise { usage(); } + if (first === "character" && second === "qa" && third === "eye-stress") { + const [project, ...extra] = process.argv.slice(5); + if (!project || project.startsWith("--") || extra.length) usage(); + print(await createEyeStressReview(project)); + return; + } + if (first === "character" && second === "qa" && third === "blink") { const [project, ...flags] = process.argv.slice(5); if (!project || project.startsWith("--")) usage(); diff --git a/tests/eye-stress.test.ts b/tests/eye-stress.test.ts new file mode 100644 index 0000000..49c46bf --- /dev/null +++ b/tests/eye-stress.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, it } from "vitest"; +import { eyeStressCases } from "../src/character/eye-stress.js"; + +const controls = [ + ["EyeLOpen", 0, 1], ["EyeROpen", 0, 1], ["EyeBallX", -1, 1], + ["EyeBallY", -1, 1], ["AngleX", -30, 30], ["AngleY", -30, 30], + ["MouthOpen", 0, 1], ["MouthForm", -1, 1], +].map(([id, min, max]) => ({ id: id as string, min: min as number, max: max as number })); + +describe("combined eye, gaze, face and head diagnostic plan", () => { + it("exposes exactly 16 named, distinct combinations and preserves input", () => { + const original = structuredClone(controls); + const cases = eyeStressCases(controls); + expect(controls).toEqual(original); + expect(cases).toHaveLength(16); + expect(new Set(cases.map(c => c.id)).size).toBe(16); + expect(new Set(cases.map(c => JSON.stringify(c.params))).size).toBe(16); + expect(cases[0]!.id).toBe("open-neutral"); + expect(cases[3]!.params.EyeLOpen).toBe(0); + expect(cases[3]!.params.EyeROpen).toBe(0); + expect(cases.at(-1)!.params.EyeLOpen).toBe(0); + expect(cases.at(-1)!.params.EyeROpen).toBe(1); + expect(cases.some(c => c.params.AngleX !== 0 && c.params.EyeBallX !== 0 && + c.params.EyeLOpen < 1 && c.params.MouthOpen > 0)).toBe(true); + }); + + it("never produces a parameter outside the model's declared ranges", () => { + const cases = eyeStressCases(controls); + const map = new Map(controls.map(p => [p.id, p])); + for (const c of cases) for (const [name, value] of Object.entries(c.params)) { + expect(value).toBeGreaterThanOrEqual(map.get(name)!.min); + expect(value).toBeLessThanOrEqual(map.get(name)!.max); + } + }); + + it("rejects missing, duplicate or restricted controls rather than silently clamping", () => { + const missing = controls.filter(p => p.id !== "EyeBallX"); + expect(() => eyeStressCases(missing)).toThrow(/EyeBallX/); + expect(() => eyeStressCases([...controls, controls[0]!])).toThrow(/Duplicate/); + expect(() => eyeStressCases(controls.map(p => p.id === "AngleX" ? + { ...p, min: -4, max: 4 } : p))).toThrow(/AngleX/); + expect(() => eyeStressCases(controls.map(p => p.id === "EyeLOpen" ? + { ...p, min: 0.2, max: 1 } : p))).toThrow(/EyeLOpen/); + expect(() => eyeStressCases([])).toThrow(/missing|Missing/i); + }); +}); diff --git a/tests/vector-review-evidence.test.ts b/tests/vector-review-evidence.test.ts index 12e98b1..1d85dcb 100644 --- a/tests/vector-review-evidence.test.ts +++ b/tests/vector-review-evidence.test.ts @@ -55,12 +55,14 @@ async function fixture() { await writeFile(join(review, name), png()); } await writeFile(join(review, "synthetic-timed.mp4"), mp4()); + await writeFile(join(review, "stress.png"), png()); await writeAvatarBuildRecord(dir, await createAvatarBuildRecord(dir, DERIVED)); return { dir, root, review, model: join(dir, DERIVED), inputs: { eyeSheet: join(review, "eyes.png"), fullSheet: join(review, "poses.png"), timedVideo: join(review, "synthetic-timed.mp4"), + stressSheet: join(review, "stress.png"), }, }; } @@ -97,12 +99,12 @@ describe("vectorized eyelid current-model approval evidence", () => { const { dir, root, inputs } = await fixture(); const result = await publishVectorReviewProof(dir, DERIVED, inputs, root); expect(result.proofFile).toContain("review"); - expect((await verifyVectorReviewProof(dir, DERIVED))).toHaveLength(3); + expect((await verifyVectorReviewProof(dir, DERIVED))).toHaveLength(4); const manifestBefore = JSON.parse(await readFile(join(dir, "character.json"), "utf8")); expect(manifestBefore.poseFamilies[0].approved).toBe(false); const approval = await approveImportedAvatar(dir, true, root); expect(approval.status).toBe("approved"); - expect(approval.reviewArtifacts).toHaveLength(7); + expect(approval.reviewArtifacts).toHaveLength(8); }); it("rejects a modified image after the review proof was produced", async () => { @@ -113,6 +115,27 @@ describe("vectorized eyelid current-model approval evidence", () => { .rejects.toMatchObject({ code: "VECTOR_REVIEW_PROOF_MISSING_OR_STALE" }); }); + it("rejects old version-one evidence that lacks the combined-control stress sheet", async () => { + const { dir, root, inputs } = await fixture(); + const result = await publishVectorReviewProof(dir, DERIVED, inputs, root); + const old = JSON.parse(await readFile(result.proofFile, "utf8")); + old.version = 1; + delete old.artifacts.stress; + await writeFile(result.proofFile, JSON.stringify(old)); + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "VECTOR_REVIEW_PROOF_MISSING_OR_STALE" }); + const manifest = JSON.parse(await readFile(join(dir, "character.json"), "utf8")); + expect(manifest.poseFamilies[0].approved).toBe(false); + }); + + it("rejects tampered combined-control eye stress evidence", async () => { + const { dir, root, inputs } = await fixture(); + const proof = await publishVectorReviewProof(dir, DERIVED, inputs, root); + await writeFile(proof.stress, Buffer.concat([await readFile(proof.stress), Buffer.from("changed")])); + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "VECTOR_REVIEW_PROOF_MISSING_OR_STALE" }); + }); + it("rejects a proof after a new model build record replaces the previous model hash", async () => { const { dir, root, model, inputs } = await fixture(); await publishVectorReviewProof(dir, DERIVED, inputs, root);