Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/puppetcontrol/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Read repo-root AGENTS.md, docs/TECHNICAL_CONCEPT.md, and docs/CHARACTER_IMPORT.m
5. Stage the proposal using `node dist/cli/index.js character landmarks stage <project-dir> --proposal <proposal.json>`. Rev2D validates the isolated candidate and returns a work/overlay.png. **Actually inspect the overlay** before `character landmarks accept <project-dir> --candidate <returned-dir> --reviewed`. Accepting replaces avatar.json with a backup and revokes all existing pose approvals. Never auto-pass --reviewed solely because a validator passed.
6. Build using node dist/cli/index.js character build <project-dir> (use --force if prior output exists). Inspect any geometryWarnings, particularly BODY_PIVOT_TOO_LOW and IRIS_LANDMARK_MISSING; these do not replace pixel inspection. Source/landmark/model/texture hashes are recorded in avatar-build.json and scene preflight rejects stale assets. Inspect every returned review/poses.png, turn.png, eyes.png and mouth.png. If incorrect, repair avatar.json and rebuild using --force; this resets the generated manifest and any edited scaffold.
7. To test optional drawn eye/mouth variants on a built actor, request edit masks, locally supply reviewed `variants/<id>.png`, and explicitly import. With reviewed closed+half eye drawings, `character variants import <project> eyes_closed eyes_half --experimental-short-fade --reviewed-art` shortens fade windows but makes poses change faster and may still ghost; it is a visual experiment, never a default or automatic quality fix. The import revokes prior approval and refreshes provenance only after re-rendering review sheets.
8. For a compatible imported avatar whose extracted lash warps look unnatural, optionally run `character eyes vectorize <project> --experimental`. This creates a **new provenance-tracked vector eyelid model**, preserves source drawings, revokes existing approval, and renders real 30 FPS blink evidence. It uses the eye-mask vertices for a continuously deforming line, not alpha-switching. Derived rigs require fresh SHA-256-bound eyes/full/video evidence before `character approve`; if missing or stale, run `character eyes review <project>` and inspect the returned real frames. Neither evidence nor software can prove artistic acceptance or user consent. See `docs/VECTOR_EYELIDS.md`.
8. For a compatible imported avatar whose extracted lash warps look unnatural, optionally run `character eyes vectorize <project> --experimental`. This creates a **new provenance-tracked vector eyelid model**, preserves source drawings, revokes existing approval, and renders real 30 FPS blink evidence. It uses the eye-mask vertices for a continuously deforming line, not alpha-switching. Derived rigs require fresh SHA-256-bound eyes/full/video plus 16-case head/gaze/expression evidence before `character approve`; if missing or stale, run `character eyes review <project>` and inspect the returned real frames. Neither evidence nor software can prove artistic acceptance or user consent. Also run `character qa eye-stress <project>` to render 16 actual combined blink/gaze/head/expression states and inspect both contact sheets. The current-model proof includes that sheet, but a validator cannot confirm its artistic quality; Satoko's independent MIT trial still shows closed-eye residue. See `docs/VECTOR_EYELIDS.md` and `docs/SECOND_STYLE_EYE_QA.md`.
9. For a deliberately non-blended eye-art experiment, `character variants import <project> eyes_closed eyes_half --experimental-stepped-eyes --reviewed-art` selects only one complete eye appearance outside tiny threshold windows. It can remove ghosted doubled lashes but creates visible pose pops; do not call it seamless or automatically approve. See `docs/EYE_TRANSITIONS.md` and compare actual frames.
10. Before approving a verified imported rig, run `node dist/cli/index.js character qa blink <project-dir>` to generate an ordered 25-frame actual eyelid transition contact sheet (`blink-eyes.png` and `blink-full.png`); inspect EyeOpen=0.19 and 0.69 and the real eye artwork seams. The QA report flags motion stalls but never passes visual approval. Also run `character qa blink <project-dir> --timed` for a 36-frame synthetic 30 FPS video and review the actual moving result. Different fade policies may produce identical frames when their transition windows fall between sampled timestamps; see `docs/EYE_TIMING.md`.
11. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA may an agent/user explicitly run `node dist/cli/index.js character approve <project-dir> --reviewed` (the flag is an acknowledgement, not evidence of review) to record neutral-pose approval. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The long-term goal: **generate artwork when necessary, animate and reuse it inde
- Local PNG pixel and ordered-frame QA (full frame or explicit region) reports RGB activity, unexpected transparency, candidate motion jumps and changed-control stalls; these metrics never auto-approve a rig.
- Import review can render a 25-frame parameter-sweep contact sheet and a separate **36-frame, 30 FPS synthetic timed-blink MP4** with motion warnings; see [Character Review](docs/CHARACTER_REVIEW.md) and [Time-domain Blink QA](docs/EYE_TIMING.md). The opt-in [stepped-eye experiment](docs/EYE_TRANSITIONS.md) removes sampled opacity ghosting but does not provide a seamless morph. The two modes even produced byte-identical frames in one real-time test.
- Imported flat-image actors start with narrower editable head/gaze motion limits; see [Look Motion](docs/LOOK_MOTION.md).
- Optional **experimental continuous vector eyelids** replace warped raster lash outlines with thin mask-following Rev2D mesh strips while keeping the iris clipped. This improves one real Hana sample without automatic approval. A **current-model SHA-256-bound eye/full/video proof** is required before explicit approval; use `character eyes review <project>` to refresh it. See [Vector Eyelids](docs/VECTOR_EYELIDS.md).
- Optional **experimental continuous vector eyelids** use mask-following Rev2D strips instead of warped raster lashes. **Two independent MIT samples have been tested**: Hana improved, while Satoko still has visible eyelid/iris residue and remains unapproved. [See matched before/after frames](docs/media/satoko-vector-eye-comparison.png) and [second-style findings](docs/SECOND_STYLE_EYE_QA.md). `character qa eye-stress <project>` now renders 16 eye/gaze/head/expression cases; derived-rig approval requires these plus the current-model SHA-256-bound blink/MP4 sheets. Refresh via `character eyes review <project>`. [Vector eyelid details](docs/VECTOR_EYELIDS.md).
- Optional SHA-256 Rhubarb cue bindings can detect when audio or mouth-cue bytes are replaced after alignment; a bound scene fails validation/render instead of silently using stale lip sync.
- Agent-readable CLI output, tests and CI.

Expand Down
2 changes: 1 addition & 1 deletion docs/CHARACTER_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ For a genuinely deforming eyelid outline instead of alpha-switching whole-eye pi
node dist/cli/index.js character eyes vectorize avatars/portrait --experimental
~~~

It uses the existing Rev2D eye-white mask to deform upper/lower lashes continuously, keeps the iris clipped, preserves the previous model, writes new fingerprints, and revokes approval. Unlike the sprite transitions it simplifies the painted lashes; visually inspect the resulting video. **The generated model requires a model-hash-bound eye/full/MP4 review proof before approval.** If evidence is missing, run `character eyes review <project>` and examine all fresh frames. See [VECTOR_EYELIDS.md](VECTOR_EYELIDS.md).
It uses Rev2D's existing eye-white mask to deform upper/lower lashes continuously, preserves the previous model, writes new fingerprints and revokes approval. **Approval now requires a model-hash-bound blink MP4 plus full/eyes sheets and a 16-case head/gaze/expression eye sheet.** If evidence is missing or old, run `character eyes review <project>`; visually inspect every case. The separate `character qa eye-stress <project>` command works on any provenance-tracked imported rig. The [second MIT character study](SECOND_STYLE_EYE_QA.md) still shows a greenish closed-eye trace and remains unapproved. See [VECTOR_EYELIDS.md](VECTOR_EYELIDS.md).

## 3. Inspect exact blink transition frames

Expand Down
2 changes: 2 additions & 0 deletions docs/IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@
- [x] Add a 36-frame, 30 FPS time-domain blink QA export with deterministic synthetic timing, unique Rev2D state caching, and JSON motion warnings. The actual short-fade and stepped 30 FPS frames were SHA-256 identical for this one schedule, showing threshold tuning alone is insufficient. Four mesh/overlay variants failed visual QA and were not integrated.
- [x] Build an opt-in, provenance-verified **continuous vector eyelid** rig: replace distorted raster lash warps with curved vertex strips driven by Rev2D's existing eye mask, preserving iris clipping and source art. Inspect real 9-state, 30 FPS blink and head/gaze sweeps; 90-frame end-to-end H.264 export verified. The Hana experiment remains unapproved and artists may prefer their own lash styling.
- [x] Make derived vector rig approvals require a SHA-256-bound current-model full/eye/30 FPS MP4 review package. Reject stale/missing/tampered files and renamed vector geometry; add `character eyes review` recovery without changing approval. Real licensed Hana test regenerated evidence successfully and remained unapproved.
- [x] Import independent MIT Satoko chibi artwork, annotate/build under the pinned Rev2D backend, apply and verify continuous vector eyelids; record actual good and bad frames versus original. The second style is compatible but green/gray lash leakage remains and it is **not approved**.
- [x] Add 16-case combined eye/gaze/head/expression visual QA with model-range validation and real engine renders, and require its hashed contact sheet alongside 30 FPS blink evidence in version-2 vector review proof. Regression tests and local stress renders on Hana + Satoko.
- [ ] Produce and visually approve high-quality closed/half eye and mouth artwork; demonstrate seamless real rendered transitions across full blink playback and multiple character styles. The local proof drawing stays unapproved.

## First video milestone — reproducible usable slice (2026-11-07)
Expand Down
48 changes: 48 additions & 0 deletions docs/SECOND_STYLE_EYE_QA.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Second illustration and combined-expression eye QA (2026-11-08)

**Status:** second-style integration **PASS**; subjective character-quality approval **FAIL / NOT APPROVED**. Do not turn this into a generic one-click importer claim.

## Independent licensed artwork

The second illustration is **Satoko**, an original code-drawn chibi character from Henrik Hanssen's [anime-character-creator](https://github.com/rhedak/anime-character-creator), MIT-licensed, at pinned commit `8a52cd146445b1aee4d890df5e44bcf9102cceef`. This differs substantially from RevStudio's Hana: simple vector strokes, flat-colored hair, large green irises, tiny nose and mouth, full body and a short stylized face. It is not a recolor or re-render of Hana. The original PNG checkout and all generated model files remain under ignored `.cache/`; this repository includes only a small attributed **derived comparison crop**.

The source `ref-out/satoko.png` (800×1000) was locally resized to 512×640 and imported without network processing. Coordinates were placed manually from the source-pixel grid and inspected on a real landmark overlay. Rev2D's initial landmark validator reported **zero warnings**, then the full build reported **two nonblocking PuppetControl triage warnings**: body pivot too low and body fade starting below the chest. Neither was suppressed; they need a proper body rig correction if we ever use Satoko for a full performance.

## Actual imported vector rig

- `character import` → source/spec created, unapproved.
- `character build` → model, textures, review PNGs and SHA-256 record, unapproved.
- `character eyes vectorize --experimental` → four continuous upper/lower eyelid strips, a separate model, fresh hash record and current-model blink review package, **approved:false**.
- `character qa eye-stress` → 16 real pinned Rev2D snapshots combining eye closure, asymmetric wink, gaze left/right/up/down, head yaw/pitch, happy/confused mouth, speaking and combined head+gaze+expression. Contact sheets and a JSON manifest written to a unique ignored QA directory.
- `character eyes review` → now regenerates model-bound `blink-eyes.png`, `blink-full.png`, 36-frame 30 FPS H.264 blink and the **16-case combined-control eye sheet**. The review proof is version 2, binding all four artifacts and exact model SHA-256. Old version-1 proofs are rejected; existing vector avatars can regenerate evidence without changing their approval.

Sample contact sheets were checked visually in both Satoko and Hana. The combination cases are bounded diagnostics, not human-quality certification; static pictures do not prove motion continuity.

## Actual image comparison

![Satoko original raster eye deformation (top) versus experimental vector eyelids (bottom), six matched Rev2D states](media/satoko-vector-eye-comparison.png)

The top row exhibits pointed lash tips, dark overlaps and iris trace during quarter closure. The vector bottom row reduces those artifacts and closes more cleanly. **Remaining defects:** green/gray iris residue beneath almost-closed lids, simplified or doubled fine lines at the lash boundary, and body/head deformation risks from the first landmark pass. The vector eye technique *runs* on a second independent art style, but the current output is **not good enough to automatically approve or describe as polished Live2D quality**.

The original Hana sample has cleaner near-closed geometry under vector eyelids, but its facial art also remains experimental. The difference is why independent styles matter.

## Reproducing locally

Requires a pinned Rev2D checkout and FFmpeg in PATH. Reuse an existing imported, provenance-verified, **unapproved** character rig:

~~~powershell
node dist/cli/index.js character qa eye-stress .cache/avatar-onboarding/satoko-second-20261108
node dist/cli/index.js character qa eye-stress .cache/avatar-onboarding/vector-review-proof-20261108
~~~

Each invocation creates `outputs/qa/eye-stress/<uuid>/` with `frames/frame-000.png` through `frame-015.png`, `expression-full.png`, `expression-eyes.png`, and `eye-stress.json`. No input model or approval bit is changed, and missing/limited controls are rejected. For a vector rig, `character eyes review <project>` regenerates the model-bound proof with both a neutral timed blink and expression stress case.

## Full-scene integration check

For renderer integration only, a separate **disposable copy** of Satoko was marked `approved:true` by a QA script. The original imported source project and its model-bound review package remain `approved:false`. On the QA copy, the normal `scene new` → `scene validate` → `scene render` workflow produced a real **3.000-second silent H.264 MP4 at 512×640, 30 FPS, 90 frames**, passing the project's FFprobe acceptance checker. Frames at 0, 1 and 2 seconds were also visually inspected, with the same cartoon character identity preserved. This tests software integration **only**, not approval or animation quality.

## Still blocked

- A human/artist must repair Satoko's closed-lid color/iris leakage and review gaze/face/neck. **It is not approved.**
- Both artworks still lack broad pose and full-speaking quality certification. We need a third, independently prepared character style before enabling any universal default.
- The proof verifies artifact bytes and model identity; it does **not** prove a human inspected a sheet or that it is aesthetically acceptable. Never automatically run `character approve --reviewed`.
32 changes: 32 additions & 0 deletions docs/THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,36 @@ SOFTWARE.

---

## Anime Character Creator — Satoko illustration

- Source: https://github.com/rhedak/anime-character-creator
- Pinned commit: 8a52cd146445b1aee4d890df5e44bcf9102cceef
- Original source file: `ref-out/satoko.png` (800×1000).
- Use: a locally resized, manually landmarked, **unapproved** second-style import for vector eyelid QA. Only an attributed derived eye comparison crop is included in `docs/media/satoko-vector-eye-comparison.png`; the original full picture and models are not bundled.
- Original license: MIT License, Copyright (c) 2026 Henrik Hanssen.

### MIT License (Henrik Hanssen)

Copyright (c) 2026 Henrik Hanssen

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

---

Rhubarb, FFmpeg, and optional image models are not bundled into the PuppetControl project. Their binaries and weights have their own licenses; see project documentation and upstream sources.
Loading
Loading