Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions .agents/skills/puppetcontrol/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,15 @@ Read repo-root AGENTS.md, docs/TECHNICAL_CONCEPT.md, and docs/CHARACTER_IMPORT.m
5. Stage the proposal using `node dist/cli/index.js character landmarks stage <project-dir> --proposal <proposal.json>`. Rev2D validates the isolated candidate and returns a work/overlay.png. **Actually inspect the overlay** before `character landmarks accept <project-dir> --candidate <returned-dir> --reviewed`. Accepting replaces avatar.json with a backup and revokes all existing pose approvals. Never auto-pass --reviewed solely because a validator passed.
6. Build using node dist/cli/index.js character build <project-dir> (use --force if prior output exists). Inspect any geometryWarnings, particularly BODY_PIVOT_TOO_LOW and IRIS_LANDMARK_MISSING; these do not replace pixel inspection. Source/landmark/model/texture hashes are recorded in avatar-build.json and scene preflight rejects stale assets. Inspect every returned review/poses.png, turn.png, eyes.png and mouth.png. If incorrect, repair avatar.json and rebuild using --force; this resets the generated manifest and any edited scaffold.
7. To test optional drawn eye/mouth variants on a built actor, request edit masks, locally supply reviewed `variants/<id>.png`, and explicitly import. With reviewed closed+half eye drawings, `character variants import <project> eyes_closed eyes_half --experimental-short-fade --reviewed-art` shortens fade windows but makes poses change faster and may still ghost; it is a visual experiment, never a default or automatic quality fix. The import revokes prior approval and refreshes provenance only after re-rendering review sheets.
8. Before approving a verified imported rig, run `node dist/cli/index.js character qa blink <project-dir>` to generate an ordered 25-frame actual eyelid transition contact sheet (`blink-eyes.png` and `blink-full.png`); inspect EyeOpen=0.19 and 0.69 and the real eye artwork seams. The QA report flags motion stalls but never passes visual approval.
9. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA may an agent/user explicitly run `node dist/cli/index.js character approve <project-dir> --reviewed` (the flag is an acknowledgement, not evidence of review) to record neutral-pose approval. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction.
10. For a reviewed reusable rig, use `node dist/cli/index.js scene new <new-scene.json> --character <character.json> --seconds 12 --fps 15` to create a strict editable scene. Reuse the same character file in a second scene without rebuilding artwork. The generated preview-scene.json is also editable. Use node dist/cli/index.js scene validate <scene.json> and scene plan before rendering. Preview PNG frames, inspect key timestamps, then render and inspect the MP4.
11. Within a compatible shared rig, semantic pose events may interpolate through validated parameter presets (the neutral-to-pointing demo uses shoulder and elbow controls). Distinct pose models require explicit scene camera cuts; an authored bridge is NOT implemented. Never silently crossfade meshes or call a body cut a smooth physical transition.
12. For speech, prepare WAV, install Rhubarb separately, and use node dist/cli/index.js audio align <voice.wav> --out <mouth.json> [--binary <rhubarb>]. For exact-byte continuity, optionally run `audio bind <voice.wav> --cues <mouth.json> --out <mouth.binding.json>` and include `--binding <mouth.binding.json>` in `scene new`; bound scenes fail validation/render if cue or voice bytes change. A `scene new` voice track requires both `--voice` and the actual `--cues` file; no fabricated speech sync. Reference audio.path, audio.startMs and audio.cuesPath from the scene; review the actual mouth/audio timing.
13. Do not commit private character illustrations, full generated models, user voice recordings or secrets without checking redistribution rights. Demo:avatar generates pinned MIT-licensed sample artwork into ignored .cache.
14. For animation QA, compare real ordered PNGs with `node dist/cli/index.js character qa sequence <f0.png> <f1.png> ... [--region x,y,w,h] [--control-values v0,v1,...] [--cut-after i,j]`. Supply sampled control values when investigating invisible or stalled motion; label known cuts. Inspect the PNGs/video and report possible ghosting even if the motion statistics look normal. The QA report is not an approval.
15. Verified imported rigs and explicitly bound voice tracks are now consumed from fresh per-render copies, hashed against recorded inputs; see `docs/RENDER_INTEGRITY.md`. This is not protection against a malicious same-user process modifying temporary files, nor proof of visual quality.
16. State what actually rendered and which controls remain unsupported. Do not claim a flat PNG is automatically converted into production-quality animation without landmark review.
8. For a deliberately non-blended eye-art experiment, `character variants import <project> eyes_closed eyes_half --experimental-stepped-eyes --reviewed-art` selects only one complete eye appearance outside tiny threshold windows. It can remove ghosted doubled lashes but creates visible pose pops; do not call it seamless or automatically approve. See `docs/EYE_TRANSITIONS.md` and compare actual frames.
9. Before approving a verified imported rig, run `node dist/cli/index.js character qa blink <project-dir>` to generate an ordered 25-frame actual eyelid transition contact sheet (`blink-eyes.png` and `blink-full.png`); inspect EyeOpen=0.19 and 0.69 and the real eye artwork seams. The QA report flags motion stalls but never passes visual approval.
10. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA may an agent/user explicitly run `node dist/cli/index.js character approve <project-dir> --reviewed` (the flag is an acknowledgement, not evidence of review) to record neutral-pose approval. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction.
11. For a reviewed reusable rig, use `node dist/cli/index.js scene new <new-scene.json> --character <character.json> --seconds 12 --fps 15` to create a strict editable scene. Reuse the same character file in a second scene without rebuilding artwork. The generated preview-scene.json is also editable. Use node dist/cli/index.js scene validate <scene.json> and scene plan before rendering. Preview PNG frames, inspect key timestamps, then render and inspect the MP4.
12. Within a compatible shared rig, semantic pose events may interpolate through validated parameter presets (the neutral-to-pointing demo uses shoulder and elbow controls). Distinct pose models require explicit scene camera cuts; an authored bridge is NOT implemented. Never silently crossfade meshes or call a body cut a smooth physical transition.
13. For speech, prepare WAV, install Rhubarb separately, and use node dist/cli/index.js audio align <voice.wav> --out <mouth.json> [--binary <rhubarb>]. For exact-byte continuity, optionally run `audio bind <voice.wav> --cues <mouth.json> --out <mouth.binding.json>` and include `--binding <mouth.binding.json>` in `scene new`; bound scenes fail validation/render if cue or voice bytes change. A `scene new` voice track requires both `--voice` and the actual `--cues` file; no fabricated speech sync. Reference audio.path, audio.startMs and audio.cuesPath from the scene; review the actual mouth/audio timing.
14. Do not commit private character illustrations, full generated models, user voice recordings or secrets without checking redistribution rights. Demo:avatar generates pinned MIT-licensed sample artwork into ignored .cache.
15. For animation QA, compare real ordered PNGs with `node dist/cli/index.js character qa sequence <f0.png> <f1.png> ... [--region x,y,w,h] [--control-values v0,v1,...] [--cut-after i,j]`. Supply sampled control values when investigating invisible or stalled motion; label known cuts. Inspect the PNGs/video and report possible ghosting even if the motion statistics look normal. The QA report is not an approval.
16. Verified imported rigs and explicitly bound voice tracks are now consumed from fresh per-render copies, hashed against recorded inputs; see `docs/RENDER_INTEGRITY.md`. This is not protection against a malicious same-user process modifying temporary files, nor proof of visual quality.
17. State what actually rendered and which controls remain unsupported. Do not claim a flat PNG is automatically converted into production-quality animation without landmark review.

Keep edits small and testable. Run npm run verify and inspect visual evidence for any animation/asset changes.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ The long-term goal: **generate artwork when necessary, animate and reuse it inde
- Guarded **local drawn eye/mouth variant** request, status and import commands using Rev2D masks and sprites. Imported drawings revoke approval and require a fresh visual review; no image generation happens automatically.
- An optional **experimental** short-fade profile for imported half/closed eye drawings; avoids prolonged duplicate lash outlines at the cost of faster pose changes, revokes approval and regenerates review sheets.
- Local PNG pixel and ordered-frame QA (full frame or explicit region) reports RGB activity, unexpected transparency, candidate motion jumps and changed-control stalls; these metrics never auto-approve a rig.
- Import review can now render a 25-frame blink contact sheet focused on the actual eyelid transition boundaries, with pixel/motion diagnostics that never auto-approve a rig; see [Character Review](docs/CHARACTER_REVIEW.md).
- Import review can now render a 25-frame blink contact sheet focused on actual eyelid transition boundaries, with pixel/motion diagnostics that never auto-approve a rig; see [Character Review](docs/CHARACTER_REVIEW.md). An optional [stepped-eye visibility experiment](docs/EYE_TRANSITIONS.md) eliminates sampled double-sprite ghosting on one MIT sample but introduces noticeable pose switching; it is not a seamless eyelid morph.
- Imported flat-image actors start with narrower editable head/gaze motion limits, with consistent vertical look direction and unchanged amplitude for existing layered rigs; see [Look Motion](docs/LOOK_MOTION.md).
- Optional SHA-256 Rhubarb cue bindings can detect when audio or mouth-cue bytes are replaced after alignment; a bound scene fails validation/render instead of silently using stale lip sync.
- Agent-readable CLI output, tests and CI.
Expand Down
8 changes: 7 additions & 1 deletion docs/CHARACTER_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,13 @@ An **optional** short-fade experiment can reduce double contours by narrowing sp
node dist/cli/index.js character variants import avatars/portrait eyes_closed eyes_half --experimental-short-fade --reviewed-art
~~~

Both paths revoke any earlier approval and generate new review sheets and fingerprints.
Another **opt-in, sharp-switch** experiment avoids showing two eyelid drawings simultaneously. It removes the sampled translucent/double-lash effect in the Hana trial, but **introduces abrupt eye-pose jumps** and is *not* a smooth blink:

~~~powershell
node dist/cli/index.js character variants import avatars/portrait eyes_closed eyes_half --experimental-stepped-eyes --reviewed-art
~~~

Choose **only one** experimental profile. Every import revokes earlier approval, regenerates review sheets and refreshes fingerprints. See [EYE_TRANSITIONS.md](EYE_TRANSITIONS.md) for the actual rendered comparison and trade-offs.

## 3. Inspect exact blink transition frames

Expand Down
70 changes: 70 additions & 0 deletions docs/EYE_TRANSITIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Experimental eye transitions: real visual comparison

**Status:** opt-in mitigation, not a production-quality blink solution.
**Sample:** locally imported, MIT-licensed RevStudio/Rev2D Hana art (see [third-party notices](THIRD_PARTY_NOTICES.md)).
**Date:** 2026-11-07.

## Why a shorter fade still ghosted

The imported Rev2D eye rig has an open mesh plus separately drawn half-closed and fully closed sprites. The earlier short-fade experiment still made two complete, **misaligned eye drawings simultaneously translucent**:

- Closed → half at `EyeOpen=0.18–0.20`.
- Half → open mesh at `EyeOpen=0.68–0.70`.

Blending two different lash geometries yields duplicate contours, see-through eyes and pale iris fragments. Narrowing the fade merely shortens the duration of this artifact.

## Opt-in stepped-eye profile

`--experimental-stepped-eyes` replaces just those opacity controls with near-instant, complementary visibility switches:

| EyeOpen parameter | Visible eye appearance |
|---|---|
| Below 0.195 | Closed drawing |
| 0.195 to below 0.69 | Half-closed drawing |
| At/above 0.69 | Original open mesh |

A tiny `0.0001`-wide ramp is used instead of duplicate key coordinates because Rev2D requires well-defined interpolation keys. All other rig parameters, meshes, textures, expressions and mouth controls remain unchanged. The transformer accepts only the previously validated Rev2D two-eye sprite topology; unfamiliar rig bindings are rejected.

**This stops the sampled double-image opacity problem, but it is *not smooth geometry morphing*.** The entire eye appearance can pop at either switch. Use only when that tradeoff actually looks better for the character and intended blink speed.

To use it with **reviewed, user-provided** full-canvas half/closed drawings:

~~~powershell
node dist/cli/index.js character variants import avatars/portrait eyes_closed eyes_half --experimental-stepped-eyes --reviewed-art
node dist/cli/index.js character qa blink avatars/portrait
~~~

Only one experimental profile is permitted; `--experimental-short-fade` and `--experimental-stepped-eyes` cannot be combined. The import revokes prior character approval, rebuilds the review images and refreshes provenance hashes. The program never grants approval or claims that a character has been visually reviewed.

## Observed comparison

![MIT-licensed derived Hana crop: old short fade (top) and new stepped profile (bottom) at four EyeOpen values](media/stepped-eye-before-after.png)

*Source: actual Rev2D rendered pixels at identical `EyeOpen` values and identical source illustration. Top row: short fade. Bottom row: stepped. The original Hana asset is MIT-licensed (RevStudio).*

In the local test the unapproved experimental Hana character's **translucent double lashes at 0.69** and **ghosted irises at 0.19** were absent in the matched stepped renders. The prior half-open and closed drawings still have a visibly stiff, imperfect outline.

The same 25-frame, 13-real-render plus mirrored opening review sequence gave:

| Pixel QA diagnostic | Short fade | Stepped |
|---|---:|---:|
| Possible abrupt changes | 0 | 4 |
| Possible changed-control motion stalls | 10 | 12 |

The stepped profile deliberately creates jumps; the automated warning is expected and **should not be suppressed or relabeled as an artistic pass**. The stalled intervals reflect static sprite holds (some intentional). Neither diagnostic alone assesses visual acceptability.

A separate **mesh-only** experiment—no replacement eye sprites, continuously deforming the original mesh—had fewer opacity artifacts, but noticeably squeezed/distorted the eyelids and irises near full closure. That was not selected.

The full locally ignored A/B sequence video is:

`outputs/qa/eye-profile-comparison-20261107/ordered-blink-comparison.mp4`

This is a **non-uniform parameter-sweep review clip** (25 frames at 30 FPS), **not** a natural-timing performance demo.

A separate disposable, **QA-only** copy of the stepped actor was temporarily marked approved solely for renderer integration testing. It exported a valid 3-second, 15 FPS, 512×640, 45-frame H.264 MP4 via the verified-rig snapshot path. That marker was never applied to the experimental source character. The source remains `approved:false`.

## Acceptance boundary and next work

Actual quality requires better curved, consistent eye artwork and/or a continuous eyelid-specific mesh with properly hidden irises. A reversible stepped visibility option is useful for stylized quick blinks, but shouldn't become a default or be described as Live2D-grade deformation.

Do **not** approve this Hana trial, infer quality from geometry checks, or publish copied test art without preserving the MIT notice. Before approval, inspect the entire blink video, seam/iris/color consistency, gaze extremes, head movement and the surrounding face at target FPS. Compare at least two additional rights-cleared illustration styles.
1 change: 1 addition & 0 deletions docs/IMPLEMENTATION_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
- [x] Wrap local Rev2D drawn eye/mouth variant request/status/import with explicit reviewed-art selection, approval revocation before import, post-import provenance refresh, and new review sheets. A real Hana avatar produced correct edit masks; unchanged fake drawings were rejected.
- [x] Test and expose an opt-in, validated short eye-sprite fade as a local derived-model experiment. Compare 13 actual Hana blink frames with native, immediate-switch and shortened fades: shorter fades visibly reduce double lashes at sampled midpoints, but introduce faster pose changes; never auto-approve.
- [x] Add bounded per-character semantic head/eye look amplitudes with conservative imported-PNG starting values, schema validation, compiler tests and real Hana ±yaw/pitch/gaze frame comparisons. This does not establish visual approval.
- [x] Test an optional stepped-eye visibility profile that removes sampled double-sprite ghosting at EyeOpen 0.19/0.69 on refined MIT Hana artwork, with a 25-frame A/B render, explicit abrupt-change warnings and local 45-frame end-to-end smoke video. The source remains unapproved; **this trades ghosting for pose popping**, not a seamless morph.
- [ ] Produce and visually approve high-quality closed/half eye and mouth artwork; demonstrate seamless real rendered transitions across full blink playback and multiple character styles. The local proof drawing stays unapproved.

## First video milestone — reproducible usable slice (2026-11-07)
Expand Down
2 changes: 1 addition & 1 deletion docs/THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- Source project: https://github.com/RevStudio/Rev2D
- Pinned source commit: f144581cdf1f12d781379c4df90d262b8c2dfa3d
- File: examples/hana/hana.r2d.json in Rev2D upstream
- Use: local demo pose-rig generation and attributed sample MP4. PuppetControl's own source code does not bundle Hana's model/artwork.
- Use: local demo pose-rig generation, attributed sample MP4, and the derived eye-transition crop in `docs/media/stepped-eye-before-after.png`. PuppetControl's own source code does not bundle the original full Hana model/artwork.
- Original license: MIT License, Copyright (c) 2026 RevStudio.

### MIT License (RevStudio)
Expand Down
Binary file added docs/media/stepped-eye-before-after.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading