diff --git a/README.md b/README.md index 1efcd42..2603633 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ The long-term goal: **generate artwork when necessary, animate and reuse it inde - An optional **experimental** short-fade profile for imported half/closed eye drawings; avoids prolonged duplicate lash outlines at the cost of faster pose changes, revokes approval and regenerates review sheets. - Local PNG pixel and ordered-frame QA (full frame or explicit region) reports RGB activity, unexpected transparency, candidate motion jumps and changed-control stalls; these metrics never auto-approve a rig. - Import review can now render a 25-frame blink contact sheet focused on the actual eyelid transition boundaries, with pixel/motion diagnostics that never auto-approve a rig; see [Character Review](docs/CHARACTER_REVIEW.md). +- Imported flat-image actors start with narrower editable head/gaze motion limits, with consistent vertical look direction and unchanged amplitude for existing layered rigs; see [Look Motion](docs/LOOK_MOTION.md). - Optional SHA-256 Rhubarb cue bindings can detect when audio or mouth-cue bytes are replaced after alignment; a bound scene fails validation/render instead of silently using stale lip sync. - Agent-readable CLI output, tests and CI. diff --git a/docs/CHARACTER_REVIEW.md b/docs/CHARACTER_REVIEW.md index 1f418d3..95f22fc 100644 --- a/docs/CHARACTER_REVIEW.md +++ b/docs/CHARACTER_REVIEW.md @@ -66,4 +66,6 @@ Approval records a **user acknowledgement**, not proof of visual suitability. Ch On 2026-11-07, the short-fade Hana trial retained valid source/rig fingerprint integrity but **remained unapproved**. A 25-frame blink review rendered successfully, found control-motion plateaus and showed unnatural skin-tone patches and doubled eyelid contours at transition frames. The short-fade profile reduced one type of overlap but did not repair the underlying drawings. The ability to detect and inspect these issues is real; attractive universal blinking from arbitrary flat illustrations is still an open milestone. -Future work: human-quality intermediate eyelid drawings, review-safe eye/head motion envelopes, additional licensed character styles and realistic mouth/voice assessment. +The latest imported-character build uses deliberately smaller head/eye look amplitudes; configure the specific character after inspecting the head and gaze sheets, as described in [LOOK_MOTION.md](LOOK_MOTION.md). Small limits do not repair torn eyelid sprites or certify visual suitability. + +Future work: human-quality intermediate eyelid drawings, properly reviewed eye/head motion envelopes across styles, additional licensed character styles and realistic mouth/voice assessment. diff --git a/docs/IMPLEMENTATION_PLAN.md b/docs/IMPLEMENTATION_PLAN.md index 24b77fa..67a783b 100644 --- a/docs/IMPLEMENTATION_PLAN.md +++ b/docs/IMPLEMENTATION_PLAN.md @@ -82,6 +82,7 @@ - [ ] Add head/neck seam-specific QA and review multiple rights-cleared character styles; metrics must never auto-approve a rig. - [x] Wrap local Rev2D drawn eye/mouth variant request/status/import with explicit reviewed-art selection, approval revocation before import, post-import provenance refresh, and new review sheets. A real Hana avatar produced correct edit masks; unchanged fake drawings were rejected. - [x] Test and expose an opt-in, validated short eye-sprite fade as a local derived-model experiment. Compare 13 actual Hana blink frames with native, immediate-switch and shortened fades: shorter fades visibly reduce double lashes at sampled midpoints, but introduce faster pose changes; never auto-approve. +- [x] Add bounded per-character semantic head/eye look amplitudes with conservative imported-PNG starting values, schema validation, compiler tests and real Hana ±yaw/pitch/gaze frame comparisons. This does not establish visual approval. - [ ] Produce and visually approve high-quality closed/half eye and mouth artwork; demonstrate seamless real rendered transitions across full blink playback and multiple character styles. The local proof drawing stays unapproved. ## First video milestone — reproducible usable slice (2026-11-07) diff --git a/docs/LOOK_MOTION.md b/docs/LOOK_MOTION.md new file mode 100644 index 0000000..fd6ccaf --- /dev/null +++ b/docs/LOOK_MOTION.md @@ -0,0 +1,37 @@ +# Authoring reviewable head and eye movement + +PuppetControl's semantic scene `look` event uses normalized **screen-space** coordinates: `x=1` looks right, `x=-1` looks left, `y=1` looks **down**, and `y=-1` looks **up**. Both the head and irises follow the same direction. Rev2D itself declares positive `AngleY` and `EyeBallY` as up, so the compiler reverses both when scene `y` is positive. + +## Per-character look limits + +An imported single-image actor gets conservative **starting amplitudes** in its generated `character.json`: + +~~~json +"lookLimits": { + "headYawDegrees": 8, + "headPitchDegrees": 5, + "eyeX": 0.2, + "eyeY": 0.2 +} +~~~ + +The property lives inside the `render` object, alongside `expressionPresets`. These are **only intensity limits, not quality scores or proof of approval**. A character may still have broken eyelids, detached hair or an ugly neck at an 8-degree head turn. Always inspect actual pose sheets and video. + +Before this feature, a `look` event at the extremes drove `AngleX=±22`, `AngleY=±12`, `EyeBallX=±0.5`, and `EyeBallY=±0.5`. Existing fully authored or upstream pre-rigged actors without `render.lookLimits` keep these original amplitudes. Previous provenance-tracked flat-image imports that lack the new field automatically use the conservative imported profile at render time. New `character build` manifests include the values explicitly. + +All four limits are mandatory when the object is present. They are independently customizable in `character.json`, but can only **decrease** from the wider pre-rigged defaults. Setting any limit to `0` disables that channel for semantic `look` events. `character inspect ` shows the currently applied values and whether they came from the manifest, imported fallback, or legacy pre-rigged fallback. + +For example, `look: { x: 0.5, y: -0.25 }` with the imported profile compiles to `AngleX=4`, `AngleY=+1.25`, `EyeBallX=+0.1`, `EyeBallY=+0.05`. The keyframes are eased over the event's specified duration, and expressions/mouth cues are unaffected. + +## How to judge a setting + +1. Render the imported character's head turn and gaze review sheets, and inspect small and extreme positions. Check for iris clipping, asymmetry, detached hair, shifting neck outlines and unintended background movement. +2. Start with the small imported amplitudes and only increase a channel after comparing actual frames of the same character. Reduce or disable channels that fail. +3. Use two `look` events in an editable scene, then render the whole transition at the target FPS. A clean endpoint does not prove intermediate frames are smooth. +4. **Keep the avatar unapproved** when any required pose is visually unacceptable. Altering the authored look configuration after approval warrants renewed review; the present asset hashes track PNG/landmarks/model/textures, not arbitrary manifest settings. + +On 2026-11-07, the reviewed **experimental, unapproved** Hana import was directly rendered at yaw 22°/8°, pitch 12°/5° and gaze 0.5/0.2. The small states remain recognizable and exhibit less drastic movement, but the character still has unacceptable eyelid art and does not pass visual acceptance. The preview outputs are ignored under `outputs/qa/look-envelope-20261107`. + +## Current boundary + +This is a deliberate constraint on *how far* existing rig controls move. It does not create new eye sprites, compensate 3D head parallax, decompose layers or invent a physically believable pose transition. Per-character values are editable human/agent authored metadata; future work should associate a motion envelope's approval with its reviewed frame evidence. diff --git a/src/character/avatar.ts b/src/character/avatar.ts index 13b9988..9806d60 100644 --- a/src/character/avatar.ts +++ b/src/character/avatar.ts @@ -4,6 +4,7 @@ import { readFile, realpath, stat, lstat, mkdir, writeFile, rename, unlink } fro import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { promisify } from "node:util"; import { characterSchema, sceneSchema, type Character, type Scene } from "../scene/schemas.js"; +import { IMPORTED_INITIAL_LOOK_LIMITS } from "../scene/look-limits.js"; import { AVATAR_BUILD_RECORD, createAvatarBuildRecord, writeAvatarBuildRecord, revokeAvatarApprovalBeforeBuild } from "./provenance.js"; import { assessAvatarGeometry } from "./geometry-quality.js"; @@ -102,7 +103,8 @@ export function createImportedCharacter(id: string, model: Rev2DModelParameters) expressions: Object.keys(expressionPresets), motions: [], transitions: [], - render: { backend: "rev2d", modelPath: id + ".r2d.json", expressionPresets }, + render: { backend: "rev2d", modelPath: id + ".r2d.json", expressionPresets, + lookLimits: { ...IMPORTED_INITIAL_LOOK_LIMITS } }, }; return characterSchema.parse(manifest); } diff --git a/src/cli/index.ts b/src/cli/index.ts index 83da677..06d75cf 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -22,6 +22,7 @@ import { requestDrawnVariants, statusDrawnVariants, importDrawnVariants } from " import { comparePngFiles, parsePixelRegionArgument } from "../character/pixel-qa.js"; import { comparePngSequence } from "../character/frame-sequence-qa.js"; import { createBlinkReview } from "../character/blink-review.js"; +import { DEFAULT_LOOK_LIMITS, IMPORTED_INITIAL_LOOK_LIMITS } from "../scene/look-limits.js"; import { createCueBinding, verifyCueBinding, CueBindingError } from "../audio/cue-provenance.js"; function print(value: unknown): void { @@ -285,7 +286,13 @@ async function run(): Promise { expressions: parsed.data.expressions, motions: parsed.data.motions, transitions: parsed.data.transitions, - note: "Schema validation does not prove that a visual rig/asset can render.", + ...(parsed.data.render ? { + lookLimits: parsed.data.render.lookLimits ?? (parsed.data.provenance + ? IMPORTED_INITIAL_LOOK_LIMITS : DEFAULT_LOOK_LIMITS), + lookLimitsSource: parsed.data.render.lookLimits ? "manifest" : parsed.data.provenance + ? "imported-initial-fallback" : "legacy-layered-fallback", + } : {}), + note: "Look limits bound movement, not visual quality; approval still requires real frame inspection.", }); return; } diff --git a/src/render/scene-plan.ts b/src/render/scene-plan.ts index 8dcb886..c94e768 100644 --- a/src/render/scene-plan.ts +++ b/src/render/scene-plan.ts @@ -1,6 +1,7 @@ import type { Character, Scene, SceneEvent } from "../scene/schemas.js"; import type { RhubarbMouthCue } from "../audio/rhubarb.js"; import { mapRhubarbCuesToRev2DVisemes } from "../audio/visemes.js"; +import { DEFAULT_LOOK_LIMITS, IMPORTED_INITIAL_LOOK_LIMITS } from "../scene/look-limits.js"; export interface AnimationKey { t: number; @@ -30,8 +31,13 @@ export class PreviewUnsupportedError extends Error { type ExpressionEvent = Extract; type LookEvent = Extract; -const HEAD_YAW_LIMIT = 22; -const HEAD_PITCH_LIMIT = 12; + + + +/** Preserve exact zero for disabled look channels (never emit -0). */ +function scaledLook(value: number, limit: number): number { + return value === 0 || limit === 0 ? 0 : value * limit; +} /** Piecewise linear evaluation is only used when selecting a new animation endpoint. */ function valueAt(keys: AnimationKey[], timestamp: number): number { @@ -209,11 +215,18 @@ export function compilePerformance( if (event.kind === "look") { const start = event.atMs / 1000; const end = Math.min(durationSeconds, (event.atMs + event.durationMs) / 1000); + // Imported flat art starts with restrained motion; reviewers can edit + // this envelope after inspecting actual eye, hair and neck poses. + // Pre-rigged layered characters keep the wider legacy mapping. + const bounds = render.lookLimits ?? (character.provenance + ? IMPORTED_INITIAL_LOOK_LIMITS : DEFAULT_LOOK_LIMITS); const mappings = { - AngleX: event.x * HEAD_YAW_LIMIT, - AngleY: -event.y * HEAD_PITCH_LIMIT, - EyeBallX: event.x * 0.5, - EyeBallY: event.y * 0.5, + AngleX: scaledLook(event.x, bounds.headYawDegrees), + AngleY: scaledLook(-event.y, bounds.headPitchDegrees), + EyeBallX: scaledLook(event.x, bounds.eyeX), + // Scene coordinates are screen-space (+y down); Rev2D +Y is up. + // Head and eyes must agree vertically instead of counter-steering. + EyeBallY: scaledLook(-event.y, bounds.eyeY), }; for (const [parameter, goal] of Object.entries(mappings)) { if (!rigDefaults.has(parameter)) { diff --git a/src/scene/look-limits.ts b/src/scene/look-limits.ts new file mode 100644 index 0000000..018afa8 --- /dev/null +++ b/src/scene/look-limits.ts @@ -0,0 +1,34 @@ +/** + * Semantic look controls -> Rev2D rig parameters. + * + * These are *amplitude limits*, not an assessment that an illustration looks + * good at any particular pose. Every imported flat PNG needs actual visual + * review of iris clipping and head/neck distortion. + */ +export interface LookLimits { + headYawDegrees: number; + headPitchDegrees: number; + eyeX: number; + eyeY: number; +} + +export const DEFAULT_LOOK_LIMITS: Readonly = Object.freeze({ + headYawDegrees: 22, + headPitchDegrees: 12, + eyeX: 0.5, + eyeY: 0.5, +}); + +/** + * Conservative initial envelope for imported single-layer illustrations. + * + * Not artist-approved. This is deliberately smaller than the engine's + * supported parameter bounds and should be changed only after reviewing + * actual parameter sweeps for the specific character. + */ +export const IMPORTED_INITIAL_LOOK_LIMITS: Readonly = Object.freeze({ + headYawDegrees: 8, + headPitchDegrees: 5, + eyeX: 0.2, + eyeY: 0.2, +}); diff --git a/src/scene/schemas.ts b/src/scene/schemas.ts index cd6ad0e..42133ac 100644 --- a/src/scene/schemas.ts +++ b/src/scene/schemas.ts @@ -1,4 +1,5 @@ import { z } from "zod"; +import { DEFAULT_LOOK_LIMITS } from "./look-limits.js"; const identifier = z.string().trim().min(1).max(100).regex(/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/); const milliseconds = z.number().int().nonnegative(); @@ -38,6 +39,13 @@ export const characterSchema = z.strictObject({ expressionPresets: z.record(z.string(), z.record(z.string(), z.number())), /** Controls for multiple poses represented by one deformable rig. */ posePresets: z.record(z.string(), z.record(z.string(), z.number())).optional(), + /** Authored look amplitudes; limits alone do not certify visual quality. */ + lookLimits: z.strictObject({ + headYawDegrees: z.number().finite().min(0).max(DEFAULT_LOOK_LIMITS.headYawDegrees), + headPitchDegrees: z.number().finite().min(0).max(DEFAULT_LOOK_LIMITS.headPitchDegrees), + eyeX: z.number().finite().min(0).max(DEFAULT_LOOK_LIMITS.eyeX), + eyeY: z.number().finite().min(0).max(DEFAULT_LOOK_LIMITS.eyeY), + }).optional(), }).optional(), id: identifier, defaultPose: identifier, @@ -98,6 +106,7 @@ export const sceneEventSchema = z.discriminatedUnion("kind", [ z.strictObject({ ...actorEvent, kind: z.literal("look"), + /** Normalized screen-space target: +x right, +y down. */ x: z.number().min(-1).max(1), y: z.number().min(-1).max(1), durationMs: milliseconds.default(250), diff --git a/tests/avatar-import.test.ts b/tests/avatar-import.test.ts index 486ae38..f772afc 100644 --- a/tests/avatar-import.test.ts +++ b/tests/avatar-import.test.ts @@ -45,6 +45,9 @@ describe("agent-first avatar import contracts", () => { expect(result.expressions).toEqual(["neutral", "happy", "confused"]); expect(result.render?.expressionPresets.happy).toEqual({ MouthForm: 0.6 }); expect(result.render?.expressionPresets.confused).toEqual({ MouthForm: -0.3 }); + expect(result.render?.lookLimits).toEqual({ + headYawDegrees: 8, headPitchDegrees: 5, eyeX: 0.2, eyeY: 0.2, + }); expect(result.transitions).toEqual([]); }); diff --git a/tests/scene-plan.test.ts b/tests/scene-plan.test.ts index 3c54039..747215b 100644 --- a/tests/scene-plan.test.ts +++ b/tests/scene-plan.test.ts @@ -60,6 +60,67 @@ describe("performance compiler (one rig)", () => { ]); }); + it("keeps legacy look amplitude for existing layered characters", () => { + const animation = compilePerformance(build([ + { atMs: 200, actor: "hana", kind: "look", x: 1, y: -1, durationMs: 400 }, + ]), actor, "hana", defaultRig); + const values = new Map(animation.tracks.map((track) => + [track.target, track.keys.find((key) => key.t === 0.6)?.v])); + expect(values.get("param:AngleX")).toBe(22); + expect(values.get("param:AngleY")).toBe(12); + expect(values.get("param:EyeBallX")).toBe(0.5); + expect(values.get("param:EyeBallY")).toBe(0.5); + }); + + it("restrains look for legacy flat-image imports even without a new manifest field", () => { + const imported = characterSchema.parse({ + ...actor, provenance: { file: "avatar-build.json" }, + }); + const animation = compilePerformance(build([ + { atMs: 200, actor: "hana", kind: "look", x: 1, y: -1, durationMs: 400 }, + ]), imported, "hana", defaultRig); + const values = new Map(animation.tracks.map((track) => + [track.target, track.keys.find((key) => key.t === 0.6)?.v])); + expect(values.get("param:AngleX")).toBe(8); + expect(values.get("param:AngleY")).toBe(5); + expect(values.get("param:EyeBallX")).toBe(0.2); + expect(values.get("param:EyeBallY")).toBe(0.2); + }); + + it("obeys each explicitly authored look limit independently, including zero", () => { + const authored = characterSchema.parse({ + ...actor, render: { + ...actor.render, + lookLimits: { headYawDegrees: 4, headPitchDegrees: 1, + eyeX: 0, eyeY: 0.1 }, + }, + }); + const animation = compilePerformance(build([ + { atMs: 200, actor: "hana", kind: "look", x: -1, y: 1, durationMs: 400 }, + ]), authored, "hana", defaultRig); + const values = new Map(animation.tracks.map((track) => + [track.target, track.keys.find((key) => key.t === 0.6)?.v])); + expect(values.get("param:AngleX")).toBe(-4); + expect(values.get("param:AngleY")).toBe(-1); + expect(values.get("param:EyeBallX")).toBe(0); + expect(values.get("param:EyeBallY")).toBe(-0.1); + }); + + it("rejects malformed or wider-than-supported look limits", () => { + const limits = { headYawDegrees: 8, headPitchDegrees: 5, eyeX: 0.2, eyeY: 0.2 }; + for (const bad of [ + { ...limits, eyeX: 0.8 }, + { ...limits, headYawDegrees: -1 }, + { ...limits, headPitchDegrees: 30 }, + { ...limits, eyeY: Number.POSITIVE_INFINITY }, + { eyeX: 0.2 }, + { ...limits, extra: true }, + ]) { + expect(characterSchema.safeParse({ ...actor, render: { ...actor.render, lookLimits: bad } }).success) + .toBe(false); + } + }); + it("rejects unsupported pose changes rather than faking a seamless transition", () => { expect(() => compilePerformance(build([ { atMs: 300, actor: "hana", kind: "pose", name: "crossed" },