From f8c509aaba1ec509fc89ae8cc5dff19398c4e502 Mon Sep 17 00:00:00 2001 From: Navneet Chaudhary Date: Sat, 7 Nov 2026 14:40:47 +0530 Subject: [PATCH 1/3] feat: deliver reviewed first-video workflow --- .agents/skills/puppetcontrol/SKILL.md | 6 +- README.md | 8 + docs/CHARACTER_IMPORT.md | 2 +- docs/FIRST_VIDEO.md | 118 +++++++ docs/IMPLEMENTATION_PLAN.md | 15 + package.json | 1 + scripts/check-first-video-video.mjs | 423 ++++++++++++++++++++++++++ scripts/demo-first-video.mjs | 292 ++++++++++++++++++ src/character/avatar.ts | 2 +- src/character/landmarks.ts | 2 +- src/character/review-approval.ts | 241 +++++++++++++++ src/cli/index.ts | 80 ++++- src/render/renderability.ts | 64 ++++ src/render/rev2d.ts | 50 ++- src/scene/create-first-video-file.ts | 118 +++++++ src/scene/first-video.ts | 122 ++++++++ src/scene/load.ts | 9 +- tests/first-video-file.test.ts | 90 ++++++ tests/first-video-video.test.ts | 178 +++++++++++ tests/first-video.test.ts | 79 +++++ tests/renderability.test.ts | 65 ++++ tests/review-approval.test.ts | 141 +++++++++ tests/video-output-atomic.test.ts | 44 +++ 23 files changed, 2118 insertions(+), 32 deletions(-) create mode 100644 docs/FIRST_VIDEO.md create mode 100644 scripts/check-first-video-video.mjs create mode 100644 scripts/demo-first-video.mjs create mode 100644 src/character/review-approval.ts create mode 100644 src/render/renderability.ts create mode 100644 src/scene/create-first-video-file.ts create mode 100644 src/scene/first-video.ts create mode 100644 tests/first-video-file.test.ts create mode 100644 tests/first-video-video.test.ts create mode 100644 tests/first-video.test.ts create mode 100644 tests/renderability.test.ts create mode 100644 tests/review-approval.test.ts create mode 100644 tests/video-output-atomic.test.ts diff --git a/.agents/skills/puppetcontrol/SKILL.md b/.agents/skills/puppetcontrol/SKILL.md index 0b4c8d5..7cba3d2 100644 --- a/.agents/skills/puppetcontrol/SKILL.md +++ b/.agents/skills/puppetcontrol/SKILL.md @@ -13,10 +13,10 @@ Read repo-root AGENTS.md, docs/TECHNICAL_CONCEPT.md, and docs/CHARACTER_IMPORT.m 4. For optional image-model-generated landmarks, run `node dist/cli/index.js character landmarks propose --provider codex` only when the user accepts sending artwork to their configured Codex provider. The command produces a review candidate or diagnostic output, never an approved rig. Otherwise, for a manual agent task, run `node dist/cli/index.js character landmarks assist `. Open source.png, work/grid.png and work/landmark-agent-task.md. Fill the generated JSON proposal template with original source-pixel coordinates; do not guess from resized images. Read .cache/rev2d/docs/AVATAR.md. 5. Stage the proposal using `node dist/cli/index.js character landmarks stage --proposal `. Rev2D validates the isolated candidate and returns a work/overlay.png. **Actually inspect the overlay** before `character landmarks accept --candidate --reviewed`. Accepting replaces avatar.json with a backup and revokes all existing pose approvals. Never auto-pass --reviewed solely because a validator passed. 6. Build using node dist/cli/index.js character build (use --force if prior output exists). Inspect any geometryWarnings, particularly BODY_PIVOT_TOO_LOW and IRIS_LANDMARK_MISSING; these do not replace pixel inspection. Source/landmark/model/texture hashes are recorded in avatar-build.json and scene preflight rejects stale assets. Inspect every returned review/poses.png, turn.png, eyes.png and mouth.png. If incorrect, repair avatar.json and rebuild using --force; this resets the generated manifest and any edited scaffold. -7. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA should an agent/user mark the neutral pose approved. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction. -8. The generated preview-scene.json is editable. Use node dist/cli/index.js scene validate and scene plan before rendering. Preview PNG frames, inspect key timestamps, then render and inspect the MP4. +7. A successful build intentionally leaves character.json with approved:false. A --force rebuild revokes approval *before* writing model/texture files, even if the rebuild later fails. Only after actual visual QA may an agent/user explicitly run `node dist/cli/index.js character approve --reviewed` (the flag is an acknowledgement, not evidence of review) to record neutral-pose approval. If there are torn eyelids, missing areas, extra contours or identity drift, LEAVE IT UNAPPROVED and report what needs correction. +8. For a reviewed reusable rig, use `node dist/cli/index.js scene new --character --seconds 12 --fps 15` to create a strict editable scene. Reuse the same character file in a second scene without rebuilding artwork. The generated preview-scene.json is also editable. Use node dist/cli/index.js scene validate and scene plan before rendering. Preview PNG frames, inspect key timestamps, then render and inspect the MP4. 9. Within a compatible shared rig, semantic pose events may interpolate through validated parameter presets (the neutral-to-pointing demo uses shoulder and elbow controls). Distinct pose models require explicit scene camera cuts; an authored bridge is NOT implemented. Never silently crossfade meshes or call a body cut a smooth physical transition. -10. For speech, prepare WAV, install Rhubarb separately, and use node dist/cli/index.js audio align --out [--binary ]. Reference audio.path, audio.startMs and audio.cuesPath from the scene; review the actual mouth/audio timing. +10. For speech, prepare WAV, install Rhubarb separately, and use node dist/cli/index.js audio align --out [--binary ]. A `scene new` voice track requires both `--voice` and the actual `--cues` file; no fabricated speech sync. Reference audio.path, audio.startMs and audio.cuesPath from the scene; review the actual mouth/audio timing. 11. Do not commit private character illustrations, full generated models, user voice recordings or secrets without checking redistribution rights. Demo:avatar generates pinned MIT-licensed sample artwork into ignored .cache. 12. State what actually rendered and which controls remain unsupported. Do not claim a flat PNG is automatically converted into production-quality animation without landmark review. diff --git a/README.md b/README.md index 5ec9f50..3fbdc10 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,9 @@ The long-term goal: **generate artwork when necessary, animate and reuse it inde - Assisted single-PNG character onboarding through Rev2D landmarks, local rig build, review sheets, unapproved-by-default manifest, and starter scene. - Agent-guided landmark proposals: source-hash-bound task, optional **tested** Codex image-based suggestion, isolated geometry validation, visual overlay and explicit review/acceptance that revokes old rig approval. - Imported avatar asset provenance: source/spec/model/texture fingerprints checked by scene validation and render; untracked alternate imported pose models are refused. Force rebuild revokes approval before modifying art. Suspicious landmarks generate nonblocking geometry warnings. +- An opt-in `character approve --reviewed` gate for visually inspected imported rigs; asset fingerprints and review sheets are checked before approval. +- `scene new` creates editable expression/look/dialogue scenes from an already approved rig without changing the character; voice requires genuine Rhubarb cues. +- Non-destructive MP4 export: output files are promoted only after successful FFmpeg encoding, with overwrite requiring explicit `--force`. - Agent-readable CLI output, tests and CI. **Not implemented yet:** fully automatic landmark placement or reliable one-click rigging, AI-generated arbitrary new pose rigs, convincing smooth bridges between unrelated meshes (especially crossed arms), automatic transcription/TTS, captions, B-roll and multiple actors. The arm-raise gesture is genuinely animated, but its artwork is still prototype-quality. Other pose families may require edited hard cuts. The pose manifest is capability metadata; visual poses become usable only after their rig files are generated, validated and inspected. @@ -42,6 +45,9 @@ npm run demo:directed # Apply edited camera framing and gradual punch-in npm run demo:gesture # Generate two-joint gesture rig and export edited 1080p demo npm run demo:gesture-cycle # Render continuous raise/lower cycles without cuts npm run demo:avatar # Import a freshly rendered flat PNG and create an unapproved rig + review sheets +npm run demo:first-video # Prepare an ignored, unapproved Hana fixture for review +# After actually reviewing the pinned Hana test fixture only: +# node scripts/demo-first-video.mjs --reviewed-sample ~~~ The sample uses Rev2D's own **Hana** rig from the pinned MIT-licensed upstream checkout; generated character rig JSON is not checked into this repository. The derived example video is separately attributed and licensed under the original Rev2D terms. @@ -54,6 +60,8 @@ The sample uses Rev2D's own **Hana** rig from the pinned MIT-licensed upstream c [Watch the reversible gesture demo](docs/media/hana-gesture-cycle.mp4) (5 seconds, silent 30 fps; the same arm moves between neutral and pointing four times with no cuts). +For the first reproducible **one-character video workflow**, see [First Video](docs/FIRST_VIDEO.md). With an already reviewed rig, run `character approve --reviewed` for provenance-tracked imported characters, then `scene new --character `, `scene validate` and `scene render`. Create another scene with the same character path to reuse its rig, without regenerating artwork. To attach your own voice, use a real WAV and aligned Rhubarb cue JSON; no TTS is implied. + Run a custom scene through the CLI: ~~~sh diff --git a/docs/CHARACTER_IMPORT.md b/docs/CHARACTER_IMPORT.md index c2ca564..35913c5 100644 --- a/docs/CHARACTER_IMPORT.md +++ b/docs/CHARACTER_IMPORT.md @@ -68,7 +68,7 @@ This revokes prior approval **before** the rendering backend touches the rig, an Inspect every review sheet for torn eye closures, halos, doubled contours, missing hair/skin, incorrect mouth shapes, and awkward head movement. Correct the landmark source and rebuild as needed. -When the rendered rig is genuinely acceptable, change the neutral pose family's approved field to true in character.json. Only then: +When the rendered rig is genuinely acceptable, run `node dist/cli/index.js character approve avatars/portrait --reviewed`. This requires four existing review PNGs and a current source/spec/model/texture provenance record. The flag acknowledges review but cannot prove that someone genuinely inspected the artwork; never pass it automatically for user images. Only then: ~~~sh node dist/cli/index.js scene validate avatars/portrait/preview-scene.json diff --git a/docs/FIRST_VIDEO.md b/docs/FIRST_VIDEO.md new file mode 100644 index 0000000..d41c52f --- /dev/null +++ b/docs/FIRST_VIDEO.md @@ -0,0 +1,118 @@ +# First usable video: two scenes, one reviewed rig + +This page documents the smallest truthful PuppetControl flow that produces real MP4 files while reusing one already-reviewed Rev2D actor. + +The integration demo uses the pinned MIT-licensed Rev2D **Hana** sample only. It never approves user artwork. Its approval marker lives in an ignored .cache/first-video-demo/... fixture and is created only after an explicit --reviewed-sample flag. + +## Prerequisites + +Use Node.js 22 or newer and install the repository dependencies. FFmpeg and FFprobe must be available on PATH. + +Prepare the pinned rig backend and build the current CLI: + + npm run rig:setup + npm run build + +The expected Rev2D commit is f144581cdf1f12d781379c4df90d262b8c2dfa3d. + +## Reproduce the two-video Hana demo + +First prepare an **unapproved** ignored fixture and a PNG for visual review: + + node scripts/demo-first-video.mjs + +The command prints a path such as: + + .cache/first-video-demo/run-XXXXXX/hana-review.png + +Open that PNG and inspect the sample. PuppetControl cannot technically verify that a human actually reviewed or consented to an asset. + +Only for this pinned Hana QA fixture, explicitly continue with: + + node scripts/demo-first-video.mjs --reviewed-sample + +That second command uses the current CLI scene new, scene validate, and scene render commands to create two separate 5-second scenes that both point at the same copied Hana rig. It then runs the FFprobe acceptance checker on both MP4s. + +Each run is isolated under ignored .cache/first-video-demo/run-XXXXXX/. Expected outputs are: + + outputs/hana-scene-a.mp4 + outputs/hana-scene-b.mp4 + +Both videos are silent H.264 MP4s at 512x640, 15 fps, 75 frames, and 5 seconds. The scenes differ in supported expression/look timing while keeping the same actor and rig bytes. + +The checker validates container/stream metadata, dimensions, fps, duration, frame count, expected lack of audio, and availability of FFmpeg/FFprobe. That is **not** a visual-quality approval; inspect actual frames/video as part of QA. + +## Use your own approved rig + +The scene new command is for an actor that has already passed the project's real review/approval workflow. It refuses an unapproved default pose and does not auto-approve artwork. For a single-image Rev2D import, after genuinely inspecting all four review sheets, acknowledge approval explicitly with: + + node dist/cli/index.js character approve path/to/avatar-project --reviewed + +This validates source/landmark/rig/texture fingerprints and presence of review sheets before recording approval. It cannot technically verify that a human looked at them. Do not script approval for unreviewed user art. + +After npm run build, create an editable scene: + + node dist/cli/index.js scene new .cache/my-video/scene-a.json \ + --character path/to/character.json \ + --id scene-a \ + --seconds 5 \ + --fps 15 \ + --size 512x640 + +Edit only events that the character actually supports, then validate and render: + + node dist/cli/index.js scene validate .cache/my-video/scene-a.json + node dist/cli/index.js scene render .cache/my-video/scene-a.json --out .cache/my-video/scene-a.mp4 + +For a second scene, run scene new again with a different output scene path and ID while keeping the same --character path. That reuses the same approved rig rather than rebuilding or regenerating the actor. + +## Optional WAV + Rhubarb cues + +Voice is optional. PuppetControl does not synthesize speech or pretend that lip sync exists when cues are missing. + +If Rhubarb is installed, align a WAV first: + + node dist/cli/index.js audio align path/to/voice.wav \ + --out .cache/my-video/mouth.json \ + --binary rhubarb + +Then create the scene with both the WAV and cue JSON: + + node dist/cli/index.js scene new .cache/my-video/voiced.json \ + --character path/to/character.json \ + --id voiced \ + --seconds 5 \ + --fps 15 \ + --size 512x640 \ + --voice path/to/voice.wav \ + --cues .cache/my-video/mouth.json + +The current first-video path requires --voice and --cues together. Validate and render the resulting scene normally. A real 12-second test using local nonredistributed voice input successfully generated 36 Rhubarb cues and a 180-frame H.264/AAC MP4 at 512×640, 15 FPS. The audio recording, cues and MP4 are intentionally kept only in ignored .cache/first-video-demo/qa-12s-voiced/. This proves file/cue/mux integration, not perceptual lip-sync quality on every speaker. + +## Non-destructive rendering + +By default, PuppetControl refuses to replace an existing MP4 at the chosen output path. The renderer writes to a unique temporary MP4 and publishes it only after FFmpeg successfully completes. To explicitly re-render over the same output path after editing a scene, use `--force`: + + node dist/cli/index.js scene render .cache/my-video/scene-a.json --out .cache/my-video/scene-a.mp4 --force + +For ordinary revisions, prefer a new `--out` name so the prior render remains available. Scene IDs by default follow the scene filename, reducing collisions between different scenes. + +## Verify an exported MP4 + +For a 5-second 15-fps 512x640 silent export: + + node scripts/check-first-video-video.mjs .cache/my-video/scene-a.mp4 \ + --fps 15 --width 512 --height 640 --frames 75 --video-codec h264 + +For a voiced export, add --audio --audio-codec aac. + +## Current limitations + +- The first-video flow is intentionally one reviewed reusable actor; it is not a reliable one-click importer or automatic approval system. +- Approval state is a technical gate, not proof that a human reviewed the art or that the creator has rights/consent. +- The Hana --reviewed-sample path is a test-only exception isolated under ignored .cache; it must not be copied into production approval logic. +- Voice requires an existing WAV plus valid Rhubarb cue JSON. There is no automatic transcription or TTS in this flow. +- Current rendering supports at most one audio track and the first-video scaffold is aimed at one actor. +- Smooth arbitrary between-pose bridges are not implemented. Do not present hard cuts or unrelated-rig switches as fluid motion. +- Metadata acceptance with FFprobe does not establish animation quality. Inspect the rendered frames/video. +- Native rig rendering and presentation/output scaling are separate concerns; use dimensions that the current renderer has actually been tested with. diff --git a/docs/IMPLEMENTATION_PLAN.md b/docs/IMPLEMENTATION_PLAN.md index 22fd6c1..65ac334 100644 --- a/docs/IMPLEMENTATION_PLAN.md +++ b/docs/IMPLEMENTATION_PLAN.md @@ -79,6 +79,21 @@ - [ ] Add pixel-level motion QA probes and a reviewable report; automated geometry warnings are not proof of visual quality. - [ ] Wrap Rev2D's existing drawn closed-eye/mouth variants to reduce visible deformation artifacts without writing another mesh engine. +## First video milestone — reproducible usable slice (2026-11-07) + +This milestone is deliberately **narrower** than M5's universal single-image promise. The verified workflow uses one previously reviewed Rev2D character, not an arbitrary one-click PNG rig. Commands and proof: [FIRST_VIDEO.md](FIRST_VIDEO.md). + +- [x] Reuse an approved rig in two independently authored 5-second scenes, without changing character model, texture or manifest. +- [x] Create strict editable scene files via `scene new`, instead of hand-editing TypeScript; reject unsupported events and unapproved characters. +- [x] Add an explicit `character approve --reviewed` gate for one provenance-tracked imported rig with required review sheets. Software cannot prove human review, so this is a record of an explicit acknowledgement, never an automatic decision. +- [x] Validate the actual 75-frame, 15fps, 512×640 H.264 output of both scenes using FFprobe and visually inspect representative frames. +- [x] Render an additional 12-second, 180-frame H.264/AAC sample using 36 real Rhubarb cues from a local test WAV; verify both streams and inspect representative frames. Local audio and video are ignored, not published. +- [x] Prevent silent overwrite of existing MP4s by encoding to a temporary sibling file; `--force` explicitly replaces only after successful encoding. +- [x] Make `scene validate` distinguish planning-valid bridge/caption/motion documents from scenes the current renderer can actually export, with actionable errors. +- [ ] Achieve visibly polished eye/mouth deformation and user-friendly assisted rig correction on **diverse** artwork; the imported Hana avatar previously showed artifacts and remains unapproved. +- [ ] Verify acceptable lip synchronization with multiple natural speakers, and bind Rhubarb cue data to its source-audio fingerprint. +- [ ] Produce a convincing, reproducible 10–15s video **using an independently imported and genuinely approved custom illustration** without developer intervention. The 12s test so far uses a pre-rigged licensed sample. + ## Acceptance benchmark A single 45-second 1080p/30 video with human/AI voice, four pose families (three minimum), convincing continuous head/face movement, visually acceptable lip-sync, smooth or intentionally hidden pose changes, consistent identity, and a successful second export after a one-line expression edit. diff --git a/package.json b/package.json index d43886b..e0e7981 100644 --- a/package.json +++ b/package.json @@ -26,6 +26,7 @@ "demo:gesture-cycle": "npm run demo:gesture:rig && tsx src/cli/index.ts scene render examples/scenes/hana-gesture-cycle.json", "demo:benchmark": "npm run demo:gesture:rig && tsx src/cli/index.ts scene render examples/scenes/hana-45s-benchmark.json", "demo:avatar": "npm run build && node scripts/demo-avatar-import.mjs", + "demo:first-video": "npm run build && node scripts/demo-first-video.mjs", "test": "vitest run", "typecheck": "tsc --noEmit", "build": "tsc", diff --git a/scripts/check-first-video-video.mjs b/scripts/check-first-video-video.mjs new file mode 100644 index 0000000..deb8a0d --- /dev/null +++ b/scripts/check-first-video-video.mjs @@ -0,0 +1,423 @@ +#!/usr/bin/env node + +import { execFileSync } from "node:child_process"; +import { realpath, stat } from "node:fs/promises"; +import { dirname, extname, isAbsolute, relative, resolve, sep } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; + +const scriptDir = dirname(fileURLToPath(import.meta.url)); +export const projectRoot = resolve(scriptDir, ".."); +export const DEFAULT_MAX_BYTES = 2 * 1024 * 1024 * 1024; +export const DEFAULT_FPS_TOLERANCE = 0.001; +export const DEFAULT_DURATION_TOLERANCE_MS = 50; +export const DEFAULT_FRAME_TOLERANCE = 0; + +function requireValue(argv, index, option) { + const value = argv[index + 1]; + if (value === undefined || value.startsWith("--")) { + throw new Error(`${option} requires a value`); + } + return value; +} + +function parseFinitePositive(name, value) { + const parsed = Number(value); + if (!Number.isFinite(parsed) || parsed <= 0) { + throw new Error(`${name} must be a finite positive number`); + } + return parsed; +} + +function parseNonNegative(name, value) { + const parsed = Number(value); + if (!Number.isFinite(parsed) || parsed < 0) { + throw new Error(`${name} must be a finite non-negative number`); + } + return parsed; +} + +function parsePositiveInteger(name, value) { + const parsed = Number(value); + if (!Number.isSafeInteger(parsed) || parsed <= 0) { + throw new Error(`${name} must be a positive integer`); + } + return parsed; +} + +function parseNonNegativeInteger(name, value) { + const parsed = Number(value); + if (!Number.isSafeInteger(parsed) || parsed < 0) { + throw new Error(`${name} must be a non-negative integer`); + } + return parsed; +} + +export function parseArgs(argv) { + const options = { + path: undefined, + fps: undefined, + width: undefined, + height: undefined, + frames: undefined, + audio: false, + fpsTolerance: DEFAULT_FPS_TOLERANCE, + durationToleranceMs: DEFAULT_DURATION_TOLERANCE_MS, + frameTolerance: DEFAULT_FRAME_TOLERANCE, + maxBytes: DEFAULT_MAX_BYTES, + videoCodec: undefined, + audioCodec: undefined, + help: false, + }; + const seen = new Set(); + + const setOnce = (key, option, value) => { + if (seen.has(key)) throw new Error(`${option} may only be specified once`); + seen.add(key); + options[key] = value; + }; + + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + + if (arg === "-h" || arg === "--help") { + options.help = true; + continue; + } + if (arg === "--audio") { + if (seen.has("audio")) throw new Error("--audio may only be specified once"); + seen.add("audio"); + options.audio = true; + continue; + } + + if (arg.startsWith("--")) { + const value = requireValue(argv, i, arg); + i += 1; + switch (arg) { + case "--fps": + setOnce("fps", arg, parseFinitePositive(arg, value)); + break; + case "--width": + setOnce("width", arg, parsePositiveInteger(arg, value)); + break; + case "--height": + setOnce("height", arg, parsePositiveInteger(arg, value)); + break; + case "--frames": + setOnce("frames", arg, parsePositiveInteger(arg, value)); + break; + case "--fps-tolerance": + setOnce("fpsTolerance", arg, parseNonNegative(arg, value)); + break; + case "--duration-tolerance-ms": + setOnce("durationToleranceMs", arg, parseNonNegative(arg, value)); + break; + case "--frame-tolerance": + setOnce("frameTolerance", arg, parseNonNegativeInteger(arg, value)); + break; + case "--max-bytes": + setOnce("maxBytes", arg, parsePositiveInteger(arg, value)); + break; + case "--video-codec": + if (!/^[A-Za-z0-9_.-]+$/.test(value)) throw new Error("--video-codec contains invalid characters"); + setOnce("videoCodec", arg, value.toLowerCase()); + break; + case "--audio-codec": + if (!/^[A-Za-z0-9_.-]+$/.test(value)) throw new Error("--audio-codec contains invalid characters"); + setOnce("audioCodec", arg, value.toLowerCase()); + break; + default: + throw new Error(`unknown option: ${arg}`); + } + continue; + } + + if (options.path !== undefined) { + throw new Error(`unexpected positional argument: ${arg}`); + } + if (arg.includes("\0")) throw new Error("path must not contain a null byte"); + options.path = arg; + } + + if (options.help) return options; + + for (const key of ["path", "fps", "width", "height", "frames"]) { + if (options[key] === undefined) { + throw new Error(`missing required argument: ${key === "path" ? "" : `--${key}`}`); + } + } + if (options.audioCodec && !options.audio) { + throw new Error("--audio-codec requires --audio"); + } + + return options; +} + +export function usage() { + return `Usage: + node scripts/check-first-video-video.mjs --fps --width --height --frames [--audio] [options] + +Checks an already-exported local MP4 without modifying it. + +Required: + MP4 path inside this PuppetControl workspace + --fps Expected video frame rate + --width Expected coded width + --height Expected coded height + --frames Expected decoded video frame count + +Audio: + --audio Require exactly one audio stream + (without it, require zero audio streams) + +Optional stricter/limit checks: + --fps-tolerance Absolute FPS tolerance (default: ${DEFAULT_FPS_TOLERANCE}) + --duration-tolerance-ms Duration tolerance vs frames/fps (default: ${DEFAULT_DURATION_TOLERANCE_MS}) + --frame-tolerance Allowed frame-count difference (default: ${DEFAULT_FRAME_TOLERANCE}) + --max-bytes Maximum accepted file size (default: ${DEFAULT_MAX_BYTES}) + --video-codec Require a specific ffprobe video codec name, e.g. h264 + --audio-codec With --audio, require a specific audio codec name, e.g. aac + -h, --help Show this help + +Acceptance is metadata/stream validation only. It does not approve animation or visual quality. +`; +} + +export function isPathInsideRoot(root, candidate) { + const rel = relative(resolve(root), resolve(candidate)); + return rel === "" || (rel !== ".." && !rel.startsWith(`..${sep}`) && !isAbsolute(rel)); +} + +async function resolveConfinedVideo(inputPath, maxBytes) { + const lexicalRoot = resolve(projectRoot); + const candidate = resolve(lexicalRoot, inputPath); + if (!isPathInsideRoot(lexicalRoot, candidate)) { + throw new Error("video path must stay inside the PuppetControl workspace"); + } + if (extname(candidate).toLowerCase() !== ".mp4") { + throw new Error("video path must point to an .mp4 file"); + } + + const [realRoot, realCandidate] = await Promise.all([ + realpath(lexicalRoot), + realpath(candidate), + ]); + if (!isPathInsideRoot(realRoot, realCandidate)) { + throw new Error("video path resolves outside the PuppetControl workspace"); + } + + const info = await stat(realCandidate); + if (!info.isFile()) throw new Error("video path must point to a regular file"); + if (info.size <= 0) throw new Error("video file is empty"); + if (info.size > maxBytes) { + throw new Error(`video file is too large: ${info.size} bytes exceeds limit ${maxBytes}`); + } + return { path: realCandidate, size: info.size }; +} + +function toolVersion(command) { + try { + const stdout = execFileSync(command, ["-version"], { + cwd: projectRoot, + encoding: "utf8", + windowsHide: true, + timeout: 5000, + maxBuffer: 256 * 1024, + }); + return String(stdout).split(/\r?\n/, 1)[0]?.trim() || `${command} available`; + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + throw new Error(`${command} is required but unavailable: ${detail}`); + } +} + +export function parseRate(value) { + if (typeof value !== "string" || value.length === 0 || value === "N/A") return undefined; + const slash = value.indexOf("/"); + if (slash >= 0) { + const numerator = Number(value.slice(0, slash)); + const denominator = Number(value.slice(slash + 1)); + if (!Number.isFinite(numerator) || !Number.isFinite(denominator) || denominator === 0) return undefined; + const rate = numerator / denominator; + return Number.isFinite(rate) && rate > 0 ? rate : undefined; + } + const parsed = Number(value); + return Number.isFinite(parsed) && parsed > 0 ? parsed : undefined; +} + +function parseCount(value) { + if (typeof value !== "string" && typeof value !== "number") return undefined; + const parsed = Number(value); + return Number.isSafeInteger(parsed) && parsed >= 0 ? parsed : undefined; +} + +function parseDuration(value) { + if (typeof value !== "string" && typeof value !== "number") return undefined; + const parsed = Number(value); + return Number.isFinite(parsed) && parsed >= 0 ? parsed : undefined; +} + +export function evaluateProbe(probe, expected) { + const errors = []; + const streams = Array.isArray(probe?.streams) ? probe.streams : []; + const videos = streams.filter((stream) => stream?.codec_type === "video"); + const audios = streams.filter((stream) => stream?.codec_type === "audio"); + + if (videos.length !== 1) { + errors.push(`expected exactly one video stream, found ${videos.length}`); + } + const video = videos[0]; + + const width = Number(video?.width); + const height = Number(video?.height); + if (!Number.isSafeInteger(width) || width <= 0) errors.push("ffprobe did not report a valid video width"); + else if (width !== expected.width) errors.push(`width mismatch: expected ${expected.width}, got ${width}`); + if (!Number.isSafeInteger(height) || height <= 0) errors.push("ffprobe did not report a valid video height"); + else if (height !== expected.height) errors.push(`height mismatch: expected ${expected.height}, got ${height}`); + + const fps = parseRate(video?.avg_frame_rate) ?? parseRate(video?.r_frame_rate); + if (fps === undefined) errors.push("ffprobe did not report a usable video frame rate"); + else if (Math.abs(fps - expected.fps) > expected.fpsTolerance) { + errors.push(`fps mismatch: expected ${expected.fps} ± ${expected.fpsTolerance}, got ${fps}`); + } + + const frameCount = parseCount(video?.nb_read_frames) ?? parseCount(video?.nb_frames); + if (frameCount === undefined) errors.push("ffprobe did not report a decoded/container video frame count"); + else if (Math.abs(frameCount - expected.frames) > expected.frameTolerance) { + errors.push(`frame count mismatch: expected ${expected.frames} ± ${expected.frameTolerance}, got ${frameCount}`); + } + + const duration = parseDuration(video?.duration) ?? parseDuration(probe?.format?.duration); + const expectedDuration = expected.frames / expected.fps; + if (duration === undefined) errors.push("ffprobe did not report a usable video duration"); + else { + const deltaMs = Math.abs(duration - expectedDuration) * 1000; + if (deltaMs > expected.durationToleranceMs) { + errors.push( + `duration mismatch: expected ${expectedDuration.toFixed(6)}s ± ${expected.durationToleranceMs}ms, got ${duration.toFixed(6)}s`, + ); + } + } + + const formatNames = typeof probe?.format?.format_name === "string" + ? probe.format.format_name.split(",").map((name) => name.trim().toLowerCase()) + : []; + if (!formatNames.includes("mp4")) { + errors.push(`container mismatch: expected MP4, ffprobe reported ${probe?.format?.format_name ?? "unknown"}`); + } + + if (expected.audio) { + if (audios.length !== 1) errors.push(`expected exactly one audio stream, found ${audios.length}`); + } else if (audios.length !== 0) { + errors.push(`expected no audio stream, found ${audios.length}`); + } + + const videoCodec = typeof video?.codec_name === "string" ? video.codec_name.toLowerCase() : undefined; + if (expected.videoCodec && videoCodec !== expected.videoCodec) { + errors.push(`video codec mismatch: expected ${expected.videoCodec}, got ${videoCodec ?? "unknown"}`); + } + + const audioCodec = audios.length === 1 && typeof audios[0]?.codec_name === "string" + ? audios[0].codec_name.toLowerCase() + : undefined; + if (expected.audioCodec && audioCodec !== expected.audioCodec) { + errors.push(`audio codec mismatch: expected ${expected.audioCodec}, got ${audioCodec ?? "unknown"}`); + } + + return { + ok: errors.length === 0, + errors, + observed: { + width: Number.isSafeInteger(width) ? width : null, + height: Number.isSafeInteger(height) ? height : null, + fps: fps ?? null, + frames: frameCount ?? null, + durationSeconds: duration ?? null, + expectedDurationSeconds: expectedDuration, + videoStreams: videos.length, + audioStreams: audios.length, + format: probe?.format?.format_name ?? null, + videoCodec: videoCodec ?? null, + audioCodec: audioCodec ?? null, + }, + }; +} + +function probeVideo(videoPath) { + const stdout = execFileSync("ffprobe", [ + "-v", "error", + "-count_frames", + "-show_entries", + "stream=index,codec_type,codec_name,width,height,avg_frame_rate,r_frame_rate,nb_frames,nb_read_frames,duration:format=format_name,duration,size", + "-of", "json", + videoPath, + ], { + cwd: projectRoot, + encoding: "utf8", + windowsHide: true, + timeout: 30000, + maxBuffer: 2 * 1024 * 1024, + }); + return JSON.parse(stdout); +} + +async function main() { + let options; + try { + options = parseArgs(process.argv.slice(2)); + if (options.help) { + process.stdout.write(usage()); + return; + } + + const video = await resolveConfinedVideo(options.path, options.maxBytes); + const ffmpegVersion = toolVersion("ffmpeg"); + const ffprobeVersion = toolVersion("ffprobe"); + const probe = probeVideo(video.path); + const result = evaluateProbe(probe, options); + + process.stdout.write(JSON.stringify({ + ok: result.ok, + checker: "first-video-video", + path: relative(projectRoot, video.path).replaceAll("\\", "/"), + sizeBytes: video.size, + tools: { + ffmpeg: ffmpegVersion, + ffprobe: ffprobeVersion, + }, + expected: { + fps: options.fps, + width: options.width, + height: options.height, + frames: options.frames, + audio: options.audio, + fpsTolerance: options.fpsTolerance, + durationToleranceMs: options.durationToleranceMs, + frameTolerance: options.frameTolerance, + maxBytes: options.maxBytes, + videoCodec: options.videoCodec ?? null, + audioCodec: options.audioCodec ?? null, + }, + ...result, + note: "Metadata/stream acceptance only; this does not approve animation or visual quality.", + }, null, 2) + "\n"); + + if (!result.ok) process.exitCode = 1; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + process.stderr.write(JSON.stringify({ + ok: false, + checker: "first-video-video", + error: message, + note: "Metadata/stream acceptance only; this does not approve animation or visual quality.", + }, null, 2) + "\n"); + process.exitCode = 1; + } +} + +const invokedDirectly = process.argv[1] + ? pathToFileURL(resolve(process.argv[1])).href === import.meta.url + : false; +if (invokedDirectly) { + await main(); +} diff --git a/scripts/demo-first-video.mjs b/scripts/demo-first-video.mjs new file mode 100644 index 0000000..1659dfd --- /dev/null +++ b/scripts/demo-first-video.mjs @@ -0,0 +1,292 @@ +#!/usr/bin/env node + +/** + * Reproducible first-video integration demo. + * + * The demo never approves user artwork. It copies the pinned MIT-licensed + * Rev2D Hana sample into an ignored .cache fixture, renders a review PNG, and + * only marks that ephemeral sample fixture approved when --reviewed-sample is + * supplied explicitly. + */ + +import { execFileSync } from "node:child_process"; +import { + access, + copyFile, + mkdir, + mkdtemp, + readFile, + writeFile, +} from "node:fs/promises"; +import { dirname, join, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const REV2D_SHA = "f144581cdf1f12d781379c4df90d262b8c2dfa3d"; +const FPS = 15; +const DURATION_SECONDS = 5; +const WIDTH = 512; +const HEIGHT = 640; +const FRAME_COUNT = FPS * DURATION_SECONDS; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const rev2dRoot = join(root, ".cache", "rev2d"); +const rev2dModel = join(rev2dRoot, "examples", "hana", "hana.r2d.json"); +const rev2dCli = join(rev2dRoot, "bin", "rev2d.js"); +const cli = join(root, "dist", "cli", "index.js"); +const checker = join(root, "scripts", "check-first-video-video.mjs"); + +function repoRelative(path) { + return relative(root, path).replaceAll("\\", "/"); +} + +function parseArgs(argv) { + if (argv.includes("--help") || argv.includes("-h")) { + process.stdout.write( + "Usage: node scripts/demo-first-video.mjs [--reviewed-sample]\n\n" + + "Without --reviewed-sample, only an ignored Hana review fixture is prepared.\n" + + "After visually reviewing the generated PNG, rerun with --reviewed-sample to\n" + + "create two 5-second scenes, render both MP4s, and verify them with FFprobe.\n", + ); + return { help: true, reviewedSample: false }; + } + const unknown = argv.filter((arg) => arg !== "--reviewed-sample"); + if (unknown.length > 0) throw new Error("Unknown argument: " + unknown[0]); + return { help: false, reviewedSample: argv.includes("--reviewed-sample") }; +} + +function execJson(command, args, label, maxBuffer = 32 * 1024 * 1024) { + try { + const stdout = execFileSync(command, args, { + cwd: root, + encoding: "utf8", + windowsHide: true, + timeout: 180_000, + maxBuffer, + }); + const parsed = JSON.parse(stdout); + if (parsed.valid === false || parsed.ok === false) { + throw new Error(JSON.stringify(parsed)); + } + return parsed; + } catch (error) { + const stdout = error && typeof error === "object" && "stdout" in error + ? String(error.stdout ?? "").trim() + : ""; + const stderr = error && typeof error === "object" && "stderr" in error + ? String(error.stderr ?? "").trim() + : ""; + const detail = [stdout, stderr, error instanceof Error ? error.message : String(error)] + .filter(Boolean) + .join("\n"); + throw new Error(label + " failed:\n" + detail); + } +} + +function invokeCli(args) { + return execJson(process.execPath, [cli, ...args], "puppetctl " + args.join(" ")); +} + +function invokeChecker(videoPath) { + return execJson(process.execPath, [ + checker, + repoRelative(videoPath), + "--fps", String(FPS), + "--width", String(WIDTH), + "--height", String(HEIGHT), + "--frames", String(FRAME_COUNT), + "--video-codec", "h264", + ], "first-video FFprobe acceptance"); +} + +function assertPinnedRev2D() { + const actual = execFileSync("git", ["-C", rev2dRoot, "rev-parse", "HEAD"], { + cwd: root, + encoding: "utf8", + windowsHide: true, + timeout: 10_000, + }).trim(); + if (actual !== REV2D_SHA) { + throw new Error( + "Pinned Rev2D checkout required. Expected " + REV2D_SHA + + " but found " + actual + ". Run npm run rig:setup.", + ); + } +} + +function assertBuiltCliHasSceneNew() { + const help = execFileSync(process.execPath, [cli, "--help"], { + cwd: root, + encoding: "utf8", + windowsHide: true, + timeout: 10_000, + }); + if (!help.includes("scene new")) { + throw new Error("Built CLI does not contain 'scene new'. Run npm run build on the integrated first-video branch."); + } +} + +function renderReviewPng(modelPath, outputPath) { + return execJson(process.execPath, [ + rev2dCli, + "render", modelPath, + "-o", outputPath, + "--json", + ], "Rev2D Hana review render"); +} + +async function createDemoCharacter(characterFile, modelFile, reviewedSample) { + const template = JSON.parse( + await readFile(join(root, "examples", "characters", "hana", "character.json"), "utf8"), + ); + + const fixture = { + ...template, + id: "hana-first-video-demo", + defaultPose: "neutral", + poseFamilies: [{ + name: "neutral", + rigId: "hana-first-video-demo-neutral", + head: "shared", + approved: reviewedSample, + }], + transitions: [], + render: { + ...template.render, + modelPath: "hana.r2d.json", + }, + }; + + await copyFile(rev2dModel, modelFile); + await writeFile(characterFile, JSON.stringify(fixture, null, 2) + "\n", "utf8"); +} + +async function replaceSceneEvents(sceneFile, events) { + const scene = JSON.parse(await readFile(sceneFile, "utf8")); + scene.events = events; + await writeFile(sceneFile, JSON.stringify(scene, null, 2) + "\n", "utf8"); +} + +async function main() { + const options = parseArgs(process.argv.slice(2)); + if (options.help) return; + + await Promise.all([access(rev2dModel), access(rev2dCli), access(cli), access(checker)]); + assertPinnedRev2D(); + + const generatedRoot = join(root, ".cache", "first-video-demo"); + await mkdir(generatedRoot, { recursive: true }); + const runDir = await mkdtemp(join(generatedRoot, "run-")); + const characterDir = join(runDir, "character"); + const scenesDir = join(runDir, "scenes"); + const outputsDir = join(runDir, "outputs"); + await Promise.all([ + mkdir(characterDir, { recursive: true }), + mkdir(scenesDir, { recursive: true }), + mkdir(outputsDir, { recursive: true }), + ]); + + const modelFile = join(characterDir, "hana.r2d.json"); + const characterFile = join(characterDir, "character.json"); + const reviewPng = join(runDir, "hana-review.png"); + await createDemoCharacter(characterFile, modelFile, options.reviewedSample); + const review = renderReviewPng(modelFile, reviewPng); + + if (!options.reviewedSample) { + process.stdout.write(JSON.stringify({ + valid: true, + status: "review-required", + runDir: repoRelative(runDir), + reviewPng: repoRelative(reviewPng), + reviewDimensions: { width: review.width, height: review.height }, + approved: false, + next: "Visually inspect the review PNG, then rerun: node scripts/demo-first-video.mjs --reviewed-sample", + note: "Human visual review/consent cannot be technically verified. This fixture is ignored and test-only; no user artwork is approved.", + }, null, 2) + "\n"); + return; + } + + assertBuiltCliHasSceneNew(); + + const sceneA = join(scenesDir, "hana-scene-a.json"); + const sceneB = join(scenesDir, "hana-scene-b.json"); + const videoA = join(outputsDir, "hana-scene-a.mp4"); + const videoB = join(outputsDir, "hana-scene-b.mp4"); + + invokeCli([ + "scene", "new", repoRelative(sceneA), + "--character", repoRelative(characterFile), + "--id", "hana-scene-a", + "--seconds", String(DURATION_SECONDS), + "--fps", String(FPS), + "--size", WIDTH + "x" + HEIGHT, + ]); + invokeCli([ + "scene", "new", repoRelative(sceneB), + "--character", repoRelative(characterFile), + "--id", "hana-scene-b", + "--seconds", String(DURATION_SECONDS), + "--fps", String(FPS), + "--size", WIDTH + "x" + HEIGHT, + ]); + + await replaceSceneEvents(sceneA, [ + { atMs: 500, actor: "hana-first-video-demo", kind: "expression", name: "happy", intensity: 0.75 }, + { atMs: 1400, actor: "hana-first-video-demo", kind: "look", x: 0.38, y: -0.12, durationMs: 700 }, + { atMs: 3000, actor: "hana-first-video-demo", kind: "expression", name: "confused", intensity: 0.6 }, + { atMs: 3900, actor: "hana-first-video-demo", kind: "look", x: -0.3, y: 0.18, durationMs: 700 }, + ]); + await replaceSceneEvents(sceneB, [ + { atMs: 400, actor: "hana-first-video-demo", kind: "expression", name: "annoyed", intensity: 0.55 }, + { atMs: 1150, actor: "hana-first-video-demo", kind: "look", x: -0.42, y: 0.12, durationMs: 650 }, + { atMs: 2600, actor: "hana-first-video-demo", kind: "expression", name: "happy", intensity: 0.65 }, + { atMs: 3650, actor: "hana-first-video-demo", kind: "look", x: 0.32, y: -0.2, durationMs: 800 }, + ]); + + const validationA = invokeCli(["scene", "validate", repoRelative(sceneA)]); + const validationB = invokeCli(["scene", "validate", repoRelative(sceneB)]); + const renderA = invokeCli(["scene", "render", repoRelative(sceneA), "--out", repoRelative(videoA)]); + const renderB = invokeCli(["scene", "render", repoRelative(sceneB), "--out", repoRelative(videoB)]); + const probeA = invokeChecker(videoA); + const probeB = invokeChecker(videoB); + + process.stdout.write(JSON.stringify({ + valid: true, + status: "first-video-demo-rendered", + runDir: repoRelative(runDir), + fixture: { + character: repoRelative(characterFile), + model: repoRelative(modelFile), + sourceRev2DCommit: REV2D_SHA, + approved: true, + scope: "ignored test-only Hana fixture", + humanReviewTechnicallyVerified: false, + }, + scenes: [ + { + scene: repoRelative(sceneA), + output: repoRelative(videoA), + validation: validationA, + render: renderA, + ffprobe: probeA, + }, + { + scene: repoRelative(sceneB), + output: repoRelative(videoB), + validation: validationB, + render: renderB, + ffprobe: probeB, + }, + ], + expected: { + seconds: DURATION_SECONDS, + fps: FPS, + width: WIDTH, + height: HEIGHT, + frames: FRAME_COUNT, + audio: false, + }, + note: "Both scenes reuse one copied pinned Hana rig. The explicit sample-review flag is not a production approval mechanism and never approves user artwork.", + }, null, 2) + "\n"); +} + +await main(); diff --git a/src/character/avatar.ts b/src/character/avatar.ts index 2e46f1b..c591a0c 100644 --- a/src/character/avatar.ts +++ b/src/character/avatar.ts @@ -274,6 +274,6 @@ export async function buildAvatar(projectDir: string, overwrite = false, root = summary: join(project, "review", "summary.json"), }, notes: built.notes ?? [], - next: "Inspect every review image, repair avatar.json if needed, rebuild with --force, and only then set poseFamilies[0].approved=true in character.json.", + next: "Inspect poses.png, turn.png, eyes.png and mouth.png; fix defects and rebuild with --force if needed. Only after real visual review, run puppetctl character approve --reviewed.", }; } diff --git a/src/character/landmarks.ts b/src/character/landmarks.ts index 0a3da5a..220ae0e 100644 --- a/src/character/landmarks.ts +++ b/src/character/landmarks.ts @@ -319,6 +319,6 @@ export async function acceptLandmarks( valid: true, status: "rebuild-required", approved: false, avatarSpecPath: project.specFile, backup, approvalRevoked, geometryWarnings: assessAvatarGeometry(candidateSpec), - next: "Run character build --force, visually review sheets, then manually approve the rig if acceptable.", + next: "Run character build --force, inspect all review sheets, then explicitly run character approve --reviewed only if the visual rig is genuinely acceptable.", }; } diff --git a/src/character/review-approval.ts b/src/character/review-approval.ts new file mode 100644 index 0000000..2c2fecd --- /dev/null +++ b/src/character/review-approval.ts @@ -0,0 +1,241 @@ +import { randomUUID } from "node:crypto"; +import { lstat, open, readFile, rename, unlink, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { AvatarWorkflowError, checkedPath } from "./avatar.js"; +import { checkAvatarPoseCoverage, verifyAvatarBuildRecord } from "./provenance.js"; +import { characterSchema, type Character } from "../scene/schemas.js"; + +const REVIEW_FILES = ["poses.png", "turn.png", "eyes.png", "mouth.png"] as const; +const PNG_SIGNATURE = Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]); +const MAX_MANIFEST_BYTES = 1024 * 1024; +const MAX_REVIEW_BYTES = 100 * 1024 * 1024; +const MIN_REVIEW_DIMENSION = 64; +const MAX_REVIEW_DIMENSION = 32_768; +const MAX_REVIEW_PIXELS = 268_435_456; + +export interface ImportedAvatarApprovalResult { + valid: true; + status: "approved" | "already-approved"; + approved: true; + changed: boolean; + characterPath: string; + reviewArtifacts: string[]; + message: string; +} + +function workflowError(code: string, message: string): never { + throw new AvatarWorkflowError(code, message); +} + +function asRecord(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? value as Record + : undefined; +} + +function parseManifest(contents: string): { raw: Record; character: Character } { + let raw: unknown; + try { + raw = JSON.parse(contents) as unknown; + } catch (error) { + workflowError("AVATAR_MANIFEST_INVALID", "character.json is not valid JSON: " + String(error)); + } + const record = asRecord(raw); + if (!record) workflowError("AVATAR_MANIFEST_INVALID", "character.json must contain one character object"); + const parsed = characterSchema.safeParse(record); + if (!parsed.success) { + workflowError( + "AVATAR_MANIFEST_INVALID", + "character.json does not match the character schema: " + parsed.error.message, + ); + } + return { raw: record, character: parsed.data }; +} + +function requireSupportedImportedAvatar(character: Character): void { + if (!character.provenance) { + workflowError( + "AVATAR_PROVENANCE_REQUIRED", + "Only provenance-tracked imported avatars can be approved by this workflow.", + ); + } + if (!character.render || character.render.backend !== "rev2d") { + workflowError( + "AVATAR_RENDER_UNSUPPORTED", + "This approval workflow only supports already-built Rev2D imported avatars.", + ); + } + if (character.poseFamilies.length !== 1 || + character.poseFamilies[0]?.name !== character.defaultPose) { + workflowError( + "AVATAR_POSE_FAMILIES_UNSUPPORTED", + "Imported-avatar approval currently supports exactly one built pose family. Additional pose families need separate provenance and review.", + ); + } + const coverage = checkAvatarPoseCoverage(character); + if (!coverage.valid) workflowError(coverage.code, coverage.message); + const family = character.poseFamilies[0]!; + if ((family.modelPath ?? character.render.modelPath) !== character.render.modelPath) { + workflowError( + "AVATAR_POSE_FAMILIES_UNSUPPORTED", + "The single imported pose must use the provenance-tracked primary Rev2D model.", + ); + } +} + +function parseReviewPngHeader(header: Buffer, filename: string): { width: number; height: number } { + if (header.length < 24 || !header.subarray(0, 8).equals(PNG_SIGNATURE) || + header.readUInt32BE(8) !== 13 || header.toString("ascii", 12, 16) !== "IHDR") { + workflowError( + "AVATAR_REVIEW_ASSET_INVALID", + filename + " is not a plausible PNG review artifact with a valid IHDR header.", + ); + } + const width = header.readUInt32BE(16); + const height = header.readUInt32BE(20); + const pixels = width * height; + if (width < MIN_REVIEW_DIMENSION || height < MIN_REVIEW_DIMENSION || + width > MAX_REVIEW_DIMENSION || height > MAX_REVIEW_DIMENSION || + !Number.isSafeInteger(pixels) || pixels > MAX_REVIEW_PIXELS) { + workflowError( + "AVATAR_REVIEW_ASSET_INVALID", + filename + " has implausible review dimensions " + width + "x" + height + ".", + ); + } + return { width, height }; +} + +async function validateReviewArtifacts(project: string): Promise { + const paths: string[] = []; + for (const filename of REVIEW_FILES) { + const path = await checkedPath(project, join(project, "review", filename)); + let info; + try { + info = await lstat(path); + } catch (error) { + if (error instanceof Error && "code" in error && error.code === "ENOENT") { + workflowError( + "AVATAR_REVIEW_ASSET_MISSING", + "Missing required review image " + filename + ". Rebuild the avatar and inspect all four review PNGs before approval.", + ); + } + throw error; + } + if (!info.isFile() || info.isSymbolicLink() || info.size < 24 || info.size > MAX_REVIEW_BYTES) { + workflowError( + "AVATAR_REVIEW_ASSET_INVALID", + filename + " must be a non-empty regular PNG review file of a plausible size.", + ); + } + const handle = await open(path, "r"); + try { + const header = Buffer.alloc(24); + const { bytesRead } = await handle.read(header, 0, header.length, 0); + if (bytesRead !== header.length) { + workflowError("AVATAR_REVIEW_ASSET_INVALID", filename + " is truncated."); + } + parseReviewPngHeader(header, filename); + } finally { + await handle.close(); + } + paths.push(path); + } + return paths; +} + +async function verifyCurrentBuild(characterPath: string, character: Character): Promise { + const modelPath = character.render?.modelPath; + if (!modelPath) { + workflowError("AVATAR_RENDER_UNSUPPORTED", "Imported avatar is missing its primary Rev2D model path."); + } + const integrity = await verifyAvatarBuildRecord(characterPath, modelPath); + if (!integrity.valid) workflowError(integrity.code, integrity.message); +} + +/** + * Record explicit approval for one already-built, provenance-tracked imported avatar. + * + * reviewedFlag is only an acknowledgement supplied by the caller. Software cannot + * prove that a human actually inspected the review PNGs or consented to approval. + */ +export async function approveImportedAvatar( + projectDir: string, + reviewedFlag: boolean, + root = process.cwd(), +): Promise { + if (reviewedFlag !== true) { + workflowError( + "AVATAR_REVIEW_REQUIRED", + "Approval requires reviewed=true only after a human has actually inspected poses.png, turn.png, eyes.png and mouth.png. This acknowledgement cannot technically prove visual review or consent.", + ); + } + + const project = await checkedPath(root, resolve(root, projectDir)); + const characterPath = await checkedPath(project, join(project, "character.json")); + const manifestInfo = await lstat(characterPath).catch((error: unknown) => { + if (error instanceof Error && "code" in error && error.code === "ENOENT") { + workflowError("AVATAR_MANIFEST_INVALID", "Missing character.json; build the imported avatar before approval."); + } + throw error; + }); + if (!manifestInfo.isFile() || manifestInfo.isSymbolicLink() || + manifestInfo.size <= 0 || manifestInfo.size > MAX_MANIFEST_BYTES) { + workflowError("AVATAR_MANIFEST_INVALID", "character.json must be a non-empty regular file smaller than 1 MB."); + } + + const originalContents = await readFile(characterPath, "utf8"); + const { raw, character } = parseManifest(originalContents); + requireSupportedImportedAvatar(character); + await verifyCurrentBuild(characterPath, character); + const reviewArtifacts = await validateReviewArtifacts(project); + + if (character.poseFamilies[0]!.approved) { + return { + valid: true, + status: "already-approved", + approved: true, + changed: false, + characterPath, + reviewArtifacts, + message: "Avatar was already approved. Provenance and required review artifacts were rechecked; human review itself cannot be technically verified.", + }; + } + + const rawPoses = raw.poseFamilies; + if (!Array.isArray(rawPoses) || rawPoses.length !== 1 || !asRecord(rawPoses[0])) { + workflowError("AVATAR_MANIFEST_INVALID", "character.json poseFamilies changed during approval validation."); + } + const rawPose = asRecord(rawPoses[0])!; + rawPose.approved = true; + const nextContents = JSON.stringify(raw, null, 2) + "\n"; + const tempPath = await checkedPath(project, characterPath + ".next-" + randomUUID()); + + try { + await writeFile(tempPath, nextContents, { encoding: "utf8", flag: "wx" }); + + const currentContents = await readFile(characterPath, "utf8"); + if (currentContents !== originalContents) { + workflowError( + "AVATAR_MANIFEST_CHANGED", + "character.json changed while approval was being prepared; no approval was written. Re-run after reviewing the current file.", + ); + } + + // Recheck the exact build and review artifacts immediately before the atomic manifest swap. + await verifyCurrentBuild(characterPath, character); + await validateReviewArtifacts(project); + await rename(tempPath, characterPath); + } finally { + await unlink(tempPath).catch(() => undefined); + } + + return { + valid: true, + status: "approved", + approved: true, + changed: true, + characterPath, + reviewArtifacts, + message: "Approval recorded from an explicit reviewed=true acknowledgement. PuppetControl cannot technically verify human visual review or consent.", + }; +} diff --git a/src/cli/index.ts b/src/cli/index.ts index 0f127df..2cf257e 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -7,6 +7,7 @@ import { readFile } from "node:fs/promises"; import { characterSchema } from "../scene/schemas.js"; import { loadAndValidateScene } from "../scene/load.js"; import { encodeFrames, renderRev2DFrames } from "../render/rev2d.js"; +import { checkSceneRenderability } from "../render/renderability.js"; import { PreviewUnsupportedError } from "../render/scene-plan.js"; import { alignAudioWithRhubarb } from "../audio/align.js"; import { planPoseTimeline, PoseTimelineError } from "../animation/pose-timeline.js"; @@ -14,6 +15,9 @@ import { AvatarWorkflowError, importAvatar, buildAvatar } from "../character/ava import { prepareLandmarkTask, stageLandmarks, acceptLandmarks } from "../character/landmarks.js"; import { proposeCodexLandmarks, codexExecutableForPlatform } from "../character/vision-codex.js"; import { verifyAvatarBuildRecord } from "../character/provenance.js"; +import { approveImportedAvatar } from "../character/review-approval.js"; +import { FirstVideoSceneError } from "../scene/first-video.js"; +import { createFirstVideoFile } from "../scene/create-first-video-file.js"; function print(value: unknown): void { process.stdout.write(JSON.stringify(value, null, 2) + "\n"); @@ -27,15 +31,17 @@ function usage(): never { " puppetctl character inspect \n" + " puppetctl character import --out --id \n" + " puppetctl character build [--force]\n" + + " puppetctl character approve --reviewed\n" + " puppetctl character landmarks assist \n" + " puppetctl character landmarks propose --provider codex\n" + " puppetctl character landmarks stage --proposal \n" + " puppetctl character landmarks accept --candidate --reviewed\n" + " puppetctl audio align --out [--binary rhubarb] [--force]\n" + + " puppetctl scene new --character [--id name] [--seconds 12] [--fps 24] [--size 1920x1080] [--voice voice.wav --cues mouth.json]\n" + " puppetctl scene validate \n" + " puppetctl scene plan \n" + " puppetctl scene preview \n" + - " puppetctl scene render [--out output.mp4]\n" + + " puppetctl scene render [--out output.mp4] [--force]\n" + " puppetctl --help\n" + "\nPreview/render require a configured Rev2D rig. Single-image imports require reviewed landmarks and explicit approval.\n", ); @@ -47,7 +53,7 @@ async function run(): Promise { if (first === "--help" || first === "help") { process.stdout.write( "PuppetControl (experimental)\n" + - "Available: doctor, character inspect/import/build, landmark assist/propose/stage/accept, audio align, scene validate/plan, single-rig scene preview/render.\n" + + "Available: doctor, character inspect/import/build/approve, landmark assist/propose/stage/accept, audio align, scene new/validate/plan, single-rig scene preview/render.\n" + "Not yet available: reliable one-click landmark inference, animated pose bridges, captions, automatic transcription/TTS.\n", ); return; @@ -72,7 +78,7 @@ async function run(): Promise { ffmpegAvailable: ffmpegFound, rev2dAvailable, codexCliAvailable, - availableFeatures: ["character-inspect", "scene-validate", "transition-policy", "rhubarb-cue-parsing", "rhubarb-align-if-installed", ...(rev2dAvailable ? ["assisted-png-avatar-import", "agent-landmark-proposal-staging", ...(codexCliAvailable ? ["optional-codex-vision-proposals"] : []), "avatar-review-sheets", "single-rig-preview", "single-rig-voice-render", "pose-family-hard-cut-render", "experimental-shot-framing"] : [])], + availableFeatures: ["character-inspect", "scene-create", "scene-validate", "transition-policy", "rhubarb-cue-parsing", "rhubarb-align-if-installed", ...(rev2dAvailable ? ["assisted-png-avatar-import", "explicit-visual-approval", "agent-landmark-proposal-staging", ...(codexCliAvailable ? ["optional-codex-vision-proposals"] : []), "avatar-review-sheets", "single-rig-preview", "single-rig-voice-render", "pose-family-hard-cut-render", "experimental-shot-framing"] : [])], unavailableFeatures: ["animated-pose-bridges", "automatic-landmark-placement", "automatic-transcription", "text-to-speech", "multi-audio-mixing"], }); return; @@ -121,6 +127,13 @@ async function run(): Promise { return; } + if (first === "character" && second === "approve" && third) { + const flags = process.argv.slice(5); + if (flags.length !== 1 || flags[0] !== "--reviewed") usage(); + print(await approveImportedAvatar(third, true)); + return; + } + if (first === "character" && second === "landmarks" && third) { const [project, ...flags] = process.argv.slice(5); if (!project) usage(); @@ -178,17 +191,47 @@ async function run(): Promise { return; } + if (first === "scene" && second === "new" && third) { + const flags = process.argv.slice(5); + const allowed = new Set(["--character", "--id", "--seconds", "--fps", "--size", "--voice", "--cues"]); + const values = new Map(); + for (let index = 0; index < flags.length; index += 2) { + const flag = flags[index]; + const value = flags[index + 1]; + if (!flag || !value || !allowed.has(flag) || values.has(flag)) usage(); + values.set(flag, value); + } + const characterFile = values.get("--character"); + if (!characterFile) usage(); + const size = values.get("--size"); + if (size && !/^\d+x\d+$/.test(size)) usage(); + const dimensions = size?.split("x").map(Number); + print(await createFirstVideoFile({ + outputSceneFile: third, + characterFile, + ...(values.has("--id") ? { id: values.get("--id")! } : {}), + ...(values.has("--seconds") ? { seconds: Number(values.get("--seconds")) } : {}), + ...(values.has("--fps") ? { fps: Number(values.get("--fps")) } : {}), + ...(dimensions ? { outputSize: { width: dimensions[0]!, height: dimensions[1]! } } : {}), + ...(values.has("--voice") ? { voiceFile: values.get("--voice")! } : {}), + ...(values.has("--cues") ? { cuesFile: values.get("--cues")! } : {}), + })); + return; + } + if (first === "scene" && second === "validate" && third) { - const result = await loadAndValidateScene(third); + const result = await loadAndValidateScene(third, { root: process.cwd() }); + const renderability = result.valid ? checkSceneRenderability(result.scene, result.characters) : null; + if (renderability && !renderability.valid) process.exitCode = 1; print(result.valid - ? { valid: true, scene: result.scene.id, diagnostics: result.check.diagnostics, transitions: result.check.transitions } + ? { valid: renderability!.valid, scene: result.scene.id, diagnostics: [...result.check.diagnostics, ...renderability!.diagnostics], transitions: result.check.transitions } : { valid: false, diagnostics: result.diagnostics }); if (!result.valid) process.exitCode = 1; return; } if (first === "scene" && second === "plan" && third) { - const result = await loadAndValidateScene(third); + const result = await loadAndValidateScene(third, { root: process.cwd() }); if (!result.valid) { print({ valid: false, diagnostics: result.diagnostics }); process.exitCode = 1; @@ -203,12 +246,18 @@ async function run(): Promise { } if (first === "scene" && (second === "preview" || second === "render") && third) { - const result = await loadAndValidateScene(third); + const result = await loadAndValidateScene(third, { root: process.cwd() }); if (!result.valid) { print({ valid: false, diagnostics: result.diagnostics }); process.exitCode = 1; return; } + const renderability = checkSceneRenderability(result.scene, result.characters); + if (!renderability.valid) { + print({ valid: false, diagnostics: renderability.diagnostics }); + process.exitCode = 1; + return; + } const actor = result.scene.actors[0]; if (!actor) throw new PreviewUnsupportedError("ACTOR_REQUIRED", "Scene needs one actor."); const character = result.characters.get(actor.id); @@ -229,10 +278,17 @@ async function run(): Promise { } const args = process.argv.slice(5); const outputIndex = args.indexOf("--out"); - if (outputIndex !== -1 && !args[outputIndex + 1]) usage(); - if (args.length > 0 && (outputIndex !== 0 || args.length !== 2)) usage(); - const outputFile = args[outputIndex + 1] ?? resolve("outputs", result.scene.id + ".mp4"); - const mp4 = await encodeFrames(frameResult, outputFile); + const overwrite = args.includes("--force"); + if (args.filter((value) => value === "--force").length > 1 || + args.filter((value) => value === "--out").length > 1 || + args.some((value, i) => value.startsWith("--") && value !== "--force" && value !== "--out") || + (outputIndex >= 0 && (!args[outputIndex + 1] || args[outputIndex + 1]!.startsWith("--"))) || + (outputIndex < 0 && args.length !== (overwrite ? 1 : 0)) || + (outputIndex >= 0 && args.length !== (overwrite ? 3 : 2))) usage(); + // Render policy: never overwrite a video without explicit --force. + if (second === "render" && outputIndex >= 0 && args[outputIndex + 1] === "--force") usage(); + const outputFile = outputIndex >= 0 ? args[outputIndex + 1]! : resolve("outputs", result.scene.id + ".mp4"); + const mp4 = await encodeFrames(frameResult, outputFile, { overwrite }); print({ valid: true, kind: frameResult.audio ? "voiced-mp4" : "silent-mp4", output: mp4, ...frameResult, limitations: ["Shot framing is experimental; no captions, smooth between-pose bridges or automatic Rhubarb invocation yet."] }); return; } @@ -242,7 +298,7 @@ async function run(): Promise { run().catch((error: unknown) => { print({ valid: false, - code: error instanceof PreviewUnsupportedError || error instanceof PoseTimelineError || error instanceof AvatarWorkflowError ? error.code : "CLI_UNEXPECTED_ERROR", + code: error instanceof PreviewUnsupportedError || error instanceof PoseTimelineError || error instanceof AvatarWorkflowError || error instanceof FirstVideoSceneError ? error.code : "CLI_UNEXPECTED_ERROR", message: error instanceof Error ? error.message : String(error), }); process.exitCode = 1; diff --git a/src/render/renderability.ts b/src/render/renderability.ts new file mode 100644 index 0000000..3cf1748 --- /dev/null +++ b/src/render/renderability.ts @@ -0,0 +1,64 @@ +/** + * Renderability preflight is deliberately separate from the wider scene + * capability validator: a scene may be a legal planning document but still + * request features the current MP4 renderer does not implement. + */ +import { planPoseTimeline, PoseTimelineError } from "../animation/pose-timeline.js"; +import type { Character, Scene } from "../scene/schemas.js"; + +export interface RenderabilityIssue { + severity: "error"; + code: string; + path: string; + message: string; +} + +export function checkSceneRenderability( + scene: Scene, + characters: ReadonlyMap, +): { valid: boolean; diagnostics: RenderabilityIssue[] } { + const diagnostics: RenderabilityIssue[] = []; + const error = (code: string, path: string, message: string) => + diagnostics.push({ severity: "error", code, path, message }); + + if (scene.actors.length !== 1) { + error("MULTI_ACTOR_UNSUPPORTED", "actors", + "MP4 rendering currently supports exactly one actor; multi-actor scenes can be planned but not rendered."); + } + for (const [index, audio] of scene.audio.entries()) { + if (!audio.cuesPath) { + error("LIP_CUES_REQUIRED", "audio." + index + ".cuesPath", + "Voice rendering requires validated Rhubarb cues. Generate them with audio align before rendering."); + } + } + for (const [index, event] of scene.events.entries()) { + if (event.kind === "caption") { + error("COMPOSITION_NOT_IMPLEMENTED", "events." + index, + "Caption composition is not implemented in the MP4 renderer."); + } else if (event.kind === "motion") { + error("MOTION_NOT_IMPLEMENTED", "events." + index, + "Semantic motion events cannot be rendered yet; use a supported rig pose control instead."); + } + } + if (scene.actors.length === 1) { + const actor = scene.actors[0]!; + const character = characters.get(actor.id); + if (!character) { + error("ACTOR_NOT_LOADED", "actors.0", "Actor manifest has not been loaded."); + } else { + try { + const timeline = planPoseTimeline(scene, character, actor.id); + for (const [index, segment] of timeline.segments.entries()) { + if (segment.kind === "transition" && segment.mode === "bridge") { + error("BRIDGE_RENDER_NOT_IMPLEMENTED", "timeline.segments." + index, + "A planned authored between-rig bridge has no render implementation. Use a supported rig interpolation or explicit camera cut."); + } + } + } catch (cause) { + error(cause instanceof PoseTimelineError ? cause.code : "POSE_PLAN_UNSUPPORTED", + "events", cause instanceof Error ? cause.message : String(cause)); + } + } + } + return { valid: diagnostics.length === 0, diagnostics }; +} diff --git a/src/render/rev2d.ts b/src/render/rev2d.ts index 14335a7..687863e 100644 --- a/src/render/rev2d.ts +++ b/src/render/rev2d.ts @@ -1,8 +1,8 @@ import { randomUUID } from "node:crypto"; import { execFile } from "node:child_process"; import { promisify } from "node:util"; -import { copyFile, mkdir, readFile, stat, writeFile, unlink } from "node:fs/promises"; -import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; +import { copyFile, link, lstat, mkdir, readFile, rename, stat, writeFile, unlink } from "node:fs/promises"; +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import type { Character, Scene } from "../scene/schemas.js"; import { compilePerformance, PreviewUnsupportedError } from "./scene-plan.js"; import { parseRhubarbMouthCuesJson } from "../audio/rhubarb.js"; @@ -11,7 +11,7 @@ import type { VideoOutput } from "../video/output.js"; import type { SceneEvent } from "../scene/schemas.js"; import { planPoseTimeline, PoseTimelineError } from "../animation/pose-timeline.js"; import { verifyAvatarBuildRecord, checkAvatarPoseCoverage } from "../character/provenance.js"; -import { pinnedRev2D } from "../character/avatar.js"; +import { checkedPath, pinnedRev2D } from "../character/avatar.js"; const execFileAsync = promisify(execFile); @@ -114,8 +114,8 @@ async function loadSpeech( throw new PreviewUnsupportedError("LIP_CUES_REQUIRED", "Audio entry needs Rhubarb cuesPath JSON."); } const directory = dirname(resolve(sceneFilePath)); - const source = insideRoot(resolve(directory, entry.path)); - const cuesFile = insideRoot(resolve(directory, entry.cuesPath)); + const source = await checkedPath(process.cwd(), resolve(directory, entry.path)); + const cuesFile = await checkedPath(process.cwd(), resolve(directory, entry.cuesPath)); try { const file = await stat(source); @@ -284,7 +284,7 @@ export async function renderRev2DFrames( throw new PreviewUnsupportedError("FRAME_LIMIT", "Current renderer supports at most 2000 frames per scene."); } - const rev2dRoot = insideRoot(process.env.PUPPETCONTROL_REV2D_ROOT ?? ".cache/rev2d"); + const rev2dRoot = await checkedPath(process.cwd(), resolve(process.env.PUPPETCONTROL_REV2D_ROOT ?? ".cache/rev2d")); const framesDir = insideRoot(outputDirectory); const { speech, audio } = await loadSpeech(scene, sceneFilePath); const familyNames: string[] = []; @@ -302,7 +302,7 @@ export async function renderRev2DFrames( // Different artwork must use separately declared rig assets; sharing rigId // alone does not create a visual pose change. const rigFile = family.modelPath ?? character.render.modelPath; - const modelPath = insideRoot(resolve(dirname(characterPath), rigFile)); + const modelPath = await checkedPath(process.cwd(), resolve(dirname(characterPath), rigFile)); const model = await readRig(modelPath, scene); const segmentDir = join(framesDir, "segment_" + String(segmentIndex).padStart(3, "0")); const files = await renderRigSegment({ @@ -361,19 +361,47 @@ export async function renderRev2DFrames( export async function encodeFrames( frames: FrameResult, - out: string, + out: string, options: { overwrite?: boolean } = {}, ): Promise { - const outputFile = insideRoot(out); + const outputFile = await checkedPath(process.cwd(), resolve(out)); await mkdir(dirname(outputFile), { recursive: true }); + if (!outputFile.toLowerCase().endsWith(".mp4")) { + throw new PreviewUnsupportedError("VIDEO_OUTPUT_INVALID", "Output file must use the .mp4 extension."); + } + const existing = await lstat(outputFile).catch((error: unknown) => { + if (error instanceof Error && "code" in error && error.code === "ENOENT") return null; + throw error; + }); + if (existing && (!existing.isFile() || existing.isSymbolicLink())) { + throw new PreviewUnsupportedError("VIDEO_OUTPUT_INVALID", "Output must not be a symlink or directory."); + } + if (existing && !options.overwrite) { + throw new PreviewUnsupportedError("VIDEO_OUTPUT_EXISTS", + "Output exists; choose another --out path or use --force to replace after successful encoding."); + } + const tempFile = join(dirname(outputFile), ".puppetcontrol-" + randomUUID() + ".mp4"); const args = buildFfmpegArgs({ framesPattern: join(frames.framesDir, "frame_%04d.png"), - outputPath: outputFile, + outputPath: tempFile, fps: frames.fps, frameCount: frames.count, ...(frames.audio ? { audio: frames.audio } : {}), ...(frames.presentation ? { output: frames.presentation } : {}), ...(frames.camera ? { camera: frames.camera } : {}), }); - await execute("ffmpeg", args, process.cwd(), 240_000); + try { + // FFmpeg can overwrite its unique temporary file only, never the user's old video. + await execute("ffmpeg", args, process.cwd(), 240_000); + if (options.overwrite) await rename(tempFile, outputFile); + else await link(tempFile, outputFile); // EEXIST fails atomically without touching the old file. + } catch (error) { + if (error instanceof Error && "code" in error && error.code === "EEXIST") { + throw new PreviewUnsupportedError("VIDEO_OUTPUT_EXISTS", + "Another video already exists at this path; no existing file was overwritten."); + } + throw error; + } finally { + await unlink(tempFile).catch(() => undefined); + } return outputFile; } diff --git a/src/scene/create-first-video-file.ts b/src/scene/create-first-video-file.ts new file mode 100644 index 0000000..18eaf62 --- /dev/null +++ b/src/scene/create-first-video-file.ts @@ -0,0 +1,118 @@ +/** + * Filesystem boundary for a first video scene: an already-reviewed reusable + * avatar plus an editable JSON scene. No implicit rig approval or TTS. + */ +import { mkdir, readFile, stat, writeFile } from "node:fs/promises"; +import { basename, dirname, extname, relative, resolve } from "node:path"; +import { characterSchema } from "./schemas.js"; +import { createFirstVideoScene } from "./first-video.js"; +import { checkedPath, AvatarWorkflowError } from "../character/avatar.js"; +import { checkAvatarPoseCoverage, verifyAvatarBuildRecord } from "../character/provenance.js"; +import { parseRhubarbMouthCuesJson } from "../audio/rhubarb.js"; + +export interface CreateFirstVideoFileOptions { + outputSceneFile: string; + characterFile: string; + id?: string; + seconds?: number; + fps?: number; + voiceFile?: string; + cuesFile?: string; + outputSize?: { width: number; height: number }; +} + +async function nonemptyFile(path: string, label: string, max = 30 * 1024 * 1024) { + const info = await stat(path); + if (!info.isFile() || info.size < 1 || info.size > max) { + throw new AvatarWorkflowError("FIRST_VIDEO_INPUT_INVALID", label + " must be a nonempty file under " + max + " bytes."); + } +} + +function relativeForScene(sceneDir: string, target: string): string { + const value = relative(sceneDir, target).replaceAll("\\", "/"); + return value.startsWith(".") ? value : "./" + value; +} + +export async function createFirstVideoFile( + options: CreateFirstVideoFileOptions, root = process.cwd(), +) { + const project = resolve(root); + const outputFile = await checkedPath(project, resolve(project, options.outputSceneFile)); + const characterFile = await checkedPath(project, resolve(project, options.characterFile)); + await nonemptyFile(characterFile, "Character manifest", 2 * 1024 * 1024); + const character = characterSchema.parse(JSON.parse(await readFile(characterFile, "utf8")) as unknown); + if (!character.render || character.render.backend !== "rev2d") { + throw new AvatarWorkflowError("FIRST_VIDEO_RIG_REQUIRED", "Use a real Rev2D character with a configured rig."); + } + const defaultPose = character.poseFamilies.find((pose) => pose.name === character.defaultPose); + if (!defaultPose?.approved) { + throw new AvatarWorkflowError("FIRST_VIDEO_NOT_REVIEWED", + "This actor is not approved. Inspect review/poses.png, turn.png, eyes.png and mouth.png; explicitly approve the rig only after actual visual review."); + } + const coverage = checkAvatarPoseCoverage(character); + if (!coverage.valid) throw new AvatarWorkflowError(coverage.code, coverage.message); + if (character.provenance) { + const integrity = await verifyAvatarBuildRecord(characterFile, character.render.modelPath); + if (!integrity.valid) throw new AvatarWorkflowError(integrity.code, integrity.message); + } + const characterDir = dirname(characterFile); + const modelPath = await checkedPath(character.provenance ? characterDir : project, + resolve(characterDir, character.render.modelPath)); + await nonemptyFile(modelPath, "Rig model", 20 * 1024 * 1024); + const model = JSON.parse(await readFile(modelPath, "utf8")) as { + meta?: { width?: number; height?: number }; + }; + const width = model.meta?.width; + const height = model.meta?.height; + if (!Number.isSafeInteger(width) || !Number.isSafeInteger(height) || + width! < 160 || height! < 160 || width! > 4096 || height! > 4096) { + throw new AvatarWorkflowError("FIRST_VIDEO_RIG_INVALID", "Rig metadata needs valid native canvas dimensions."); + } + const durationMs = Math.round((options.seconds ?? 12) * 1000); + const fps = options.fps ?? 24; + if (!Number.isSafeInteger(durationMs) || durationMs < 1000 || durationMs > 60_000 || + !Number.isSafeInteger(fps) || fps < 1 || fps > 60) { + throw new AvatarWorkflowError("FIRST_VIDEO_TIMING_INVALID", + "Use a scene duration of 1–60 seconds and an integer FPS of 1–60."); + } + const outputDir = dirname(outputFile); + let audio: { path: string; cuesPath: string; startMs: number } | undefined; + if (Boolean(options.voiceFile) !== Boolean(options.cuesFile)) { + throw new AvatarWorkflowError("FIRST_VIDEO_VOICE_CUES_REQUIRED", + "Voice requires a Rhubarb cues JSON file; generate it first with puppetctl audio align. No simulated lip sync."); + } + if (options.voiceFile && options.cuesFile) { + const wav = await checkedPath(project, resolve(project, options.voiceFile)); + const cues = await checkedPath(project, resolve(project, options.cuesFile)); + await Promise.all([nonemptyFile(wav, "Voice audio", 200 * 1024 * 1024), + nonemptyFile(cues, "Rhubarb cues", 5 * 1024 * 1024)]); + try { + parseRhubarbMouthCuesJson(await readFile(cues, "utf8")); + } catch (error) { + throw new AvatarWorkflowError("FIRST_VIDEO_LIP_CUES_INVALID", + "Rhubarb cue file is invalid: " + (error instanceof Error ? error.message : String(error))); + } + audio = { path: relativeForScene(outputDir, wav), cuesPath: relativeForScene(outputDir, cues), startMs: 0 }; + } + const scene = createFirstVideoScene(character, { + id: options.id ?? basename(outputFile, extname(outputFile)).replace(/[^A-Za-z0-9_-]/g, "-").replace(/^[^A-Za-z0-9]+/, "").slice(0, 100), + actorId: character.id, + characterPath: relativeForScene(outputDir, characterFile), + durationMs, + fps, + resolution: { width: width!, height: height! }, + ...(options.outputSize ? { + output: { ...options.outputSize, background: "#f0eaf4" }, + } : {}), + ...(audio ? { audio } : {}), + }); + await mkdir(outputDir, { recursive: true }); + // Never overwrite previous edits or a prior scene, even on a retry. + await writeFile(outputFile, JSON.stringify(scene, null, 2) + "\n", { flag: "wx" }); + return { + valid: true, status: "editable-scene-created", scene: scene.id, outputFile, + character: character.id, rig: modelPath, reusedExistingRig: true, + hasVoice: Boolean(audio), durationMs: scene.durationMs, fps: scene.fps, + next: "Edit the scene's supported events, then run scene validate and scene render. The existing approved character and its art are reused unchanged.", + }; +} diff --git a/src/scene/first-video.ts b/src/scene/first-video.ts new file mode 100644 index 0000000..b764687 --- /dev/null +++ b/src/scene/first-video.ts @@ -0,0 +1,122 @@ +/** + * A small, honest scene composer for one existing reviewed character rig. + * It never infers landmarks, draws missing poses, synthesizes speech or + * approves artwork. Output remains an editable, strict scene JSON document. + */ +import { + characterSchema, sceneSchema, type Character, type Scene, type SceneEvent, +} from "./schemas.js"; +import { validateSceneCapabilities } from "./validate.js"; + +export interface FirstVideoOptions { + id: string; + actorId: string; + characterPath: string; + durationMs: number; + fps: number; + resolution: { width: number; height: number }; + output?: { width: number; height: number; background: string }; + audio?: { path: string; cuesPath: string; startMs: number }; + /** Supported semantic events in millisecond timestamps. */ + events?: SceneEvent[]; +} + +export class FirstVideoSceneError extends Error { + constructor(public readonly code: string, message: string) { + super(message); + this.name = "FirstVideoSceneError"; + } +} + +/** Choose only capabilities that the current character genuinely declares. */ +function gentleDefaultEvents(character: Character, actor: string, ms: number): SceneEvent[] { + const expression = character.expressions.find( + (name) => name !== "neutral" && character.render?.expressionPresets[name] !== undefined, + ); + const result: SceneEvent[] = [{ + atMs: 0, actor, kind: "pose", name: character.defaultPose, transition: "auto", + }]; + if (ms >= 2_000) { + result.push({ + atMs: Math.min(ms - 1, Math.round(ms * 0.17)), + actor, kind: "look", x: 0.22, y: -0.08, durationMs: 450, + }); + } + if (expression && ms >= 3_000) { + result.push({ + atMs: Math.min(ms - 1, Math.round(ms * 0.34)), + actor, kind: "expression", name: expression, intensity: 0.65, + }); + } + if (ms >= 4_000) { + result.push({ + atMs: Math.min(ms - 1, Math.round(ms * 0.61)), + actor, kind: "look", x: -0.2, y: 0.06, durationMs: 500, + }); + } + if (expression && character.expressions.includes("neutral") && + character.render?.expressionPresets.neutral !== undefined && ms >= 5_000) { + result.push({ + atMs: Math.min(ms - 1, Math.round(ms * 0.82)), + actor, kind: "expression", name: "neutral", intensity: 1, + }); + } + return result; +} + +/** Validates both scene schema and semantic capabilities before returning. */ +export function createFirstVideoScene(characterInput: Character, options: FirstVideoOptions): Scene { + const validCharacter = characterSchema.safeParse(characterInput); + if (!validCharacter.success) { + throw new FirstVideoSceneError("FIRST_VIDEO_CHARACTER_INVALID", + "Supply a valid character manifest with real declared rig capabilities."); + } + const character = validCharacter.data; + if (!character.render || character.render.backend !== "rev2d") { + throw new FirstVideoSceneError("FIRST_VIDEO_RIG_REQUIRED", "The character must have a real Rev2D rig."); + } + if (!character.poseFamilies.find((p) => p.name === character.defaultPose)?.approved) { + throw new FirstVideoSceneError("FIRST_VIDEO_NOT_REVIEWED", + "Review the actual character pose sheets and explicitly approve its default pose first."); + } + if (!Number.isSafeInteger(options.durationMs) || options.durationMs < 1_000 || + options.durationMs > 60_000 || !Number.isSafeInteger(options.fps) || + options.fps < 1 || options.fps > 60) { + throw new FirstVideoSceneError("FIRST_VIDEO_TIMING_INVALID", + "Use a duration from 1 to 60 seconds and an integer FPS between 1 and 60."); + } + const events = options.events ?? gentleDefaultEvents(character, options.actorId, options.durationMs); + // A first-video MVP cannot claim captions, animated motions or bridge clips + // that its current renderer does not implement. + if (events.some((event) => event.kind === "caption" || event.kind === "motion" || + event.kind === "camera" || (event.kind === "pose" && + (event.name !== character.defaultPose || event.transition === "bridge")))) { + throw new FirstVideoSceneError("FIRST_VIDEO_EVENT_UNSUPPORTED", + "First-video scenes support only the reviewed neutral pose, gaze and supported expressions. Other poses, camera edits and captions require separate explicit authoring."); + } + const parsed = sceneSchema.safeParse({ + schemaVersion: 1, id: options.id, durationMs: options.durationMs, + fps: options.fps, resolution: options.resolution, + ...(options.output ? { output: options.output } : {}), + actors: [{ id: options.actorId, characterPath: options.characterPath }], + audio: options.audio + ? [{ ...options.audio, actor: options.actorId }] + : [], + allowCameraCuts: false, events, + }); + if (!parsed.success) { + throw new FirstVideoSceneError("FIRST_VIDEO_SCENE_INVALID", + "Scene is invalid: " + parsed.error.issues.map((issue) => issue.path.join(".") + ": " + issue.message).join("; ")); + } + const validated = validateSceneCapabilities(parsed.data, new Map([[options.actorId, character]])); + if (!validated.valid) { + throw new FirstVideoSceneError("FIRST_VIDEO_CAPABILITY_MISMATCH", + validated.diagnostics.filter((diagnostic) => diagnostic.severity === "error") + .map((diagnostic) => diagnostic.code + ": " + diagnostic.message).join("; ")); + } + if (options.audio && !options.audio.cuesPath) { + throw new FirstVideoSceneError("FIRST_VIDEO_VOICE_CUES_REQUIRED", + "Precomputed Rhubarb audio cues are required before attaching speech."); + } + return parsed.data; +} diff --git a/src/scene/load.ts b/src/scene/load.ts index f6bdccc..8914d7e 100644 --- a/src/scene/load.ts +++ b/src/scene/load.ts @@ -3,6 +3,7 @@ import { dirname, resolve } from "node:path"; import { characterSchema, sceneSchema, type Character, type Scene } from "./schemas.js"; import { validateSceneCapabilities, type Diagnostic, type SceneValidation } from "./validate.js"; import { verifyAvatarBuildRecord, checkAvatarPoseCoverage } from "../character/provenance.js"; +import { checkedPath } from "../character/avatar.js"; async function readJson(path: string): Promise { const text = await readFile(path, "utf8"); @@ -13,8 +14,8 @@ export type LoadResult = | { valid: true; scene: Scene; characters: Map; check: SceneValidation } | { valid: false; diagnostics: Diagnostic[] }; -export async function loadAndValidateScene(filePath: string): Promise { - const absolutePath = resolve(filePath); +export async function loadAndValidateScene(filePath: string, options: { root?: string } = {}): Promise { + const absolutePath = options.root ? await checkedPath(options.root, resolve(filePath)) : resolve(filePath); let raw: unknown; try { raw = await readJson(absolutePath); @@ -41,7 +42,9 @@ export async function loadAndValidateScene(filePath: string): Promise(); const diagnostics: Diagnostic[] = []; for (const actor of parsed.data.actors) { - const characterFile = resolve(dirname(absolutePath), actor.characterPath); + const characterFile = options.root + ? await checkedPath(options.root, resolve(dirname(absolutePath), actor.characterPath)) + : resolve(dirname(absolutePath), actor.characterPath); try { const character = characterSchema.safeParse(await readJson(characterFile)); if (!character.success) { diff --git a/tests/first-video-file.test.ts b/tests/first-video-file.test.ts new file mode 100644 index 0000000..6d9cd0d --- /dev/null +++ b/tests/first-video-file.test.ts @@ -0,0 +1,90 @@ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { join, relative, resolve } from "node:path"; +import { createFirstVideoFile } from "../src/scene/create-first-video-file.js"; +import { loadAndValidateScene } from "../src/scene/load.js"; + +let root = ""; +let working = ""; +beforeEach(async () => { + root = resolve(process.cwd()); + await mkdir(resolve(root, ".cache"), { recursive: true }); + working = await mkdtemp(join(resolve(root, ".cache"), "first-video-file-")); +}); +afterEach(async () => { + if (working) await rm(working, { recursive: true, force: true }); +}); + +describe("first-video file creation", () => { + const hana = resolve("examples/characters/hana/character.json"); + it("creates two editable scenes using exactly one already approved rig without mutating the character", async () => { + const before = await readFile(hana, "utf8"); + const outputs = [join(working, "first.json"), join(working, "revision.json")]; + for (let i = 0; i < outputs.length; i++) { + const result = await createFirstVideoFile({ + outputSceneFile: outputs[i]!, characterFile: hana, + id: "hana-test-" + i, seconds: 4, fps: 15, + }, root); + expect(result).toMatchObject({ + valid: true, reusedExistingRig: true, + durationMs: 4000, fps: 15, hasVoice: false, + }); + const parsed = await loadAndValidateScene(outputs[i]!); + expect(parsed.valid).toBe(true); + if (parsed.valid) { + expect(parsed.scene.actors[0]?.id).toBe("hana"); + expect(resolve(working, parsed.scene.actors[0]!.characterPath)).toBe(hana); + expect(parsed.scene.events.every((event) => event.kind !== "motion" && event.kind !== "caption")).toBe(true); + } + } + expect(await readFile(hana, "utf8")).toBe(before); + const first = JSON.parse(await readFile(outputs[0]!, "utf8")); + const second = JSON.parse(await readFile(outputs[1]!, "utf8")); + expect(first.id).not.toBe(second.id); + expect(first.actors[0].characterPath).toBe(second.actors[0].characterPath); + }); + + it("derives distinct default scene IDs from distinct filenames", async () => { + const first = join(working, "first-clip.json"); + const second = join(working, "second-clip.json"); + await createFirstVideoFile({ outputSceneFile: first, characterFile: hana, seconds: 3 }, root); + await createFirstVideoFile({ outputSceneFile: second, characterFile: hana, seconds: 3 }, root); + expect(JSON.parse(await readFile(first, "utf8")).id).toBe("first-clip"); + expect(JSON.parse(await readFile(second, "utf8")).id).toBe("second-clip"); + }); + + it("refuses silent overwrite of a previously authored scene", async () => { + const output = join(working, "existing.json"); + await writeFile(output, "{\"author\":\"do not overwrite\"}"); + await expect(createFirstVideoFile({ outputSceneFile: output, characterFile: hana }, root)) + .rejects.toThrow(); + expect(await readFile(output, "utf8")).toContain("do not overwrite"); + }); + + it("requires cues when voice is requested and never fakes alignment", async () => { + const output = join(working, "voice.json"); + await writeFile(join(working, "voice.wav"), Buffer.from([1, 2, 3])); + await expect(createFirstVideoFile({ + outputSceneFile: output, characterFile: hana, + voiceFile: join(working, "voice.wav"), + }, root)).rejects.toMatchObject({ code: "FIRST_VIDEO_VOICE_CUES_REQUIRED" }); + }); + + it("refuses unapproved imported-style characters rather than silently granting approval", async () => { + const manifest = JSON.parse(await readFile(hana, "utf8")); + manifest.poseFamilies[0].approved = false; + manifest.render.modelPath = relative(working, resolve(".cache/rev2d/examples/hana/hana.r2d.json")); + const file = join(working, "unapproved.json"); + await writeFile(file, JSON.stringify(manifest)); + await expect(createFirstVideoFile({ + outputSceneFile: join(working, "blocked.json"), characterFile: file, + }, root)).rejects.toMatchObject({ code: "FIRST_VIDEO_NOT_REVIEWED" }); + }); + + it("refuses output files outside the workspace", async () => { + await expect(createFirstVideoFile({ + outputSceneFile: resolve(root, "..", "outside-project.scene.json"), + characterFile: hana, + }, root)).rejects.toMatchObject({ code: "PATH_OUTSIDE_WORKSPACE" }); + }); +}); diff --git a/tests/first-video-video.test.ts b/tests/first-video-video.test.ts new file mode 100644 index 0000000..f93a535 --- /dev/null +++ b/tests/first-video-video.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, it } from "vitest"; +import { + DEFAULT_DURATION_TOLERANCE_MS, + DEFAULT_FPS_TOLERANCE, + DEFAULT_FRAME_TOLERANCE, + DEFAULT_MAX_BYTES, + evaluateProbe, + isPathInsideRoot, + parseArgs, + parseRate, +} from "../scripts/check-first-video-video.mjs"; + +function probeFixture({ + audio = false, + width = 1920, + height = 1080, + fps = "30/1", + frames = "90", + duration = "3.000000", +}: { + audio?: boolean; + width?: number; + height?: number; + fps?: string; + frames?: string; + duration?: string; +} = {}) { + return { + streams: [ + { + index: 0, + codec_type: "video", + codec_name: "h264", + width, + height, + avg_frame_rate: fps, + r_frame_rate: fps, + nb_frames: frames, + nb_read_frames: frames, + duration, + }, + ...(audio ? [{ + index: 1, + codec_type: "audio", + codec_name: "aac", + duration, + }] : []), + ], + format: { + format_name: "mov,mp4,m4a,3gp,3g2,mj2", + duration, + size: "12345", + }, + }; +} + +function expected(overrides: Record = {}) { + return { + fps: 30, + width: 1920, + height: 1080, + frames: 90, + audio: false, + fpsTolerance: DEFAULT_FPS_TOLERANCE, + durationToleranceMs: DEFAULT_DURATION_TOLERANCE_MS, + frameTolerance: DEFAULT_FRAME_TOLERANCE, + videoCodec: undefined, + audioCodec: undefined, + ...overrides, + }; +} + +describe("first video MP4 acceptance checker", () => { + it("parses required expectations and bounded defaults without implying quality approval", () => { + expect(parseArgs([ + "outputs/first.mp4", + "--fps", "30", + "--width", "1920", + "--height", "1080", + "--frames", "90", + "--audio", + "--video-codec", "h264", + "--audio-codec", "aac", + ])).toMatchObject({ + path: "outputs/first.mp4", + fps: 30, + width: 1920, + height: 1080, + frames: 90, + audio: true, + maxBytes: DEFAULT_MAX_BYTES, + fpsTolerance: DEFAULT_FPS_TOLERANCE, + durationToleranceMs: DEFAULT_DURATION_TOLERANCE_MS, + frameTolerance: DEFAULT_FRAME_TOLERANCE, + videoCodec: "h264", + audioCodec: "aac", + }); + + expect(() => parseArgs(["outputs/first.mp4", "--fps", "30"])).toThrow("missing required argument"); + expect(() => parseArgs([ + "outputs/first.mp4", + "--fps", "30", + "--width", "1920", + "--height", "1080", + "--frames", "90", + "--audio-codec", "aac", + ])).toThrow("--audio-codec requires --audio"); + }); + + it("accepts exact video metadata and ffprobe rational frame rates", () => { + expect(parseRate("30000/1001")).toBeCloseTo(29.97002997, 8); + const result = evaluateProbe(probeFixture(), expected()); + expect(result.ok).toBe(true); + expect(result.errors).toEqual([]); + expect(result.observed).toMatchObject({ + width: 1920, + height: 1080, + fps: 30, + frames: 90, + durationSeconds: 3, + videoStreams: 1, + audioStreams: 0, + videoCodec: "h264", + }); + }); + + it("requires audio iff requested and supports exact codec checks", () => { + const unexpected = evaluateProbe(probeFixture({ audio: true }), expected()); + expect(unexpected.ok).toBe(false); + expect(unexpected.errors).toContain("expected no audio stream, found 1"); + + const missing = evaluateProbe(probeFixture(), expected({ audio: true })); + expect(missing.ok).toBe(false); + expect(missing.errors).toContain("expected exactly one audio stream, found 0"); + + const accepted = evaluateProbe( + probeFixture({ audio: true }), + expected({ audio: true, videoCodec: "h264", audioCodec: "aac" }), + ); + expect(accepted.ok).toBe(true); + }); + + it("reports dimensions, fps, frame-count, duration and container mismatches independently", () => { + const probe = probeFixture({ + width: 1280, + height: 720, + fps: "24/1", + frames: "89", + duration: "4.000000", + }); + probe.format.format_name = "matroska,webm"; + + const result = evaluateProbe(probe, expected()); + expect(result.ok).toBe(false); + expect(result.errors.join("\n")).toContain("width mismatch"); + expect(result.errors.join("\n")).toContain("height mismatch"); + expect(result.errors.join("\n")).toContain("fps mismatch"); + expect(result.errors.join("\n")).toContain("frame count mismatch"); + expect(result.errors.join("\n")).toContain("duration mismatch"); + expect(result.errors.join("\n")).toContain("container mismatch"); + }); + + it("keeps lexical paths confined to the workspace root", () => { + const root = process.platform === "win32" ? "S:\\PuppetControl" : "/repo/PuppetControl"; + const inside = process.platform === "win32" + ? "S:\\PuppetControl\\outputs\\first.mp4" + : "/repo/PuppetControl/outputs/first.mp4"; + const outside = process.platform === "win32" + ? "S:\\other\\first.mp4" + : "/repo/other/first.mp4"; + + expect(isPathInsideRoot(root, inside)).toBe(true); + expect(isPathInsideRoot(root, outside)).toBe(false); + if (process.platform === "win32") { + expect(isPathInsideRoot(root, "T:\\elsewhere\\first.mp4")).toBe(false); + } + }); +}); diff --git a/tests/first-video.test.ts b/tests/first-video.test.ts new file mode 100644 index 0000000..cc36f6d --- /dev/null +++ b/tests/first-video.test.ts @@ -0,0 +1,79 @@ +import { describe, expect, it } from "vitest"; +import { createFirstVideoScene } from "../src/scene/first-video.js"; +import { characterSchema, type Character, type SceneEvent } from "../src/scene/schemas.js"; + +function character(approved = true): Character { + return characterSchema.parse({ + schemaVersion: 1, id: "hana", defaultPose: "neutral", + poseFamilies: [{ name: "neutral", rigId: "hana-test", approved, head: "shared" }], + expressions: ["neutral", "happy", "confused"], + motions: [], transitions: [], + render: { backend: "rev2d", modelPath: "hana.r2d.json", + expressionPresets: { neutral: {}, happy: { MouthForm: 0.6 }, confused: { MouthForm: -0.4 } }, + }, + }); +} +const base = { + id: "first-video", actorId: "hana", characterPath: "./character.json", + durationMs: 10_000, fps: 24, resolution: { width: 512, height: 640 }, +}; + +describe("usable first-video scene composition", () => { + it("creates a supported strict 10-second scene, reusable without mutating the character", () => { + const actor = character(); + const before = JSON.stringify(actor); + const one = createFirstVideoScene(actor, base); + const two = createFirstVideoScene(actor, { ...base, id: "second-scene" }); + expect(one.id).toBe("first-video"); + expect(two.id).toBe("second-scene"); + expect(one.actors[0]?.characterPath).toBe(two.actors[0]?.characterPath); + expect(one.events.some((event) => event.kind === "expression")).toBe(true); + expect(one.events.some((event) => event.kind === "look")).toBe(true); + expect(one.events.every((event) => event.atMs < one.durationMs)).toBe(true); + expect(JSON.stringify(actor)).toBe(before); + }); + + it("enforces explicit reviewed pose and refuses unsupported actions", () => { + expect(() => createFirstVideoScene(character(false), base)) + .toThrow(/approve/i); + const bad: SceneEvent[] = [ + { kind: "caption", atMs: 1000, text: "not yet", durationMs: 700 }, + ]; + expect(() => createFirstVideoScene(character(), { ...base, events: bad })) + .toThrow(/support/i); + const motion: SceneEvent[] = [ + { kind: "motion", atMs: 1000, actor: "hana", name: "jump", intensity: 1 }, + ]; + expect(() => createFirstVideoScene(character(), { ...base, events: motion })) + .toThrow(/support/i); + }); + + it("validates explicit expressions, timing, IDs and voice cue paths", () => { + const wrong: SceneEvent[] = [ + { kind: "expression", atMs: 2000, actor: "hana", name: "invisible", intensity: 0.5 }, + ]; + expect(() => createFirstVideoScene(character(), { ...base, events: wrong })) + .toThrow(/not supported|UNSUPPORTED_EXPRESSION/i); + expect(() => createFirstVideoScene(character(), { ...base, fps: 0 })) + .toThrow(/duration|FPS/i); + expect(() => createFirstVideoScene(character(), { ...base, durationMs: 61_000 })) + .toThrow(/duration/i); + expect(() => createFirstVideoScene(character(), { ...base, id: "../../bad" })) + .toThrow(/scene/i); + const voice = createFirstVideoScene(character(), { + ...base, audio: { path: "./voice.wav", cuesPath: "./mouth.json", startMs: 0 }, + }); + expect(voice.audio).toEqual([{ actor: "hana", path: "./voice.wav", cuesPath: "./mouth.json", startMs: 0 }]); + expect(() => createFirstVideoScene(character(), { + ...base, audio: { path: "./voice.wav", cuesPath: "", startMs: 0 }, + })).toThrow(); + }); + + it("does not fabricate facial expressions for a rig that only supports neutral", () => { + const actor = character(); + actor.expressions = ["neutral"]; + actor.render!.expressionPresets = { neutral: {} }; + const scene = createFirstVideoScene(actor, base); + expect(scene.events.filter((event) => event.kind === "expression")).toHaveLength(0); + }); +}); diff --git a/tests/renderability.test.ts b/tests/renderability.test.ts new file mode 100644 index 0000000..0e38501 --- /dev/null +++ b/tests/renderability.test.ts @@ -0,0 +1,65 @@ +import { describe, expect, it } from "vitest"; +import { characterSchema, sceneSchema } from "../src/scene/schemas.js"; +import { checkSceneRenderability } from "../src/render/renderability.js"; + +const actor = characterSchema.parse({ + schemaVersion: 1, id: "person", defaultPose: "neutral", + poseFamilies: [{ name: "neutral", rigId: "rig", approved: true, head: "shared" }], + expressions: ["neutral"], motions: ["wave"], transitions: [], + render: { backend: "rev2d", modelPath: "rig.r2d.json", + expressionPresets: { neutral: {} } }, +}); + +const base = { + schemaVersion: 1, id: "clip", durationMs: 3000, fps: 15, + resolution: { width: 512, height: 640 }, + actors: [{ id: "person", characterPath: "./character.json" }], + audio: [], allowCameraCuts: false, events: [], +}; + +const asScene = (patch: Record = {}) => sceneSchema.parse({ ...base, ...patch }); +const loaded = new Map([["person", actor]]); + +describe("strict MP4 renderability preflight", () => { + it("accepts a simple supported single-rig actor scene", () => { + expect(checkSceneRenderability(asScene(), loaded)).toEqual({ valid: true, diagnostics: [] }); + }); + it("refuses audio without required lip-cue mapping", () => { + const result = checkSceneRenderability(asScene({ + audio: [{ path: "voice.wav", startMs: 0, actor: "person" }], + }), loaded); + expect(result.valid).toBe(false); + expect(result.diagnostics[0]?.code).toBe("LIP_CUES_REQUIRED"); + }); + it("refuses a caption and declared motion even when general schema permits them", () => { + const result = checkSceneRenderability(asScene({ + events: [ + { kind: "caption", atMs: 0, text: "Title", durationMs: 500 }, + { kind: "motion", atMs: 1000, actor: "person", name: "wave" }, + ], + }), loaded); + expect(result.diagnostics.map((d) => d.code)) + .toEqual(["COMPOSITION_NOT_IMPLEMENTED", "MOTION_NOT_IMPLEMENTED"]); + }); + it("refuses multiple actors because the current renderer can only draw one", () => { + const result = checkSceneRenderability(asScene({ + actors: [{ id: "person", characterPath: "./character.json" }, + { id: "other", characterPath: "./other.json" }], + }), new Map([["person", actor], ["other", actor]])); + expect(result.diagnostics.some((d) => d.code === "MULTI_ACTOR_UNSUPPORTED")).toBe(true); + }); + it("detects a bridge that is plan-valid but cannot be rendered", () => { + const alternative = characterSchema.parse({ + ...actor, + poseFamilies: [ + { name: "neutral", rigId: "rig", approved: true, head: "shared", modelPath: "rig.r2d.json" }, + { name: "folded", rigId: "folded-rig", approved: true, head: "shared", modelPath: "folded.r2d.json" }, + ], + transitions: [{ kind: "bridge", from: "neutral", to: "folded", clip: "folded-bridge.anim", durationMs: 250 }], + }); + const result = checkSceneRenderability(asScene({ + events: [{ kind: "pose", atMs: 500, actor: "person", name: "folded", transition: "bridge" }], + }), new Map([["person", alternative]])); + expect(result.diagnostics.some((d) => d.code === "BRIDGE_RENDER_NOT_IMPLEMENTED")).toBe(true); + }); +}); diff --git a/tests/review-approval.test.ts b/tests/review-approval.test.ts new file mode 100644 index 0000000..82c11d4 --- /dev/null +++ b/tests/review-approval.test.ts @@ -0,0 +1,141 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { basename, dirname, join } from "node:path"; +import { tmpdir } from "node:os"; +import { createImportedCharacter } from "../src/character/avatar.js"; +import { createAvatarBuildRecord, writeAvatarBuildRecord } from "../src/character/provenance.js"; +import { approveImportedAvatar } from "../src/character/review-approval.js"; + +const dirs: string[] = []; + +afterEach(async () => { + await Promise.all(dirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true }))); +}); + +function pngHeader(width = 512, height = 640): Buffer { + const header = Buffer.alloc(24); + Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]).copy(header, 0); + header.writeUInt32BE(13, 8); + header.write("IHDR", 12); + header.writeUInt32BE(width, 16); + header.writeUInt32BE(height, 20); + return header; +} + +async function fixture(options: { approved?: boolean; reviews?: boolean } = {}) { + const dir = await mkdtemp(join(tmpdir(), "puppetcontrol-approval-")); + dirs.push(dir); + await mkdir(join(dir, "textures")); + await mkdir(join(dir, "review")); + await writeFile(join(dir, "source.png"), "source-art-v1"); + await writeFile(join(dir, "avatar.json"), '{"format":"rev2d-avatar","name":"test"}\n'); + await writeFile(join(dir, "textures", "base.png"), "rendered-layer-v1"); + await writeFile(join(dir, "test.r2d.json"), JSON.stringify({ + textures: [{ id: "base", path: "textures/base.png", width: 512, height: 640 }], + })); + + const parameters = ["AngleX", "AngleY", "EyeBallX", "EyeBallY", "MouthOpen", "MouthForm"] + .map((id) => ({ id, min: -1, max: 1, default: 0 })); + const character = createImportedCharacter("test", { parameters }); + character.poseFamilies[0]!.approved = options.approved ?? false; + // Preserve a legitimate user edit while approval changes only the approved bit. + character.expressions = ["neutral", "happy", "confused", "serious"]; + await writeFile(join(dir, "character.json"), JSON.stringify(character, null, 2) + "\n"); + + const record = await createAvatarBuildRecord(dir, "test.r2d.json"); + await writeAvatarBuildRecord(dir, record); + + if (options.reviews !== false) { + for (const name of ["poses.png", "turn.png", "eyes.png", "mouth.png"]) { + await writeFile(join(dir, "review", name), pngHeader()); + } + } + return { dir, root: dirname(dir), characterPath: join(dir, "character.json") }; +} + +describe("explicit imported-avatar review approval", () => { + it("rejects approval without explicit reviewed=true and leaves the manifest untouched", async () => { + const { dir, root, characterPath } = await fixture(); + const before = await readFile(characterPath, "utf8"); + await expect(approveImportedAvatar(dir, false, root)) + .rejects.toMatchObject({ code: "AVATAR_REVIEW_REQUIRED" }); + expect(await readFile(characterPath, "utf8")).toBe(before); + }); + + it("rejects missing or implausible review artifacts without approving anything", async () => { + const { dir, root, characterPath } = await fixture({ reviews: false }); + const before = await readFile(characterPath, "utf8"); + await writeFile(join(dir, "review", "poses.png"), pngHeader()); + await writeFile(join(dir, "review", "turn.png"), pngHeader()); + await writeFile(join(dir, "review", "eyes.png"), pngHeader()); + + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "AVATAR_REVIEW_ASSET_MISSING" }); + expect(await readFile(characterPath, "utf8")).toBe(before); + + await writeFile(join(dir, "review", "mouth.png"), pngHeader(1, 640)); + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "AVATAR_REVIEW_ASSET_INVALID" }); + expect(await readFile(characterPath, "utf8")).toBe(before); + }); + + it("rejects stale provenance-tracked assets and has no approval side effect", async () => { + const { dir, root, characterPath } = await fixture(); + const before = await readFile(characterPath, "utf8"); + await writeFile(join(dir, "textures", "base.png"), "changed-after-review"); + + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "AVATAR_ASSETS_STALE" }); + expect(await readFile(characterPath, "utf8")).toBe(before); + }); + + it("approves atomically while preserving other manifest edits", async () => { + const { dir, root, characterPath } = await fixture(); + const result = await approveImportedAvatar(basename(dir), true, root); + expect(result).toMatchObject({ + valid: true, status: "approved", approved: true, changed: true, + }); + expect(result.message).toMatch(/cannot technically verify/i); + + const saved = JSON.parse(await readFile(characterPath, "utf8")) as { + poseFamilies: Array<{ approved: boolean; modelPath?: string }>; + expressions: string[]; + render: { modelPath: string }; + }; + expect(saved.poseFamilies).toHaveLength(1); + expect(saved.poseFamilies[0]!.approved).toBe(true); + expect(saved.poseFamilies[0]!.modelPath).toBe("test.r2d.json"); + expect(saved.render.modelPath).toBe("test.r2d.json"); + expect(saved.expressions).toEqual(["neutral", "happy", "confused", "serious"]); + }); + + it("is explicitly idempotent for an already-approved avatar after rechecking safety gates", async () => { + const { dir, root, characterPath } = await fixture({ approved: true }); + const before = await readFile(characterPath, "utf8"); + const result = await approveImportedAvatar(dir, true, root); + expect(result).toMatchObject({ + valid: true, status: "already-approved", approved: true, changed: false, + }); + expect(await readFile(characterPath, "utf8")).toBe(before); + }); + + it("rejects additional pose families that are not covered by this single-pose approval flow", async () => { + const { dir, root, characterPath } = await fixture(); + const raw = JSON.parse(await readFile(characterPath, "utf8")) as { + poseFamilies: Array>; + }; + raw.poseFamilies.push({ + name: "pointing", + rigId: "test-avatar", + modelPath: "test.r2d.json", + head: "shared", + approved: false, + }); + await writeFile(characterPath, JSON.stringify(raw, null, 2) + "\n"); + const before = await readFile(characterPath, "utf8"); + + await expect(approveImportedAvatar(dir, true, root)) + .rejects.toMatchObject({ code: "AVATAR_POSE_FAMILIES_UNSUPPORTED" }); + expect(await readFile(characterPath, "utf8")).toBe(before); + }); +}); diff --git a/tests/video-output-atomic.test.ts b/tests/video-output-atomic.test.ts new file mode 100644 index 0000000..7d9d8be --- /dev/null +++ b/tests/video-output-atomic.test.ts @@ -0,0 +1,44 @@ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { encodeFrames, type FrameResult } from "../src/render/rev2d.js"; + +let directory = ""; +beforeEach(async () => { + await mkdir(resolve(".cache"), { recursive: true }); + directory = await mkdtemp(join(resolve(".cache"), "video-atomic-")); +}); +afterEach(async () => { + if (directory) await rm(directory, { recursive: true, force: true }); +}); + +function fixtureFrames(): FrameResult { + return { + framesDir: join(directory, "missing-frames"), + count: 2, fps: 15, width: 512, height: 640, + manifestPath: "", characterId: "fixture", poseFamilies: ["neutral"], + }; +} + +describe("non-destructive MP4 output", () => { + it("refuses an existing output before launching FFmpeg or altering its bytes", async () => { + const output = join(directory, "original.mp4"); + const original = Buffer.from("previous good video must survive even when source frames do not exist"); + await writeFile(output, original); + await expect(encodeFrames(fixtureFrames(), output)) + .rejects.toMatchObject({ code: "VIDEO_OUTPUT_EXISTS" }); + expect(await readFile(output)).toEqual(original); + expect((await readdir(directory)).filter((entry) => entry.endsWith(".mp4"))) + .toEqual(["original.mp4"]); + }); + + it("rejects unsupported extension before attempting FFmpeg", async () => { + await expect(encodeFrames(fixtureFrames(), join(directory, "movie.txt"))) + .rejects.toMatchObject({ code: "VIDEO_OUTPUT_INVALID" }); + }); + + it("refuses an output path outside the project", async () => { + await expect(encodeFrames(fixtureFrames(), resolve("..", "export-outside.mp4"))) + .rejects.toMatchObject({ code: "PATH_OUTSIDE_WORKSPACE" }); + }); +}); From b4881f06e350030d5b406f8b64226c89472b574e Mon Sep 17 00:00:00 2001 From: Navneet Chaudhary Date: Sat, 7 Nov 2026 14:43:50 +0530 Subject: [PATCH 2/3] test: make first-video scene fixtures self-contained --- tests/first-video-file.test.ts | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/tests/first-video-file.test.ts b/tests/first-video-file.test.ts index 6d9cd0d..edbcb0f 100644 --- a/tests/first-video-file.test.ts +++ b/tests/first-video-file.test.ts @@ -1,22 +1,34 @@ import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; -import { join, relative, resolve } from "node:path"; +import { join, resolve } from "node:path"; import { createFirstVideoFile } from "../src/scene/create-first-video-file.js"; import { loadAndValidateScene } from "../src/scene/load.js"; let root = ""; let working = ""; +let hana = ""; beforeEach(async () => { root = resolve(process.cwd()); await mkdir(resolve(root, ".cache"), { recursive: true }); working = await mkdtemp(join(resolve(root, ".cache"), "first-video-file-")); + // Hermetic fixture: CI intentionally does not install the ignored .cache/rev2d + // backend. We only test scene-file composition, not real model rendering. + const character = JSON.parse( + await readFile(resolve(root, "examples/characters/hana/character.json"), "utf8"), + ) as { render: { modelPath: string } }; + character.render.modelPath = "fixture.r2d.json"; + hana = join(working, "character.json"); + await writeFile(hana, JSON.stringify(character)); + await writeFile(join(working, "fixture.r2d.json"), JSON.stringify({ + meta: { name: "test-rig", width: 512, height: 640 }, + parameters: [], animations: [], + })); }); afterEach(async () => { if (working) await rm(working, { recursive: true, force: true }); }); describe("first-video file creation", () => { - const hana = resolve("examples/characters/hana/character.json"); it("creates two editable scenes using exactly one already approved rig without mutating the character", async () => { const before = await readFile(hana, "utf8"); const outputs = [join(working, "first.json"), join(working, "revision.json")]; @@ -73,7 +85,7 @@ describe("first-video file creation", () => { it("refuses unapproved imported-style characters rather than silently granting approval", async () => { const manifest = JSON.parse(await readFile(hana, "utf8")); manifest.poseFamilies[0].approved = false; - manifest.render.modelPath = relative(working, resolve(".cache/rev2d/examples/hana/hana.r2d.json")); + manifest.render.modelPath = "fixture.r2d.json"; const file = join(working, "unapproved.json"); await writeFile(file, JSON.stringify(manifest)); await expect(createFirstVideoFile({ From 0333419aeb7a33ee76ca83c505de988b57159760 Mon Sep 17 00:00:00 2001 From: Navneet Chaudhary Date: Sat, 7 Nov 2026 14:45:20 +0530 Subject: [PATCH 3/3] docs: make first-video commands PowerShell friendly --- docs/FIRST_VIDEO.md | 27 ++++++--------------------- 1 file changed, 6 insertions(+), 21 deletions(-) diff --git a/docs/FIRST_VIDEO.md b/docs/FIRST_VIDEO.md index d41c52f..eb651ac 100644 --- a/docs/FIRST_VIDEO.md +++ b/docs/FIRST_VIDEO.md @@ -31,7 +31,7 @@ Only for this pinned Hana QA fixture, explicitly continue with: node scripts/demo-first-video.mjs --reviewed-sample -That second command uses the current CLI scene new, scene validate, and scene render commands to create two separate 5-second scenes that both point at the same copied Hana rig. It then runs the FFprobe acceptance checker on both MP4s. +That second command creates a **separate disposable copy** of the same pinned Hana sample. Its flag records acknowledgement for a test fixture, not verified review of that newly created copy. It uses the current CLI scene new, scene validate, and scene render commands to create two separate 5-second scenes that both point at the same copied Hana rig. It then runs the FFprobe acceptance checker on both MP4s. Each run is isolated under ignored .cache/first-video-demo/run-XXXXXX/. Expected outputs are: @@ -50,14 +50,9 @@ The scene new command is for an actor that has already passed the project's real This validates source/landmark/rig/texture fingerprints and presence of review sheets before recording approval. It cannot technically verify that a human looked at them. Do not script approval for unreviewed user art. -After npm run build, create an editable scene: +After `npm run build`, create an editable scene. Commands below are each a single line so they work in **PowerShell**, Command Prompt, bash and other supported shells: - node dist/cli/index.js scene new .cache/my-video/scene-a.json \ - --character path/to/character.json \ - --id scene-a \ - --seconds 5 \ - --fps 15 \ - --size 512x640 + node dist/cli/index.js scene new .cache/my-video/scene-a.json --character path/to/character.json --id scene-a --seconds 5 --fps 15 --size 512x640 Edit only events that the character actually supports, then validate and render: @@ -72,20 +67,11 @@ Voice is optional. PuppetControl does not synthesize speech or pretend that lip If Rhubarb is installed, align a WAV first: - node dist/cli/index.js audio align path/to/voice.wav \ - --out .cache/my-video/mouth.json \ - --binary rhubarb + node dist/cli/index.js audio align path/to/voice.wav --out .cache/my-video/mouth.json --binary rhubarb Then create the scene with both the WAV and cue JSON: - node dist/cli/index.js scene new .cache/my-video/voiced.json \ - --character path/to/character.json \ - --id voiced \ - --seconds 5 \ - --fps 15 \ - --size 512x640 \ - --voice path/to/voice.wav \ - --cues .cache/my-video/mouth.json + node dist/cli/index.js scene new .cache/my-video/voiced.json --character path/to/character.json --id voiced --seconds 5 --fps 15 --size 512x640 --voice path/to/voice.wav --cues .cache/my-video/mouth.json The current first-video path requires --voice and --cues together. Validate and render the resulting scene normally. A real 12-second test using local nonredistributed voice input successfully generated 36 Rhubarb cues and a 180-frame H.264/AAC MP4 at 512×640, 15 FPS. The audio recording, cues and MP4 are intentionally kept only in ignored .cache/first-video-demo/qa-12s-voiced/. This proves file/cue/mux integration, not perceptual lip-sync quality on every speaker. @@ -101,8 +87,7 @@ For ordinary revisions, prefer a new `--out` name so the prior render remains av For a 5-second 15-fps 512x640 silent export: - node scripts/check-first-video-video.mjs .cache/my-video/scene-a.mp4 \ - --fps 15 --width 512 --height 640 --frames 75 --video-codec h264 + node scripts/check-first-video-video.mjs .cache/my-video/scene-a.mp4 --fps 15 --width 512 --height 640 --frames 75 --video-codec h264 For a voiced export, add --audio --audio-codec aac.