Long renders: one-GOP chunks taken in turn, resume a stopped or failed render, an estimate first; --fps vs the page's, --out's base

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 ## Unreleased
 
+**Long renders: one-GOP chunks taken in turn, a stopped or failed render resumes, an estimate before a long one; `--fps` against the page's own; where a relative `--out` goes**
+- Why: `bin/vh render` split the frames evenly by count across its workers, so one heavy chapter (3D at 4K) kept one worker busy long after the others were done, and a frame that timed out late in a long 4K render threw the whole render away. The way round was to render the film as two `--range` runs and join them by hand, which leaves no sidecar, so `bin/vh patch` could not be used on the result.
+- The span is now cut into chunks of one GOP each, and the `--workers` browsers take the next chunk as they finish one. Each chunk goes into an x264 of its own, so the video no longer depends on `--workers` or on which browser drew what: 1, 2 and 4 workers give the same decoded frames, where before each worker count gave other bytes. The keyframe grid, the concat of whole-GOP chunks that patch cuts on, and the sidecar are as before (`chunks` now lists one per GOP; new `reused_chunks`). The join gives each chunk its exact length, as patch's join does. The frames drawn are the same as before. The encoded bytes are the same as an earlier render that also had one GOP per chunk; where it had several GOPs per chunk they differ at x264's rate-control level (54 dB PSNR on the canvas-film template, which was cut into 4 chunks of 3 GOPs).
+- Stopped (Ctrl-C, SIGTERM) or failed (a frame past `--frame-timeout`, an encoder that stopped), a render keeps the chunks it finished in `.<out>.parts/`. `plan.json` there records what they were made from: the page and the sha256 of every file the browsers were sent, the options that shape the video (all but `--workers`, `--frame-timeout` and the estimate), the browser, ffmpeg and the render tools. The same command again draws only the rest; when any of that changed, it names what changed and starts over. A chunk in which the page reported an error is never kept. After a failure, the other workers finish the chunk they are on first. With this and the queue, neither reason for hand-joined range renders is left, so no join command was added.
+- The estimate: a span of more than 600 frames to draw (any span with `--estimate`) first draws six places of three consecutive frames in one browser and encodes them with the film's settings. It prints the time a frame takes, the time with the workers, the video's size as a range, and the disk needed at the peak, and stops (exit 2) when even the low end does not fit. About a second for the template at 1080p; `--no-estimate` skips it, and `bin/vh patch` skips it. On the canvas-film template it estimated 9 s and took 11 s, and gave 6–12 MB for a 9.0 MB video at 60 fps and 10–19 MB for 9.3 MB at 30 fps.
+- `--fps` other than the page's own frame rate is said, naming both. The page contract has an optional `fps` field (`engines/CONTRACT.md`). The canvas-film template gives `FILM.fps` there; for a copy made before that, render reads `FILM.fps` directly.
+- A relative `--out` stays relative to the current folder, as the output paths of the other tools are (frames, textcheck, storyboard, deliver, mux). When that puts the video outside the project (bin/vh run from the harness with `--out out/x.mp4`), the full path is printed at the start and at the end.
+- `bin/vh render -h`, the tool's own help and `engines/README.md` (canvas-film) describe all this.
+- Regression coverage: a page whose last second is slow, with one very slow frame, renders with 3 workers to the video 1 worker makes (PSNR). With `--frame-timeout 1` it keeps 5 of 6 chunks; after an edit to the page the next run starts over; the same command then resumes and gives the whole render's video. The template at `--fps 60`, run from another folder with a relative `--out`, prints the `--out` note, the fps note and the estimate, and is refused on a disk with room only for a flat film.
+
 **The other type docs' 禁止 sections split into floors and common misuses; a few general notes**
 - Why: every film reads its type doc (quick reads its 禁止 section), and those lists mixed floors (captions in the platform's UI zone, placeholder data, an unlicensed logo, a title the film does not keep) with taste (blue-purple gradients, six equal cards, 3D boxes in a hand-drawn film). One hand-drawn item, "every shot a different world with nothing linking them", read as forbidding a film that changes world each chapter on purpose.
 - Types 01–04 and 06–09, as 05 already was: 底线 (and 会出错的, for code and encode errors), then 常见误用（参考）, each with its reason. The heading stays 禁止, so quick still reads it. Prompt blocks keep their defaults but drop the fixed "never"s (a plain crossfade, a screenshot of the PDF, a joke in under a second, every cut on a beat: declared hits stay ±1 frame), and 02's "one line each in DECISIONS.md" is gone.

bin/vh

@@ -90,9 +90,15 @@
 #                               --covers: the page's covers (window.film.covers; canvas-film COVERS, picked by cue) in <project>/out/covers
 #                               (--no-title: without their titles, <id>-clean.png; --ratios: each ratio a crop of the cover with its own title, <id>-9x16.png)
 #   render <project|page.html> [--out f.mp4] [--fps 30] [--range a:b | --frames k0:k1] [--dpr 2] [--workers 4] [--crf 16] [--gop 60] [--png]
+#          [--estimate | --no-estimate]
 #                               an MP4 drawn through the page contract (the canvas engine, ClaudeAnimationBase, any page that writes
-#                               window.film): parallel chunks into x264, BT.709 tv range, a keyframe every --gop frames, <out>.render.json
-#                               records how; video only (sound: mux). HyperFrames: npx hyperframes render (--force to render it here anyway).
+#                               window.film): one-GOP chunks that the --workers browsers take in turn, each into its own x264, BT.709 tv
+#                               range, a keyframe every --gop frames, <out>.render.json records how; video only (sound: mux). Stopped or
+#                               failed, the finished chunks stay: the same command resumes while the page, its files and the options are
+#                               unchanged. Over 600 frames it first draws a few and prints the time, size and disk the render needs
+#                               (refused when the disk cannot hold it; --no-estimate skips). Says when --fps is not the page's own
+#                               (window.film.fps). A relative --out is from the current folder. HyperFrames: npx hyperframes render
+#                               (--force to render it here anyway).
 #                               A whole standard or studio film waits for gate ① on record (tools/gatecheck.py); --range and --frames do not
 #   patch <film.mp4> (--frames a-b | --seconds s-e) [--sidecar f.render.json] [--out f | --in-place] [--workers 4] [--verify seams|full]
 #                               draw frames a–b of a bin/vh render film again and put them back, the rest untouched: cut on its keyframes
@@ -594,7 +600,7 @@ arg_dir() { # the folder of the project or page among a tool's arguments: the fi
   for a in "$@"; do
     if [ $skip = 1 ]; then skip=0; continue; fi
     case $a in
-      --soft-gl|--real-clock|--png|--force|--save|--export|--strict|--in-place|--keep|--snapshot|--*=*) ;;
+      --soft-gl|--real-clock|--png|--force|--save|--export|--strict|--in-place|--keep|--snapshot|--estimate|--no-estimate|--*=*) ;;
       -*) skip=1 ;;
       *) if [ -d "$a" ]; then echo "$a"; else case $a in */*) echo "${a%/*}" ;; *) echo . ;; esac; fi; return 0 ;;
     esac

engines/CONTRACT.md

@@ -39,6 +39,7 @@ window.film = {
 | `timeline` | 这些 cue 挂在哪份旁白时间轴上：相对页面的路径（`audio/timeline.zh.json`），没有旁白时是 `null` | `bin/vh cues` 记下它的 sha256，认出换过的配音 |
 | `music` | `{path, sha256}`，相对页面的配乐 beats 文件路径，以及页面实际加载字节的 SHA-256；无配乐时为 `null` | `bin/vh cues --save` 保存来源，重跑时提示配乐表变化 |
 | `timemap` | 可选。`{path, sha256}`，相对页面的时间映射文件（数据轴的折点和事件，`tools/audio/time_map.py` 说明格式），以及页面实际加载字节的 SHA-256；没有时为 `null` 或不提供 | `bin/vh cues --save` 保存来源，重跑时提示映射变化 |
+| `fps` | 可选。页面是按哪个帧率做的（数字）：每帧变一次的东西（颗粒、抖动）按它变 | `bin/vh render`：`--fps` 和它不同时提醒一句，两个数都写出来 |
 | `ui(t)` | 方法：t 秒时常驻界面（进度条、署名、字幕）的字和线在哪：`[{name, box: [x0, y0, x1, y1]}]`，画面的 CSS px；只框界面自己的字和线，垫在下面的底板、渐变不算。写了它，`frame(t, { layer: "scene" })` 就要画出不带界面的这一帧（不认这个参数的页面照常画整帧，宿主会看出来、说一声） | `bin/vh textcheck`：两种画法各画一遍，场景自己在这些框里画了线、边、字的，报出来（警告） |
 
 这几项都是数据，在 `ready` 里算好：OpenFilm 的说法是"跟着声音走的时间是数据，不是接口"。`cover(id)` 和 `ui(t)` 是方法，同样是 t 的纯函数。

engines/README.md

@@ -34,6 +34,12 @@ bin/vh frames projects/<date>-<slug> --at 1,5,9   # 模板带一支 23 秒的示
 - 一页 `film.html` 加 `src/*.js`，不用构建。它按页面合约写了 `window.film`，`bin/vh frames` 取帧、`bin/vh textcheck` 查字直接能用；reads 和字幕（`texts`）、画面事件（`events`）、章节和 cue 表都在 `ready` 里算好。
 - 字号下限在画字的最底层兜住，故意要小的字写 `texture: true`；绕过它写下的小字，`ready` 时会被报出来。
 - 成片画面用 `bin/vh render <项目>`（4K 加 `--dpr 2`），再用 `bin/vh mux` 套上混音。出片后只改了一镜，用 `bin/vh patch <成片> --seconds <起>-<止>` 只重画那一段、在关键帧处接回去（套过混音的成片加 `--sidecar out/film.mp4.render.json`）。现在只有 2D canvas，没有 Three.js 底层。
+- 长片的 `bin/vh render`（也适用于别的写了 `window.film` 的页面）：
+  - 片子按 GOP 切段，一个 GOP 一段；`--workers` 个浏览器谁画完谁取下一段，重的一章不会让别的 worker 干等。每段单独进一个 x264，所以成片和 `--workers` 是几无关，sidecar 照旧，`bin/vh patch` 照旧能用。
+  - 停下（Ctrl-C）或失败（比如一帧超过 `--frame-timeout`）时，画完的段留在视频旁边的 `.<视频名>.parts/` 里，原样再跑同一条命令只画剩下的。页面和它载入的每个文件、决定成片的参数（`--workers`、`--frame-timeout` 和估算之外的都算）、浏览器和渲染工具本身都没变，才接着用；变了会说哪里变了，从头渲。所以不用再分几段 `--range` 渲完手拼：手拼的成片没有一份完整的 sidecar，`bin/vh patch` 用不上。
+  - 要画的帧超过 600 帧时，先在片中均匀取几帧画出来、按成片的设置编码，打印每帧耗时、预计用时、视频大概多大、渲染时磁盘最多要多少（各段加合并出的那份，约两倍），磁盘放不下就停（exit 2）。几秒钟；`--no-estimate` 跳过，`--estimate` 让短片也估。用时是按一个浏览器的速度除以 worker 数算的，几个浏览器共用一块 GPU 或机器忙时会更久。
+  - 页面给出 `window.film.fps` 时（模板给的是 `FILM.fps`，颗粒每帧换一次），`--fps` 和它不一样会提醒，两个数都写出来；改帧率时两处一起改。之前复制的项目没有这个字段，`render` 直接读 `FILM.fps`，不用改。
+  - `--out` 的相对路径和别的 `bin/vh` 命令一样，从当前目录算起。在仓库根目录跑 `--out out/x.mp4`，视频会在仓库的 `out/` 里，不在项目里，这时会把完整路径打出来；要放进项目，写项目里的路径，或者不写 `--out`（默认 `<项目>/out/<页面名>.mp4`）。
 
 ## HyperFrames（HTML + GSAP）
 

engines/canvas-film/template/src/90-main.js

@@ -191,6 +191,7 @@ window.film = {
   ready: READY,
   frame,
   get duration() { return DUR; },
+  get fps() { return FILM.fps; },                                                // the rate it is made for (bin/vh render says when --fps differs)
   get shots() {                                                                 // actual cut boundaries, resolved by ready
     return SHOT_T.map((S) => {
       const s = S.shot, chapter = CHAPTER_T.filter((c) => c.t <= S.a).at(-1);

tools/ci.sh

@@ -1149,6 +1149,42 @@ PY
     a=$(OVH_FREE_BYTES=1000 vh render tools/film/fixture --out "$t/r/full.mp4" 2>&1); rc=$?
     case "$rc:$a" in 2:*"needed at the peak"*"GB free on that disk"*) ok "render: a disk too full for even a flat film stops it before a frame is drawn (exit 2)" ;;
       *) bad "render on a full disk: exit $rc: $(printf '%s' "$a" | tail -c 400)" ;; esac
+    # the queue and the resume, on a page whose frames cost unevenly (tools/film/fixture/uneven.html: from 2 s on 30 ms
+    # each, frame 75 1.5 s): chunks of one GOP, so 3 workers make the video 1 makes (PSNR: separate browsers). A render
+    # whose frame 75 passes --frame-timeout 1 keeps the 5 chunks it finished; with the page edited they are not used;
+    # the same command then draws only the last chunk, and the video is the one a whole render makes
+    mkdir -p "$t/q" && cp tools/film/fixture/uneven.html "$t/q/u.html"
+    vh render "$t/q/u.html" --workers 1 --gop 15 --out "$t/q/one.mp4" >/dev/null 2>&1
+    a=$(vh render "$t/q/u.html" --workers 3 --gop 15 --out "$t/q/three.mp4" 2>&1); rc=$?
+    b=$(vh render "$t/q/u.html" --workers 2 --gop 15 --frame-timeout 1 --out "$t/q/r.mp4" 2>&1); rc2=$?
+    printf '<!-- edited -->\n' >> "$t/q/u.html"
+    c=$(vh render "$t/q/u.html" --workers 2 --gop 15 --frame-timeout 1 --out "$t/q/r.mp4" 2>&1); rc3=$?
+    d=$(vh render "$t/q/u.html" --workers 2 --gop 15 --out "$t/q/r.mp4" 2>&1); rc4=$?
+    if [ "$rc:$rc2:$rc3:$rc4" = 0:1:1:0 ] && printf '%s' "$b" | grep -q '5 of 6 chunks are kept' && printf '%s' "$c" | grep -q 'not used (changed: .*u\.html)' \
+      && printf '%s' "$d" | grep -q 'resuming: 5 of 6 chunks kept' && [ ! -e "$t/q/.r.mp4.parts" ] && python3 - "$t/q" <<'PY'
+import json, re, subprocess, sys
+d = sys.argv[1]
+def psnr(a, b):
+    r = subprocess.run(["ffmpeg", "-v", "info", "-i", a, "-i", b, "-lavfi", "psnr", "-f", "null", "-"], capture_output=True, text=True).stderr
+    m = re.findall(r"average:(inf|[\d.]+)", r)
+    return float(m[-1]) if m else 0
+side = json.load(open(d + "/r.mp4.render.json"))
+ok = side["chunks"] == [[k, k + 15] for k in range(0, 90, 15)] and side["reused_chunks"] == 5 and side["frames"] == 90
+sys.exit(0 if ok and psnr(d + "/one.mp4", d + "/three.mp4") >= 45 and psnr(d + "/one.mp4", d + "/r.mp4") >= 45 else 1)
+PY
+    then ok "render: one-GOP chunks taken in turn (3 workers make what 1 makes); a timed-out frame keeps the finished chunks, an edited page starts over, the same command resumes"
+    else bad "render queue and resume: exit $rc:$rc2:$rc3:$rc4: $(printf '%s' "$b$c$d" | tail -c 900)"; fi
+    # the estimate, the page's own fps and a relative --out, from another folder: the canvas-film template (made for 30 fps)
+    # at --fps 60 is over 600 frames, so it is estimated; the disk has room for a flat film (7.2 MB at the peak) but not
+    # for what was measured (about 12 MB)
+    mkdir -p "$t/e"
+    a=$(cd "$t/e" && OVH_FREE_BYTES=8000000 "$VH_BASH" "$ROOT/bin/vh" render "$ROOT/engines/canvas-film/template" --fps 60 --out rel/f.mp4 2>&1); rc=$?
+    case "$rc:$a" in
+      2:*"--out rel/f.mp4 is from the current folder: "*"/e/rel/f.mp4, outside the project"*"drawn at 60 fps (--fps), the page says 30"*"estimate ("*" frames in about"*"is needed at the peak and 8 MB is free"*)
+        [ ! -e "$t/e/rel/.f.mp4.parts" ] && ok "render: a long span is estimated first and refused on a disk that cannot hold it; --fps unlike the page's and an --out outside the project are said" \
+          || bad "render: the refused estimate left $t/e/rel/.f.mp4.parts behind" ;;
+      *) bad "render estimate, fps and --out (exit $rc): $(printf '%s' "$a" | tail -c 900)" ;;
+    esac
     # bin/vh patch: the fixture rendered from 0.5 s with a keyframe every 20 frames (not a whole number of milliseconds) in
     # a folder whose name a filter graph would choke on; then its square turned blue from 1.4 s up to 1.7 s (the film's
     # frames 27–35) and those frames put back: 20–39 drawn again, the rest the film's own bytes and times. Naming only 27–29

tools/film/fixture/uneven.html

@@ -0,0 +1,25 @@
+<!doctype html>
+<html>
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=320, height=180">
+<style>html, body { margin: 0; background: #000; } canvas { display: block; }</style>
+</head>
+<body>
+<canvas id="c" width="320" height="180"></canvas>
+<script>
+  // bin/vh render's fixture (tools/ci.sh): frames that cost unevenly, as a heavy chapter does. From 2 s on each frame
+  // takes 30 ms of real time, and frame 75 (2.5 s at 30 fps) 1.5 s, past a --frame-timeout of 1. The picture is still a
+  // function of t alone. It gives the frame rate it is made for (window.film.fps).
+  const c = document.getElementById("c"), x = c.getContext("2d");
+  window.film = {
+    duration: 3, fps: 30,
+    frame(t) {
+      x.fillStyle = `hsl(${Math.round(t * 120)}, 60%, 30%)`; x.fillRect(0, 0, 320, 180);
+      x.fillStyle = "#fc0"; x.fillRect(20 + t * 80, 90, 40, 40);
+      if (t >= 2) return new Promise((ok) => setTimeout(ok, Math.round(t * 30) === 75 ? 1500 : 30));
+    },
+  };
+</script>
+</body>
+</html>