Skip to content

feat(layers): highlight parts of a text layer while the video plays through them - #876

Merged
hm21 merged 2 commits into
stablefrom
feat/text-layer-highlights
Sep 29, 2026
Merged

hm21 merged 2 commits into
stablefrom
feat/text-layer-highlights

Conversation

@hm21

@hm21 hm21 commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

Description

Word-by-word ("karaoke") captions light up each word while it is spoken. A video editor built on this package can already time a text layer, but a text layer has one look for its whole time range, and a layer is exported as a single image. Neither the preview nor the export could change the color of one word at a time.

This adds timed highlights to text layers: a part of the text that is drawn in a highlight color while the video plays through it.

Model

  • TextHighlight is a UTF-16 range of TextLayer.text plus a startTime/endTime. It is @immutable, with toMap/fromMap, copyWith and value equality. fromMap is lenient like LayerAnimation.fromMap; TextLayer.fromMap drops highlights that can never show.
  • TextLayer.highlights and TextLayer.highlightColor (default: none, a warm yellow). They are serialized only when there are highlights, with minified keys hl / hc, and diffed in toMapFromReference.
  • Highlight times count from the layer's startTime, not from the start of the video, so moving a layer along the timeline keeps its highlights on their words. When highlights overlap, the last one in the list wins (TextLayer.highlightIndexAt).
  • LayerCopyManager.createCopyTextLayer copies them, so they survive edit, duplicate, group and crop/rotate like every other field.

Preview

LayerWidget hands its play-time notifier to the text item, which draws the active highlight. It rebuilds only when the active highlight changes, not on every frame. Only the color of a span changes, so the layout and the layer's size stay the same whichever word is lit. Without a play time (image editing) no highlight is drawn.

Export

  • The on-screen boundary shows whichever word the play head is on, so a text layer with highlights is captured from its model instead: the base image with no highlight, and one image per highlight in ExportedLayer.highlightBytes. All of them have the same size, so each can stand in for the base at the same position. This goes through the existing LayerRepaintBoundary.renderContent hook plus a new renderHighlight hook, and RoundedBackgroundText.toImage paints the text exactly as the widget does. A test pins that the model render is pixel-identical to the on-screen capture.
  • ExportedLayer.frames lays the images out over the layer's time range, without gaps: a highlight image while its highlight is active and the base image in between. A renderer that draws each frame as its own timed overlay, such as pro_video_editor's ImageLayer, then matches the preview. Enter and leave animations belong on the first and last frame.
  • To share the exact paint between build and toImage, RoundedBackgroundText now builds its painter in one _layout method. The outline, silhouette shadows and reserved effect space from feat(text-editor): add an outline to text layers and scale shadows with the text #875 are unchanged, and toImage draws them too.
  • Layers without highlights take the capture path they took before. They don't even add an await: an extra one left LayerRasterizer's widget tests waiting on a microtask the fake-async zone never ran.

Related Issue: Closes #none. This is needed for word-by-word captions in the Divine video editor (divinevideo/divine-mobile#9564).

Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • 🛠️ Bug fix (non-breaking change which fixes an issue)
  • ❌ Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 Code refactor
  • ✅ Build configuration change
  • 📝 Documentation
  • 🗑️ Chore

Tests

  • TextHighlight: active window (start inclusive, end exclusive), lenient fromMap, toMap round trip, copyWith, equality.
  • TextLayer: defaults, highlightIndexAt counting from the layer start with the later highlight winning, toMap/fromMap and minified round trips, invalid highlights dropped on import, and the reference diff.
  • ExportedLayer.frames: a single frame without highlights, alternating highlight and base frames, back-to-back highlights, clipping to the layer range, open-ended layers, and the base image standing in for an uncaptured highlight.
  • Widgets: the preview lights the word under the play head and nothing without a play time. captureAllLayers captures an unlit base while the screen shows a lit word, and each highlight image lights the right half of "Hello world". The model render matches the on-screen capture pixel for pixel.
  • RoundedBackgroundTextPainter repaints when only a span color changes. feat(text-editor): add an outline to text layers and scale shadows with the text #875 added that comparison; this pins it, since the highlight preview depends on it.
  • Full suite green locally (730 tests), flutter analyze clean.

Tested in the Divine app on an iPhone 12 Pro (iOS 18.5) and the iOS Simulator: the words light up one after another in the editor preview and in the exported MP4, where the frames line up exactly with the base image.

Review fixes

  • Import lost a custom highlightColor. toMap wrote the color only alongside highlights. If a history step recolored a layer before a later step added highlights, the diff left the unchanged color out and the import fell back to the default yellow. The color is now written whenever it isn't the default, the same way outlineColor is (feat(text-editor): add an outline to text layers and scale shadows with the text #875).
  • The built-in text editor dropped highlights, and also the layer's time range and animations. TextEditor.done() built a new TextLayer from its own fields only, so editing a timed text layer made it show for the whole video. It now keeps startTime/endTime, the enter/exit settings, animations and highlightColor. It keeps highlights only while the text is unchanged, since their offsets point into the old text.
  • ExportedLayer.frames produced a zero-length first frame (null → 0) for a layer without a startTime when a highlight starts at zero. Edges at or before the video start no longer create a frame.
  • Shortened the CHANGELOG entry.

Tests were added for each fix. The full suite passes locally (734 tests) and flutter analyze is clean.

…hrough them

Word-by-word ("karaoke") captions need the spoken word lit up in the preview
and burned into the export. A layer is exported as one image, so the export
also captures one image per highlight and ExportedLayer.frames lays them out
over the layer's time range for a renderer that draws timed overlays.

# Conflicts:
#	CHANGELOG.md
#	lib/core/models/layers/text_layer.dart
#	lib/features/main_editor/services/layer_copy_manager.dart
#	lib/features/text_editor/utils/rounded_background_painter.dart
#	lib/features/text_editor/widgets/rounded_background_text/rounded_background_text.dart
#	test/core/models/layers/text_layer_test.dart
@hm21 hm21 self-assigned this Sep 29, 2026
… drop an empty first frame

- TextLayer.toMap writes a custom highlightColor even without highlights, so
  a later history step that adds highlights no longer imports the default.
- The text editor keeps the layer's time range, animations and highlight
  color, and its highlights while the text is unchanged.
- ExportedLayer.frames no longer starts a layer without a startTime with an
  empty frame when a highlight starts at zero.
@hm21
hm21 marked this pull request as ready for review September 29, 2026 09:23
@hm21
hm21 merged commit bb89f6f into stable Sep 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant