feat(layers): highlight parts of a text layer while the video plays through them - #876
Merged
Merged
Conversation
…hrough them
Word-by-word ("karaoke") captions need the spoken word lit up in the preview
and burned into the export. A layer is exported as one image, so the export
also captures one image per highlight and ExportedLayer.frames lays them out
over the layer's time range for a renderer that draws timed overlays.
# Conflicts:
# CHANGELOG.md
# lib/core/models/layers/text_layer.dart
# lib/features/main_editor/services/layer_copy_manager.dart
# lib/features/text_editor/utils/rounded_background_painter.dart
# lib/features/text_editor/widgets/rounded_background_text/rounded_background_text.dart
# test/core/models/layers/text_layer_test.dart
… drop an empty first frame - TextLayer.toMap writes a custom highlightColor even without highlights, so a later history step that adds highlights no longer imports the default. - The text editor keeps the layer's time range, animations and highlight color, and its highlights while the text is unchanged. - ExportedLayer.frames no longer starts a layer without a startTime with an empty frame when a highlight starts at zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Word-by-word ("karaoke") captions light up each word while it is spoken. A video editor built on this package can already time a text layer, but a text layer has one look for its whole time range, and a layer is exported as a single image. Neither the preview nor the export could change the color of one word at a time.
This adds timed highlights to text layers: a part of the text that is drawn in a highlight color while the video plays through it.
Model
TextHighlightis a UTF-16 range ofTextLayer.textplus astartTime/endTime. It is@immutable, withtoMap/fromMap,copyWithand value equality.fromMapis lenient likeLayerAnimation.fromMap;TextLayer.fromMapdrops highlights that can never show.TextLayer.highlightsandTextLayer.highlightColor(default: none, a warm yellow). They are serialized only when there are highlights, with minified keyshl/hc, and diffed intoMapFromReference.startTime, not from the start of the video, so moving a layer along the timeline keeps its highlights on their words. When highlights overlap, the last one in the list wins (TextLayer.highlightIndexAt).LayerCopyManager.createCopyTextLayercopies them, so they survive edit, duplicate, group and crop/rotate like every other field.Preview
LayerWidgethands its play-time notifier to the text item, which draws the active highlight. It rebuilds only when the active highlight changes, not on every frame. Only the color of a span changes, so the layout and the layer's size stay the same whichever word is lit. Without a play time (image editing) no highlight is drawn.Export
ExportedLayer.highlightBytes. All of them have the same size, so each can stand in for the base at the same position. This goes through the existingLayerRepaintBoundary.renderContenthook plus a newrenderHighlighthook, andRoundedBackgroundText.toImagepaints the text exactly as the widget does. A test pins that the model render is pixel-identical to the on-screen capture.ExportedLayer.frameslays the images out over the layer's time range, without gaps: a highlight image while its highlight is active and the base image in between. A renderer that draws each frame as its own timed overlay, such aspro_video_editor'sImageLayer, then matches the preview. Enter and leave animations belong on the first and last frame.buildandtoImage,RoundedBackgroundTextnow builds its painter in one_layoutmethod. The outline, silhouette shadows and reserved effect space from feat(text-editor): add an outline to text layers and scale shadows with the text #875 are unchanged, andtoImagedraws them too.await: an extra one leftLayerRasterizer's widget tests waiting on a microtask the fake-async zone never ran.Related Issue: Closes #none. This is needed for word-by-word captions in the Divine video editor (divinevideo/divine-mobile#9564).
Type of Change
Tests
TextHighlight: active window (start inclusive, end exclusive), lenientfromMap,toMapround trip,copyWith, equality.TextLayer: defaults,highlightIndexAtcounting from the layer start with the later highlight winning,toMap/fromMapand minified round trips, invalid highlights dropped on import, and the reference diff.ExportedLayer.frames: a single frame without highlights, alternating highlight and base frames, back-to-back highlights, clipping to the layer range, open-ended layers, and the base image standing in for an uncaptured highlight.captureAllLayerscaptures an unlit base while the screen shows a lit word, and each highlight image lights the right half of "Hello world". The model render matches the on-screen capture pixel for pixel.RoundedBackgroundTextPainterrepaints when only a span color changes. feat(text-editor): add an outline to text layers and scale shadows with the text #875 added that comparison; this pins it, since the highlight preview depends on it.flutter analyzeclean.Tested in the Divine app on an iPhone 12 Pro (iOS 18.5) and the iOS Simulator: the words light up one after another in the editor preview and in the exported MP4, where the frames line up exactly with the base image.
Review fixes
highlightColor.toMapwrote the color only alongside highlights. If a history step recolored a layer before a later step added highlights, the diff left the unchanged color out and the import fell back to the default yellow. The color is now written whenever it isn't the default, the same wayoutlineColoris (feat(text-editor): add an outline to text layers and scale shadows with the text #875).TextEditor.done()built a newTextLayerfrom its own fields only, so editing a timed text layer made it show for the whole video. It now keepsstartTime/endTime, the enter/exit settings,animationsandhighlightColor. It keepshighlightsonly while the text is unchanged, since their offsets point into the old text.ExportedLayer.framesproduced a zero-length first frame (null→0) for a layer without astartTimewhen a highlight starts at zero. Edges at or before the video start no longer create a frame.Tests were added for each fix. The full suite passes locally (734 tests) and
flutter analyzeis clean.