Repository navigation
fix(translate): strip labels per line, re-ask only around a lone label - #14
Closed
huangzhenyuan-bytetrade wants to merge 1 commit into
Closed
huangzhenyuan-bytetrade wants to merge 1 commit into
huangzhenyuan-bytetrade wants to merge 1 commit into
Conversation
The transcript route shows the model each turn as "[speaker] text", and
Hy-MT2-1.8B mirrors the labels back however plainly rule 2 tells it not to.
The cleanup guessed: it stripped only when every line of a reply opened with
a bracket and none was left empty. A real meeting broke both conditions in
one group. Speaker separation cut "…as a leader, engineer, and" / "Innovator."
into two turns, Chinese puts that tail mid-sentence, so the model folded it
into the previous line and answered its own turn with the label alone. All
sixteen turns of that group reached the caller still prefixed, the folded
one as a bare "[SPEAKER_01]". The same group failed this way on three runs
of the same recording.
Labels are now removed line by line, and only the labels the call actually
showed (its turns and its lead-in), so a bracket that is part of what was
said stays. A line left with nothing but a label is a hole. The rest of that
reply is kept. The hole is asked for again together with the turn on either
side of it, since its text sits in a neighbour; that answer replaces the
window only if it has no hole of its own. Otherwise the hole is asked for on
its own, where there is nothing to fold it into. It comes back empty only
when that call fails too: unaligned, or upstream/deadline when the engine or
the budget is why. An engine failing the window call is not asked again turn
by turn. A reply that is all holes still gets the retry and the halving.
Each reply that lines up gets one round of this and those calls owe none, so
the number of calls stays bounded.
Measured against Hy-MT2-1.8B on the groups of that meeting with the hole
forced. Re-asking the whole group instead of the window never produced
"Innovator." in 30 runs, and 17 of its apparent successes put the next
turn's text in its place, which nothing downstream can detect. The window
answered it 6 of 30 and misplaced it 3 of 30; with the turn asked on its own
after a window that still folds it, 20 runs gave "创新者。" 19 times and a
misplaced line once, and none came back empty. The turn on its own repeats
a word the previous turn already carries ("…工程师和创新者…" then "创新者。"),
which is taken over an empty line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
先关闭:领导认为根因在上层(Note 的切轮和说话人归属),这个修复会把上层问题遮住。等 Note 侧排查出结论后再决定是否需要。分支 fix/translate-strip-sent-labels 保留。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
/translate/transcript一次翻译会议转写的一组轮次(Note 每组最多 32 轮)。发给模型时,每一轮编一个号,前面拼上说话人,例如:拼说话人,是为了让模型知道谁在说话,代词才不会翻错。提示词里要求模型不要把标签写进译文,但 Hy-MT2-1.8B 经常照抄回来,所以拿到回复后要把标签剥掉,函数是
stripEchoedLabels。出了什么问题
原来的
stripEchoedLabels是一票否决:一组回复里,只要有一行不以[开头,或者有一行剥完是空的,就认定模型没回吐标签,整组原样交回。一场真实会议撞上了第二个条件。说话人分离把 "Innovator." 这个词单独切成了一轮(上面的第 3 行)。中文要把它放到句子中间,模型就把它并进了第 2 行,第 3 行只回了一个标签:
第 3 行剥完是空的,整组 16 轮就带着
[SPEAKER_xx]前缀交给了 Note,界面上直接显示出来。同一段录音跑了三次,三次都在这一组出问题。改了什么
1. 剥标签:一行一行剥,只剥这次发出去的标签
先把这次调用发给模型的说话人名字收集成一张名单(本组各轮加上文里的),每一行开头的
[名字]在名单里就剥掉。剥不剥只看这一行自己,跟其它行无关。名单外的方括号不动,比如原话里的[笑]。2. 某一行只剩标签:只补这一行,其它行照收
只剩标签说明这一轮的内容被模型挪进了邻行,这一行本身没有译文。处理分三步,每一步只在上一步不成功时才做:
error里标出原因:unaligned、upstream或deadline。其它行都用第一次的译文。唯一的例外是第 1 步重翻成功时,一起重翻的那几行邻行会换成新译文。
两种特殊情况:
upstream,不再逐行单独去翻,免得引擎故障时请求翻倍。调用次数有上限:每个对得上的回复最多走一轮上面的补救,补救时发出的调用不会再触发补救。
对调用方的影响
[SPEAKER_xx]前缀[SPEAKER_01],没有译文副作用:最坏情况是内容重复
补救之后最坏的结果不是丢内容,而是同一段内容出现两次。有两种:
和改之前比:这一行原来只有一个标签,现在可能显示一句放错位置的译文。两种都是错的,但现在这种看得出来是重复。错位目前在任何地方都拦不住,下面「不在本 PR」那一节准备另开的内容校验(相邻两行的译文几乎一样、原文却不一样,就当作对不上),能把它拦下来。
为什么这样选
在 yaotest005 的 Hy-MT2-1.8B 上,用这场会议里出问题的那一组,每次都强制第一次回复的第 3 行只剩标签,比较几种补救方式。「错位」指这一行显示的其实是下一轮的译文:行数照样对得上,下游发现不了,比空着更糟。
整组重翻的「成功」大多是错位,所以没有用它。
端到端:上一版(只有第 1 步)装到 yaotest005,同一段录音完整重跑了一遍,结果是带前缀的译文 0 条、留空 1 轮(就是 "Innovator.")。加上第 2 步之后,只按上表做了实测,没有再完整重跑一遍。
不在本 PR
模型在行数对得上的情况下,把下一轮的译文放进这一轮。端到端重跑里有 5 处这种错位,都不在补救过的组里,是原来就有的问题,这次没有处理。要拦它,需要检查译文内容是否对得上,另开 PR。
验证
go test -race ./internal/translate/、go vet ./...、golangci-lint run ./...都通过。🤖 Generated with Claude Code