Skip to content

修复订阅词库的列解析和词频丢失 - #24

Closed
takeshi264 wants to merge 1 commit into
zoolapp:mainfrom
takeshi264:fix/vocabulary-parser
Closed

takeshi264 wants to merge 1 commit into
zoolapp:mainfrom
takeshi264:fix/vocabulary-parser

Conversation

@takeshi264

Copy link
Copy Markdown

看订阅词库的解析时,发现原来固定按“词条、编码、词频”读取,没有处理 columns 声明。遇到列顺序不同的词库,就会读错编码或丢掉词频。

这次修了几处:

  • 按 columns 读取各列,空列也保留位置。
  • 兼容文件开头的 BOM。
  • 同一个词、同一种读音重复出现时,保留较高词频;不同读音仍分别保留。
  • 补上没有列声明的两列词频表,例如“词条 + 100”,不再把词频丢掉。
  • 已有订阅在下次部署时重新生成词表,不用重新下载。

补了对应的回归测试,本地测试通过。

Co-Authored-By: Codex <noreply@openai.com>
@foru17

foru17 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

感谢 @takeshi264!已按补丁合入内部 main(公开仓库是从内部仓库导出的镜像,没法直接在这里点 merge),提交作者保留为你,会随下个版本发布,CHANGELOG 里也记了你的名字。

合入前做了这些验证:

  • 你补的 VocabularyParserTests 和已有的订阅、双拼测试全部通过,内部全量测试通过。
  • columns 声明顺序不同、空列、BOM、同词同音重复取较高词频、「词条 + 词频」两列表这几种情况都对过;已有订阅下次部署时会重新生成词表,不用重新下载。

因为公开仓库由私有仓库导出,这个 PR 无法直接合并,我先关闭它。

@foru17 foru17 closed this Oct 19, 2026
foru17 added a commit that referenced this pull request Oct 20, 2026
…ary columns

Imported Squirrel skins apply again (empty app_options and missing includes no longer drop the appearance patch), negative borders no longer clip the first candidate, Squirrel import follows the configured sync directory and really merges learned frequencies, subscribed vocabularies honor declared columns and work with Ziranma, and upgrades stop the stale input method process. Refs #22, refs #23, refs #24.

Co-authored-by: takeshi <11970938+takeshi264@users.noreply.github.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants