Skip to content

[feat] providers: add GPT Image quality selection - #243

Merged
FayeSpica merged 4 commits into
ThinkFlowLab:mainfrom
leekaimao:feat/gpt-image-quality
Nov 16, 2026
Merged

FayeSpica merged 4 commits into
ThinkFlowLab:mainfrom
leekaimao:feat/gpt-image-quality

Conversation

@leekaimao

@leekaimao leekaimao commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • 为 GPT Image 图片生成增加 auto / low / medium / high 质量选择,UI 和 Agent 共用模型能力与后端校验。
  • 质量随生成规格保存,并固定到任务输入、执行记录和适配器参数;文生图 JSON 与参考图 multipart 请求均透传显式质量。
  • auto 保持原请求和历史指纹;切换到不支持质量参数的模型时清除设置,后端在创建任务前拒绝不支持的显式质量。

Linked issue

Refs #220。真实服务验收仍未完成,因此保留 Issue 开启。

  • Refs #220 is included

Labels

  • Labels applied: type:feat, area:provider, area:canvas

Test plan

  • Local: macOS arm64,Node 24.22.1,Python 3.12.13,验证实现提交 7390034ed70cc40b7ca44a76620514ad96b5e1ae。
    • npm test:696 项通过。
    • .venv/bin/python -m pytest tests -q:1291 项通过,6 项可选运行时集成测试跳过。
    • npm run build、npm run check:i18n、npm run check:browser-locale、npm run check:browser-manifest、.venv/bin/lint-imports 和 .venv/bin/python scripts/generate_contracts.py --check 均通过。
    • 在隔离 API 与 Vite 服务上运行 node tests/browser/image_quality_test.mjs,中英文流程通过。覆盖 High 保存与刷新恢复、语言切换、模型切换、Auto 清除、任务输入固定和实际 PNG 解码。
    • HTTP 模型替身测试覆盖 JSON、multipart、Agent 参数、旧任务指纹及恢复不重复提交。
  • Left to CI: 仓库的 Release checks,包括其完整浏览器及平台检查。本地未运行整套 Release checks;更新描述时 PR 尚无已报告的 CI checks。

上述测试计数来自单次执行,没有重复运行样本,无法计算标准差;未作性能结论。真实 OpenAI 调用未运行,见下文。

Evidence that it works

  • Media attached: 中英文截图和操作录屏,均来自隔离模拟服务。

证据于 2026-11-10 从上述实现提交录制。视频内持续标注 Mock provider, no live OpenAI calls,生成的纯色 PNG 是夹具结果。

查看或下载 MP4 操作演示 · 证据文件与环境说明

录屏依次展示 Low / Medium / High 切换、保存并刷新后保留 High、切换英文、模拟生成、切换到不支持质量的模型以及恢复 Auto。持久化规格、任务执行输入和适配器参数的断言由验证脚本完成。

中文界面,保存并刷新后仍选中“高”:

中文质量设置保存与恢复

英文界面,切换语言后仍选中 High:

English quality settings retain High

证据存放于 fork 的独立 evidence/issue-220 分支;以上链接固定到证据提交。

Provider verification

  • Real-provider calls are described honestly: none were made.

浏览器演示使用隔离 creation server 与模型替身;HTTP 测试使用请求/响应替身验证质量字段传递。PNG 文件经过真实解码,但不能证明真实模型的出图质量或真实响应行为。

仍需具备相应图片模型访问权限的真实 OpenAI API key,验证真实生成/编辑调用,以及服务返回的 quality 是否与请求匹配。该部分尚未验收。

Bilingual UI

  • Both zh-CN and en-US are updated.

新增质量选项、历史参数展示、错误提示及用户文档均包含两种语言。check:i18n 已通过;浏览器检查和录屏覆盖中英文即时切换。截图中的项目名与提示词是测试内容。

## summary

- Add Auto, Low, Medium and High image quality controls with matching
  Chinese and English copy and shared Agent parameter capabilities.
- Persist quality in generation intent and frozen execution inputs,
  forwarding explicit values in generation and reference-edit requests.
- Keep Auto requests and historical fingerprints unchanged, and clear
  unsupported selections when changing image models.
- Cover parameter persistence, history restoration, recovery, provider
  isolation and bilingual browser workflows.

## validation

- Frontend and Python test suites passed; optional runtime tests skipped.
- Production build, generated contracts, import boundaries, i18n and
  browser locale/manifest checks passed.
- Chinese and English browser workflows passed with isolated providers
  and real PNG decoding. No live OpenAI API request was made.

Refs: ThinkFlowLab#220
@hsliuustc0106

Copy link
Copy Markdown
Contributor

check the new PR template, adding the verification evidence with screenshot and video demos

@hsliuustc0106 hsliuustc0106 added type:feat New capability or behaviour area:provider Model providers and generation protocols area:canvas Canvas, nodes and direct manipulation labels Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@hsliuustc0106 Updated the PR description using the latest template as requested, with Chinese and English screenshots, an MP4 demo, and the local test results. The provider verification section explicitly states that the demo uses an isolated mock provider and that real OpenAI API verification remains outstanding. Thanks for the guidance.

@hsliuustc0106

Copy link
Copy Markdown
Contributor

fix conflicts

## summary

- Merge main while retaining GPT Image quality controls, validation,
  provider parameters and fixed execution snapshots.
- Preserve upstream Kling support and editable timeline capabilities.
- Combine bilingual errors and browser entries, then regenerate the
  shared contracts from the merged Python definitions.

## validation

- Frontend tests: 752 passed.
- Python tests: 1605 passed, 98 skipped with absolute source paths.
- Focused quality, contract, localization and Kling tests: 177 passed.
- Contracts, i18n, browser manifest and locale checks passed.
- Four import boundaries and the production build passed.
- Chinese and English quality workflows passed in dev and preview
  builds with isolated provider fixtures and actual PNG outputs.

Refs: ThinkFlowLab#220

Copy link
Copy Markdown
Contributor Author

@hsliuustc0106 Regarding your conflict comment: the conflicts were resolved and pushed in beae0b7, merging main at 1eaa873 while preserving both sets of changes. The frontend and Python test suites, build, contract checks, and bilingual image-quality browser flows passed, with conditional Python skips.

I checked the current status before replying: main has since advanced to af428f9, and GitHub is reporting conflicts again. The earlier conflict fix is pushed, but the branch now needs another sync with main.

## summary

- Merge upstream provider, Agent runtime and validation updates while
  retaining GPT Image quality behavior across UI and Agent entry points.
- Keep vLLM sampling parameters and MiniMax response formats alongside
  quality in accepted parameters and immutable execution snapshots.
- Regenerate shared contracts with both quality and upstream fields.

## validation

- Frontend tests: 774 passed.
- Full Python suite: 1692 passed, 100 skipped.
- Focused quality, contracts and affected provider tests: 162 passed.
- Build, generated contracts, i18n, browser locale and manifest passed.
- Four import boundaries passed; bundled OpenCode reports 1.19.5.
- Chinese and English quality workflows passed in dev and production
  preview using isolated providers and actual PNG decoding.

Refs: ThinkFlowLab#220
@github-actions github-actions Bot added area:agent Agent runtime, assistant chat and tools area:i18n Localisation and bilingual UI modality:image Image generation or editing provider:openai OpenAI and OpenAI-compatible services labels Nov 12, 2026

@allendyu allendyu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: code quality looks good, but not mergeable yet

I reviewed the diff and ran the suites myself on aea6331a. The implementation is sound and I have no code-level objections. Requesting changes only because the branch cannot go green in its current state.

What I verified locally

Checked out the PR head and ran, on Windows (Python 3.12, Node 24):

Check Result
pytest tests/test_image_quality_test.py tests/test_creation_parameters.py -q 60 passed
pytest tests/ -q (full suite) 1765 passed, 26 skipped
node --test tests/imageQuality_test.test.ts 8 passed
tsc --noEmit clean
python scripts/generate_contracts.py --check verified
lint-imports 4 contracts kept

The single failure, tests/test_model_reasoning_trace.py::test_reasoning_trace_handles_fragmented_sse_without_changing_response, is a pre-existing Windows locale issue (the test reads a UTF-8 fixture with the gbk default codec). It fails the same way on clean main, so it is unrelated to this PR.

Design points I checked and agree with

  • auto is excluded from the serialized parameters via exclude_if, so historical snapshot fingerprints and existing jobs are untouched. test_auto_keeps_historical_snapshot_fingerprints_and_wire_parameters_unchanged locks this down.
  • quality is only added to the provider payload when the profile is openai-gpt-image and the value is not auto, and both the JSON and multipart paths are covered.
  • Recovery and queued restarts use the quality frozen in execution, not the current spec, so changing settings mid-flight cannot alter an in-flight request.
  • Unsupported values are rejected before job creation with a dedicated image_quality_unsupported code, leaving the snapshot unchanged.
  • imageQualities stays None for non-GPT-Image models, so the UI control does not appear where it means nothing.

Blocking: merge conflicts

GitHub reports CONFLICTING. hsliuustc0106 asked for a conflict fix on 2026-11-10; beae0b7e resolved it against 1eaa8737, but main has since moved to fcac231a and the conflicts are back. I reproduced the merge and there are six conflicted files:

apps/api/src/opendirector_api/localization.py
packages/core/src/opendirector/application/creation_generation.py
packages/core/src/opendirector/domain/creation/commands.py
src/contracts/creation.schema.json
src/contracts/generated.ts
tests/browser/manifest.json

Good news: these are all mechanical neighbours, not semantic disagreements. The bigmodel_* codes and limits from #254 landed in the same spots where this PR adds image_quality_unsupported and the quality parameter, so both sides simply need to be kept. The two src/contracts/* files are generated, so python scripts/generate_contracts.py should resolve them rather than hand-editing.

Blocking: CI has not run

gh pr checks 243 currently shows only the route job passing. The heavier jobs (static, python, frontend, browser) did not report at all, which is consistent with the branch being unmergeable. So the "Left to CI" note in the description has not actually been exercised yet.

One thing for the record

The description is honest about this and I appreciate it: no real OpenAI calls were made, and the quality echo in the mock responses does not prove that the live service honors the parameter. That is correctly left to #220, which stays open.

What I need before approving

  1. Merge current main and resolve the six conflicts, keeping both #254's codes and this PR's additions. Regenerate the contract files rather than editing them.
  2. Push and let the full Release checks run on the merge commit.
  3. Reply here once checks are green.

I will approve as soon as the branch is mergeable and CI is green. Nothing in the code itself needs to change.

## summary

- Merge the current main branch while preserving published history.
- Keep GPT Image quality alongside BigModel validation, execution
  snapshots, localized errors, and browser workflow coverage.
- Regenerate shared contracts and normalize the Agent tool file ending
  for the shared static checks.
@github-actions github-actions Bot added the area:docs Documentation / 文档 label Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

@allendyu The six conflicts are resolved in 4dc84c4 by merging main at 526ece2. Both the BigModel definitions/limits and the image-quality changes are preserved, and the contracts were regenerated from the merged Python definitions. GitHub now reports the branch as mergeable.

Local frontend and Python suites passed, with conditional Python skips. The build, applicable static checks, contract checks, bilingual quality flows, and isolated browser workflows also passed. No live OpenAI calls were made.

The new head's Release checks and CodeQL are awaiting maintainer approval and have not executed, so I cannot report CI as green yet. Could a maintainer approve these workflow runs? My account does not have write access to the upstream repository. The full CI result remains pending until those runs can execute.

@FayeSpica FayeSpica left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FayeSpica
FayeSpica merged commit 0cc82f1 into ThinkFlowLab:main Nov 16, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent runtime, assistant chat and tools area:canvas Canvas, nodes and direct manipulation area:docs Documentation / 文档 area:i18n Localisation and bilingual UI area:provider Model providers and generation protocols modality:image Image generation or editing provider:openai OpenAI and OpenAI-compatible services type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants