Skip to content

refactor: UI Toolkit screen elements move into a shared module behind one entry point - #3311

Merged
hatayama merged 1 commit into
feature/screenshot-ui-toolkit-annotationfrom
refactor/uitk-screen-elements-module
Oct 13, 2026
Merged

hatayama merged 1 commit into
feature/screenshot-ui-toolkit-annotationfrom
refactor/uitk-screen-elements-module

Conversation

@hatayama

@hatayama hatayama commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Moves the UI Toolkit screen-element code that screenshot annotations use into a shared module, so simulate-mouse-ui can find and press the same controls next. No behaviour change.

User Impact

  • None. screenshot --annotate-elements reports the same UI Toolkit entries with the same coordinates.

Changes

  • New assembly UnityCLILoop.FirstPartyTools.Common.UIToolkit.Editor (Editor only, not auto-referenced, references Common.MouseUi).
    • Public entry point UIToolkitScreenElements.CollectControls(probePointsForTopLeftBounds) and result type UIToolkitScreenControl. Both work in top-left Game View pixels.
    • Internal parts: panel gathering and picking, element paths, the control walk, plus the classifier, panel kind, screen mapping and reachability probe moved with git mv. Internals are visible to the test assemblies only.
  • Common.MouseUi exposes its internals to the new module, which uses GameObjectPathUtility.
  • Screenshot keeps UIToolkitElementCollector.Collect(elements, gameViewHeight) as an adapter. It supplies the uGUI probe points and converts the results back to bottom-left entries, using the same conversions in the same order as before.
  • Screenshot and the EditMode / PlayMode test assemblies reference the new module.

Verification

  • uloop compile: 0 errors. The one warning is an existing one in a test fixture.
  • EditMode, one class at a time: UIToolkitElementClassifierTests 13/13, UIToolkitPanelKindTests 4/4, UIToolkitReachabilityProbeTests 6/6, UIToolkitScreenMappingTests 9/9, UIElementAnnotatorCollectTests 11/11, OnionAssemblyDependencyTests 83/83.
  • PlayMode UIToolkitElementCollectorTests 18/18.
  • check-asmdef-policy: no violations.
  • scripts/test-screenshot-uitoolkit-annotation.sh: PASS, with the coordinate checks against the elements-only capture (±1 px) passing. git status was unchanged after the run.

View guided diff

simulate-mouse-ui is about to find and press UI Toolkit controls the
same way screenshot annotations list them, and one tool may not
reference another. The panel gathering, picking, element paths and
control walk therefore move out of the Screenshot assembly into
Common.UIToolkit, behind a single public entry point that works in
top-left Game View pixels. Its parts stay internal and are visible
only to the tests.

Screenshot keeps UIToolkitElementCollector.Collect with the same
signature as an adapter: it supplies the uGUI probe points and turns
the results back into bottom-left annotation entries with the same
conversions in the same order, so annotations are unchanged.
@coderabbitai

coderabbitai Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository: hatayama/unity-cli-loop/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cb22aa11-1f36-427a-84c8-9ef3c3e32cc9
📥 Commits

Reviewing files that changed from the base of the PR and between 33457e4 and f7a88b6.

⛔ Files ignored due to path filters (16)
  • Assets/Tests/Editor/UnityCLILoop.Tests.Editor.asmdef is excluded by none and included by none
  • Assets/Tests/PlayMode/UnityCLILoop.Tests.PlayMode.asmdef is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/AssemblyInfo.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitControlCollector.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitElementClassifier.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitElementPath.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitPanelKind.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitReachabilityProbe.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenControl.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenElements.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenMapping.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenPanels.cs.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UnityCLILoop.FirstPartyTools.Common.UIToolkit.Editor.asmdef is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UnityCLILoop.FirstPartyTools.Common.UIToolkit.Editor.asmdef.meta is excluded by none and included by none
  • Packages/src/Editor/FirstPartyTools/Screenshot/UnityCLILoop.FirstPartyTools.Screenshot.Editor.asmdef is excluded by none and included by none
📒 Files selected for processing (12)
  • Packages/src/Editor/FirstPartyTools/Common/MouseUi/AssemblyInfo.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/AssemblyInfo.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitControlCollector.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitElementClassifier.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitElementPath.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitPanelKind.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitReachabilityProbe.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenControl.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenElements.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenMapping.cs
  • Packages/src/Editor/FirstPartyTools/Common/UIToolkit/UIToolkitScreenPanels.cs
  • Packages/src/Editor/FirstPartyTools/Screenshot/UIToolkitAnnotation/UIToolkitElementCollector.cs
 ______________________________________________________________________________
< Dr. CodeRabbit, or: How I Learned to Stop Worrying and Love the Code Review. >
 ------------------------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hatayama
hatayama merged commit 63ce71f into feature/screenshot-ui-toolkit-annotation Oct 13, 2026
10 of 11 checks passed
@hatayama
hatayama deleted the refactor/uitk-screen-elements-module branch October 13, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant