feat: simulate-mouse-ui drags reach UI Toolkit sliders and scrollbars

.agents/skills/uloop-screenshot/references/annotated-elements.md

@@ -53,7 +53,7 @@ Limits:
 
 Clicking:
 
-- `simulate-mouse-ui --action Click --x <SimX> --y <SimY>` (or `LongPress`) reaches UI Toolkit controls, with or without an EventSystem. Its `--target-path` does not accept UI Toolkit `Path` values.
+- `simulate-mouse-ui --action Click --x <SimX> --y <SimY>` (or `LongPress`, or a drag from that point) reaches UI Toolkit controls, with or without an EventSystem. Its `--target-path` does not accept UI Toolkit `Path` values.
 - `simulate-mouse-input --action Click --x <SimX> --y <SimY>` also reaches them when the scene has an EventSystem with an Input System UI input module.
 - Without an EventSystem the `simulate-mouse-input` click does not reach UI Toolkit controls.
 

.agents/skills/uloop-simulate-mouse-ui/SKILL.md

@@ -64,7 +64,7 @@ uloop simulate-mouse-ui --action <action> --x <x> --y <y> [options]
 - Get coordinates from `AnnotatedElements` JSON (`SimX`/`SimY`) — do NOT look up GameObject positions
 - `--bypass-raycast` still uses coordinates for pointer event positions, but chooses the clicked, long-pressed, or dragged GameObject by `--target-path`
 - If `--target-path` or `--drop-target-path` matches multiple active GameObjects, the command fails instead of choosing an arbitrary duplicate
-- UI Toolkit entries (`UiSystem` `UIToolkit`) can be clicked and long-pressed at `SimX`/`SimY`; `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObjects only. Details: `references/ui-toolkit.md`.
+- UI Toolkit entries (`UiSystem` `UIToolkit`) can be clicked, long-pressed, and dragged at `SimX`/`SimY`; `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObjects only. Details: `references/ui-toolkit.md`.
 - Device Simulator play view is supported. Prefer `uloop screenshot --capture-mode rendering --annotate-elements` for coordinates; they use the simulated device resolution (`Handles.GetMainGameViewSize()` / `Screen`), not the Simulator chrome scale.
 
 ## Pause Point Inspection (Standard for E2E)
@@ -86,8 +86,8 @@ uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300
 # Long-press a button for 3 seconds
 uloop simulate-mouse-ui --action LongPress --x 400 --y 300 --duration 3.0
 
-# Slow drag for visual inspection
-uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300 --drag-speed 200
+# One-shot drag (add --drag-speed 200 to slow it for visual inspection)
+uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300
 
 # Split drag with hold (for inspection between steps)
 uloop simulate-mouse-ui --action DragStart --x 400 --y 300

.agents/skills/uloop-simulate-mouse-ui/references/output.md

@@ -5,9 +5,9 @@ Returns JSON with:
 - `Success`: Whether the operation succeeded
 - `Message`: Status message (e.g. "Hit element: ButtonStart" or "No UI element under (x, y)")
 - `Action`: Echoes which action was executed (`Click`, `Drag`, `DragStart`, `DragMove`, `DragEnd`, or `LongPress`)
-- `HitGameObjectName`: Name of the GameObject that received the pointer events (nullable string; null if nothing was hit). It is the event-handler target resolved up the hierarchy from the raycast hit (the first `IPointerDownHandler`, else the first `IPointerClickHandler`, for `Click`/`LongPress`; the first `IDragHandler` for drags), so it can be an ancestor of the topmost raycast hit — a Button's child Text reports the Button. With `--bypass-raycast` it is the handler resolved the same way from `--target-path` (also possibly an ancestor), not the raycast hit and not necessarily the raw path GameObject itself. For a UI Toolkit hit it is the control's `name`, the same as the annotation's `Name`
-- `UiSystem`: Which UI system received the press (nullable string; `Click`/`LongPress` only, null if nothing was hit): `UGUI` for a target under a Canvas, `UIToolkit` for a UI Toolkit control, and `""` for a non-UI target such as a physics object
-- `HitPath`: Path of the hit target in the form `AnnotatedElements[].Path` uses (nullable string; `Click`/`LongPress` only): a hierarchy path for GameObjects, `Document/Path#element` for UI Toolkit controls
+- `HitGameObjectName`: Name of the GameObject that received the pointer events (nullable string; null if nothing was hit). It is the event-handler target resolved up the hierarchy from the raycast hit (the first `IPointerDownHandler`, else the first `IPointerClickHandler`, for `Click`/`LongPress`; the first `IDragHandler` for drags), so it can be an ancestor of the topmost raycast hit — a Button's child Text reports the Button. With `--bypass-raycast` it is the handler resolved the same way from `--target-path` (also possibly an ancestor), not the raycast hit and not necessarily the raw path GameObject itself. For a UI Toolkit hit it is the name of the outermost control an annotation would list (the same as the annotation's `Name`); when the picked element is not inside such a control (a bare `Label` or `VisualElement`), it is that element's own `name`, or its type name when empty, and the element is not annotated. Unlike uGUI, which reports null when no handler is found, UI Toolkit reports the picked element whether or not it handles pointer events
+- `UiSystem`: Which UI system received the press (nullable string; `Click`/`LongPress` and drags, null if nothing was hit): `UGUI` for a target under a Canvas, `UIToolkit` for a UI Toolkit control, and `""` for a non-UI target such as a physics object
+- `HitPath`: Path of the hit target in the form `AnnotatedElements[].Path` uses (nullable string; `Click`/`LongPress` and drags): a hierarchy path for GameObjects, `Document/Path#element` for UI Toolkit controls. Null when nothing was hit, or when the hit element belongs to no document, such as the menu of an open dropdown
 - `PositionX`: Target X coordinate that was used
 - `PositionY`: Target Y coordinate that was used
 - `EndPositionX`: Drag end X coordinate (nullable float; populated for drag actions only)
@@ -20,4 +20,4 @@ Note: Click and LongPress on empty space (no UI element) still return `Success =
 
 ## UI Toolkit Elements
 
-`Click` and `LongPress` reach UI Toolkit controls at their annotated `SimX`/`SimY`. How presses are delivered and what is not supported: `ui-toolkit.md`.
+`Click`, `LongPress`, and the drag actions reach UI Toolkit controls at their annotated `SimX`/`SimY` (`--from-x`/`--from-y` for `Drag`). How presses and drags are delivered and what is not supported: `ui-toolkit.md`.

.agents/skills/uloop-simulate-mouse-ui/references/ui-toolkit.md

@@ -1,9 +1,9 @@
 # simulate-mouse-ui and UI Toolkit
 
-`Click` and `LongPress` reach UI Toolkit controls on screen-space panels — the
-`AnnotatedElements` entries with `UiSystem` `UIToolkit`. Pass the entry's `SimX`/`SimY` as
-`--x`/`--y`; the response reports `UiSystem` `UIToolkit`, the control's `Name` as
-`HitGameObjectName`, and its annotation `Path` as `HitPath`.
+`Click`, `LongPress`, and the drag actions reach UI Toolkit controls on screen-space panels —
+the `AnnotatedElements` entries with `UiSystem` `UIToolkit`. Pass the entry's `SimX`/`SimY` as
+`--x`/`--y` (`--from-x`/`--from-y` for `Drag`); the response reports `UiSystem` `UIToolkit`, the
+control's `Name` as `HitGameObjectName`, and its annotation `Path` as `HitPath`.
 
 ## How a press is delivered
 
@@ -29,18 +29,33 @@ A UI Toolkit `Button` fires `clicked` once when the long press is released (UI T
 `Clickable` behaviour); a uGUI long press sends no click. A `RepeatButton` fires repeatedly
 while held.
 
+## Drags
+
+- A drag whose start point resolves to a UI Toolkit control (by the rule above) presses it
+  with the left button, moves the pointer, and releases at the end, so a `Slider` or a
+  `Scroller` follows the drag. The start point is what counts: the drag stays with that panel
+  wherever it moves.
+- A labelled slider's bounds include its label; start the drag on the track.
+- `--drop-target-path` applies to uGUI drags only. A `Drag` with it fails before anything is
+  sent; a `DragEnd` with it fails and leaves the drag in progress, so call `DragEnd` again
+  without it.
+- If the control's document is disabled or destroyed during a split drag, the next `DragMove`
+  or `DragEnd` fails and the drag ends. A one-shot `Drag` stops sending and still succeeds.
+- If a UI Toolkit handler throws, the button is released and the exception ends the command.
+  A `DragMove` that throws leaves the drag in progress, as a cancelled `DragMove` does.
+
 ## Not supported
 
 - `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObject paths only;
   a UI Toolkit `Path` (it contains `#`) is reported as not found.
-- Drag actions go through the uGUI EventSystem only.
 - World-space panels (Unity 6), as in the annotation.
 - Two panels with the same `sortingOrder` overlapping at the point: Unity orders them by
   creation order, which is not exposed, so either one may be chosen.
 
 ## Notes
 
-- Move the real mouse cursor out of the Game View while simulating. Real pointer input over
-  the Game View reaches the same panels and can interleave with the simulated press.
+- Move the real mouse cursor out of the Game View while simulating, and keep it away between
+  `DragStart` and `DragEnd`. Real pointer input over the Game View reaches the same panels and
+  can interleave with the simulated press or move a held slider.
 - Device Simulator has the same limits as the annotation: coordinates use the simulated device
   resolution.

.claude/skills/uloop-screenshot/references/annotated-elements.md

@@ -53,7 +53,7 @@ Limits:
 
 Clicking:
 
-- `simulate-mouse-ui --action Click --x <SimX> --y <SimY>` (or `LongPress`) reaches UI Toolkit controls, with or without an EventSystem. Its `--target-path` does not accept UI Toolkit `Path` values.
+- `simulate-mouse-ui --action Click --x <SimX> --y <SimY>` (or `LongPress`, or a drag from that point) reaches UI Toolkit controls, with or without an EventSystem. Its `--target-path` does not accept UI Toolkit `Path` values.
 - `simulate-mouse-input --action Click --x <SimX> --y <SimY>` also reaches them when the scene has an EventSystem with an Input System UI input module.
 - Without an EventSystem the `simulate-mouse-input` click does not reach UI Toolkit controls.
 

.claude/skills/uloop-simulate-mouse-ui/SKILL.md

@@ -64,7 +64,7 @@ uloop simulate-mouse-ui --action <action> --x <x> --y <y> [options]
 - Get coordinates from `AnnotatedElements` JSON (`SimX`/`SimY`) — do NOT look up GameObject positions
 - `--bypass-raycast` still uses coordinates for pointer event positions, but chooses the clicked, long-pressed, or dragged GameObject by `--target-path`
 - If `--target-path` or `--drop-target-path` matches multiple active GameObjects, the command fails instead of choosing an arbitrary duplicate
-- UI Toolkit entries (`UiSystem` `UIToolkit`) can be clicked and long-pressed at `SimX`/`SimY`; `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObjects only. Details: `references/ui-toolkit.md`.
+- UI Toolkit entries (`UiSystem` `UIToolkit`) can be clicked, long-pressed, and dragged at `SimX`/`SimY`; `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObjects only. Details: `references/ui-toolkit.md`.
 - Device Simulator play view is supported. Prefer `uloop screenshot --capture-mode rendering --annotate-elements` for coordinates; they use the simulated device resolution (`Handles.GetMainGameViewSize()` / `Screen`), not the Simulator chrome scale.
 
 ## Pause Point Inspection (Standard for E2E)
@@ -86,8 +86,8 @@ uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300
 # Long-press a button for 3 seconds
 uloop simulate-mouse-ui --action LongPress --x 400 --y 300 --duration 3.0
 
-# Slow drag for visual inspection
-uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300 --drag-speed 200
+# One-shot drag (add --drag-speed 200 to slow it for visual inspection)
+uloop simulate-mouse-ui --action Drag --from-x 400 --from-y 300 --x 600 --y 300
 
 # Split drag with hold (for inspection between steps)
 uloop simulate-mouse-ui --action DragStart --x 400 --y 300

.claude/skills/uloop-simulate-mouse-ui/references/output.md

@@ -5,9 +5,9 @@ Returns JSON with:
 - `Success`: Whether the operation succeeded
 - `Message`: Status message (e.g. "Hit element: ButtonStart" or "No UI element under (x, y)")
 - `Action`: Echoes which action was executed (`Click`, `Drag`, `DragStart`, `DragMove`, `DragEnd`, or `LongPress`)
-- `HitGameObjectName`: Name of the GameObject that received the pointer events (nullable string; null if nothing was hit). It is the event-handler target resolved up the hierarchy from the raycast hit (the first `IPointerDownHandler`, else the first `IPointerClickHandler`, for `Click`/`LongPress`; the first `IDragHandler` for drags), so it can be an ancestor of the topmost raycast hit — a Button's child Text reports the Button. With `--bypass-raycast` it is the handler resolved the same way from `--target-path` (also possibly an ancestor), not the raycast hit and not necessarily the raw path GameObject itself. For a UI Toolkit hit it is the control's `name`, the same as the annotation's `Name`
-- `UiSystem`: Which UI system received the press (nullable string; `Click`/`LongPress` only, null if nothing was hit): `UGUI` for a target under a Canvas, `UIToolkit` for a UI Toolkit control, and `""` for a non-UI target such as a physics object
-- `HitPath`: Path of the hit target in the form `AnnotatedElements[].Path` uses (nullable string; `Click`/`LongPress` only): a hierarchy path for GameObjects, `Document/Path#element` for UI Toolkit controls
+- `HitGameObjectName`: Name of the GameObject that received the pointer events (nullable string; null if nothing was hit). It is the event-handler target resolved up the hierarchy from the raycast hit (the first `IPointerDownHandler`, else the first `IPointerClickHandler`, for `Click`/`LongPress`; the first `IDragHandler` for drags), so it can be an ancestor of the topmost raycast hit — a Button's child Text reports the Button. With `--bypass-raycast` it is the handler resolved the same way from `--target-path` (also possibly an ancestor), not the raycast hit and not necessarily the raw path GameObject itself. For a UI Toolkit hit it is the name of the outermost control an annotation would list (the same as the annotation's `Name`); when the picked element is not inside such a control (a bare `Label` or `VisualElement`), it is that element's own `name`, or its type name when empty, and the element is not annotated. Unlike uGUI, which reports null when no handler is found, UI Toolkit reports the picked element whether or not it handles pointer events
+- `UiSystem`: Which UI system received the press (nullable string; `Click`/`LongPress` and drags, null if nothing was hit): `UGUI` for a target under a Canvas, `UIToolkit` for a UI Toolkit control, and `""` for a non-UI target such as a physics object
+- `HitPath`: Path of the hit target in the form `AnnotatedElements[].Path` uses (nullable string; `Click`/`LongPress` and drags): a hierarchy path for GameObjects, `Document/Path#element` for UI Toolkit controls. Null when nothing was hit, or when the hit element belongs to no document, such as the menu of an open dropdown
 - `PositionX`: Target X coordinate that was used
 - `PositionY`: Target Y coordinate that was used
 - `EndPositionX`: Drag end X coordinate (nullable float; populated for drag actions only)
@@ -20,4 +20,4 @@ Note: Click and LongPress on empty space (no UI element) still return `Success =
 
 ## UI Toolkit Elements
 
-`Click` and `LongPress` reach UI Toolkit controls at their annotated `SimX`/`SimY`. How presses are delivered and what is not supported: `ui-toolkit.md`.
+`Click`, `LongPress`, and the drag actions reach UI Toolkit controls at their annotated `SimX`/`SimY` (`--from-x`/`--from-y` for `Drag`). How presses and drags are delivered and what is not supported: `ui-toolkit.md`.

.claude/skills/uloop-simulate-mouse-ui/references/ui-toolkit.md

@@ -1,9 +1,9 @@
 # simulate-mouse-ui and UI Toolkit
 
-`Click` and `LongPress` reach UI Toolkit controls on screen-space panels — the
-`AnnotatedElements` entries with `UiSystem` `UIToolkit`. Pass the entry's `SimX`/`SimY` as
-`--x`/`--y`; the response reports `UiSystem` `UIToolkit`, the control's `Name` as
-`HitGameObjectName`, and its annotation `Path` as `HitPath`.
+`Click`, `LongPress`, and the drag actions reach UI Toolkit controls on screen-space panels —
+the `AnnotatedElements` entries with `UiSystem` `UIToolkit`. Pass the entry's `SimX`/`SimY` as
+`--x`/`--y` (`--from-x`/`--from-y` for `Drag`); the response reports `UiSystem` `UIToolkit`, the
+control's `Name` as `HitGameObjectName`, and its annotation `Path` as `HitPath`.
 
 ## How a press is delivered
 
@@ -29,18 +29,33 @@ A UI Toolkit `Button` fires `clicked` once when the long press is released (UI T
 `Clickable` behaviour); a uGUI long press sends no click. A `RepeatButton` fires repeatedly
 while held.
 
+## Drags
+
+- A drag whose start point resolves to a UI Toolkit control (by the rule above) presses it
+  with the left button, moves the pointer, and releases at the end, so a `Slider` or a
+  `Scroller` follows the drag. The start point is what counts: the drag stays with that panel
+  wherever it moves.
+- A labelled slider's bounds include its label; start the drag on the track.
+- `--drop-target-path` applies to uGUI drags only. A `Drag` with it fails before anything is
+  sent; a `DragEnd` with it fails and leaves the drag in progress, so call `DragEnd` again
+  without it.
+- If the control's document is disabled or destroyed during a split drag, the next `DragMove`
+  or `DragEnd` fails and the drag ends. A one-shot `Drag` stops sending and still succeeds.
+- If a UI Toolkit handler throws, the button is released and the exception ends the command.
+  A `DragMove` that throws leaves the drag in progress, as a cancelled `DragMove` does.
+
 ## Not supported
 
 - `--bypass-raycast`, `--target-path`, and `--drop-target-path` take GameObject paths only;
   a UI Toolkit `Path` (it contains `#`) is reported as not found.
-- Drag actions go through the uGUI EventSystem only.
 - World-space panels (Unity 6), as in the annotation.
 - Two panels with the same `sortingOrder` overlapping at the point: Unity orders them by
   creation order, which is not exposed, so either one may be chosen.
 
 ## Notes
 
-- Move the real mouse cursor out of the Game View while simulating. Real pointer input over
-  the Game View reaches the same panels and can interleave with the simulated press.
+- Move the real mouse cursor out of the Game View while simulating, and keep it away between
+  `DragStart` and `DragEnd`. Real pointer input over the Game View reaches the same panels and
+  can interleave with the simulated press or move a held slider.
 - Device Simulator has the same limits as the annotation: coordinates use the simulated device
   resolution.

Assets/Tests/Demo/UIToolkitAnnotationDemo.cs

@@ -21,6 +21,13 @@ public int GetClickCount(string elementName)
             return _clickCounts.TryGetValue(elementName, out int count) ? count : 0;
         }
 
+        // NaN when the back panel has no slider by that name, so a missing control cannot pass as a value.
+        public float GetSliderValue(string elementName)
+        {
+            Slider? slider = backDocument.rootVisualElement?.Q<Slider>(elementName);
+            return slider != null ? slider.value : float.NaN;
+        }
+
         // Start runs after every Awake and OnEnable, so both documents have created their roots by now
         // regardless of component order.
         private void Start()
@@ -33,7 +40,8 @@ private void BuildBackPanel(VisualElement root)
         {
             AddPlaced(root, CreateCountingButton("play"), 100f, 100f, 200f, 50f);
             AddPlaced(root, new Toggle("Sound") { name = "sound" }, 100f, 180f, 200f, 30f);
-            AddPlaced(root, new Slider("Volume", 0f, 1f) { name = "volume" }, 100f, 240f, 300f, 30f);
+            // No label, so the slider's bounds are its track and the E2E can drag between its edges.
+            AddPlaced(root, new Slider(0f, 1f) { name = "volume" }, 100f, 240f, 300f, 30f);
             AddPlaced(root, new TextField("Player") { name = "player-name" }, 100f, 300f, 300f, 40f);
             DropdownField quality = new DropdownField(
                 "Quality", new List<string> { "Low", "Medium", "High" }, 1) { name = "quality" };