Skip to content

Sprint 1C: App-only capture & circular floating control - #4

Merged
xeriomy merged 24 commits into
mainfrom
sprint-1c-app-only-capture
Nov 16, 2026
Merged

xeriomy merged 24 commits into
mainfrom
sprint-1c-app-only-capture

Conversation

@xeriomy

@xeriomy xeriomy commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Stacked on PR #3 (sprint-1b-hero-recognition, unmerged). Do not merge before #3; nothing here merges automatically.

Sprint 1C lets the user freeze/resume the MLBB capture without leaving the game: prefer Single app / app window → Mobile Legends (Android 14+ chooser; never forced), keep the last valid game frame when MLBB is hidden, and toggle LIVE/FROZEN with one small circular overlay button. No recommendation UI, panel, menu, toolbar, fullscreen overlay, accessibility service or game interaction.

What is in this PR

  • Capture mode guidance — debug screen + Vision Lab describe both modes; the stale "choose Entire screen" instruction is gone. Single-app preferred (API 34+); full-display documented honestly as a fallback that may include the shade/system UI/other apps.
  • App-only visibility lifecycle — onCapturedContentVisibilityChanged (API 34+) wired into the CAPTURING guard; pure CaptureVisibilityPolicy (JVM-tested) gates FramePipeline: hidden window ⇒ images acquired/closed, never converted/published; last valid game frame (and a pinned frame) preserved; visible again ⇒ live publication resumes unless pinned. Resume requested while hidden flips intent to LIVE, effective on visibility return. Freeze/resume never recreate the projection/VirtualDisplay and never create a second VirtualDisplay; resize keeps the gate state.
  • Circular floating button — 52dp, LIVE green / FROZEN orange, TYPE_APPLICATION_OVERLAY + FLAG_NOT_FOCUSABLE; tap toggles via the shared CaptureFreeze path; 400ms long-press drag; nearest-edge snap; normalized position in DataStore; re-anchored on rotation; attached only during capture; removed on every teardown path.
  • Permission fallback — SYSTEM_ALERT_WINDOW declared; requested only from the Enable toggle via Android's overlay-settings screen; denial ⇒ capture + in-app Freeze/Resume unaffected; screen explains how to grant it later.
  • Capture stability — Sprint 1B landscape fixes untouched (resize path, reader generations, stride/crop handling, freeze-safe resize, pinned-frame ownership). Frame aspect-ratio and Vision Lab rendering unchanged.
  • Tests — visibility policy truth table + freeze/resume/visibility sequences; freeze/stop edge cases kept; overlay position math + edge snap; enablement incl. denial; instrumented overlay attach/remove (compile-only in CI).

Verification (CI is the build of record; authoring host has no Android toolchain)

  • Code head of record (single-app first-frame fix) 3c7b58c · CI run 43936524131: unit tests 926, 0 failures / 0 errors / 0 skipped; strict lint ✅; assembleAndroidTest ✅; assembleDebug ✅. Docs-only head 9c6270f re-verified green as run 43937268586.
  • Debug APK: app-debug.apk, 10,876,851 bytes — sha256 ade9ec10c5d6b9d4e372b325e6b521f87170d0fa5273c9b30e9ea9063a763070 (artifact draftlens-debug on run 43936524131). SYSTEM_ALERT_WINDOW present in the merged manifest (utf-16-le string scan); foregroundServiceType=0x20 (mediaProjection) confirmed by the CI manifest check.
  • Instrumentation tests: compiled, not executed (no KVM on the runner).
  • Physical device: NOT RUN — maintainer-run checklist, rows 33–43 in docs/TESTING_PHYSICAL_DEVICE.md. Overlay-vs-capture isolation (single-app and full-display) is an on-device finding only; no exclusion claim is made and no capture region is masked.
  • Landscape/resize/freeze device rows: remain NOT RUN (inherited from 1B).

Known limitations

  • API < 34: no visibility signal; app-only mode undetectable — full-display semantics.
  • Full-display fallback may capture the notification shade, Quick Settings, other apps and DraftLens itself; the freeze controls still work.
  • Overlay appearance in single-app vs full-display captures is per-OEM device-dependent (rows 34/35); if it contaminates hero-slot crops on the fallback path, the remedy is single-app capture or disabling the floating control — not region masking.
  • Scope restrictions honored (spec §10): no draft-phase detection, slot detection, hero models, scoring, overlays beyond the one button, or automated game interaction.

Follow-up (this push): notification Pause, permissions-only UI, overlay independence

Root cause of the overlay "freezing" during app-only capture: the overlay lifecycle was already decoupled from capture visibility (render, touch, window lifetime have no frame input; onCapturedContentVisibilityChanged gates only the FramePipeline). Three real defects produced the observed behavior: (1) a silent no-op tap on NoFrame in fresh app-only sessions; (2) the glyph and the notification derived only from frozenFrame, so a non-pin publication block (PAUSED) still showed LIVE/green; (3) no shared derivation of "is live publication happening?". Fixes: a single CapturePublishState derivation (pin wins over pause for presentation) now collected from a combined frozen-frame + session-state flow and shared by overlay and notification; toast feedback on NoFrame; overlay tap is the unified pause/freeze toggle. No OEM masking, no mode fallback — OEM/immersive-game overlay coverage remains a hardware finding (rows 44–53).

Notification: Stop + exactly one state action — Pause while LIVE, Resume while PAUSED or FROZEN. Pause maps onto the existing CaptureStatus.PAUSED pipeline gate (projection and VirtualDisplay persist; latest valid frame preserved; nothing replaces it). Resume = resumeLive() + setPaused(false). No Freeze wording remains in the notification.

UI slim-down: removed the capture-mode tutorial card, the floating-control tutorial card, all in-game help texts, and the stale Sprint-0 marker. Kept, in one compact Permissions card: notification permission (status + grant action) and floating-overlay permission (status + switch + Android settings action). Capture controls, live preview and sampling interval unchanged.

Tests: JVM-tested CapturePublishState matrix, holder pause-preservation tests, pause-visibility sequence case, instrumented overlaySurvivesPublicationStateFlips (compile-only; plus the existing attach/remove and orphan-attach regression tests). Instrumented tests compiled, not executed.


Follow-up 2: single-app MLBB first-frame bug

Root cause (code-level): Sprint 1C registers MediaProjection.Callback before
FramePipeline.start(), createVirtualDisplay() and beginCapture(). In single-app mode
the initial onCapturedContentVisibilityChanged(false) (MLBB hidden while the user is in
DraftLens) and onCapturedContentResize(W,H) (selected app's dimensions) both arrive in
that STARTING window. The old boolean gate either got overwritten by start()'s unconditional
true or closed the gate before any frame existed; the early resize was ignored or treated
as fatal ("VirtualDisplay disappeared during resize"); frames published in STARTING were
dropped from the holder. Combined → zero published frames and "No frame captured yet".

Diagnostics added first (CaptureDiagnostics, rate-limited first-4/every-20; no pixels):
callback seq/value/pipelineInitialized/status, resize seq/readiness, per-acquisition image
dims/generation/isActive/generationCurrent/frozen/contentVisible, exact gate-reject reason,
sampling accept/reject, conversion result + bitmap dims, publication and holder delivery.
Run adb logcat -v time -s CaptureSessionService FramePipeline per the A–E checklist — it
answers whether ImageReader received images that our code rejected or the platform sent none.

Fix: explicit UNKNOWN/VISIBLE/HIDDEN policy owned by the service (UNKNOWN deliberately
open; first published frame promotes to VISIBLE so later false is authoritative); callbacks
before pipeline creation preserved, stale-session ones rejected, transitions idempotent;
pipeline no longer resets the gate; PendingResize buffers the latest valid selected-app
dimensions and sizes both VirtualDisplay and ImageReader at startup, applies later values
idempotently, and no longer kills the session when the VD is not ready; holder records
frames published during STARTING. Visibility protection, Pause/Freeze, in-app controls and
overlay behavior otherwise unchanged.

Tests: JVM policy state machine, pending-resize retention, first-frame/visibility
sequences; instrumented suite compiled only. Instrumented and device tests not executed.
The A–E first-frame checklist (full-display baseline, single-app MLBB, transitions, overlay
disabled, resize incl. landscape) remains NOT RUN — this fix is NOT claimed to work on a
device from CI alone.

@xeriomy
xeriomy changed the base branch from sprint-1b-hero-recognition to main November 16, 2026 19:20
@xeriomy
xeriomy merged commit 60dd412 into main Nov 16, 2026
1 check passed
@xeriomy
xeriomy deleted the sprint-1c-app-only-capture branch November 16, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant