diff --git a/README.md b/README.md index 4e8dc3d..cf78132 100644 --- a/README.md +++ b/README.md @@ -30,16 +30,26 @@ and [docs/VISION_TESTING.md](docs/VISION_TESTING.md). - Hero recognition v0 (Sprint 1B, PR #3): hero catalog sync from public static data, explicit Identify action for the selected slot, ranked candidates with similarity scores, manual confirm/correct — **accuracy on real game screens is not measured yet** -- Freeze the game frame from the notification (Sprint 1B): tap **Freeze current frame** on the - capture notification while you are in MLBB — no need to leave the game; Vision Lab shows the - pinned game frame, not the DraftLens screen; **Resume live capture** from the notification or - in the Vision Lab +- Capture notification with **Stop** and **Pause** (Sprint 1C): pausing keeps the latest + valid frame, blocks new frames replacing it, keeps MediaProjection running, and the same + Resume action recovers from pause or from a pinned frame; the notification reflects + LIVE / PAUSED / FROZEN and never shows both actions +- App-only capture guidance (Sprint 1C): prefer **Single app / app window → Mobile Legends** + (Android 14+ chooser); the captured window's visibility drives publication — + MLBB hidden ⇒ the last valid game frame is kept, never DraftLens/shade/system UI; + full-display capture remains the documented fallback (it may include the shade/system UI) +- Circular floating freeze button (Sprint 1C): one 52dp circular `TYPE_APPLICATION_OVERLAY` + button shown only while capture runs — tap toggles LIVE (green) / orange (paused or frozen), + long-press-and-drag moves it, it snaps to a screen edge, position persists normalized in + DataStore; notifications/in-app Freeze/Resume keep working without it; the overlay + permission is requested only when you enable the button ## What is deliberately NOT here (later milestones) -ML hero recognition, recommendation engine, counter/synergy/role scoring, floating overlays, -API sync, player profiling, builds. The Vision Lab says **"Detection: not implemented"** — -manual slots are calibration data, never detections. See `tasks/plan.md` and -`docs/ARCHITECTURE.md`. +Recommendation engine, counter/synergy/role scoring — and no recommendation overlays +(Sprint 1C ships exactly one capture-freeze overlay button, and nothing else). +ML hero recognition is Sprint 1B (PR #3). API sync, player profiling, builds. +The Vision Lab says **"Detection: not implemented"** — manual slots are calibration data, +never detections. See `tasks/plan.md` and `docs/ARCHITECTURE.md`. ## Build Requires JDK 17 and Android SDK platform 35, build-tools 35. @@ -58,11 +68,13 @@ build of record.** See `docs/VISION_TESTING.md` for exactly what ran. validates compilation, not MediaProjection reliability). 2. Open the app; the **Debug Capture** tab is first. 3. Tap **Start Capture** → grant the system screen-capture consent dialog. When Android asks - what to capture, choose **Entire screen / full device display** — app-window capture shows - this app, not Mobile Legends, and the notification freeze workflow would be pointless. + what to capture, prefer **Single app / app window → Mobile Legends: Bang Bang** (Android 14+); + if that option is missing, choose **Entire screen / full device display** (it can capture the + shade and system UI). 4. Open Mobile Legends and wait for the draft screen you want to annotate. -5. Pull down the notification shade and tap **Freeze current frame** on the DraftLens capture - notification (notifications must be enabled — the app checks and tells you). +5. With MLBB on screen, tap the circular **Floating freeze button** (enable it on the capture + screen first) to freeze the frame — or, if you never granted the overlay permission, pull + down the notification shade and tap **Pause** (notifications must be enabled). 6. Switch back to **DraftLens → Vision Lab**. Capture kept running; the Vision Lab shows the pinned MLBB frame instead of the DraftLens screen. 7. Draw a slot rectangle, then run **Identify hero** on the frozen frame. Candidates are diff --git a/app/src/androidTest/kotlin/com/draftlens/FloatingControlOverlayInstrumentedTest.kt b/app/src/androidTest/kotlin/com/draftlens/FloatingControlOverlayInstrumentedTest.kt new file mode 100644 index 0000000..44f822f --- /dev/null +++ b/app/src/androidTest/kotlin/com/draftlens/FloatingControlOverlayInstrumentedTest.kt @@ -0,0 +1,132 @@ +package com.draftlens + +import android.content.Context +import android.content.pm.PackageManager +import android.os.SystemClock +import android.provider.Settings +import androidx.test.core.app.ApplicationProvider +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.draftlens.datastore.AppPreferencesRepository +import com.draftlens.overlay.OverlayController +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.SupervisorJob +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.runBlocking +import org.junit.After +import org.junit.Assert.assertFalse +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotNull +import org.junit.Assert.assertTrue +import org.junit.Assume.assumeTrue +import org.junit.Test +import org.junit.runner.RunWith + +/** + * Compile-only in CI (no device farm): real WindowManager behavior needs a device with the + * overlay appop granted. To execute on a physical device or emulator: + * + * adb shell appops set --uid $(adb shell pm list package -U com.draftlens.debug | cut -d: -f3 | cut -d\; -f2) SYSTEM_ALERT_WINDOW allow + * ./gradlew connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.draftlens.FloatingControlOverlayInstrumentedTest + * + * What can never be asserted here: whether the overlay appears inside a captured image — + * that is a physical-device finding, recorded in docs/TESTING_PHYSICAL_DEVICE.md. + */ +@RunWith(AndroidJUnit4::class) +class FloatingControlOverlayInstrumentedTest { + + private val context: Context = ApplicationProvider.getApplicationContext() + private val scope = CoroutineScope(SupervisorJob() + Dispatchers.Main) + private val preferences = AppPreferencesRepository(context) + private var controller: OverlayController? = null + + @After fun tearDown() { + controller?.detach() + controller = null + runBlocking { preferences.setFloatingControlEnabled(false) } + } + + @Test fun overlayPermissionIsDeclaredInTheManifest() { + val info = context.packageManager.getPackageInfo(context.packageName, PackageManager.GET_PERMISSIONS) + val requested = info.requestedPermissions?.toSet().orEmpty() + assertTrue( + "SYSTEM_ALERT_WINDOW must be declared; granting happens at runtime", + requested.contains(android.Manifest.permission.SYSTEM_ALERT_WINDOW), + ) + } + + @Test fun overlayAttachesWhenEnabledAndPermittedAndDetachesCleanly() { + assumeTrue( + "overlay appop not granted on this device; see class comment", + Settings.canDrawOverlays(context), + ) + runBlocking { preferences.setFloatingControlEnabled(true) } + + val created = OverlayController(context, preferences, scope) + controller = created + created.show(1080, 2400) + + assertTrue("overlay did not attach within 5s", waitUntil { created.isAttached }) + created.detach() + assertFalse(created.isAttached) + } + + @Test fun aPendingAttachIsInvalidatedByDetachSoNoOverlayIsOrphaned() { + assumeTrue( + "overlay appop not granted on this device; see class comment", + Settings.canDrawOverlays(context), + ) + runBlocking { preferences.setFloatingControlEnabled(true) } + + val created = OverlayController(context, preferences, scope) + controller = created + created.show(1080, 2400) + // Teardown lands while the DataStore read inside show() is still in flight. + created.detach() + Thread.sleep(1_000) + + assertFalse("a detached controller must never attach afterwards", created.isAttached) + } + + @Test fun floatingPositionRoundTripsThroughDataStore() = runBlocking { + // Note: writes the shared preferences store on the test device; restored at the end. + val prefs = AppPreferencesRepository(context) + prefs.setFloatingPosition(com.draftlens.overlay.OverlayPosition.Normalized(0.15f, 0.3f)) + + val read = prefs.floatingPosition.first() + + assertNotNull("position must persist", read) + assertEquals(0.15f, read!!.x, 0.001f) + assertEquals(0.3f, read.y, 0.001f) + // Restore a sane default for the app on this device. + prefs.setFloatingPosition(com.draftlens.overlay.OverlayPosition.Normalized(1f, 0.18f)) + } + + @Test fun overlaySurvivesPublicationStateFlips() { + assumeTrue( + "overlay appop not granted on this device; see class comment", + Settings.canDrawOverlays(context), + ) + runBlocking { preferences.setFloatingControlEnabled(true) } + + val created = OverlayController(context, preferences, scope) + controller = created + created.show(1080, 2400) + assertTrue("overlay did not attach within 5s", waitUntil { created.isAttached }) + + // Publication-state changes (freeze/resume/pause/visibility gates) must never detach it. + created.setNotLive(true) + created.setNotLive(false) + + assertTrue("overlay must stay attached across publication-state flips", created.isAttached) + } + + private fun waitUntil(timeoutMs: Long = 5_000, condition: () -> Boolean): Boolean { + val deadline = SystemClock.elapsedRealtime() + timeoutMs + while (SystemClock.elapsedRealtime() < deadline) { + if (condition()) return true + SystemClock.sleep(100) + } + return condition() + } +} diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml index 66e937a..b2634b5 100644 --- a/app/src/main/AndroidManifest.xml +++ b/app/src/main/AndroidManifest.xml @@ -5,6 +5,8 @@ + + diff --git a/app/src/main/kotlin/com/draftlens/capture/CaptureController.kt b/app/src/main/kotlin/com/draftlens/capture/CaptureController.kt index 3cef19b..c6a5290 100644 --- a/app/src/main/kotlin/com/draftlens/capture/CaptureController.kt +++ b/app/src/main/kotlin/com/draftlens/capture/CaptureController.kt @@ -19,4 +19,7 @@ object CaptureController { fun stop(context: Context) = context.startService(CaptureSessionService.stopIntent(context)) fun pause(context: Context) = context.startService(CaptureSessionService.pauseIntent(context)) fun resume(context: Context) = context.startService(CaptureSessionService.resumeIntent(context)) + + /** Re-evaluates the floating control after the user toggled it or granted the permission. */ + fun refreshOverlay(context: Context) = context.startService(CaptureSessionService.overlayUpdateIntent(context)) } diff --git a/app/src/main/kotlin/com/draftlens/capture/CaptureDiagnostics.kt b/app/src/main/kotlin/com/draftlens/capture/CaptureDiagnostics.kt new file mode 100644 index 0000000..fe9f583 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/capture/CaptureDiagnostics.kt @@ -0,0 +1,31 @@ +package com.draftlens.capture + +/** + * Rate limiter for bounded capture diagnostics: the first few occurrences of each event + * and then one line every [every] occurrences. No pixel data is ever logged. + * + * Keyed by event name, so "gate-reject:contentHidden" is counted separately from + * "gate-reject:staleGeneration" — a log line can then answer whether ImageReader received + * images and which gate rejected them. + */ +class CaptureDiagnostics( + private val firstEvents: Int = 4, + private val every: Int = 20, +) { + private val counts = LinkedHashMap() + + /** Increments the event's counter and returns the new count. */ + fun sample(event: String): Int { + val next = (counts[event] ?: 0) + 1 + counts[event] = next + return next + } + + fun shouldLog(count: Int): Boolean = count <= firstEvents || count % every == 0 + + fun total(event: String): Int = counts[event] ?: 0 + + fun reset() { + counts.clear() + } +} diff --git a/app/src/main/kotlin/com/draftlens/capture/CapturePublishState.kt b/app/src/main/kotlin/com/draftlens/capture/CapturePublishState.kt new file mode 100644 index 0000000..5a48ed7 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/capture/CapturePublishState.kt @@ -0,0 +1,36 @@ +package com.draftlens.capture + +/** + * Single source of truth for the "is live publication happening?" presentation used by + * the capture notification, the floating overlay glyph and the toggle logic. + * + * Three states, never independent: + * - LIVE: images convert and publish (no pin, not paused). + * - PAUSED: the user (or the notification) paused publication; projection stays valid. + * - FROZEN: a frame snapshot is pinned (in-app freeze / floating button). + * + * Pure Kotlin so the decision matrix is unit-testable; every UI surface derives its + * action from here instead of keeping parallel state. + */ +enum class CapturePublishState { LIVE, PAUSED, FROZEN } + +/** What the notification's one freeze/pause action shows. */ +enum class CapturePublishAction { PAUSE, RESUME } + +enum class FloatingTapOutcome { FREEZE, RESUME } + +/** Derivation both surfaces share; a pin wins over pause for presentation. */ +fun publishStateFor(pinInPlace: Boolean, isPaused: Boolean): CapturePublishState = when { + pinInPlace -> CapturePublishState.FROZEN + isPaused -> CapturePublishState.PAUSED + else -> CapturePublishState.LIVE +} + +fun publishActionFor(state: CapturePublishState): CapturePublishAction = + if (state == CapturePublishState.LIVE) CapturePublishAction.PAUSE else CapturePublishAction.RESUME + +/** The overlay tap toggles the shared pause/freeze gate; it never invents a new one. */ +fun floatingTapFor(state: CapturePublishState): FloatingTapOutcome = + if (state == CapturePublishState.LIVE) FloatingTapOutcome.FREEZE else FloatingTapOutcome.RESUME + +fun publishStateBlocksPublication(state: CapturePublishState): Boolean = state != CapturePublishState.LIVE diff --git a/app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt b/app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt index 0e7da44..d40c9fc 100644 --- a/app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt +++ b/app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt @@ -25,7 +25,12 @@ import androidx.core.app.ServiceCompat import com.draftlens.MainActivity import com.draftlens.R import com.draftlens.models.CaptureStatus +import com.draftlens.overlay.OverlayController +import com.draftlens.overlay.OverlayEnablement import com.draftlens.vision.model.FrameRotation +import kotlinx.coroutines.flow.combine +import kotlinx.coroutines.flow.distinctUntilChanged +import kotlinx.coroutines.flow.first import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Job @@ -50,16 +55,25 @@ class CaptureSessionService : Service() { ) private val startupExecutor = CaptureStartupExecutor() private lateinit var pipeline: FramePipeline + /** Service-owned so visibility callbacks before pipeline creation are never lost. */ + private val visibilityPolicy = CaptureVisibilityPolicy() + /** Service-owned so an early selected-app resize callback is never lost. */ + private val pendingResize = PendingResize() private var projection: MediaProjection? = null private var virtualDisplay: VirtualDisplay? = null private var displayListener: DisplayManager.DisplayListener? = null private var captureJob: Job? = null private var frameJob: Job? = null - private var frozenJob: Job? = null + private var publishedStateJob: Job? = null private var foregroundPromoted = false + /** Non-null only while the floating control may be attached (user-enabled + permitted + capturing). */ + private var overlayController: OverlayController? = null /** Increments once per capture session; tags every structured log line. */ private var sessionCounter = 0 + private var visibilityCallbackSeq = 0L + private var resizeCallbackSeq = 0L + private val diagnostics = CaptureDiagnostics() private val projectionCallback = object : MediaProjection.Callback() { override fun onStop() { @@ -68,17 +82,59 @@ class CaptureSessionService : Service() { } override fun onCapturedContentResize(width: Int, height: Int) { + resizeCallbackSeq += 1 + Log.i( + TAG, + "session=$sessionCounter resize-callback seq=$resizeCallbackSeq ${width}x${height} " + + "status=${CaptureStateHolder.state.value.status} virtualDisplayReady=${virtualDisplay != null} " + + "pipelineReady=${::pipeline.isInitialized && pipeline.surface() != null}", + ) // The captured-content dimensions are authoritative; they are not the same as // the display-window metrics the session started with (that is the landscape bug). - recreateDisplay(width, height, source = "captured-content-resize") + val seq = pendingResize.offer(width, height) + if (CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING && virtualDisplay != null) { + applyPendingResize() + } else { + Log.i(TAG, "session=$sessionCounter resize buffered seq=$seq (status=${CaptureStateHolder.state.value.status} virtualDisplayReady=${virtualDisplay != null})") + } + } + + /** + * API 34+: the window chosen in the chooser left or returned to the screen. + * The projection stays valid: while hidden, images are acquired and closed but + * never published, so DraftLens/shade/other apps can never replace the last + * valid game frame. Returning to MLBB reopens publication unless a frame is pinned. + * Fed to the pipeline regardless of session status: a pause must not stale the + * gate (a hide during PAUSED would otherwise publish foreign frames on resume). + */ + override fun onCapturedContentVisibilityChanged(isVisible: Boolean) { + visibilityCallbackSeq += 1 + Log.i( + TAG, + "session=$sessionCounter visibility-callback seq=$visibilityCallbackSeq visible=$isVisible " + + "pipelineInitialized=${::pipeline.isInitialized} status=${CaptureStateHolder.state.value.status} " + + "virtualDisplayReady=${virtualDisplay != null}", + ) + if (!::pipeline.isInitialized) { + Log.i(TAG, "session=$sessionCounter visibility callback arrived before pipeline init: BUFFERED by service policy") + } + val transition = visibilityPolicy.onContentVisibilityChanged(isVisible, sessionCounter) + Log.i( + TAG, + "session=$sessionCounter visibility applied seq=${transition.sequence} accepted=${transition.accepted} " + + "${transition.previous.name}->${transition.current.name} changed=${transition.changed} " + + "state=${CaptureStateHolder.state.value.status}", + ) } } override fun onCreate() { super.onCreate() - pipeline = FramePipeline(sampler) { - CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING - } + pipeline = FramePipeline( + sampler, + isActive = { CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING }, + visibility = visibilityPolicy, + ) createNotificationChannel() } @@ -134,29 +190,10 @@ class CaptureSessionService : Service() { CaptureStateHolder.setPaused(false) START_NOT_STICKY } - ACTION_FREEZE -> { - // Never re-promotes, never restarts MediaProjection: no-op without an active pipeline. - if (!CaptureFreeze.hasActivePipeline()) { - Log.i(TAG, "Freeze command ignored: no active capture session") - stopSelf() // do not linger for a stale command - return START_NOT_STICKY - } - when (val outcome = CaptureFreeze.freezeCurrentFrame()) { - is FreezeOutcome.Frozen -> - Log.i(TAG, "Freeze command honoured (${outcome.frame.width}x${outcome.frame.height})") - FreezeOutcome.AlreadyFrozen -> - Log.i(TAG, "Freeze command: a frame is already frozen; keeping it") - FreezeOutcome.NoFrame -> - Log.i(TAG, "Freeze command ignored: no frame captured yet") - FreezeOutcome.NoPipeline -> { - Log.i(TAG, "Freeze command ignored: no active capture session") - stopSelf() // do not linger for a stale command - } - } - START_NOT_STICKY - } ACTION_RESUME_LIVE -> { + // Unified resume: clears any pinned frame AND any pause. Safe in either state. CaptureFreeze.resumeLive() + CaptureStateHolder.setPaused(false) if (CaptureStateHolder.state.value.status != CaptureStatus.CAPTURING && CaptureStateHolder.state.value.status != CaptureStatus.PAUSED ) { @@ -164,6 +201,15 @@ class CaptureSessionService : Service() { } START_NOT_STICKY } + ACTION_OVERLAY_UPDATE -> { + // Re-evaluates the floating control against the current preference/permission/session state. + scope.launch { syncFloatingControl() } + val status = CaptureStateHolder.state.value.status + if (status != CaptureStatus.CAPTURING && status != CaptureStatus.PAUSED && status != CaptureStatus.STARTING) { + stopSelf() // nothing to update; don't linger for a stale refresh + } + START_NOT_STICKY + } else -> START_NOT_STICKY } } @@ -176,6 +222,8 @@ class CaptureSessionService : Service() { } sessionCounter += 1 + visibilityPolicy.resetForNewSession(sessionCounter) + pendingResize.clear() sampler.setIntervalMs(intervalMs) releaseCaptureResources(releaseForeground = false) @@ -197,16 +245,21 @@ class CaptureSessionService : Service() { proj.registerCallback(projectionCallback, null) val wm = getSystemService(Context.WINDOW_SERVICE) as WindowManager - val (captureWidth, captureHeight) = captureDimensions(wm) + val (windowWidth, windowHeight) = captureDimensions(wm) + // An early captured-content resize carries the selected app's real size. + // Use it for BOTH the ImageReader and the VirtualDisplay from the start. + val (captureWidth, captureHeight) = pendingResize.peek() ?: (windowWidth to windowHeight) val dpi = resources.configuration.densityDpi val displayRotation = currentDisplayRotation() - Log.i(TAG, "session=$sessionCounter capture-start: dimensions=${captureWidth}x${captureHeight} dpi=$dpi rotation=$displayRotation source=windowMetrics") + Log.i(TAG, "session=$sessionCounter capture-start: dimensions=${captureWidth}x${captureHeight} dpi=$dpi rotation=$displayRotation source=${if (pendingResize.peek() != null) "pending-captured-content-resize" else "windowMetrics"} windowMetrics=${windowWidth}x${windowHeight} visibility=${visibilityPolicy.currentState().name} visibilitySeq=${visibilityPolicy.sequence()}") - pipeline.start(captureWidth, captureHeight, dpi, displayRotation) + pipeline.start(captureWidth, captureHeight, dpi, displayRotation, sessionId = sessionCounter) CaptureFreeze.attachPipeline(pipeline) createVirtualDisplay(proj, captureWidth, captureHeight, dpi) registerDisplayListener() CaptureStateHolder.beginCapture(captureWidth, captureHeight, dpi) + applyPendingResize() // idempotent no-op when the early value was already used + attachFloatingControl() collectFrames() } ) @@ -220,6 +273,47 @@ class CaptureSessionService : Service() { } } + /** Creates the overlay controller and asks it to show; the controller re-checks the + * preference and the Android overlay permission asynchronously. No-op when present. */ + private fun attachFloatingControl() { + if (overlayController != null) return + val app = application as? com.draftlens.DraftLensApp ?: run { + Log.w(TAG, "floating control unavailable: unexpected application type") + return + } + val wm = getSystemService(Context.WINDOW_SERVICE) as WindowManager + val (width, height) = captureDimensions(wm) + overlayController = OverlayController(applicationContext, app.preferences, scope) + overlayController?.show(width, height) + } + + /** Adds or removes the overlay to match preference + permission + session state. */ + private suspend fun syncFloatingControl() { + val app = application as? com.draftlens.DraftLensApp ?: return + val enabled = runCatching { app.preferences.floatingControlEnabled.first() }.getOrElse { false } + val canDraw = runCatching { android.provider.Settings.canDrawOverlays(this) }.getOrDefault(false) + val running = CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING || + CaptureStateHolder.state.value.status == CaptureStatus.PAUSED + if (OverlayEnablement.shouldAttachOverlay(enabled, canDraw, running)) { + if (overlayController == null) attachFloatingControl() + } else { + Log.i(TAG, "floating control detached (enabled=$enabled canDraw=$canDraw running=$running)") + overlayController?.detach() + overlayController = null + } + } + + /** Applies the latest buffered selected-app dimensions once the session is ready. */ + private fun applyPendingResize() { + val pending = pendingResize.consume() ?: return + if (CaptureStateHolder.state.value.status != CaptureStatus.CAPTURING || virtualDisplay == null) { + Log.w(TAG, "session=$sessionCounter pending resize ${pending.first}x${pending.second} could not be applied yet; keeping it buffered") + pendingResize.offer(pending.first, pending.second) + return + } + recreateDisplay(pending.first, pending.second, source = "pending-captured-content-resize") + } + private fun captureDimensions(wm: WindowManager): Pair { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) { val bounds = wm.maximumWindowMetrics.bounds @@ -263,7 +357,7 @@ class CaptureSessionService : Service() { } createNotificationChannel() - val notification = buildNotification() + val notification = buildNotification(currentPublishState()) return try { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) { ServiceCompat.startForeground( @@ -293,16 +387,29 @@ class CaptureSessionService : Service() { frameJob?.cancel() frameJob = scope.launch { pipeline.latestFrame.collectLatest { frame -> - if (frame != null && CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING) { + if (frame != null) { + // The pipeline owns freshness: stop() clears latestFrame. Do not drop a + // frame that was published during STARTING, before beginCapture(). CaptureStateHolder.recordFrame(frame) + val delivered = diagnostics.sample("holder-delivered") + if (diagnostics.shouldLog(delivered)) { + Log.i(TAG, "session=$sessionCounter recordFrame delivered count=$delivered frame=${frame.width}x${frame.height} id=${frame.frame.id.value} status=${CaptureStateHolder.state.value.status}") + } } } } - frozenJob?.cancel() - frozenJob = scope.launch { - CaptureStateHolder.frozenFrame.collectLatest { frozen -> - updateNotificationState(frozen != null) - } + publishedStateJob?.cancel() + publishedStateJob = scope.launch { + combine( + CaptureStateHolder.frozenFrame, + CaptureStateHolder.state, + ) { pin, st -> publishStateFor(pinInPlace = pin != null, isPaused = st.isPaused) } + .distinctUntilChanged() + + .collectLatest { publishState -> + updateNotificationState(publishState) + overlayController?.setNotLive(publishStateBlocksPublication(publishState)) + } } } @@ -336,8 +443,12 @@ class CaptureSessionService : Service() { private fun recreateDisplay(width: Int, height: Int, source: String) { val densityDpi = resources.configuration.densityDpi val display = virtualDisplay ?: run { - Log.e(TAG, "session=$sessionCounter resize requested ($source) but VirtualDisplay is missing") - failProjectionSession("VirtualDisplay disappeared during resize", null) + if (CaptureStateHolder.state.value.status == CaptureStatus.CAPTURING) { + Log.e(TAG, "session=$sessionCounter resize requested ($source) but VirtualDisplay is missing") + failProjectionSession("VirtualDisplay disappeared during resize", null) + } else { + Log.i(TAG, "session=$sessionCounter resize requested ($source) ${width}x${height} before VirtualDisplay creation; buffered") + } return } if (CaptureStateHolder.state.value.status != CaptureStatus.CAPTURING) { @@ -421,6 +532,8 @@ class CaptureSessionService : Service() { Log.i(TAG, "session=$sessionCounter display metrics changed ${current.displayWidth}x${current.displayHeight} -> ${width}x${height}; resyncing capture size") recreateDisplay(width, height, source = "display-listener") } + // Keep the overlay anchored to the (possibly rotated) free axis. + overlayController?.onDisplayMetricsChanged(width, height) } } } @@ -461,13 +574,17 @@ class CaptureSessionService : Service() { } private fun releaseCaptureResources(releaseForeground: Boolean) { - Log.i(TAG, "session=$sessionCounter release: captureJob/frameJob/frozenJob cancelled=${captureJob != null || frameJob != null || frozenJob != null}") + pendingResize.clear() + // The overlay goes first: after stop the control has nothing to toggle and must never linger. + overlayController?.detach() + overlayController = null + Log.i(TAG, "session=$sessionCounter release: captureJob/frameJob/publishedStateJob cancelled=${captureJob != null || frameJob != null || publishedStateJob != null}") captureJob?.cancel() captureJob = null frameJob?.cancel() frameJob = null - frozenJob?.cancel() - frozenJob = null + publishedStateJob?.cancel() + publishedStateJob = null try { virtualDisplay?.release() } catch (e: Exception) { Log.w(TAG, "session=$sessionCounter VirtualDisplay release failed", e) } if (virtualDisplay != null) Log.i(TAG, "session=$sessionCounter virtual-display released") virtualDisplay = null @@ -529,8 +646,11 @@ class CaptureSessionService : Service() { .createNotificationChannel(channel) } - /** Re-posts the ongoing notification with the current LIVE/FROZEN state and its action. */ - private fun updateNotificationState(frozen: Boolean) { + private fun currentPublishState(): CapturePublishState = + publishStateFor(pinInPlace = CaptureStateHolder.frozenFrame.value != null, isPaused = CaptureStateHolder.state.value.isPaused) + + /** Re-posts the ongoing notification with the current LIVE/PAUSED/FROZEN state and its actions. */ + private fun updateNotificationState(publishState: CapturePublishState) { if (!foregroundPromoted) { Log.d(TAG, "Skipping notification update: service is not promoted") return @@ -541,7 +661,7 @@ class CaptureSessionService : Service() { } try { (getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager) - .notify(NOTIFICATION_ID, buildNotification(frozen)) + .notify(NOTIFICATION_ID, buildNotification(publishState)) } catch (e: Exception) { Log.w(TAG, "Failed to update the capture notification", e) } @@ -553,7 +673,7 @@ class CaptureSessionService : Service() { .areNotificationsEnabled() }.getOrDefault(true) - private fun buildNotification(frozen: Boolean = CaptureStateHolder.frozenFrame.value != null): Notification { + private fun buildNotification(publishState: CapturePublishState): Notification { val tapIntent = PendingIntent.getActivity( this, 0, Intent(this, MainActivity::class.java), @@ -564,18 +684,26 @@ class CaptureSessionService : Service() { Intent(this, CaptureSessionService::class.java).setAction(ACTION_STOP), PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE ) - val freezeIntent = PendingIntent.getService( + val pauseIntent = PendingIntent.getService( this, 2, - Intent(this, CaptureSessionService::class.java).setAction(ACTION_FREEZE), + Intent(this, CaptureSessionService::class.java).setAction(ACTION_PAUSE), PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE ) - val resumeCaptureIntent = PendingIntent.getService( + val resumeIntent = PendingIntent.getService( this, 3, Intent(this, CaptureSessionService::class.java).setAction(ACTION_RESUME_LIVE), PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE ) - val subText = if (frozen) getString(R.string.capture_frozen_subtext) else getString(R.string.capture_live_subtext) - val text = if (frozen) getString(R.string.capture_notification_frozen_text) else getString(R.string.capture_notification_text) + val subText = when (publishState) { + CapturePublishState.LIVE -> getString(R.string.capture_live_subtext) + CapturePublishState.PAUSED -> getString(R.string.capture_paused_subtext) + CapturePublishState.FROZEN -> getString(R.string.capture_frozen_subtext) + } + val text = when (publishState) { + CapturePublishState.LIVE -> getString(R.string.capture_notification_text) + CapturePublishState.PAUSED -> getString(R.string.capture_notification_paused_text) + CapturePublishState.FROZEN -> getString(R.string.capture_notification_frozen_text) + } val builder = NotificationCompat.Builder(this, CHANNEL_ID) .setContentTitle(getString(R.string.capture_notification_title)) .setContentText(text) @@ -584,10 +712,12 @@ class CaptureSessionService : Service() { .setContentIntent(tapIntent) .setOngoing(true) .setShowWhen(false) - if (frozen) { - builder.addAction(android.R.drawable.ic_media_play, getString(R.string.capture_resume_capture_action), resumeCaptureIntent) - } else { - builder.addAction(android.R.drawable.ic_menu_camera, getString(R.string.capture_freeze_action), freezeIntent) + // Exactly one of Pause/Resume — never both, never a stale Freeze action. + when (publishActionFor(publishState)) { + CapturePublishAction.PAUSE -> + builder.addAction(android.R.drawable.ic_media_pause, getString(R.string.capture_pause_action), pauseIntent) + CapturePublishAction.RESUME -> + builder.addAction(android.R.drawable.ic_media_play, getString(R.string.capture_resume_capture_action), resumeIntent) } return builder .addAction(android.R.drawable.ic_menu_close_clear_cancel, getString(R.string.capture_stop_action), stopIntent) @@ -599,8 +729,8 @@ class CaptureSessionService : Service() { const val ACTION_STOP = "com.draftlens.action.STOP_CAPTURE" const val ACTION_PAUSE = "com.draftlens.action.PAUSE_CAPTURE" const val ACTION_RESUME = "com.draftlens.action.RESUME_CAPTURE" - const val ACTION_FREEZE = "com.draftlens.action.FREEZE_CAPTURE" const val ACTION_RESUME_LIVE = "com.draftlens.action.RESUME_LIVE_CAPTURE" + const val ACTION_OVERLAY_UPDATE = "com.draftlens.action.OVERLAY_UPDATE" const val EXTRA_RESULT_CODE = "extra_result_code" const val EXTRA_DATA = "extra_data" const val EXTRA_INTERVAL_MS = "extra_interval_ms" @@ -623,5 +753,8 @@ class CaptureSessionService : Service() { fun resumeIntent(context: Context): Intent = Intent(context, CaptureSessionService::class.java).setAction(ACTION_RESUME) + + fun overlayUpdateIntent(context: Context): Intent = + Intent(context, CaptureSessionService::class.java).setAction(ACTION_OVERLAY_UPDATE) } } diff --git a/app/src/main/kotlin/com/draftlens/capture/CaptureVisibilityPolicy.kt b/app/src/main/kotlin/com/draftlens/capture/CaptureVisibilityPolicy.kt new file mode 100644 index 0000000..eea31e6 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/capture/CaptureVisibilityPolicy.kt @@ -0,0 +1,87 @@ +package com.draftlens.capture + +/** + * Visibility of the captured single-app window, with an explicit third state. + * + * UNKNOWN is deliberate: no platform callback has been applied yet for this session, or a + * `false` arrived before any frame existed (the captured window was hidden while the user + * was still in DraftLens choosing the app). UNKNOWN **allows** publication — full-display + * projections never call back at all, and an early single-app `false` must not block the + * first frame forever. The first successfully published frame promotes UNKNOWN → VISIBLE, + * so every later `false` is authoritative and protects the valid frame already held. + * + * HIDDEN blocks publication only; it never touches the overlay, projection or VirtualDisplay. + * Callbacks are kept even when they arrive before the pipeline exists: the policy is owned + * by the service, so ordering can no longer lose or overwrite a real callback. + * + * Thread-safe: read from the ImageReader thread, written from the callback thread. + */ +enum class CapturedContentVisibility { UNKNOWN, VISIBLE, HIDDEN } + +class CaptureVisibilityPolicy { + + private val lock = Any() + + @Volatile + private var state: CapturedContentVisibility = CapturedContentVisibility.UNKNOWN + + @Volatile + private var sessionId: Int = 0 + + @Volatile + private var sequence: Long = 0L + + fun currentState(): CapturedContentVisibility = state + + fun sequence(): Long = sequence + + fun boundSessionId(): Int = sessionId + + /** Called once per capture session, before consent/projection resources exist. */ + fun resetForNewSession(sessionId: Int) = synchronized(lock) { + state = CapturedContentVisibility.UNKNOWN + this.sessionId = sessionId + sequence = 0L + Unit + } + + /** + * Stores a platform callback, including those arriving before the pipeline is ready. + * Callbacks for a previous session are ignored (stale projection token). + */ + fun onContentVisibilityChanged(visible: Boolean, sessionId: Int): VisibilityTransition = synchronized(lock) { + if (sessionId != this.sessionId) { + VisibilityTransition(state, state, sequence, accepted = false) + } else { + sequence += 1 + val previous = state + state = when { + visible -> CapturedContentVisibility.VISIBLE + state == CapturedContentVisibility.VISIBLE -> CapturedContentVisibility.HIDDEN + state == CapturedContentVisibility.HIDDEN -> CapturedContentVisibility.HIDDEN + else -> state // UNKNOWN + early false: nothing valid to protect yet + } + VisibilityTransition(previous, state, sequence, accepted = true) + } + } + + /** A real frame makes a following `false` authoritative (UNKNOWN → VISIBLE). */ + fun onFramePublished(): Boolean = synchronized(lock) { + if (state != CapturedContentVisibility.UNKNOWN) return false + state = CapturedContentVisibility.VISIBLE + true + } + + fun shouldPublish(isActive: Boolean, generationCurrent: Boolean, frozen: Boolean): Boolean = + isActive && generationCurrent && !frozen && state != CapturedContentVisibility.HIDDEN +} + +/** Result of a stored visibility callback; [accepted] false = stale-session callback. */ +data class VisibilityTransition( + val previous: CapturedContentVisibility, + val current: CapturedContentVisibility, + val sequence: Long, + val accepted: Boolean = true, +) { + val changed: Boolean get() = previous != current +} diff --git a/app/src/main/kotlin/com/draftlens/capture/FramePipeline.kt b/app/src/main/kotlin/com/draftlens/capture/FramePipeline.kt index 5a938bd..735eecc 100644 --- a/app/src/main/kotlin/com/draftlens/capture/FramePipeline.kt +++ b/app/src/main/kotlin/com/draftlens/capture/FramePipeline.kt @@ -26,7 +26,9 @@ import kotlinx.coroutines.flow.asStateFlow */ class FramePipeline( private val sampler: FrameSampler, - private val isActive: () -> Boolean = { true } + private val isActive: () -> Boolean = { true }, + /** Owned by the service so callbacks before pipeline.start() are never lost. */ + private val visibility: CaptureVisibilityPolicy = CaptureVisibilityPolicy(), ) { private var imageReader: ImageReader? = null private var handlerThread: HandlerThread? = null @@ -43,6 +45,16 @@ class FramePipeline( @Volatile private var frozen = false + /** Bounded, event-keyed diagnostics (first 4 + every 20 per event; never pixels). */ + private val diagnostics = CaptureDiagnostics() + + /** Session id copied from the service for every structured log line. */ + @Volatile + private var sessionId: Int = 0 + + /** Effective gate state for the structured acquisition log. */ + fun visibilityState(): String = visibility.currentState().name + /** * Bumped every time a new reader/surface generation is created (start/resize). * Images captured with an older generation are dropped, so a late callback from a @@ -99,8 +111,11 @@ class FramePipeline( width: Int, height: Int, densityDpi: Int, - initialDisplayRotation: FrameRotation = FrameRotation.ROTATION_0 + initialDisplayRotation: FrameRotation = FrameRotation.ROTATION_0, + sessionId: Int = 0, ) { + this.sessionId = sessionId + diagnostics.reset() stopInternal(release = true) displayWidth = width displayHeight = height @@ -113,65 +128,98 @@ class FramePipeline( imageReader = ImageReader.newInstance(width, height, android.graphics.PixelFormat.RGBA_8888, 3).also { reader -> reader.setOnImageAvailableListener({ reader -> val image = try { reader.acquireLatestImage() } catch (e: Exception) { - e.printStackTrace() + val n = diagnostics.sample("acquire-exception") + if (diagnostics.shouldLog(n)) Log.w(TAG, "session=$sessionId acquisition exception count=$n", e) null } - if (image == null) return@setOnImageAvailableListener + if (image == null) { + val n = diagnostics.sample("acquire-null") + if (diagnostics.shouldLog(n)) Log.i(TAG, "session=$sessionId acquireLatestImage returned null count=$n") + return@setOnImageAvailableListener + } val wallStartedAtMs = System.currentTimeMillis() + val acquired = diagnostics.sample("acquired") + val activeNow = isActive() + val currentNow = generation.isCurrent(generationAtStart) + val frozenNow = frozen + val visibleNow = visibility.currentState() + val geometry = "session=$sessionId gen=$generationAtStart acquired=$acquired " + + "image=${image.width}x${image.height} isActive=$activeNow generationCurrent=$currentNow " + + "frozen=$frozenNow contentVisible=${visibleNow.name}" + if (diagnostics.shouldLog(acquired)) Log.i(TAG, "acquire: $geometry") try { - if (!isActive()) return@setOnImageAvailableListener - if (!generation.isCurrent(generationAtStart)) { - // A newer reader/surface generation owns the pipeline; drop late images. + if (!visibility.shouldPublish(isActive = activeNow, generationCurrent = currentNow, frozen = frozenNow)) { + val reason = when { + !activeNow -> "isActive" + !currentNow -> "staleGeneration" + frozenNow -> "frozen" + else -> "contentHidden" + } + val rejectCount = diagnostics.sample("gate-reject:$reason") + if (diagnostics.shouldLog(rejectCount)) { + Log.i(TAG, "drop gate=$reason count=$rejectCount $geometry") + } return@setOnImageAvailableListener } - if (frozen) { - // Frozen: images are acquired and closed in finally; no conversion, no retention. + val sampledAtMs = sampler.currentTimeMs() + if (!sampler.shouldEmit(sampledAtMs)) { + val n = diagnostics.sample("sampling-reject") + if (diagnostics.shouldLog(n)) Log.i(TAG, "drop sampling count=$n $geometry") return@setOnImageAvailableListener } - val sampledAtMs = sampler.currentTimeMs() - if (sampler.shouldEmit(sampledAtMs)) { - // Reference image only to avoid leaks; pixels are copied via the plane. - // Sprint 0 keeps this synchronous and in-memory; no disk, no upload. - val payload = imageToPayload(image, generationAtStart) - if (payload == null) { - Log.w(TAG, "generation=$generationAtStart dropped frame: payload conversion failed") - return@setOnImageAvailableListener - } - synchronized(freezeGate) { - if (frozen) { - // A freeze command landed during the conversion; the snapshot is authoritative. - Log.d(TAG, "generation=$generationAtStart dropped frame: a freeze command landed during conversion") - } else if (!generation.isCurrent(generationAtStart)) { - // An even newer generation took over while converting. - Log.d(TAG, "generation=$generationAtStart dropped frame: superseded by generation=${generation.current()} during conversion") - } else { - val processingLatencyMs = System.currentTimeMillis() - wallStartedAtMs - // One read per frame: both fields below see the same rotation. - val frameDisplayRotation = displayRotation - val visionFrame = OriginalFrame( - id = FrameId.random(), - payload = payload, - sourceWidth = payload.width, - sourceHeight = payload.height, - displayRotation = frameDisplayRotation, - // MediaProjection buffers are display-oriented; never double-rotate pixels. - rotation = FrameOrientation.correctionFor(frameDisplayRotation), - capturedAtElapsedRealtimeMs = sampledAtMs, - wallClockTimestampMs = System.currentTimeMillis(), - captureLatencyMs = processingLatencyMs, - ) - sampler.markEmitted(sampledAtMs) - _latestFrame.value = CapturedFrame( - frame = visionFrame, - processingLatencyMs = processingLatencyMs - ) - Log.d(TAG, "generation=$generationAtStart frame stored: source=${payload.width}x${payload.height} rotation=$frameDisplayRotation processingMs=$processingLatencyMs") + val acceptedCount = diagnostics.sample("sampling-accepted") + if (diagnostics.shouldLog(acceptedCount)) Log.i(TAG, "sampling accepted count=$acceptedCount $geometry") + + // Reference image only to avoid leaks; pixels are copied via the plane. + val payload = imageToPayload(image, generationAtStart) + if (payload == null) { + val n = diagnostics.sample("conversion-failed") + if (diagnostics.shouldLog(n)) Log.w(TAG, "drop conversion-failed count=$n $geometry") + return@setOnImageAvailableListener + } + val convertedCount = diagnostics.sample("conversion-ok") + if (diagnostics.shouldLog(convertedCount)) { + Log.i(TAG, "converted count=$convertedCount bitmap=${payload.width}x${payload.height} $geometry") + } + synchronized(freezeGate) { + if (frozen) { + val n = diagnostics.sample("publish-drop-frozen") + if (diagnostics.shouldLog(n)) Log.i(TAG, "drop at-publish frozen count=$n $geometry") + } else if (visibility.currentState() == CapturedContentVisibility.HIDDEN) { + val n = diagnostics.sample("publish-drop-hidden") + if (diagnostics.shouldLog(n)) Log.i(TAG, "drop at-publish contentHidden count=$n $geometry") + } else if (!generation.isCurrent(generationAtStart)) { + val n = diagnostics.sample("publish-drop-generation") + if (diagnostics.shouldLog(n)) Log.i(TAG, "drop at-publish staleGeneration count=$n $geometry") + } else { + val processingLatencyMs = System.currentTimeMillis() - wallStartedAtMs + val frameDisplayRotation = displayRotation + val visionFrame = OriginalFrame( + id = FrameId.random(), + payload = payload, + sourceWidth = payload.width, + sourceHeight = payload.height, + displayRotation = frameDisplayRotation, + rotation = FrameOrientation.correctionFor(frameDisplayRotation), + capturedAtElapsedRealtimeMs = sampledAtMs, + wallClockTimestampMs = System.currentTimeMillis(), + captureLatencyMs = processingLatencyMs, + ) + sampler.markEmitted(sampledAtMs) + visibility.onFramePublished() + _latestFrame.value = CapturedFrame( + frame = visionFrame, + processingLatencyMs = processingLatencyMs + ) + val published = diagnostics.sample("published") + if (diagnostics.shouldLog(published)) { + Log.i(TAG, "published to _latestFrame count=$published source=${payload.width}x${payload.height} rotation=$frameDisplayRotation processingMs=$processingLatencyMs $geometry") } } } } catch (e: Exception) { - // Never let a bad frame kill the pipeline; next frame recovers. - e.printStackTrace() + val n = diagnostics.sample("frame-exception") + if (diagnostics.shouldLog(n)) Log.w(TAG, "frame exception count=$n $geometry", e) } finally { try { image.close() } catch (_: Exception) {} } @@ -191,7 +239,10 @@ class FramePipeline( */ fun freezeAndSnapshot(): FreezeResult = synchronized(freezeGate) { if (frozen) return FreezeResult.AlreadyFrozen - val frame = _latestFrame.value ?: return FreezeResult.NoFrame + val frame = _latestFrame.value ?: run { + Log.w(TAG, "session=$sessionId freeze: NoFrame — pipeline has not published any frame; geometry=${displayWidth}x${displayHeight} visibility=${visibility.currentState().name} frozen=$frozen") + return FreezeResult.NoFrame + } val copy = FrameSnapshots.copyOf(frame) ?: return FreezeResult.NoFrame frozen = true Log.i(TAG, "generation=${generation.current()} frame pinned: ${frame.width}x${frame.height} id=${frame.frame.id.value}") @@ -248,15 +299,17 @@ class FramePipeline( Log.w(TAG, "generation=$generation dropped frame: plane copy failed for layout=$layout") return null } - Log.d( + val planeLog = diagnostics.sample("plane-detail") + if (diagnostics.shouldLog(planeLog)) Log.i( TAG, - "generation=$generation plane: image=${layout.imageWidth}x${layout.imageHeight} " + + "generation=$generation plane: count=$planeLog image=${layout.imageWidth}x${layout.imageHeight} " + "crop=${layout.cropLeft},${layout.cropTop} ${layout.cropWidth}x${layout.cropHeight} " + "rowStride=${layout.rowStride} pixelStride=${layout.pixelStride} " + "paddingBytes=${layout.rowPaddingBytes} fullCrop=${layout.isFullCrop} bufferBytes=${plane.buffer.remaining()}", ) val bitmap = Bitmap.createBitmap(argb, layout.cropWidth, layout.cropHeight, Bitmap.Config.ARGB_8888) - Log.d(TAG, "generation=$generation bitmap: ${bitmap.width}x${bitmap.height}") + val bitmapLog = diagnostics.sample("bitmap-detail") + if (diagnostics.shouldLog(bitmapLog)) Log.i(TAG, "generation=$generation bitmap: count=$bitmapLog ${bitmap.width}x${bitmap.height}") BitmapFramePayload(bitmap) } catch (e: Exception) { Log.w(TAG, "generation=$generation dropped frame: conversion threw", e) @@ -306,7 +359,7 @@ class FramePipeline( displayHeight = 0 try { - start(newWidth, newHeight, densityDpi, initialDisplayRotation) + start(newWidth, newHeight, densityDpi, initialDisplayRotation, sessionId = sessionId) } catch (e: Exception) { // Discard partially created replacement resources and restore the previous session. runCatching { surface?.release() } diff --git a/app/src/main/kotlin/com/draftlens/capture/PendingResize.kt b/app/src/main/kotlin/com/draftlens/capture/PendingResize.kt new file mode 100644 index 0000000..b204e85 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/capture/PendingResize.kt @@ -0,0 +1,43 @@ +package com.draftlens.capture + +/** + * Stores the latest captured-content dimensions reported during startup, when the + * VirtualDisplay/pipeline/session state are not ready for a resize. Later callbacks replace + * earlier ones; invalid dimensions are ignored so a bogus early value cannot size the reader. + */ +class PendingResize { + + private val lock = Any() + private var width: Int = 0 + private var height: Int = 0 + private var sequence: Long = 0L + + fun offer(newWidth: Int, newHeight: Int): Long = synchronized(lock) { + if (newWidth > 0 && newHeight > 0) { + width = newWidth + height = newHeight + sequence += 1 + } + sequence + } + + fun peek(): Pair? = synchronized(lock) { + if (width > 0 && height > 0) width to height else null + } + + /** Returns and clears the latest valid dimensions (call exactly when resources are ready). */ + fun consume(): Pair? = synchronized(lock) { + val pending = if (width > 0 && height > 0) width to height else null + width = 0 + height = 0 + pending + } + + fun sequence(): Long = synchronized(lock) { sequence } + + fun clear() = synchronized(lock) { + width = 0 + height = 0 + Unit + } +} diff --git a/app/src/main/kotlin/com/draftlens/datastore/AppPreferencesRepository.kt b/app/src/main/kotlin/com/draftlens/datastore/AppPreferencesRepository.kt index be0f8e9..395a230 100644 --- a/app/src/main/kotlin/com/draftlens/datastore/AppPreferencesRepository.kt +++ b/app/src/main/kotlin/com/draftlens/datastore/AppPreferencesRepository.kt @@ -1,7 +1,9 @@ package com.draftlens.datastore import android.content.Context +import androidx.datastore.preferences.core.booleanPreferencesKey import androidx.datastore.preferences.core.edit +import androidx.datastore.preferences.core.floatPreferencesKey import androidx.datastore.preferences.core.longPreferencesKey import androidx.datastore.preferences.core.stringPreferencesKey import androidx.datastore.preferences.preferencesDataStore @@ -9,6 +11,7 @@ import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.first import kotlinx.coroutines.flow.map import com.draftlens.capture.FrameSampler +import com.draftlens.overlay.OverlayPosition private val Context.dataStore by preferencesDataStore(name = "draftlens_preferences") @@ -18,6 +21,9 @@ class AppPreferencesRepository(private val context: Context) { val SAMPLING_INTERVAL = longPreferencesKey("sampling_interval_ms") val LAST_CAPTURE_AT = longPreferencesKey("last_capture_at_ms") val CONSENT_NOTE = stringPreferencesKey("consent_note") + val FLOATING_CONTROL_ENABLED = booleanPreferencesKey("floating_control_enabled") + val FLOATING_POSITION_X = floatPreferencesKey("floating_position_x") + val FLOATING_POSITION_Y = floatPreferencesKey("floating_position_y") } val samplingIntervalMs: Flow = context.dataStore.data.map { @@ -34,4 +40,25 @@ class AppPreferencesRepository(private val context: Context) { } suspend fun currentSamplingIntervalMs(): Long = samplingIntervalMs.first() + + /** True only when the user asked for the floating freeze button (and it may be shown during capture). */ + val floatingControlEnabled: Flow = context.dataStore.data.map { it[Keys.FLOATING_CONTROL_ENABLED] ?: false } + + /** Persisted normalized (0..1) position of the floating control; null until first placement. */ + val floatingPosition: Flow = context.dataStore.data.map { prefs -> + val x = prefs[Keys.FLOATING_POSITION_X] + val y = prefs[Keys.FLOATING_POSITION_Y] + if (x == null || y == null) null else OverlayPosition.Normalized(x, y) + } + + suspend fun setFloatingControlEnabled(enabled: Boolean) { + context.dataStore.edit { it[Keys.FLOATING_CONTROL_ENABLED] = enabled } + } + + suspend fun setFloatingPosition(position: OverlayPosition.Normalized) { + context.dataStore.edit { + it[Keys.FLOATING_POSITION_X] = position.x.coerceIn(0f, 1f) + it[Keys.FLOATING_POSITION_Y] = position.y.coerceIn(0f, 1f) + } + } } diff --git a/app/src/main/kotlin/com/draftlens/feature/home/DebugCaptureScreen.kt b/app/src/main/kotlin/com/draftlens/feature/home/DebugCaptureScreen.kt index 58a2665..c904141 100644 --- a/app/src/main/kotlin/com/draftlens/feature/home/DebugCaptureScreen.kt +++ b/app/src/main/kotlin/com/draftlens/feature/home/DebugCaptureScreen.kt @@ -41,9 +41,16 @@ import com.draftlens.capture.CaptureController import com.draftlens.datastore.AppPreferencesRepository import com.draftlens.models.CaptureStatus import android.content.Intent +import android.net.Uri import android.provider.Settings import android.Manifest import android.os.Build +import androidx.compose.material3.Switch +import androidx.compose.runtime.rememberCoroutineScope +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.compose.LifecycleEventEffect +import kotlinx.coroutines.flow.catch +import kotlinx.coroutines.launch import com.draftlens.vision.model.android.BitmapFramePayload @Composable @@ -68,8 +75,8 @@ fun DebugCaptureScreen( } } - // Notification permission: the capture notification carries the "Freeze current frame" - // action, so without it the MLBB freeze workflow is unreachable. + // Notification permission: the capture notification keeps a Freeze action, but the + // floating control and in-app Freeze/Resume work without it. var notificationsEnabled by remember { mutableStateOf(areCaptureNotificationsEnabled(activity)) } val notificationPermission = rememberLauncherForActivityResult( ActivityResultContracts.RequestPermission() @@ -88,10 +95,11 @@ fun DebugCaptureScreen( .padding(horizontal = 16.dp, vertical = 12.dp), verticalArrangement = Arrangement.spacedBy(12.dp) ) { - Text("DraftLens — Capture Debug (Sprint 0)", color = Color.White, style = MaterialTheme.typography.titleLarge) - Text("Detection: Not implemented in Sprint 0", color = Color.Gray) + Text("DraftLens — Capture", color = Color.White, style = MaterialTheme.typography.titleLarge) - MobileLegendsCaptureGuide( + PermissionsCard( + activity = activity, + preferences = preferences, notificationsEnabled = notificationsEnabled, onEnableNotifications = { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { @@ -192,47 +200,121 @@ fun DebugCaptureScreen( } } -/** Guidance for the Mobile Legends freeze workflow and missing-notification handling. */ +/** + * The only two permissions DraftLens needs beyond the screen-capture consent: notifications + * (the capture notification carries Stop + Pause) and the overlay permission (the floating + * freeze button). Both show clear granted/denied status and a grant/open-settings action. + */ @Composable -private fun MobileLegendsCaptureGuide( +private fun PermissionsCard( + activity: Activity, + preferences: AppPreferencesRepository, notificationsEnabled: Boolean, onEnableNotifications: () -> Unit, ) { val body = MaterialTheme.typography.bodySmall + var overlayGranted by remember { mutableStateOf(Settings.canDrawOverlays(activity)) } + var pendingEnable by remember { mutableStateOf(false) } + val floatingEnabledFlow = remember(preferences) { + preferences.floatingControlEnabled.catch { emit(false) } + } + val floatingEnabled by floatingEnabledFlow.collectAsState(initial = false) + val scope = rememberCoroutineScope() + + // Returning from the Android overlay-permission settings screen resolves a pending enable exactly once. + LifecycleEventEffect(Lifecycle.Event.ON_RESUME) { + val granted = Settings.canDrawOverlays(activity) + overlayGranted = granted + if (pendingEnable) { + pendingEnable = false + if (granted) { + scope.launch { + runCatching { preferences.setFloatingControlEnabled(true) } + .onFailure { android.util.Log.w("DraftLens", "failed to persist floating control enable", it) } + CaptureController.refreshOverlay(activity) + } + } + } + } + Card(modifier = Modifier.fillMaxWidth()) { - Column( - modifier = Modifier.padding(12.dp), - verticalArrangement = Arrangement.spacedBy(2.dp), - ) { - Text("Capturing Mobile Legends: Bang Bang", style = MaterialTheme.typography.titleMedium) - Text("1. Tap Start Capture.", style = body) - Text( - "2. When Android asks what to capture, choose Entire screen / full device display — " + - "not the DraftLens app window. App-window capture cannot see the game, " + - "so the frozen frame would show this app instead of MLBB.", - style = body, - ) - Text( - "3. Open Mobile Legends and wait for the desired draft screen.", - style = body, - ) - Text( - "4. Pull down the notification shade and tap “Freeze current frame” on the " + - "DraftLens capture notification. The latest game frame is kept in memory — no need to leave the game.", - style = body, - ) - Text( - "5. Return to DraftLens. Vision Lab shows the frozen MLBB frame, not this screen. " + - "Tap Resume there, or “Resume live capture” in the notification, to go back to live capture.", - style = body, - ) - if (!notificationsEnabled) { - Text( - "Notifications are disabled: the “Freeze current frame” action is not reachable from the shade while in game.", - style = body, - color = MaterialTheme.colorScheme.error, + Column(modifier = Modifier.padding(12.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) { + // --- Notifications --- + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = androidx.compose.ui.Alignment.CenterVertically, + ) { + Column(modifier = Modifier.weight(1f)) { + Text("Notifications", style = MaterialTheme.typography.titleMedium) + Text( + "Needed for the capture notification’s Stop and Pause actions while you are in the game.", + style = body, + ) + Text( + if (notificationsEnabled) "Status: Allowed" else "Status: Not allowed", + style = body, + color = if (notificationsEnabled) Color(0xFF7DDCA0) else MaterialTheme.colorScheme.error, + ) + } + if (!notificationsEnabled) { + TextButton(onClick = onEnableNotifications) { Text("Enable") } + } + } + + // --- Floating overlay button --- + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = androidx.compose.ui.Alignment.CenterVertically, + ) { + Column(modifier = Modifier.weight(1f)) { + Text("Floating freeze button", style = MaterialTheme.typography.titleMedium) + Text( + "Needed to freeze/resume the game frame from a small button while in MLBB. " + + "Capture works fully without it.", + style = body, + ) + Text( + if (overlayGranted) "Status: Allowed" else "Status: Not allowed", + style = body, + color = if (overlayGranted) Color(0xFF7DDCA0) else MaterialTheme.colorScheme.error, + ) + } + Switch( + checked = floatingEnabled, + onCheckedChange = { checked -> + if (checked) { + if (Settings.canDrawOverlays(activity)) { + scope.launch { + runCatching { preferences.setFloatingControlEnabled(true) } + .onFailure { android.util.Log.w("DraftLens", "failed to persist floating control enable", it) } + CaptureController.refreshOverlay(activity) + } + } else { + // The capture keeps working without the button; enable it once granted. + pendingEnable = true + activity.startActivity( + Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION) + .setData(Uri.parse("package:${activity.packageName}")), + ) + } + } else { + scope.launch { + runCatching { preferences.setFloatingControlEnabled(false) } + .onFailure { android.util.Log.w("DraftLens", "failed to persist floating control disable", it) } + CaptureController.refreshOverlay(activity) + } + } + }, ) - TextButton(onClick = onEnableNotifications) { Text("Enable notifications") } + } + if (!overlayGranted) { + TextButton(onClick = { + pendingEnable = true + activity.startActivity( + Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION) + .setData(Uri.parse("package:${activity.packageName}")), + ) + }) { Text("Open Android overlay settings") } } } } diff --git a/app/src/main/kotlin/com/draftlens/feature/visionlab/VisionLabScreen.kt b/app/src/main/kotlin/com/draftlens/feature/visionlab/VisionLabScreen.kt index 825a91e..7ffe652 100644 --- a/app/src/main/kotlin/com/draftlens/feature/visionlab/VisionLabScreen.kt +++ b/app/src/main/kotlin/com/draftlens/feature/visionlab/VisionLabScreen.kt @@ -187,22 +187,6 @@ private fun FramePreviewCard( enabled = state.isFrozen || state.processed != null, ) { Text("Clear") } } - Text( - if (state.isFrozen) { - "Resume returns to live frames." - } else { - "Freeze locks the current frame before annotation. While in game, tap " + - "“Freeze current frame” on the capture notification instead." - }, - style = MaterialTheme.typography.bodySmall, - ) - Text( - "Capturing Mobile Legends? When Android asks what to capture, choose Entire screen / " + - "full device display. App-window capture cannot see the game. The capture " + - "notification’s Freeze action needs notifications enabled.", - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) } } } diff --git a/app/src/main/kotlin/com/draftlens/overlay/FloatingControlView.kt b/app/src/main/kotlin/com/draftlens/overlay/FloatingControlView.kt new file mode 100644 index 0000000..11e4cec --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/overlay/FloatingControlView.kt @@ -0,0 +1,135 @@ +package com.draftlens.overlay + +import android.content.Context +import android.graphics.Color +import android.graphics.drawable.GradientDrawable +import android.os.SystemClock +import android.view.Gravity +import android.view.MotionEvent +import android.view.ViewConfiguration +import android.view.HapticFeedbackConstants +import android.widget.ImageView +import android.widget.FrameLayout +import com.draftlens.R + +/** + * The one and only floating UI for Sprint 1C: a circular freeze/resume button. + * + * Plain LIVE/FROZEN colors; no state is derived from incoming frames. + * Gesture contract (deliberate, so game taps near a draft slot are never eaten): + * - TAP (down+up inside the button, short): toggle freeze. + * - Long-press (400 ms hold) arms drag mode; moving only after the hold drags. + * - Moving before the long-press fires cancels the gesture entirely (no tap, no drag). + */ +class FloatingControlView( + context: Context, + private val onTap: () -> Unit, + private val onDragStart: (rawX: Float, rawY: Float) -> Unit, + private val onDragMove: (rawX: Float, rawY: Float) -> Unit, + private val onDragEnd: () -> Unit, +) : FrameLayout(context) { + + private val glyph: ImageView + private val longPressTimeoutMs = 400L + private val touchSlopPx = ViewConfiguration.get(context).scaledTouchSlop + private val dragRunnable = Runnable { + if (isPressed && !gestureCancelled && !dragging) { + dragging = true + performHapticFeedback(HapticFeedbackConstants.LONG_PRESS) + onDragStart(downRawX, downRawY) + } + } + + private var downRawX = 0f + private var downRawY = 0f + private var downTimeMs = 0L + private var dragging = false + private var gestureCancelled = false + + private val liveColor = Color.parseColor("#148F5A") + private val frozenColor = Color.parseColor("#E8890C") + private val liveStroke = Color.WHITE + + init { + val sizePx = dp(52) + layoutParams = LayoutParams(sizePx, sizePx) + background = GradientDrawable().apply { + shape = GradientDrawable.OVAL + setColor(liveColor) + setStroke(dp(2).toInt(), liveStroke) + } + elevation = dp(6).toFloat() + contentDescription = context.getString(R.string.floating_control_live_description) + + glyph = ImageView(context).apply { + setImageResource(R.drawable.ic_floating_live) + layoutParams = LayoutParams(LayoutParams.WRAP_CONTENT, LayoutParams.WRAP_CONTENT, Gravity.CENTER) + } + addView(glyph) + + isClickable = true + isFocusable = false + } + + /** LIVE (green) while publishing; orange while publication is paused or a frame is pinned. */ + fun setNotLive(notLive: Boolean) { + glyph.setImageResource(if (notLive) R.drawable.ic_floating_frozen else R.drawable.ic_floating_live) + val bg = background as GradientDrawable + bg.setColor(if (notLive) frozenColor else liveColor) + contentDescription = context.getString( + if (notLive) R.string.floating_control_frozen_description else R.string.floating_control_live_description, + ) + } + + override fun onTouchEvent(event: MotionEvent): Boolean { + when (event.actionMasked) { + MotionEvent.ACTION_DOWN -> { + downRawX = event.rawX + downRawY = event.rawY + downTimeMs = SystemClock.elapsedRealtime() + dragging = false + gestureCancelled = false + postDelayed(dragRunnable, longPressTimeoutMs) + return true + } + MotionEvent.ACTION_MOVE -> { + if (dragging) { + onDragMove(event.rawX, event.rawY) + return true + } + val dx = event.rawX - downRawX + val dy = event.rawY - downRawY + if (dx * dx + dy * dy > touchSlopPx * touchSlopPx) { + gestureCancelled = true + removeCallbacks(dragRunnable) + } + return true + } + MotionEvent.ACTION_UP -> { + removeCallbacks(dragRunnable) + if (dragging) { + onDragEnd() + } else if (!gestureCancelled && SystemClock.elapsedRealtime() - downTimeMs < longPressTimeoutMs + TAP_GRACE_MS) { + onTap() + } + dragging = false + gestureCancelled = false + return true + } + MotionEvent.ACTION_CANCEL -> { + removeCallbacks(dragRunnable) + if (dragging) onDragEnd() + dragging = false + gestureCancelled = false + return true + } + } + return super.onTouchEvent(event) + } + + private fun dp(value: Int): Int = (value * resources.displayMetrics.density).toInt() + + private companion object { + const val TAP_GRACE_MS = 200L + } +} diff --git a/app/src/main/kotlin/com/draftlens/overlay/OverlayController.kt b/app/src/main/kotlin/com/draftlens/overlay/OverlayController.kt new file mode 100644 index 0000000..787c3a4 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/overlay/OverlayController.kt @@ -0,0 +1,258 @@ +package com.draftlens.overlay + +import android.content.Context +import android.graphics.PixelFormat +import android.os.Handler +import android.os.Looper +import android.util.Log +import android.view.Gravity +import android.view.WindowManager +import com.draftlens.capture.CaptureFreeze +import com.draftlens.capture.CapturePublishState +import com.draftlens.capture.CaptureStateHolder +import com.draftlens.capture.FreezeOutcome +import com.draftlens.capture.FloatingTapOutcome +import com.draftlens.capture.floatingTapFor +import com.draftlens.capture.publishStateBlocksPublication +import com.draftlens.capture.publishStateFor +import com.draftlens.R +import com.draftlens.datastore.AppPreferencesRepository +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.launch + +/** + * Owns the WindowManager side of the single circular floating control. Attached only while + * a capture session is running (by [CaptureSessionService][com.draftlens.capture.CaptureSessionService]), always removed on teardown. Holds no + * capture state of its own; the tap toggles the shared pause/freeze gate and the glyph follows + * the same [CapturePublishState][com.draftlens.capture.CapturePublishState] the capture notification shows. + * Rendering, touch handling and visibility are independent of the captured-frame pipeline: + * hiding the captured window never removes or pauses the button. + * + * The window is exactly button-sized and non-focusable: touches outside the circle go to + * the app underneath, and the control never takes keyboard focus from the game. + */ +class OverlayController( + private val context: Context, + private val preferences: AppPreferencesRepository, + private val scope: CoroutineScope, +) { + private val windowManager = context.getSystemService(Context.WINDOW_SERVICE) as WindowManager + private val mainHandler = Handler(Looper.getMainLooper()) + + private var view: FloatingControlView? = null + private var layoutParams: WindowManager.LayoutParams? = null + private val sizePx = (context.resources.displayMetrics.density * SIZE_DP).toInt() + + /** Current display size; updated on rotation so drag/snap math never uses stale metrics. */ + @Volatile + private var screenWidthPx = 0 + + @Volatile + private var screenHeightPx = 0 + + @Volatile + private var currentNormalized = OverlayPosition.Normalized(x = 1f, y = DEFAULT_Y_FRACTION) + + val isAttached: Boolean get() = view != null + + /** + * Generation counter guarding pending attaches: every show() and every detach() bumps it, + * and attachView() refuses to run for a stale generation. Without this, a show() whose + * DataStore read settles after a teardown would add an orphaned always-on-top window + * nobody can dismiss (M1 review finding). + */ + private val attachLock = Any() + private var attachGeneration = 0 + + /** + * Adds the overlay after re-checking the user preference and the Android overlay + * permission — capture start calls this unconditionally, so the gate lives here. + * Main-thread attachment; the permission comes from the caller's toggle flow. + */ + fun show(screenWidthPx: Int, screenHeightPx: Int) { + if (view != null) { + Log.i(TAG, "overlay already attached; skipping show()") + return + } + val requestId = synchronized(attachLock) { ++attachGeneration } + scope.launch { + val enabled = runCatching { preferences.floatingControlEnabled.first() }.getOrElse { false } + val canDraw = runCatching { android.provider.Settings.canDrawOverlays(context) }.getOrDefault(false) + if (!OverlayEnablement.shouldAttachOverlay(enabled, canDraw, captureRunning = true)) { + Log.i(TAG, "overlay not attached (enabled=$enabled canDrawOverlays=$canDraw)") + return@launch + } + val persisted = runCatching { preferences.floatingPosition.first() }.getOrNull() + val normalized = persisted ?: currentNormalized + currentNormalized = normalized + mainHandler.post { attachView(screenWidthPx, screenHeightPx, normalized, requestId) } + } + } + + private fun attachView(screenWidthPx: Int, screenHeightPx: Int, normalized: OverlayPosition.Normalized, requestId: Int) { + synchronized(attachLock) { + if (requestId != attachGeneration) { + Log.i(TAG, "discarding stale overlay attach (generation $requestId, now $attachGeneration)") + return + } + if (view != null) return // a configure change raced the show() coroutine + this.screenWidthPx = screenWidthPx + this.screenHeightPx = screenHeightPx + + val px = OverlayPosition.normalizedToPixels(normalized, screenWidthPx, screenHeightPx, sizePx) + val button = FloatingControlView( + context, + onTap = { toggleFreeze() }, + onDragStart = { rawX, rawY -> beginDrag(rawX, rawY) }, + onDragMove = { rawX, rawY -> dragTo(rawX, rawY) }, + onDragEnd = { endDrag() }, + ) + button.setNotLive(publishStateBlocksPublication(currentPublishStateForOverlay())) + + val params = WindowManager.LayoutParams( + sizePx, + sizePx, + WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY, + WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE, + PixelFormat.TRANSLUCENT, + ).apply { + gravity = Gravity.TOP or Gravity.START + x = px.left + y = px.top + } + + try { + windowManager.addView(button, params) + view = button + layoutParams = params + Log.i(TAG, "overlay attached: sizePx=$sizePx x=${params.x} y=${params.y}") + } catch (e: Exception) { + // Missing overlay permission or a bad window state: capture and the in-app + // Freeze/Resume keep working; never crash the capture session over the control. + Log.w(TAG, "overlay attach failed; capture continues without the floating control", e) + view = null + layoutParams = null + } + } + } + + /** Re-derives the pixel position from the normalized one after a resize/rotation. */ + fun onDisplayMetricsChanged(screenWidthPx: Int, screenHeightPx: Int) { + this.screenWidthPx = screenWidthPx + this.screenHeightPx = screenHeightPx + val params = layoutParams ?: return + val button = view ?: return + val px = OverlayPosition.normalizedToPixels(currentNormalized, screenWidthPx, screenHeightPx, sizePx) + params.x = px.left + params.y = px.top + try { + windowManager.updateViewLayout(button, params) + } catch (e: Exception) { + Log.w(TAG, "overlay reposition failed; detaching", e) + detach() + } + } + + fun setNotLive(notLive: Boolean) { + mainHandler.post { view?.setNotLive(notLive) } + } + + /** Removes the overlay. Safe from any thread and when not attached; invalidates pending attaches. */ + fun detach() { + synchronized(attachLock) { attachGeneration++ } // invalidate any in-flight show() + val detachBlock = Runnable { + val button = view ?: return@Runnable + try { + windowManager.removeView(button) + Log.i(TAG, "overlay detached") + } catch (e: Exception) { + Log.w(TAG, "overlay detach failed", e) + } + view = null + layoutParams = null + } + if (Looper.myLooper() == Looper.getMainLooper()) { + detachBlock.run() + } else { + mainHandler.post(detachBlock) + } + } + + private var grabOffsetX = 0f + private var grabOffsetY = 0f + + private fun beginDrag(rawX: Float, rawY: Float) { + val params = layoutParams ?: return + grabOffsetX = rawX - params.x + grabOffsetY = rawY - params.y + } + + // shortcut: clamps against the metrics snapshot from the last rotation signal, so a drag that + // spans exactly the rotation event can briefly clamp to the old orientation; upgrade by reading + // Metrics in dragTo() if MLBB-in-multi-window drags become real. + private fun dragTo(rawX: Float, rawY: Float) { + val params = layoutParams ?: return + val button = view ?: return + val clamped = OverlayPosition.clampToScreen( + (rawX - grabOffsetX).toInt(), + (rawY - grabOffsetY).toInt(), + screenWidthPx, + screenHeightPx, + sizePx, + ) + params.x = clamped.left + params.y = clamped.top + runCatching { windowManager.updateViewLayout(button, params) } + } + + private fun endDrag() { + val params = layoutParams ?: return + val button = view ?: return + val snapped = OverlayPosition.snapToNearestEdge(params.x, params.y, screenWidthPx, screenHeightPx, sizePx) + params.x = snapped.left + params.y = snapped.top + runCatching { windowManager.updateViewLayout(button, params) } + + val normalized = OverlayPosition.pixelsToNormalized(snapped.left, snapped.top, screenWidthPx, screenHeightPx, sizePx) + currentNormalized = normalized + scope.launch { + runCatching { preferences.setFloatingPosition(normalized) } + .onFailure { Log.w(TAG, "failed to persist overlay position", it) } + } + Log.i(TAG, "overlay snapped to ${snapped.edge} (normalized ${normalized.x}, ${normalized.y})") + } + + private fun currentPublishStateForOverlay(): CapturePublishState = + publishStateFor(pinInPlace = CaptureStateHolder.frozenFrame.value != null, isPaused = CaptureStateHolder.state.value.isPaused) + + private fun toggleFreeze() { + // Unified with the notification and in-app controls: resume if live publication is + // already blocked (pin or pause), otherwise freeze (pin). Never creates a session. + when (floatingTapFor(currentPublishStateForOverlay())) { + FloatingTapOutcome.RESUME -> { + CaptureFreeze.resumeLive() + CaptureStateHolder.setPaused(false) + } + FloatingTapOutcome.FREEZE -> { + val outcome = CaptureFreeze.freezeCurrentFrame() + if (outcome == FreezeOutcome.NoFrame) { + showTransientMessage(context.getString(R.string.floating_control_no_frame_yet)) + } + } + } + // The service collects the shared state and repaints the glyph through setNotLive(). + } + + private fun showTransientMessage(message: String) { + mainHandler.post { + android.widget.Toast.makeText(context.applicationContext, message, android.widget.Toast.LENGTH_SHORT).show() + } + } + + private companion object { + const val TAG = "OverlayController" + const val SIZE_DP = 52 // within the 48–56dp visual range + const val DEFAULT_Y_FRACTION = 0.18f // upper area, away from the central draft region + } +} diff --git a/app/src/main/kotlin/com/draftlens/overlay/OverlayEnablement.kt b/app/src/main/kotlin/com/draftlens/overlay/OverlayEnablement.kt new file mode 100644 index 0000000..89d9e60 --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/overlay/OverlayEnablement.kt @@ -0,0 +1,13 @@ +package com.draftlens.overlay + +/** + * The one decision the shipped code makes about the floating control: may it be attached? + * Capture start and the UI refresh both evaluate this — the capture path calls it with a + * live session, the screen never attaches anything itself. + */ +object OverlayEnablement { + + /** True only when the user wants it, Android allows it, and capture is running. */ + fun shouldAttachOverlay(enabledPreference: Boolean, canDrawOverlays: Boolean, captureRunning: Boolean): Boolean = + enabledPreference && canDrawOverlays && captureRunning +} diff --git a/app/src/main/kotlin/com/draftlens/overlay/OverlayPosition.kt b/app/src/main/kotlin/com/draftlens/overlay/OverlayPosition.kt new file mode 100644 index 0000000..3a1f86f --- /dev/null +++ b/app/src/main/kotlin/com/draftlens/overlay/OverlayPosition.kt @@ -0,0 +1,70 @@ +package com.draftlens.overlay + +import kotlin.math.roundToInt + +/** + * Position math for the single circular floating control. + * + * Positions are stored as normalized fractions (0..1) of the *free axis* (screen span + * minus the button size), so one saved position survives rotation and density changes: + * portrait saves y against a tall screen, landscape re-derives y against a short one from + * the same fraction. Pixel math happens here, in pure Kotlin, and is unit-tested; the + * WindowManager layer only paints the result. + */ +object OverlayPosition { + + enum class HorizontalEdge { LEFT, RIGHT } + + /** Normalized position; x and y in 0..1, both measured against the free axis. */ + data class Normalized(val x: Float, val y: Float) + + /** Pixel position relative to the top-left of the screen. */ + data class Pixels(val left: Int, val top: Int) + + /** Pixel position after snapping the horizontal axis to an edge. */ + data class Snapped(val left: Int, val top: Int, val edge: HorizontalEdge) + + private fun freeSpan(screenSizePx: Int, sizePx: Int): Int = (screenSizePx - sizePx).coerceAtLeast(0) + + fun pixelsToNormalized(leftPx: Int, topPx: Int, screenWidthPx: Int, screenHeightPx: Int, sizePx: Int): Normalized { + val maxX = freeSpan(screenWidthPx, sizePx) + val maxY = freeSpan(screenHeightPx, sizePx) + val clampedLeft = leftPx.coerceIn(0, maxX) + val clampedTop = topPx.coerceIn(0, maxY) + return Normalized( + x = if (maxX == 0) 0f else clampedLeft.toFloat() / maxX, + y = if (maxY == 0) 0f else clampedTop.toFloat() / maxY, + ) + } + + fun normalizedToPixels(normalized: Normalized, screenWidthPx: Int, screenHeightPx: Int, sizePx: Int): Pixels { + val maxX = freeSpan(screenWidthPx, sizePx) + val maxY = freeSpan(screenHeightPx, sizePx) + val x = normalized.x.coerceIn(0f, 1f) + val y = normalized.y.coerceIn(0f, 1f) + return Pixels(left = (x * maxX).roundToInt(), top = (y * maxY).roundToInt()) + } + + /** Clamps an arbitrary pixel position back inside the screen. */ + fun clampToScreen(leftPx: Int, topPx: Int, screenWidthPx: Int, screenHeightPx: Int, sizePx: Int): Pixels = + Pixels( + left = leftPx.coerceIn(0, freeSpan(screenWidthPx, sizePx)), + top = topPx.coerceIn(0, freeSpan(screenHeightPx, sizePx)), + ) + + /** + * Snaps horizontally to the nearest edge by button-center comparison; the vertical + * position is preserved (already clamped to the screen). + */ + fun snapToNearestEdge(leftPx: Int, topPx: Int, screenWidthPx: Int, screenHeightPx: Int, sizePx: Int): Snapped { + val clamped = clampToScreen(leftPx, topPx, screenWidthPx, screenHeightPx, sizePx) + val center = clamped.left + sizePx / 2 + val atLeft = center <= screenWidthPx / 2 + return Snapped( + left = if (atLeft) 0 else freeSpan(screenWidthPx, sizePx), + top = clamped.top, + edge = if (atLeft) HorizontalEdge.LEFT else HorizontalEdge.RIGHT, + ) + } + +} diff --git a/app/src/main/res/drawable/ic_floating_frozen.xml b/app/src/main/res/drawable/ic_floating_frozen.xml new file mode 100644 index 0000000..c879972 --- /dev/null +++ b/app/src/main/res/drawable/ic_floating_frozen.xml @@ -0,0 +1,9 @@ + + + diff --git a/app/src/main/res/drawable/ic_floating_live.xml b/app/src/main/res/drawable/ic_floating_live.xml new file mode 100644 index 0000000..3f55a8e --- /dev/null +++ b/app/src/main/res/drawable/ic_floating_live.xml @@ -0,0 +1,9 @@ + + + diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index df1e2b8..d857d98 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -6,8 +6,12 @@ Stop LIVE FROZEN + PAUSED Frame frozen — the Vision Lab shows the pinned frame - Freeze current frame - Resume live capture - Not implemented in Sprint 0 + Capture paused — the last game frame is kept + Pause + Resume + Freeze the game frame + Resume live capture + No frame captured yet — open Mobile Legends first diff --git a/app/src/test/kotlin/com/draftlens/capture/CaptureDiagnosticsTest.kt b/app/src/test/kotlin/com/draftlens/capture/CaptureDiagnosticsTest.kt new file mode 100644 index 0000000..bf116b7 --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/capture/CaptureDiagnosticsTest.kt @@ -0,0 +1,38 @@ +package com.draftlens.capture + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class CaptureDiagnosticsTest { + + @Test fun `first events log then only periodic ones`() { + val diagnostics = CaptureDiagnostics(firstEvents = 3, every = 5) + + val logged = (1..20).map { diagnostics.sample("acquired") to diagnostics.shouldLog(it) } + .filter { it.second } + .map { it.first } + + assertEquals(listOf(1, 2, 3, 5, 10, 15, 20), logged) + } + + @Test fun `events are counted independently`() { + val diagnostics = CaptureDiagnostics(firstEvents = 1, every = 100) + + assertEquals(1, diagnostics.sample("gate-reject:contentHidden")) + assertEquals(1, diagnostics.sample("gate-reject:isActive")) + assertEquals(1, diagnostics.total("gate-reject:contentHidden")) + assertEquals(1, diagnostics.total("gate-reject:isActive")) + assertEquals(0, diagnostics.total("published")) + } + + @Test fun `reset clears all counters for a new session`() { + val diagnostics = CaptureDiagnostics() + diagnostics.sample("acquired") + diagnostics.reset() + + assertEquals(1, diagnostics.sample("acquired")) + assertEquals(0, diagnostics.total("published")) + } +} diff --git a/app/src/test/kotlin/com/draftlens/capture/CaptureStateHolderTest.kt b/app/src/test/kotlin/com/draftlens/capture/CaptureStateHolderTest.kt index 85a3790..d30eb69 100644 --- a/app/src/test/kotlin/com/draftlens/capture/CaptureStateHolderTest.kt +++ b/app/src/test/kotlin/com/draftlens/capture/CaptureStateHolderTest.kt @@ -227,6 +227,33 @@ class CaptureStateHolderTest { assertNull(CaptureStateHolder.latestFrame.value) } + @Test fun `pausing never clears the last valid frame or a pinned snapshot`() { + CaptureStateHolder.beginCapture(1080, 2400, 440) + val frame = testCapturedFrame() + CaptureStateHolder.recordFrame(frame) + CaptureStateHolder.setFrozenFrame(frame) + + CaptureStateHolder.setPaused(true) + + assertEquals(CaptureStatus.PAUSED, CaptureStateHolder.state.value.status) + assertEquals("pause must preserve the paused frame reference", frame, CaptureStateHolder.latestFrame.value) + assertEquals("pause must preserve the pinned snapshot", frame, CaptureStateHolder.frozenFrame.value) + } + + @Test fun `resume from paused returns to capturing with the pin intact`() { + CaptureStateHolder.beginCapture(1080, 2400, 440) + val frame = testCapturedFrame() + CaptureStateHolder.recordFrame(frame) + CaptureStateHolder.setFrozenFrame(frame) + CaptureStateHolder.setPaused(true) + + CaptureStateHolder.setPaused(false) + + assertEquals(CaptureStatus.CAPTURING, CaptureStateHolder.state.value.status) + assertEquals(frame, CaptureStateHolder.frozenFrame.value) + assertEquals(frame, CaptureStateHolder.latestFrame.value) + } + @Test fun `a frozen frame survives a capture stop`() { CaptureStateHolder.beginCapture(1080, 2400, 440) val frozen = testCapturedFrame() diff --git a/app/src/test/kotlin/com/draftlens/capture/CaptureVisibilityPolicyTest.kt b/app/src/test/kotlin/com/draftlens/capture/CaptureVisibilityPolicyTest.kt new file mode 100644 index 0000000..05a5fa3 --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/capture/CaptureVisibilityPolicyTest.kt @@ -0,0 +1,108 @@ +package com.draftlens.capture + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test + +class CaptureVisibilityPolicyTest { + + private lateinit var policy: CaptureVisibilityPolicy + + @Before fun setUp() { + policy = CaptureVisibilityPolicy() + policy.resetForNewSession(7) + } + + @Test fun `a session starts UNKNOWN and UNKNOWN deliberately allows publication`() { + assertEquals(CapturedContentVisibility.UNKNOWN, policy.currentState()) + assertTrue(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + } + + @Test fun `a true callback before the pipeline exists is preserved as VISIBLE`() { + val transition = policy.onContentVisibilityChanged(true, sessionId = 7) + + assertTrue(transition.accepted) + assertTrue(transition.changed) + assertEquals(CapturedContentVisibility.VISIBLE, policy.currentState()) + assertEquals(1L, policy.sequence()) + } + + @Test fun `an early false callback is kept but does not overwrite UNKNOWN with HIDDEN`() { + val transition = policy.onContentVisibilityChanged(false, sessionId = 7) + + assertTrue(transition.accepted) + assertEquals(1L, policy.sequence()) + // No frame exists yet, so there is no valid game frame to protect; bootstrap stays open. + assertEquals(CapturedContentVisibility.UNKNOWN, policy.currentState()) + assertTrue(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + } + + @Test fun `a published frame makes a later false callback authoritative`() { + assertTrue(policy.onFramePublished()) + assertEquals(CapturedContentVisibility.VISIBLE, policy.currentState()) + + val transition = policy.onContentVisibilityChanged(false, sessionId = 7) + + assertTrue(transition.changed) + assertEquals(CapturedContentVisibility.HIDDEN, policy.currentState()) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + } + + @Test fun `hidden to visible resumes publication`() { + policy.onContentVisibilityChanged(true, 7) + policy.onContentVisibilityChanged(false, 7) + assertFalse(policy.shouldPublish(true, true, false)) + + policy.onContentVisibilityChanged(true, 7) + + assertEquals(CapturedContentVisibility.VISIBLE, policy.currentState()) + assertTrue(policy.shouldPublish(true, true, false)) + } + + @Test fun `repeated callbacks are idempotent but increase the diagnostic sequence`() { + policy.onContentVisibilityChanged(true, 7) + val first = policy.onContentVisibilityChanged(false, 7) + val repeat = policy.onContentVisibilityChanged(false, 7) + + assertTrue(first.changed) + assertFalse(repeat.changed) + assertEquals(3L, policy.sequence()) + } + + @Test fun `callbacks from a previous session are rejected`() { + val transition = policy.onContentVisibilityChanged(false, sessionId = 6) + + assertFalse(transition.accepted) + assertEquals(0L, policy.sequence()) + assertEquals(CapturedContentVisibility.UNKNOWN, policy.currentState()) + } + + @Test fun `a new session resets state and sequence`() { + policy.onContentVisibilityChanged(true, 7) + policy.onContentVisibilityChanged(false, 7) + + policy.resetForNewSession(8) + + assertEquals(CapturedContentVisibility.UNKNOWN, policy.currentState()) + assertEquals(0L, policy.sequence()) + assertEquals(8, policy.boundSessionId()) + } + + @Test fun `frozen, paused and stale-generation conjuncts still block publication`() { + assertFalse(policy.shouldPublish(isActive = false, generationCurrent = true, frozen = false)) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = false, frozen = false)) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = true)) + } + + @Test fun `a resume requested while hidden opens publication on the next true callback`() { + policy.onContentVisibilityChanged(true, 7) + policy.onContentVisibilityChanged(false, 7) + // User intent switches to LIVE (pin cleared/pause cleared) while the window is hidden. + assertFalse(policy.shouldPublish(true, true, frozen = false)) + + policy.onContentVisibilityChanged(true, 7) + assertTrue(policy.shouldPublish(true, true, frozen = false)) + } +} diff --git a/app/src/test/kotlin/com/draftlens/capture/PendingResizeTest.kt b/app/src/test/kotlin/com/draftlens/capture/PendingResizeTest.kt new file mode 100644 index 0000000..7ed9b65 --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/capture/PendingResizeTest.kt @@ -0,0 +1,51 @@ +package com.draftlens.capture + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test + +class PendingResizeTest { + + private lateinit var pending: PendingResize + + @Before fun setUp() { + pending = PendingResize() + } + + @Test fun `no callback means no pending dimensions`() { + assertEquals(null, pending.peek()) + assertEquals(null, pending.consume()) + } + + @Test fun `an early resize callback is retained`() { + assertEquals(1L, pending.offer(1080, 2400)) + + assertEquals(1080 to 2400, pending.peek()) + assertEquals(1080 to 2400, pending.consume()) + assertEquals(null, pending.peek()) + } + + @Test fun `multiple early callbacks use the latest valid dimensions`() { + pending.offer(1080, 2400) + pending.offer(2400, 1080) + pending.offer(1200, 2600) + + assertEquals(3L, pending.sequence()) + assertEquals(1200 to 2600, pending.consume()) + } + + @Test fun `invalid dimensions are ignored`() { + assertEquals(0L, pending.offer(0, 2400)) + assertEquals(0L, pending.offer(1080, -1)) + assertEquals(null, pending.peek()) + } + + @Test fun `clear drops the buffered callback`() { + pending.offer(2400, 1080) + pending.clear() + + assertEquals(null, pending.consume()) + } +} diff --git a/app/src/test/kotlin/com/draftlens/capture/PublishStateTest.kt b/app/src/test/kotlin/com/draftlens/capture/PublishStateTest.kt new file mode 100644 index 0000000..d649bd5 --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/capture/PublishStateTest.kt @@ -0,0 +1,44 @@ +package com.draftlens.capture + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +/** + * Pins the shared LIVE/PAUSED/FROZEN presentation both surfaces (notification, overlay) + * derive their action from, plus the tap/toggle outcomes. + */ +class PublishStateTest { + + @Test fun `live publishing derives LIVE`() { + assertEquals(CapturePublishState.LIVE, publishStateFor(pinInPlace = false, isPaused = false)) + } + + @Test fun `paused derives PAUSED`() { + assertEquals(CapturePublishState.PAUSED, publishStateFor(pinInPlace = false, isPaused = true)) + } + + @Test fun `pinned frame wins over pause for presentation`() { + assertEquals(CapturePublishState.FROZEN, publishStateFor(pinInPlace = true, isPaused = true)) + assertEquals(CapturePublishState.FROZEN, publishStateFor(pinInPlace = true, isPaused = false)) + } + + @Test fun `live state offers Pause, every other state offers Resume`() { + assertEquals(CapturePublishAction.PAUSE, publishActionFor(CapturePublishState.LIVE)) + assertEquals(CapturePublishAction.RESUME, publishActionFor(CapturePublishState.PAUSED)) + assertEquals(CapturePublishAction.RESUME, publishActionFor(CapturePublishState.FROZEN)) + } + + @Test fun `overlay tap freezes when live and resumes otherwise`() { + assertEquals(FloatingTapOutcome.FREEZE, floatingTapFor(CapturePublishState.LIVE)) + assertEquals(FloatingTapOutcome.RESUME, floatingTapFor(CapturePublishState.PAUSED)) + assertEquals(FloatingTapOutcome.RESUME, floatingTapFor(CapturePublishState.FROZEN)) + } + + @Test fun `publication is blocked in both paused and frozen`() { + assertFalse(publishStateBlocksPublication(CapturePublishState.LIVE)) + assertTrue(publishStateBlocksPublication(CapturePublishState.PAUSED)) + assertTrue(publishStateBlocksPublication(CapturePublishState.FROZEN)) + } +} diff --git a/app/src/test/kotlin/com/draftlens/capture/VisibilityFreezeSequenceTest.kt b/app/src/test/kotlin/com/draftlens/capture/VisibilityFreezeSequenceTest.kt new file mode 100644 index 0000000..3160b0f --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/capture/VisibilityFreezeSequenceTest.kt @@ -0,0 +1,112 @@ +package com.draftlens.capture + +import com.draftlens.vision.testCapturedFrame +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test + +/** + * End-to-end JVM sequence for the app-only capture workflow, at the level the JVM can + * reach: the pure visibility gate plus the process-local holder. The ImageReader half of + * the gate lives in FramePipeline (Android-only) and is covered on-device. + */ +class VisibilityFreezeSequenceTest { + + private lateinit var policy: CaptureVisibilityPolicy + + @Before fun setUp() { + CaptureStateHolder.reset() + policy = CaptureVisibilityPolicy() + policy.resetForNewSession(3) + } + + @Test fun `hidden freeze resume visible sequence`() { + // Capture is running and a valid game frame is published. + CaptureStateHolder.beginCapture(1080, 2400, 440) + val frame = testCapturedFrame() + CaptureStateHolder.recordFrame(frame) + + // MLBB goes hidden (user is in DraftLens): nothing publishes, last valid frame stays. + policy.onContentVisibilityChanged(true, 3) + policy.onContentVisibilityChanged(false, 3) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + assertTrue(CaptureStateHolder.latestFrame.value === frame) + + // User freezes from the floating button while hidden: pin kept independently, still not published. + CaptureStateHolder.setFrozenFrame(frame) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = true)) + + // User taps Resume while hidden: the intent flips to LIVE, but publication stays + // gated until the window returns — no black/DraftLens frames may be shown. + CaptureStateHolder.clearFrozenFrame() + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + + // MLBB returns: the remembered live intent takes effect, exactly once visibility flips. + policy.onContentVisibilityChanged(true, 3) + assertTrue(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + + // Explicit freeze while visible blocks publication again (the round trip both ways). + CaptureStateHolder.setFrozenFrame(frame) + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = true)) + } + + @Test fun `a paused capture publishes nothing even when the window is visible`() { + CaptureStateHolder.beginCapture(1080, 2400, 440) + CaptureStateHolder.recordFrame(testCapturedFrame()) + policy.onContentVisibilityChanged(true, 3) + CaptureStateHolder.setPaused(true) + + // isActive() is false while PAUSED; publication stays blocked while the window is visible. + assertFalse(policy.shouldPublish(isActive = false, generationCurrent = true, frozen = false)) + + CaptureStateHolder.setPaused(false) + assertTrue(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + } + + @Test fun `an explicitly frozen session does not auto-resume on visibility`() { + CaptureStateHolder.beginCapture(1080, 2400, 440) + val frame = testCapturedFrame() + CaptureStateHolder.recordFrame(frame) + policy.onContentVisibilityChanged(true, 3) + policy.onContentVisibilityChanged(false, 3) + CaptureStateHolder.setFrozenFrame(frame) + + policy.onContentVisibilityChanged(true, 3) + + assertFalse(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = true)) + // The pin is untouched; returning to the app shows it, not live frames. + org.junit.Assert.assertEquals(frame, CaptureStateHolder.frozenFrame.value) + } + + @Test fun `early hidden callback does not block the first frame but blocks replacement afterwards`() { + // The user started capture from DraftLens: the first callback can be false while the + // selected MLBB window is not yet on screen. That must not block the bootstrap frame. + val early = policy.onContentVisibilityChanged(false, 3) + assertEquals(CapturedContentVisibility.UNKNOWN, early.current) + assertTrue(policy.shouldPublish(isActive = true, generationCurrent = true, frozen = false)) + + policy.onFramePublished() + assertEquals(CapturedContentVisibility.VISIBLE, policy.currentState()) + + policy.onContentVisibilityChanged(false, 3) + assertFalse("after a valid frame, HIDDEN must protect it", policy.shouldPublish(true, true, false)) + } + + @Test fun `hidden then paused then visible stays paused and keeps the pinned frame`() { + CaptureStateHolder.beginCapture(1080, 2400, 440) + val frame = testCapturedFrame() + CaptureStateHolder.recordFrame(frame) + policy.onContentVisibilityChanged(true, 3) + policy.onContentVisibilityChanged(false, 3) + CaptureStateHolder.setFrozenFrame(frame) + CaptureStateHolder.setPaused(true) + + policy.onContentVisibilityChanged(true, 3) + + assertFalse(policy.shouldPublish(isActive = false, generationCurrent = true, frozen = true)) + assertEquals(frame, CaptureStateHolder.frozenFrame.value) + assertTrue(CaptureStateHolder.state.value.isPaused) + } +} diff --git a/app/src/test/kotlin/com/draftlens/overlay/OverlayEnablementTest.kt b/app/src/test/kotlin/com/draftlens/overlay/OverlayEnablementTest.kt new file mode 100644 index 0000000..3117b2a --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/overlay/OverlayEnablementTest.kt @@ -0,0 +1,30 @@ +package com.draftlens.overlay + +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +/** Pins the gate both production callers (capture start, UI refresh) evaluate. */ +class OverlayEnablementTest { + + @Test fun `attach requires preference, permission and a running session`() { + assertTrue(OverlayEnablement.shouldAttachOverlay(true, true, true)) + assertFalse(OverlayEnablement.shouldAttachOverlay(false, true, true)) + assertFalse(OverlayEnablement.shouldAttachOverlay(true, false, true)) + assertFalse(OverlayEnablement.shouldAttachOverlay(true, true, false)) + } + + @Test fun `permission denial never attaches, whatever the preference says`() { + assertFalse(OverlayEnablement.shouldAttachOverlay(enabledPreference = true, canDrawOverlays = false, captureRunning = true)) + assertFalse(OverlayEnablement.shouldAttachOverlay(enabledPreference = true, canDrawOverlays = false, captureRunning = false)) + assertFalse(OverlayEnablement.shouldAttachOverlay(enabledPreference = false, canDrawOverlays = false, captureRunning = true)) + } + + @Test fun `a missing preference never attaches, even with the permission granted`() { + assertFalse(OverlayEnablement.shouldAttachOverlay(enabledPreference = false, canDrawOverlays = true, captureRunning = true)) + } + + @Test fun `a stopped capture never attaches, even when everything else is ready`() { + assertFalse(OverlayEnablement.shouldAttachOverlay(enabledPreference = true, canDrawOverlays = true, captureRunning = false)) + } +} diff --git a/app/src/test/kotlin/com/draftlens/overlay/OverlayPositionTest.kt b/app/src/test/kotlin/com/draftlens/overlay/OverlayPositionTest.kt new file mode 100644 index 0000000..c3e32ac --- /dev/null +++ b/app/src/test/kotlin/com/draftlens/overlay/OverlayPositionTest.kt @@ -0,0 +1,86 @@ +package com.draftlens.overlay + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class OverlayPositionTest { + + private val size = 150 // px, ~52dp at xxhdpi + private val portraitW = 1080 + private val portraitH = 2400 + private val landscapeW = 2400 + private val landscapeH = 1080 + + @Test fun `pixel to normalized to pixel round-trips`() { + val px = OverlayPosition.Pixels(left = 900, top = 300) + val normalized = OverlayPosition.pixelsToNormalized(px.left, px.top, portraitW, portraitH, size) + val back = OverlayPosition.normalizedToPixels(normalized, portraitW, portraitH, size) + + assertEquals(px.left, back.left) + assertEquals(px.top, back.top) + } + + @Test fun `normalized position survives portrait to landscape round-trip`() { + val portrait = OverlayPosition.Pixels(left = 40, top = 600) + val normalized = OverlayPosition.pixelsToNormalized(portrait.left, portrait.top, portraitW, portraitH, size) + val landscape = OverlayPosition.normalizedToPixels(normalized, landscapeW, landscapeH, size) + val normalizedAgain = OverlayPosition.pixelsToNormalized(landscape.left, landscape.top, landscapeW, landscapeH, size) + + assertEquals(normalized.x, normalizedAgain.x, 0.001f) + assertEquals(normalized.y, normalizedAgain.y, 0.01f) + assertTrue(landscape.top in 0..(landscapeH - size)) + } + + @Test fun `out-of-screen positions are clamped inside`() { + val normalized = OverlayPosition.pixelsToNormalized(-500, 99999, portraitW, portraitH, size) + val px = OverlayPosition.normalizedToPixels(normalized, portraitW, portraitH, size) + + assertEquals(0, px.left) + assertEquals(portraitH - size, px.top) + } + + @Test fun `snap chooses left edge when the button center is on the left half`() { + val snapped = OverlayPosition.snapToNearestEdge(leftPx = 100, topPx = 800, screenWidthPx = portraitW, screenHeightPx = portraitH, sizePx = size) + + assertEquals(0, snapped.left) + assertEquals(OverlayPosition.HorizontalEdge.LEFT, snapped.edge) + assertEquals(800, snapped.top) + } + + @Test fun `snap chooses right edge when the button center is on the right half`() { + val snapped = OverlayPosition.snapToNearestEdge(leftPx = 800, topPx = 800, screenWidthPx = portraitW, screenHeightPx = portraitH, sizePx = size) + + assertEquals(portraitW - size, snapped.left) + assertEquals(OverlayPosition.HorizontalEdge.RIGHT, snapped.edge) + } + + @Test fun `snap at the exact halfway point resolves to left (stable tie-break)`() { + val leftPx = portraitW / 2 - size / 2 // center exactly on the half + val snapped = OverlayPosition.snapToNearestEdge(leftPx, 500, portraitW, portraitH, size) + + assertEquals(OverlayPosition.HorizontalEdge.LEFT, snapped.edge) + } + + @Test fun `degenerate screen dimensions never produce NaN`() { + val normalized = OverlayPosition.pixelsToNormalized(10, 10, screenWidthPx = 0, screenHeightPx = 0, sizePx = size) + val px = OverlayPosition.normalizedToPixels(normalized, 0, 0, size) + + assertEquals(0f, normalized.x) + assertEquals(0f, normalized.y) + assertEquals(0, px.left) + assertEquals(0, px.top) + } + + @Test fun `button larger than the screen still clamps to the origin`() { + val normalized = OverlayPosition.pixelsToNormalized(50, 50, screenWidthPx = 100, screenHeightPx = 100, sizePx = 150) + val px = OverlayPosition.normalizedToPixels(normalized, 100, 100, 150) + + assertEquals(0f, normalized.x) + assertEquals(0f, normalized.y) + assertEquals(0, px.left) + assertEquals(0, px.top) + } + + +} diff --git a/docs/IMPLEMENTATION_REPORT.md b/docs/IMPLEMENTATION_REPORT.md index a6e3c91..76a22a9 100644 --- a/docs/IMPLEMENTATION_REPORT.md +++ b/docs/IMPLEMENTATION_REPORT.md @@ -386,6 +386,8 @@ review-fix pass and how it was verified. - **Part A — Freeze from the notification.** New `CaptureFreeze` bridge (service command + in-app button are one code path). The ongoing capture notification now exposes **Freeze current frame** (LIVE state) / **Resume live capture** (FROZEN state) plus + (superseded by the Sprint 1C follow-up: the notification now carries **Pause**/**Resume**) + **Stop**. The command goes to the running service via an explicit, immutable `PendingIntent.getService` (`ACTION_FREEZE` / `ACTION_RESUME_LIVE`); it never launches an Activity, never re-promotes the service, never re-requests consent and never creates a @@ -562,3 +564,167 @@ the matcher/Vision Lab are unaffected (this bug is in the capture pipeline only) - If anything still shows black after this fix, the new logs pin down whether acquisition (`generation=... plane:` line) or conversion (`... frame stored: ... bitmap:` lines) is the source — see `docs/SCREEN_CAPTURE.md` → "Diagnosing the landscape bug with logs". + +--- + +# Sprint 1C — App-Only Capture & Circular Floating Control + +**Branch:** `sprint-1c-app-only-capture` · stacked on PR #3 (`sprint-1b-hero-recognition`, unmerged) · **PR:** #4 (base `sprint-1b-hero-recognition`) +**Spec:** maintainer Sprint 1C brief · **Design:** `docs/superpowers/specs/2026-11-16-sprint-1c-app-only-capture-design.md` + +## What shipped + +- Single-app (API 34+ partial) capture is now the recommended MLBB mode; the system chooser + is never forced. `onCapturedContentVisibilityChanged` drives frame publication: while the + selected window is hidden, images are acquired and closed but never converted or published; + the last valid game frame (and, if pinned, the frozen snapshot) is preserved. Publication + resumes on visibility return unless a frame is pinned. Full-display capture remains the + documented fallback with an explicit no-notification-free-promise note. +- One circular 52dp floating button (`TYPE_APPLICATION_OVERLAY`), shown only while a capture + session runs and only after the user grants the overlay permission (requested from the + capture screen, never at install time). Tap toggles LIVE (green) / FROZEN (orange) by calling + the shared `CaptureFreeze` path; 400 ms long-press arms drag; release snaps to the nearest + horizontal edge and persists normalized coordinates (DataStore). Rotation re-anchors it. +- Overlay is non-focusable, exactly button-sized, and removed on every teardown path + (stop, error, revoke, task removal, resize failure). +- Guidance rewritten: the "choose Entire screen" instruction is gone from the debug and + Vision Lab screens. +- No recommendation UI, no panel, no menu, no accessibility service; `SYSTEM_ALERT_WINDOW` + declared, requested only on opt-in. + +## Verification + +- Implementation head `f992d45`: CI run 43784157138 — **886 unit tests, 0 failures / 0 errors / 0 skipped**; + strict lint green; `assembleDebug` green; APK `app-debug.apk` + sha256 `1d1f50629e145d48e64ed56d0cf74228e7d48d4c6f94fde81eb6c22fc4795810`. Docs-only commits after + it change no build inputs and are re-checked by CI on every push. +- Instrumentation: compiled via `assembleAndroidTest` in CI; not executed — no KVM on the runner +- Physical device (rows 33–43 of `docs/TESTING_PHYSICAL_DEVICE.md`): NOT RUN — maintainer task; + in particular the overlay-vs-capture isolation finding (rows 34/35) is an on-device question + that CI cannot answer; no exclusion claim is made. + +## Known limitations / compatibility notes + +- API < 34: no visibility signal; app-only windows are undetectable — behavior degrades to + full-display semantics. +- If the captured window is already hidden when a session starts, publication stays open + until Android's first visibility callback arrives (API 34+; typically milliseconds). The + gate defaults to open because full-display projections may never call the callback at all. +- Full-display fallback may capture the shade/system UI/other apps/DraftLens; the freeze + controls still work. +- Whether the overlay appears inside captures is unverified per device (see sprint doc). +- A resume requested while MLBB is hidden flips the user's intent to LIVE and takes effect + when MLBB returns; publication stays gated while hidden (no black/DraftLens frames). +- Resize during freeze is enforced by the pipeline's freeze gate (policy truth-table + + `CaptureVisibilityPolicyTest`) and by on-device rows 25/30; no synthetic-pixel + instrumented test exists — an ImageReader surface cannot be filled from a test Canvas. + +--- + +# Sprint 1C — Follow-up fixes (notification Pause, permissions-only UI) + +**Branch:** `sprint-1c-app-only-capture` (stacked on PR #3, unmerged) · **PR:** #4 + +## Root cause of the overlay misbehaving during app-only capture + +Full trace (callback → policy → pipeline → freeze → overlay) shows the overlay's lifecycle +was never coupled to capture visibility — render, touch and window lifetime have no frame +input. Three real defects were causing the observed "frozen" control: + +1. silent no-op tap on `NoFrame` in fresh app-only sessions (nothing published yet); +2. overlay glyph and notification derived only from `frozenFrame`, so a non-pin publication + block (PAUSED) showed LIVE/green everywhere while publishing nothing; +3. no single derivation for "is live publication happening" across surfaces. + +Fixed by: unified `CapturePublishState` (publishStateFor / publishActionFor / +floatingTapFor) shared by notification and overlay; combined frozen-frame + session-state +collector; toast feedback on `NoFrame`; OEM/immersive-game overlay coverage documented as a +hardware finding (device rows 44–53, NOT RUN), with no masking or mode fallback shipped. + +## Notification: Freeze → Pause + +Stop + one state action: Pause (LIVE) / Resume (PAUSED or FROZEN). Pause maps onto the +existing `CaptureStatus.PAUSED` pipeline gate — no second projection, no second state, latest +valid frame preserved. Resume = `resumeLive()` + `setPaused(false)`. + +## UI cleanup + +General instructions removed: capture-mode tutorial card, floating-control tutorial card, +in-game freeze help texts, stale Sprint-0 marker. Kept: notification permission (request, +status, settings action) and overlay permission (request, status, settings action) in one +compact card. + +## Verification + +- Implementation head `130ce6c`: CI run 43922482183 — **904 unit tests, 0 failures / 0 errors / 0 skipped**; + strict lint green; `assembleAndroidTest` green; `assembleDebug` green; APK `app-debug.apk` + sha256 `48b510673919fe2caa9c906747d9dc766eb27bb170c589f67b58c9a175dd7b72`. Docs-only commits after + it change no build inputs and are re-checked by CI on every push. +- Instrumented tests: compiled via `assembleAndroidTest`; not executed (no KVM). +- Physical device rows 44–53: **NOT RUN** — see `docs/TESTING_PHYSICAL_DEVICE.md`. + +--- + +# Sprint 1C — Single-app first-frame bug fix + +**Branch:** `sprint-1c-app-only-capture` (stacked on PR #3, unmerged) · **PR:** #4 + +## Root cause (code-level evidence) + +The Sprint 1B capture path had no visibility gate. Sprint 1C introduced one and registered +`MediaProjection.Callback` **before** `FramePipeline.start()`, `createVirtualDisplay()` and +`CaptureStateHolder.beginCapture()`. On single-app capture both callbacks can arrive in +that STARTING window: + +1. `onCapturedContentVisibilityChanged(false)` — the selected MLBB window is hidden while + the user is still in DraftLens. With the old boolean gate, that value either got + overwritten by `pipeline.start()`'s unconditional `true` or closed the gate before any + frame existed. A missed/first-only callback leaves publication blocked with no valid + frame to protect — the "No frame captured yet" toast. +2. `onCapturedContentResize(W,H)` with the *selected app's* dimensions — `recreateDisplay` + either ignored it (status STARTING) or terminated the session as "VirtualDisplay + disappeared during resize" when it arrived before the VirtualDisplay existed. The + reader stayed at DraftLens' window metrics. +3. `frameJob` also dropped frames published before `beginCapture()` from the holder. + +This is one ordering defect across both callbacks, not three unrelated bugs: startup events +were not buffered and the gate's initial boolean could be confused with a real callback. + +## Diagnostics (added before changing the policy) + +`CaptureDiagnostics` rate-limits every event key (first 4 + every 20; no pixel data). +`CaptureSessionService` logs callback seq/value/pipelineInitialized/status and resize +seq/dimensions/readiness. `FramePipeline` logs acquisition count, image dimensions, +generation, isActive/generationCurrent/frozen/contentVisible, the exact gate-reject reason, +sampling accept/reject, conversion success/failure and bitmap dimensions, publication and +holder delivery. `freezeAndSnapshot` logs the state behind `NoFrame`. These log lines are +the evidence the maintainer needs to answer: "did ImageReader receive images that our code +rejected, or did the platform deliver none?" — see `docs/TESTING_PHYSICAL_DEVICE.md`, +"Focused first-frame test". + +## Fix + +- `CaptureVisibilityPolicy` now uses explicit `UNKNOWN` / `VISIBLE` / `HIDDEN`. UNKNOWN + deliberately allows publication (full-display never calls back; an early single-app + `false` must not block the first frame). The first published frame promotes + UNKNOWN → VISIBLE, after which `false` is authoritative and protects the valid frame. + The policy is owned by the service, so callbacks before pipeline creation are preserved; + stale-session callbacks are rejected; duplicate callbacks are idempotent. +- `FramePipeline` no longer resets visibility in `start()`/`prepareResize`; it consults the + injected policy, promotes on publish, and no longer duplicates callback state. +- `PendingResize` buffers the latest valid captured-content dimensions during startup. + `initializeProjection` sizes both ImageReader and VirtualDisplay from the buffered value + when present, applies any later value after `beginCapture()`, stays idempotent, and no + longer fails the session for a resize that arrived before the VirtualDisplay existed. +- `frameJob` records every published frame (the pipeline clears `latestFrame` on stop). + +## Verification + +- Implementation head `3c7b58c`: CI run 43936524131 — **926 unit tests, 0 failures / 0 errors / 0 skipped**; + strict lint green; `assembleAndroidTest` green; `assembleDebug` green; APK `app-debug.apk` + sha256 `ade9ec10c5d6b9d4e372b325e6b521f87170d0fa5273c9b30e9ea9063a763070`. Docs-only commits + after it change no build inputs and are re-checked by CI on every push. +- Instrumented tests: compiled, not executed (no KVM on the runner). +- Physical device: the A–E first-frame checklist and rows 44–53 of + `docs/TESTING_PHYSICAL_DEVICE.md` are **NOT RUN**. This fix is not claimed to work on a + device from JVM tests or CI alone. diff --git a/docs/SCREEN_CAPTURE.md b/docs/SCREEN_CAPTURE.md index 407f0b5..dafd403 100644 --- a/docs/SCREEN_CAPTURE.md +++ b/docs/SCREEN_CAPTURE.md @@ -1,4 +1,4 @@ -# Screen Capture — DraftLens Sprint 0.1.3 + 1B +# Screen Capture — DraftLens Sprint 0.1.3 + 1B + 1C ## Mechanism `MediaProjectionManager.createScreenCaptureIntent()` → user consent → `ServiceCompat.startForeground()` @@ -6,24 +6,61 @@ with `ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION` → `getMediaProject `createVirtualDisplay()` → `ImageReader` surface. Consent acquisition stays in Activity; `CaptureSessionService` performs foreground promotion, projection creation, VirtualDisplay and all frame work. -## Freezing the game frame from the notification (Sprint 1B) -The ongoing capture notification carries an action (**Freeze current frame**) implemented as -an explicit `PendingIntent.getService` back to the running service — no Activity launch, no -consent re-request, no second projection or VirtualDisplay. The service snapshots the latest -validated frame into an independently owned copy (`FrameSnapshots`), keeps the projection -valid, and re-posts the notification as FROZEN with **Resume live capture**. Images arriving -while frozen are acquired and closed but not converted or published. The freeze can equally be -triggered from the Vision Lab button; both run `CaptureFreeze.freezeCurrentFrame()`. After an -Android-initiated stop the pinned frame survives in process memory for editing, but not -process death. +## The floating freeze button (Sprint 1C) + +A single circular 52dp button rendered via `WindowManager` + `TYPE_APPLICATION_OVERLAY`, +attached by the capture foreground service **only while a capture session is running** and +removed on every stop/terminate path — a detach() invalidates any in-flight attach, so a +teardown can never orphan the window. It is the primary in-game freeze control; the +notification carries Stop and Pause and is the no-overlay fallback. + +Gestures: **tap** toggles LIVE (green, publishing) ↔ orange (publication paused or a frame +pinned); the glyph and color are driven only by the shared LIVE/PAUSED/FROZEN presentation +(`CapturePublishState`), never by incoming frames — hiding the captured window cannot pause, +detach or stale the button. **Long-press (400 ms) then drag** moves it; any movement before +the long-press fires cancels the gesture entirely. On release it **snaps to the nearest +horizontal edge** and persists its position as normalized coordinates in DataStore +(`floating_control_enabled`, `floating_position_x/y`). Rotation re-anchors it via the +display-metrics path; no re-consent, no re-attachment races. The overlay lifecycle is fully +independent of capture visibility: `onCapturedContentVisibilityChanged` gates only the +frame pipeline. + +`SYSTEM_ALERT_WINDOW` is requested only when the user enables the floating button on the +capture screen (Android overlay-settings screen); the capture itself never needs it. If the +permission is missing or revoked: capture and the in-app Freeze/Resume and the notification +Stop/Pause actions are unaffected, the capture screen explains how to grant it, and no +button is shown. The button calls the exact same shared pause/freeze path as everything +else — it never creates a second VirtualDisplay or re-requests consent. + +**Isolation caveat:** the overlay sets no flags to hide itself, and no capture region is +masked. Whether it appears inside single-app or full-display captures is a +physical-device finding per OEM — not verified in CI. See +`docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md` and rows 33–43 of +`docs/TESTING_PHYSICAL_DEVICE.md`. + +## Capture notification: Stop and Pause (Sprint 1C) +The ongoing capture notification carries **Stop** and one state action — **Pause** while live, +**Resume** when paused or frozen — implemented as explicit `PendingIntent.getService`s back to +the running service — no Activity launch, no consent re-request, no second projection or +VirtualDisplay. **Pause** maps onto the shared capture pause state (`CaptureStatus.PAUSED`): +the pipeline stops converting/publishing (images are acquired and closed), the latest valid +frame is preserved, MediaProjection and the VirtualDisplay keep running, and nothing replaces +the paused frame. **Resume** clears any pinned frame (`CaptureFreeze.resumeLive()`) and clears +the pause (`setPaused(false)`), returning to live publication. The notification shows exactly +one of Pause/Resume plus Stop, reflects LIVE / PAUSED / FROZEN, and stays in sync with the +floating overlay and the in-app controls through the same `CapturePublishState` derivation. +Images arriving while frozen are acquired and closed but not converted or published. The +in-app Freeze/Resume (pinning an independently owned snapshot via `CaptureFreeze` / +`FrameSnapshots`) is unchanged. After an Android-initiated stop the pinned frame survives in +process memory for editing, but not process death. ## Android version notes (compile/target SDK 35) - Android 13+ `POST_NOTIFICATIONS` may be denied; capture still proceeds with a valid ongoing foreground notification that Android may hide. In-app status and Stop control remain - available. The **Freeze current frame** notification action is not reachable while the - notification is hidden, so the app checks the permission (and requests it on Android 13+) - and says so on the capture screen; without it, freezing is done from the Vision Lab - (or with the notification enabled first). + available. The capture notification's **Stop** and **Pause** actions are not reachable while + the notification is hidden, so the app checks the permission (and requests it on Android 13+) + and says so on the capture screen; without it, pausing/freezing is done from the in-app + controls (or with the notification enabled first). - **Android 14+**: service calls `startForeground` with the `mediaProjection` type **before** `getMediaProjection()`; the consent token is single-session and dies with the service. `MediaProjection.Callback.onStop()` (revocation) and `onCapturedContentResize()` (rotation) @@ -35,19 +72,29 @@ process death. default display's metrics change, the capture size is re-derived from the current window metrics and the same resize path runs. Both paths are idempotent and deduped by `DisplayChangePolicy`. -- **Single-app capture** (sharing one window instead of the full screen): the captured - window's size is known only through `onCapturedContentResize` (API 34+). Until the callback - fires, the VirtualDisplay uses the full-screen metrics; when it fires, capture is resized - to the window's content size. Below API 34 single-app size changes cannot be tracked and - are not tracked; use full-screen capture for the MLBB workflow. -- **onCapturedContentVisibilityChanged is not handled**: the app neither reacts to nor - depends on single-app visibility transitions; size changes flow through the resize path. +- **Single-app capture (API 34+ partial capture)**: supported and preferred for the MLBB workflow — + the system chooser offers it; DraftLens never forces it. Prefer **Single app / app window → + Mobile Legends: Bang Bang**. The captured window's size is known through + `onCapturedContentResize` (API 34+) and flows through the resize path; its on-screen + visibility flows through `onCapturedContentVisibilityChanged` (see below). Full-display + capture remains the fallback for older Android and behaves exactly as before. +- **onCapturedContentVisibilityChanged (API 34+) IS handled (Sprint 1C)**: while the + selected window is hidden, images are acquired and closed but never converted or + published; the last valid published game frame is preserved. When the window returns, + publication resumes automatically unless a frame is pinned. Full-display captures never + report hidden, so this path is inert there. Below API 34 the signal does not exist and + the behavior degrades to the previous full-display semantics. The callback is fed to the + pipeline regardless of session status, so a hide during PAUSED cannot stale the gate on + resume. If the captured window is already hidden when a session starts, publication stays + open until the first visibility callback arrives (typically milliseconds) — the gate + defaults to open because full-display projections may never call the callback at all. - **Capture dimensions:** API 30+ uses `WindowManager.maximumWindowMetrics`; lower APIs retain the legacy `defaultDisplay.getSize()` path. - **Android 12+**: partial screen capture (share one app) is possible in the system dialog. - DraftLens requests full-screen capture; choosing app-window capture for the Mobile Legends - workflow shows DraftLens itself, not the game — choose **Entire screen / full device - display**. The app tells the user this in both capture screens. + For the MLBB workflow, **prefer Single app → Mobile Legends**; choose **Entire screen / + full device display** only when that option is missing. Full-display capture may include + the notification shade, Quick Settings, other apps and DraftLens itself — DraftLens + cannot promise notification-free capture on that path. The app says this on screen too. - **minSdk 26**: `ImageReader`, `VirtualDisplay`, and foreground-service APIs used are all available since API 21–26; no version-branched capture code is needed. diff --git a/docs/TESTING_PHYSICAL_DEVICE.md b/docs/TESTING_PHYSICAL_DEVICE.md index 3c02171..540872a 100644 --- a/docs/TESTING_PHYSICAL_DEVICE.md +++ b/docs/TESTING_PHYSICAL_DEVICE.md @@ -32,11 +32,11 @@ Status values: **PASS**, **FAIL**, **PARTIAL**, **NOT RUN**. Record the commit S | 11 | Sustained capture memory/latency | Memory bounded; frame processing latency reported separately from detection latency | NOT RUN | | 12 | DraftLens foreground vs MLBB foreground | Selected capture target is clear; preview corresponds to chosen scope | NOT RUN | | 13 | Screenshots in app | System screenshot path works normally | NOT RUN | -| 14 | Full-device capture for the MLBB freeze workflow | When Android asks what to capture, **Entire screen / full device display** is chosen; live preview shows MLBB content | NOT RUN | -| 15 | Freeze from the notification while in MLBB | Tap **Freeze current frame** on the capture notification without leaving the game; the pinned frame survives returning to DraftLens; Vision Lab shows the MLBB frame, not the DraftLens UI | NOT RUN | +| 14 | Full-device capture for the MLBB freeze workflow | When Android asks what to capture, **Entire screen / full device display** is chosen (fallback mode; preferred is Single app → MLBB — see rows 33–43); live preview shows MLBB content | NOT RUN | +| 15 | Pause from the notification while in MLBB | Tap **Pause** on the capture notification without leaving the game; the last valid frame is kept, publication stops, projection persists; the notification flips to Resume | NOT RUN | | 16 | Newer live frames cannot replace the frozen frame | While frozen, DraftLens keeps acquiring/closing images but publishes none; Vision Lab keeps showing the MLBB frame | NOT RUN | | 17 | Resume from Vision Lab and from the notification | **Resume** (Vision Lab) or **Resume live capture** (notification) reveals live frames without a new projection or VirtualDisplay; notification action flips back to LIVE | NOT RUN | -| 18 | Freeze with no frame captured yet | **Freeze current frame** is a no-op (logged); nothing is pinned; no crash, no new session | NOT RUN | +| 18 | Freeze with no frame captured yet (in-app / floating button) | **Freeze** is a no-op (logged); nothing is pinned; the tap surfaces a “No frame captured yet” toast; no crash, no new session | NOT RUN | | 19 | Duplicate taps of Freeze / Resume | Idempotent: the first pinned frame is kept; notification stays FROZEN | NOT RUN | | 20 | Notifications disabled | Freeze action unreachable from the shade; the app warns and offers to enable them; in-app Freeze/Resume still work | NOT RUN | | 21 | Stop with a frame frozen | Projection and resources are released; the frozen frame remains available in Vision Lab until Resume/Clear or a new capture session; it does not survive process death | NOT RUN | @@ -51,13 +51,96 @@ Status values: **PASS**, **FAIL**, **PARTIAL**, **NOT RUN**. Record the commit S | 30 | Freeze after resize | After a portrait→landscape resize, tap Freeze: pinned frame is the landscape frame at the new dimensions, complete | NOT RUN | | 31 | Mismatch diagnosis | Intentionally start capture with the game in landscape, then force-stop rotation events via lock/unlock: collect `CaptureSessionService`/`FramePipeline` logcat and verify dimensions, cropRect, rowStride and generation lines; confirm no frame mixes generations | NOT RUN | | 32 | Single-app share resize (API 34+) | Share only the MLBB window: capture resizes to the window's content size via `onCapturedContentResize`; the app logs the resize; releasing the app restores the full share | NOT RUN | +| 33 | Floating control appears while capturing | Enable it on the capture screen (grant the overlay permission when prompted); start capture, switch to MLBB; the circular button is visible, green, ~52dp | NOT RUN | +| 34 | Overlay absent from single-app capture | With single-app MLBB capture, confirm the button is **not** visible in the captured frame (logcat: `OverlayController` + Vision Lab pinned frame) | NOT RUN | +| 35 | Overlay present in full-display fallback | Repeat the capture-frame check in full-display mode; document whether the button appears there; confirm it snaps to an edge and away from the draft area | NOT RUN | +| 36 | Tap freeze inside MLBB | In game, tap the circle: glyph becomes the pause glyph on orange; Vision Lab shows the pinned frame, not DraftLens | NOT RUN | +| 37 | Tap resume inside MLBB | Tap the circle again: live capture resumes without a new consent dialog or VirtualDisplay; glyph returns to LIVE green | NOT RUN | +| 38 | Drag, long-press and edge snap | Long-press (~0.4 s) then drag the button to the other edge; on release it snaps and stays there after rotation | NOT RUN | +| 39 | Rotation with overlay shown | Rotate portrait↔landscape with the button shown: it stays inside the screen in the same relative position; draft regions remain usable | NOT RUN | +| 40 | Overlay permission denied | Decline the overlay permission: capture still starts, in-app Freeze/Resume still work, the capture screen shows how to grant it later; no button is shown | NOT RUN | +| 41 | Stop while frozen with overlay shown | Tap Stop with a frame pinned and the button visible: button disappears, projection released, the pinned frame survives in Vision Lab | NOT RUN | +| 42 | Notification shade never replaces the pinned frame | In game with single-app MLBB capture, open the notification shade repeatedly: the pinned frame and live preview never show the shade | NOT RUN | +| 43 | App hidden resume-intent | While frozen, return to DraftLens with MLBB in the background; tap Resume in DraftLens; return to MLBB: live capture is resumed (the hidden-period intent is remembered) | NOT RUN | +| 44 | Overlay stays interactive over MLBB in app-only capture | Select Single app → MLBB; button stays visible above MLBB; taps toggle LIVE/FROZEN; long-press drag works; visibility changes (DraftLens ↔ MLBB, shade open/closed) never detach or stale it | NOT RUN | +| 45 | Overlay tap before the first frame in app-only mode | Fresh session with the game not yet rendered: tap shows “No frame captured yet” toast; button stays attached and responsive | NOT RUN | +| 46 | Overlay ↔ notification state sync | Freeze via overlay in MLBB: notification shows Resume (no Pause); pause via notification: overlay turns orange; both surface the same LIVE/PAUSED/FROZEN state | NOT RUN | +| 47 | Notification actions while LIVE | Actions are exactly Stop + Pause; no “Freeze current frame” wording remains anywhere in the notification | NOT RUN | +| 48 | Notification Pause keeps the frame | Tap Pause in MLBB: latest game frame stays shown everywhere; no new frames replace it; projection and VirtualDisplay persist; notification flips to Resume | NOT RUN | +| 49 | Notification Resume recovers | Tap Resume: live frames resume without a new consent dialog or VirtualDisplay; notification flips back to Stop + Pause | NOT RUN | +| 50 | Notification Resume while pinned in-app | Freeze via the in-app control, then Resume from the notification: pin clears and live publication resumes (gate opens unless MLBB is hidden) | NOT RUN | +| 51 | Stop from the notification | Stop is always present; removes the overlay, releases projection/VirtualDisplay/notification; Vision Lab retains the pinned frame | NOT RUN | +| 52 | Pause then process restart | Pause during MLBB; kill the app from recents; reopen: capture is gone (process death), no stale PAUSED state carried into the next session | NOT RUN | +| 53 | Single-app capture of another app | Repeat rows 44–51 with a second single-app target to confirm no MLBB-specific assumptions | NOT RUN | + +## Sprint 1C capture-isolation finding (maintainer to fill in) + +Whether the overlay appears in single-app and full-display captures is an on-device finding. +Record the outcome of rows 34/35 here (device model, Android version, OEM skin) and whether +the button contaminated hero-slot crops taken on the fallback path. Do not mark a row PASS +without evidence from that run. + +## Focused first-frame test: Single-app MLBB (Sprint 1C follow-up) + +**Status: NOT RUN.** This is the decisive test for the reported "No frame captured yet" +bug. Run each part on a physical device and record the log lines below with device model, +Android version, OEM skin, APK run ID and commit SHA. Do not mark PASS without evidence. + +Collect bounded diagnostics (first 4 + every 20 per event; no pixels are logged): + +```sh +adb logcat -c && adb logcat -v time -s CaptureSessionService FramePipeline +``` + +Key lines and what they answer: +- `visibility-callback seq=N visible=... pipelineInitialized=... status=...` — was a + callback received, with what value, before or after pipeline init, and in what order. +- `resize-callback seq=N WxH status=... virtualDisplayReady=...` and + `capture-start ... source=pending-captured-content-resize` — was the selected app's + early resize retained and applied to both reader and VirtualDisplay? +- `acquire: ... acquired=N image=WxH isActive=... generationCurrent=... frozen=... + contentVisible=...` — does ImageReader receive images at all (no lines = platform did + not deliver; lines present = our code rejected them). +- `drop gate=` (`isActive` / `staleGeneration` / `frozen` / `contentHidden`), + `drop sampling`, `drop conversion-failed`, `converted ... bitmap=WxH`, + `published to _latestFrame`, `recordFrame delivered` — where the frames stopped. +- `freeze: NoFrame ... visibility=...` — the state at the moment the button reported it. + +A. Full-display baseline +- [ ] Start capture with **Entire screen**; confirm `acquire:` and `published to + _latestFrame` lines, preview updates, and Freeze pins the current frame. (NOT RUN) + +B. Single-app MLBB +- [ ] Fresh session → **Single app** → **Mobile Legends**. +- [ ] Wait until MLBB is visibly in the foreground; record whether + `onCapturedContentVisibilityChanged(true)` arrived (and its seq/order). +- [ ] Confirm ImageReader receives images (`acquire:` lines) and at least one frame is + converted and published (`published to _latestFrame`, `recordFrame delivered`). +- [ ] Tap the floating Freeze button; confirm the game frame is pinned. (NOT RUN) + +C. Visibility transitions +- [ ] Switch MLBB → DraftLens: no hidden-window frame replaces the last valid game frame. +- [ ] Return to MLBB: publication resumes; Freeze/Resume work again. (NOT RUN) + +D. Isolate overlay effects +- [ ] Repeat B with the floating overlay **disabled**. The first-frame behavior must be + identical; if it is not, the overlay window — not the capture gate — is implicated. + (NOT RUN) + +E. Resize +- [ ] Record the selected app's reported dimensions from `resize-callback` and confirm + ImageReader/VirtualDisplay match (`virtual-display resized` / `bitmap=WxH`). +- [ ] Repeat with MLBB in landscape. (NOT RUN) + +If no `acquire:` lines appear at all in B, the platform did not deliver images to +ImageReader — report that finding with the logcat rather than changing the policy again. ## Evidence to capture each run - `git rev-parse --short HEAD` - Device model and Android version (`adb shell getprop ro.product.model`, `adb shell getprop ro.build.version.release`) - DraftLens version/APK artifact run ID -- Logcat lines tagged `CaptureSessionService`, `FramePipeline`, `VisionPreprocessor` - (`adb logcat -v time -s CaptureSessionService FramePipeline VisionPreprocessor`) for the +- Logcat lines tagged `CaptureSessionService`, `FramePipeline`, `VisionPreprocessor`, `OverlayController` + (`adb logcat -v time -s CaptureSessionService FramePipeline VisionPreprocessor OverlayController`) for the capture dimensions, resize events, plane layout and frame-generation lines - Initial resolution, frame count after 10 seconds, preview visible Y/N - For failures: full exception and steps immediately before it diff --git a/docs/VISION_ARCHITECTURE.md b/docs/VISION_ARCHITECTURE.md index 5337841..6a16048 100644 --- a/docs/VISION_ARCHITECTURE.md +++ b/docs/VISION_ARCHITECTURE.md @@ -91,8 +91,9 @@ MediaProjection -> VirtualDisplay -> ImageReader -> capture HandlerThread The freeze workflow is a side channel on the same pipeline — never a second capture system: ```text -"Freeze current frame" (notification) or in-app Freeze - -> CaptureSessionService ACTION_FREEZE (or source.freeze()) +in-app Freeze / the floating button (pinning via CaptureFreeze) or the Vision Lab Freeze button + -> CaptureFreeze.freezeCurrentFrame() (the notification issues Pause = ACTION_PAUSE) + -> CaptureFreeze.freezeCurrentFrame() -> FramePipeline freezes publication; copies the latest published frame -> CaptureStateHolder.frozenFrame (independently owned copy, at most one) diff --git a/docs/VISION_LAB.md b/docs/VISION_LAB.md index 3634f3d..021d933 100644 --- a/docs/VISION_LAB.md +++ b/docs/VISION_LAB.md @@ -166,7 +166,7 @@ The button is a compact icon in the corner of the canvas card. | Control | What it does | |---|---| -| **Freeze** | locks the current frame before annotation (also available as **Freeze current frame** in the capture notification while in another app) | +| **Freeze** | locks the current frame before annotation (the capture notification carries **Pause**/**Resume** for the same job while in another app; an independently pinned snapshot comes from the floating button or this button) | | **Resume** | returns to live frames (also as **Resume live capture** in the notification) | | **Slot** | a manually marked region where a ban/pick hero portrait is expected | | **Process Frame** | resizes and prepares the frame only; no recognition exists yet | diff --git a/docs/VISION_TESTING.md b/docs/VISION_TESTING.md index 8012c59..a8af604 100644 --- a/docs/VISION_TESTING.md +++ b/docs/VISION_TESTING.md @@ -162,7 +162,7 @@ and the `mediaProjection` service type survived the build. - **Catalog entries actually loaded** are measured at device sync time; not recorded here. - Compose screens are not UI-tested (no compose-ui-test dependency exists in this project); the interaction model is covered by the interaction state-machine tests. -- The service-level notification action (`ACTION_FREEZE` delivery through +- The service-level notification actions (`ACTION_PAUSE` / `ACTION_RESUME_LIVE` delivery through `onStartCommand`) is exercised only through the JVM-level `CaptureFreeze`/`CaptureStateHolder` paths and manual device verification; no instrumented test taps the system notification. diff --git a/docs/sprints/SPRINT_1B_HERO_RECOGNITION.md b/docs/sprints/SPRINT_1B_HERO_RECOGNITION.md index a54800b..f61a5f1 100644 --- a/docs/sprints/SPRINT_1B_HERO_RECOGNITION.md +++ b/docs/sprints/SPRINT_1B_HERO_RECOGNITION.md @@ -32,6 +32,11 @@ This branch now also contains the fixes for the PR #3 review findings: ## Capturing Mobile Legends: the freeze workflow +> **Superseded by Sprint 1C** (`docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md`): prefer +> **Single app / app window → Mobile Legends** when offered; the freeze is now primarily +> the circular floating button (rows 33–43 of `docs/TESTING_PHYSICAL_DEVICE.md`). The +> full-display guidance below remains valid as the fallback. + 1. Open DraftLens → **Debug Capture**. When Android asks what to capture, choose **Entire screen / full device display**. App-window capture shows the DraftLens screen, not the game, and cannot support this workflow. (The app says this on screen too.) diff --git a/docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md b/docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md new file mode 100644 index 0000000..0b2ec72 --- /dev/null +++ b/docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md @@ -0,0 +1,184 @@ +# Sprint 1C — App-Only Capture & Circular Floating Control + +**Branch:** `sprint-1c-app-only-capture` (stacked on `sprint-1b-hero-recognition`, PR #3 — **not merged**) · **PR:** #4 (base `sprint-1b-hero-recognition`) + +Sprint 1C makes the MLBB freeze workflow playable from inside the game: the user +captures only the MLBB window, and a single circular overlay button freezes/resumes +the capture — no need to open the notification shade and no in-app step at all. +No recommendation UI of any kind is added. + +**Dependency on Sprint 1B:** the freeze/snapshot machinery (`CaptureFreeze`, +`FrameSnapshots`), the Vision Lab pinned frame and the capture foreground service all +come from PR #3. Sprint 1C was built on top of it and cannot be reviewed meaningfully +until #3 lands. Do not merge this PR before #3 is merged and this PR is rebased onto +`main` if the maintainer prefers a clean stack. + +## What shipped + +| Piece | Where | What it does | +|---|---|---| +| Visibility gate (pure) | `capture/CaptureVisibilityPolicy.kt` | `shouldPublish(active, generationCurrent, frozen)` plus visibility transitions; JVM-tested | +| Pipeline gate | `FramePipeline.kt` | While the captured-content window is hidden, images are acquired and closed but never converted/published; the publish-time synchronized block double-checks | +| Visibility callback | `CaptureSessionService.projectionCallback.onCapturedContentVisibilityChanged` | API 34+; forwards to the pipeline; ignored unless CAPTURING; resize path unchanged | +| Overlay math (pure) | `overlay/OverlayPosition.kt` | Normalized (0..1) position on the free axis; pixel round-trip; edge snapping; clamping; JVM-tested | +| Overlay enablement (pure) | `overlay/OverlayEnablement.kt` | `shouldAttachOverlay(preference, permission, session)` gate; JVM-tested | +| Pending-attach invalidation | `OverlayController.kt` | generation counter: a detach() invalidates any in-flight show(), so a teardown during the DataStore read can never orphan the button | +| Overlay position persistence | `AppPreferencesRepository` | `floating_control_enabled`, `floating_position_x/y` in the existing preferences store | +| Floating button | `overlay/FloatingControlView.kt` | 52dp circle, LIVE green / FROZEN orange, white glyph, content description, system font | +| Overlay window | `overlay/OverlayController.kt` | `TYPE_APPLICATION_OVERLAY` + `FLAG_NOT_FOCUSABLE`; tap = freeze/resume via the shared `CaptureFreeze` path; long-press (400 ms) drag; edge snap + persist on release; re-anchored on rotation via the display listener | +| Service wiring | `CaptureSessionService.kt` | Overlay attached only while capture is running (user enabled + permission granted), repainted on freeze/resume, removed on every teardown path; `ACTION_OVERLAY_UPDATE` re-syncs from the UI | +| Enable surface | Debug Capture screen | “Floating control” card: opt-in switch, permission explanation, open Android overlay settings, live LIVE/FROZEN label; capture works fully without it | +| Guidance | Debug Capture + Vision Lab screens | Two capture modes documented (single-app preferred; full-display fallback with honest limitation) | +| Manifest | `AndroidManifest.xml` | `SYSTEM_ALERT_WINDOW` declared; **requested only when the user enables the floating control** via `ACTION_MANAGE_OVERLAY_PERMISSION` | + +## Single-app capture (Android 14+) + +DraftLens never forces the mode. The system chooser decides what the user picks; the app +only guides: + +1. Tap **Start Capture**. +2. Prefer **Single app / app window** and choose **Mobile Legends: Bang Bang**. + The projection then carries only that window: live frames while MLBB is on screen, + and the pinned last-valid frame whenever it is hidden. +3. If the single-app option is not offered (older Android), choose **Entire screen / + full device display**. That fallback captures everything, including the notification + shade, system UI and other apps. **DraftLens cannot promise notification-free + capture on the full-display path.** The freeze workflow still works. + +Below API 34 the visibility callback does not exist, so a hidden app-only window is +undetectable — frames keep flowing exactly as before (full-display behavior). + +## Visibility lifecycle + +`onCapturedContentVisibilityChanged(isVisible)` (API 34+) is wired into the existing +CAPTURING-state guard alongside `onCapturedContentResize`: + +- **Hidden:** the pipeline acquires and closes images but performs no conversion and + publishes nothing. `latestFrame` (and, if frozen, the pinned snapshot) are untouched. + The shade/DraftLens/other apps can never replace the last valid game frame. +- **Visible again:** publication resumes automatically — unless a frame is pinned + (user intent LIVE vs FROZEN wins). A resume requested while hidden flips the intent + to LIVE and takes effect exactly when visibility returns; nothing black is ever shown + in between. +- **Freeze while hidden:** the pin stores a copy of the last valid frame; still no + publication; visibility callbacks change nothing until a resume. +- **Stop:** the overlay is detached first, then the normal termination path runs; the + pinned frame survives in process memory as documented (Sprint 1B contract). +- **Resize during freeze:** unchanged — the generation-switch resize path works while + frozen; freeze/resume never recreate the projection or the VirtualDisplay and never + create a second VirtualDisplay. + +## Overlay and capture isolation (critical, not settled by code) + +**Whether the DraftLens overlay appears inside a MediaProjection capture is a physical-device +question and is NOT verified in this sprint.** The overlay does not set any flag to hide +itself from capture, and no capture region is masked. Expectations to check on device +(rows 34/35 of `docs/TESTING_PHYSICAL_DEVICE.md`): + +- Single-app MLBB capture **should** carry only the MLBB window; on many devices the + overlay (a separate window belonging to DraftLens) will not be included. +- Full-display fallback capture will very likely include the overlay. If that happens, + the control sits in the top/bottom corner, snaps away from the central draft region, + and must not be read by the hero-slot cropping (annotation rects are user-placed on + the last valid frame, and the overlay is small — but this is not automatically safe). +- If the overlay contaminates hero slot crops on the fallback path, the documented + remedy is: use single-app capture, or disable the floating control and freeze from + the notification/in-app. **Do not** crop arbitrary regions to hide it. + +No claim is made that app-only capture excludes every overlay window; only on-device runs +can establish it per device/OEM. + +## Verification + +CI (GitHub Actions, ubuntu-latest) is the build of record — the authoring host has no +Android toolchain. + +- Implementation head `f992d45` (docs-only commits after it change no build inputs): + CI run 43784157138 — **886 unit tests, 0 failures / 0 errors / 0 skipped**; strict lint green; + `assembleAndroidTest` green; `assembleDebug` green; APK `app-debug.apk` + sha256 `1d1f50629e145d48e64ed56d0cf74228e7d48d4c6f94fde81eb6c22fc4795810`. +- Instrumentation tests: **compiled via `assembleAndroidTest`; not executed** (no KVM on the runner). +- Automated coverage gaps, stated honestly: **LIVE→FROZEN pinning a real frame** (an + ImageReader surface cannot be filled from a test Canvas, and the pin path is exercised + on device — rows 36/37) and **resize during freeze** (the gate is dimension-agnostic by + construction; policy truth-table pins it — on-device rows 25/30 pin the pipeline path). +- Physical-device rows 33–43: **NOT RUN** — see `docs/TESTING_PHYSICAL_DEVICE.md` + +## Explicitly not built (scope restrictions) + +No draft-phase detection, no slot detection, no hero models, no counter/synergy scoring, +no recommendation ranking or overlays, no game memory access, no automated interaction +with MLBB, no accessibility service, no panel/menu/toolbar from the overlay button. + +## Follow-up fixes (notification Pause, permissions-only UI, overlay independence) + +### Root cause analysis (overlay "freezing" during app-only capture) + +Traced end to end: `MediaProjection.Callback` → `onCapturedContentVisibilityChanged` → +`CaptureVisibilityPolicy` → `FramePipeline` → `CaptureFreeze` → overlay/notification state. + +What the trace showed: + +1. **The overlay was never lifecycle-coupled to capture visibility** — `FloatingControlView` + renders and handles touches with no frame input; `onCapturedContentVisibilityChanged` + gates only the `FramePipeline` (by design after the 1C review fixes). Window attachment + is independent of the ImageReader generation and of the sampler. +2. **Three real defects made the button misbehave though:** + - **Silent no-op tap**: tapping freeze before any frame had ever been published returned + `NoFrame` and produced no feedback (common in app-only mode, where nothing publishes + until MLBB is first on screen). The tap now surfaces a short toast ("No frame captured + yet"), and the button is no longer perceived as dead. + - **Glyph/notification could disagree with reality**: the overlay glyph and the + notification only followed `frozenFrame`. Any publication block that was not a pin + (e.g. notification Pause) left both surfaces showing LIVE/green while publication was + blocked. Both now derive from one shared `CapturePublishState` (LIVE/PAUSED/FROZEN, + pin wins over pause for presentation) — see `CapturePublishState.kt` — collected from a + combined `frozenFrame` + session-state flow. + - **Visual workaround risk ruled out**: nothing in the fix hides the button, masks + regions, or falls back to full-display capture. +3. **Unverifiable from CI**: whether a specific OEM/immersive game covers or excludes the + overlay window on the physical screen is a hardware finding — rows 44–53 of + `docs/TESTING_PHYSICAL_DEVICE.md` (all NOT RUN). No OEM workaround is shipped; if an + OEM covers overlays during immersive games, that is a documented limitation, not a code + defect, and the notification Stop/Pause actions remain available. + +### Notification: Freeze → Pause + +The ongoing notification now carries **Stop** and exactly one of **Pause** (while LIVE) / +**Resume** (while PAUSED or FROZEN) — never both, never a Freeze action. Pause maps onto the +existing shared `CaptureStatus.PAUSED` gate (`setPaused(true)` stops conversion/publication, +keeps the projection, VirtualDisplay and latest valid frame). Resume is a unified handler: +`CaptureFreeze.resumeLive()` (clear any pin) + `setPaused(false)` (clear any pause). The +in-app and overlay Freeze/Resume (pinning a snapshot) are unchanged. + +### Permissions-only UI + +Removed: the two-mode capture tutorial card, the floating-control tutorial card, the +in-game freeze help text, and the stale "Detection: not implemented in Sprint 0" line. +Kept, compacted in one "Permissions" card: the notification permission flow (status +Allowed/Not allowed + grant action), and the floating-overlay permission flow (status, +switch, open Android overlay settings, pending-enable on return from settings). The +in-app capture controls (Start/Stop/Pause), status readout, sampling interval and live +preview are unchanged. + +### Verification + +- Implementation head `130ce6c`: CI run 43922482183 — **904 unit tests, 0 failures / 0 errors / 0 skipped**; + lint + `assembleAndroidTest` + `assembleDebug` green; APK sha256 `48b510673919fe2caa9c906747d9dc766eb27bb170c589f67b58c9a175dd7b72`. +- Physical device: rows 44–53 remain **NOT RUN**. + +## Follow-up: single-app first-frame bug + +Root cause: Sprint 1C registered MediaProjection callbacks before the pipeline, +VirtualDisplay and session state were ready. Early visibility `false` and early +captured-content resize were lost/overwritten or treated as fatal; the boolean gate could +block the first frame forever. Diagnostics were added first (`CaptureDiagnostics`, callback +seq/readiness logs, per-stage drop reasons, no pixels); then the fix: service-owned explicit +UNKNOWN/VISIBLE/HIDDEN policy (UNKNOWN intentionally open, first frame promotes to VISIBLE), +service-owned `PendingResize` applied to both VirtualDisplay and ImageReader at startup, +no pipeline-side visibility reset, and holder delivery of frames published during STARTING. +CI run 43936524131 at head `3c7b58c`: **926 unit tests, 0 failures**; lint, +`assembleAndroidTest`, `assembleDebug` green; APK sha256 +`ade9ec10c5d6b9d4e372b325e6b521f87170d0fa5273c9b30e9ea9063a763070`. See +`docs/IMPLEMENTATION_REPORT.md` and the focused A–E device test in +`docs/TESTING_PHYSICAL_DEVICE.md` (**NOT RUN**). diff --git a/docs/superpowers/plans/2026-11-16-sprint-1c-app-only-capture.md b/docs/superpowers/plans/2026-11-16-sprint-1c-app-only-capture.md new file mode 100644 index 0000000..74f6ced --- /dev/null +++ b/docs/superpowers/plans/2026-11-16-sprint-1c-app-only-capture.md @@ -0,0 +1,149 @@ +# Sprint 1C — App-Only Capture & Circular Floating Control Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A maintainer can freeze/resume MLBB capture from a small circular overlay button without leaving the game, with single-app capture preferred and the last valid game frame preserved when MLBB is hidden. + +**Architecture:** A pure-Kotlin visibility policy gates frame publication in the existing `FramePipeline` (replacing the implicit "only freeze gates" model); the capture service feeds it from `onCapturedContentVisibilityChanged`. A thin `TYPE_APPLICATION_OVERLAY` WindowManager view (attached only during capture sessions) calls the existing `CaptureFreeze` path on tap. Position/snap math is pure Kotlin + unit-tested. + +**Tech Stack:** Kotlin, Compose, WindowManager, MediaProjection (API 34+ callback), DataStore preferences, JUnit4 (JVM) + compiled-only instrumented tests. + +**Spec:** `docs/superpowers/specs/2026-11-16-sprint-1c-app-only-capture-design.md` (+ the maintainer Sprint 1C brief, expanded in `docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md`) + +## Global Constraints + +- minSdk 26, compileSdk/targetSdk 35, JDK 17, AGP 8.6.1, Gradle 8.7 — unchanged. +- Never force single-app mode; both capture modes must keep working. +- No new third-party dependencies. +- The overlay is one circular button ≤56dp; no panel, cards, menu, toolbar, fullscreen overlay, accessibility service, or game touches. +- `SYSTEM_ALERT_WINDOW` requested only when the user enables the floating control. +- Anti-goals enforced: no masking of captured content, no screenshot cropping of shade content, no silent exclusion claims. +- Build of record: GitHub Actions `test` → `assembleAndroidTest` → `lint` → `assembleDebug`; do not merge; PR base is `sprint-1b-hero-recognition`. +- Keep Sprint 1B history intact; never modify `main`. + +## Review Focus + +1. Black/hidden-app frames replacing the pinned frame (single-app capture while MLBB is backgrounded) — pinned by Task 1 policy tests + Task 2 pipeline wiring + Task 7 docs matrix row. +2. Visibility callback racing a resize re-creation (both fire on rotation/visibility flips) — Task 2 keeps `recreateDisplay` behind the same CAPTURING guard; Task 1 tests the gate truth table. +3. Overlay persisting stale position after rotation or after density change — Task 3 normalized math + Task 6 reposition on display metrics; pinned by Task 3 tests. +4. Overlay permission denied leaving a "dead" enable toggle — Task 4 decision helper tests + Task 6 screen copy. +5. Service stopped while frozen leaving an orphaned overlay attached — Task 6 removal in the shared teardown path; pinned by Task 8 holder/freeze tests where JVM-reachable. + +--- + +### Task 1: Pure visibility/publication gate + +**Files:** +- Create: `app/src/main/kotlin/com/draftlens/capture/CaptureVisibilityPolicy.kt` +- Test: `app/src/test/kotlin/com/draftlens/capture/CaptureVisibilityPolicyTest.kt` + +**Interfaces:** +- Produces: `CaptureVisibilityPolicy` (class) — `fun isContentVisible(): Boolean` (initial true), `fun onContentVisibilityChanged(visible: Boolean): VisibilityTransition`, `fun isContentVisibleNow(): Boolean`, `fun shouldPublish(isActive: Boolean, generationCurrent: Boolean, frozen: Boolean): Boolean`; `data class VisibilityTransition(val wasVisible: Boolean, val isVisible: Boolean)` with `val changed: Boolean`. Used by Tasks 2 and 6. + +- [ ] **Step 1: Write failing tests** — truth table: initially visible; hidden→visible transitions; `shouldPublish` = active && generationCurrent && !frozen && visible; `shouldPublish` false when hidden even if not frozen; transitions report was/is. +- [ ] **Step 2: Run** `./gradlew testDebugUnitTest --tests '*CaptureVisibilityPolicyTest*'` → FAIL (class missing). (Local host has no JDK — delegate this run to CI if local Gradle is unavailable; record that.) +- [ ] **Step 3: Implement** the policy as a single class with a lock, no Android imports. +- [ ] **Step 4: Re-run** the same filter → PASS. +- [ ] **Step 5: Commit** `feat: capture visibility gate (pure policy) for app-only windows` + +### Task 2: Pipeline visibility gate + service callback wiring + +**Files:** +- Modify: `app/src/main/kotlin/com/draftlens/capture/FramePipeline.kt` — add `@Volatile`-backed policy `val visibilityPolicy`, `fun onContentVisibilityChanged(visible: Boolean): VisibilityTransition`, fold `shouldPublish` into the image-available listener (next to the `frozen` checks) and into the publish-time `synchronized(freezeGate)` block. +- Modify: `app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt` — override `onCapturedContentVisibilityChanged(isVisible: Boolean)` in `projectionCallback` (API 34+); route to `pipeline.onContentVisibilityChanged`; log; call `overlayController?.onContentVisibilityChanged` if added in Task 6 (keep one call site); no release of any resource on hide (projection stays valid); ignore when not CAPTURING. +- Test: covered by Task 1 truth-table (pipeline is Android-bound) + instrumented compile in Task 7. + +**Key points:** images acquired & closed while hidden; no conversion; `_latestFrame` untouched; resize path unchanged and still allowed during freeze; the visibility gate adds a second, independent condition — never replace latest valid frame with an image acquired while hidden. + +- [ ] **Step 1:** Implement pipeline gate (one flag set, two check sites). +- [ ] **Step 2:** Implement `onCapturedContentVisibilityChanged` override (mirror `onCapturedContentResize` style; guard the CAPTURING state like the resize path does). +- [ ] **Step 3:** Commit `feat: gate frame publication on captured-content visibility (API 34+)` + +### Task 3: Overlay position math (pure) + +**Files:** +- Create: `app/src/main/kotlin/com/draftlens/overlay/OverlayPosition.kt` +- Test: `app/src/test/kotlin/com/draftlens/overlay/OverlayPositionTest.kt` + +**Interfaces:** +- Produces: `OverlayPosition` — `data class Normalized(val x: Float, val y: Float)` (0..1 fraction of free axis, edge-agnostic), `data class SnapResult(val edge: HorizontalEdge, val normalizedX: Float, val normalizedY: Float)`, `enum class HorizontalEdge { LEFT, RIGHT }`; functions: `pixelsToNormalized(pxLeft: Int, pxTop: Int, screenWidth: Int, screenHeight: Int, sizePx: Int): Normalized`, `normalizedToPixels(n: Normalized, screenWidth: Int, screenHeight: Int, sizePx: Int): Pair`, `clampToScreen(...)`, `snapToNearestEdge(leftPx: Int, topPx: Int, screenWidth: Int, sizePx: Int): Pair` (snaps x to `0` or `screenWidth - sizePx`, keeps y clamped), `snapNormalized(n: Normalized, screenWidth: Int, screenHeight: Int, sizePx: Int)`. All pure, no Android imports (density handled by callers in px). Used by Tasks 4 and 6. + +- [ ] **Step 1: Write failing tests** — round-trip px→normalized→px; clamp leaves in-screen positions untouched; out-of-screen clamps are pulled in; `snapToNearestEdge` returns LEFT below/at half and RIGHT above; y untouched by snap; landscape→portrait round-trip keeps the normalized position and changes only px; all functions handle zero/degenerate sizes without NaN. +- [ ] **Step 2:** Run filter → FAIL. +- [ ] **Step 3:** Implement. +- [ ] **Step 4:** Re-run → PASS. +- [ ] **Step 5:** Commit `feat: normalized overlay position with edge-snapping math` + +### Task 4: Overlay enablement decision + preferences + +**Files:** +- Create: `app/src/main/kotlin/com/draftlens/overlay/OverlayEnablement.kt` — pure `fun decideOverlayRequest(canDrawOverlays: Boolean, overlayEnabledPreference: Boolean): OverlayRequestOutcome` with `sealed interface` outcomes: `Show`, `RequestPermission`, `Disabled`. Used by the screen (Task 6). +- Modify: `app/src/main/kotlin/com/draftlens/datastore/AppPreferencesRepository.kt` — add keys `floating_control_enabled` (bool), `floating_position_x`/`floating_position_y` (float); flows `floatingControlEnabled: Flow`, `floatingPosition: Flow`; suspend `setFloatingControlEnabled(Boolean)`, `setFloatingPosition(OverlayPosition.Normalized)`, `clearFloatingPosition()` not needed (keep minimal). +- Test: `app/src/test/kotlin/com/draftlens/overlay/OverlayEnablementTest.kt` — grant/deny truth tables; denial never implies enablement. + +- [ ] **Step 1:** Write failing decision tests → FAIL. +- [ ] **Step 2:** Implement + wire preferences (no new DataStore instance; reuse `draftlens_preferences`). +- [ ] **Step 3:** Re-run → PASS. +- [ ] **Step 4:** Commit `feat: overlay enablement decision and persisted position keys` + +### Task 5: Floating control view + WindowManager shell + +**Files:** +- Create: `app/src/main/res/drawable/ic_floating_live.xml`, `app/src/main/res/drawable/ic_floating_frozen.xml` (simple white vector glyphs on transparent background). +- Create: `app/src/main/kotlin/com/draftlens/overlay/FloatingControlView.kt` — circular 52dp view built in code: solid circle background (LIVE green `#1DB954`-like blue/green; FROZEN orange `#F57C00`-like), white glyph `ImageView` centered, content description "Freeze game frame"/"Resume live capture", high-contrast stroke for visibility on bright and dark scenes. +- Create: `app/src/main/kotlin/com/draftlens/overlay/OverlayController.kt` — owns `WindowManager` handle; `attach()`, `detach()`, `setFrozen(Boolean)` (repaint), `repositionForDisplay(screenWidthPx, screenHeightPx, sizePx)` using Task-3 math + persisted normalized position (default: RIGHT edge, upper third), `toggleFreeze()` calling `CaptureFreeze.freezeCurrentFrame()` / `CaptureFreeze.resumeLive()`. Touch handling in the view: DOWN → 400 ms long-press timeout (Handler); MOVE beyond 12dp before timeout → cancel gesture (no drag, no toggle); after timeout → drag (update `LayoutParams.x/y`); UP → snap to nearest edge, persist normalized position, `setFrozen(CaptureStateHolder.frozenFrame.value != null)`. TAP (no cancel, short) → `toggleFreeze()`. +- No new foreground service; the controller is attached/removed by `CaptureSessionService` only. + +- [ ] **Step 1:** Implement view + vectors (no automated test; instrumented in Task 7). +- [ ] **Step 2:** Implement controller with gesture state machine; keep < 300 lines. +- [ ] **Step 3:** Commit `feat: circular floating freeze button (WindowManager overlay)` + +### Task 6: Service + screen + manifest wiring + +**Files:** +- Modify: `app/src/main/kotlin/com/draftlens/capture/CaptureSessionService.kt` — `OverlayController?` field created when (user preference enabled) && (SYSTEM_ALERT_WINDOW granted); attach after `CaptureStateHolder.beginCapture(...)`; remove in `releaseCaptureResources()`; re-attach/update on existing display-listener metrics change; observe `frozenFrame` to repaint (the existing `frozenJob` can call `overlayController?.setFrozen(...)`); gate actual attach on `Build.VERSION.SDK_INT` not needed (TYPE_APPLICATION_OVERLAY API 26+, minSdk 26 ✓). +- Modify: `app/src/main/kotlin/com/draftlens/capture/CaptureController.kt` — no change (enablement flows through the preferences/DataStore, not the service intent). +- Modify: `app/src/main/AndroidManifest.xml` — add ``. +- Modify: `app/src/main/kotlin/com/draftlens/feature/home/DebugCaptureScreen.kt` — replace the Entire-screen guidance with the two-mode guide (single-app preferred with MLBB selection steps; full-display fallback with honest limitation); add a **Floating control** card: switch ("Show the freeze button while capture runs"), current LIVE/FROZEN state when overlay attached, denial explanation + "Open Android settings to enable drawing over other apps" button (`ACTION_MANAGE_OVERLAY_PERMISSION`, package-scoped), and a note that capture works fully without it. +- Modify: `app/src/main/kotlin/com/draftlens/feature/visionlab/VisionLabScreen.kt` (only if the guidance card lives there too — check; update the same capture-mode wording if present). + +- [ ] **Step 1:** Manifest + service attach/detach (teardown removes overlay on every end path). +- [ ] **Step 2:** Debug screen rewrite of the guide + floating-control card; keep Sprint 1B content intact otherwise. +- [ ] **Step 3:** Commit `feat: enable floating freeze control during capture; two-mode capture guidance` + +### Task 7: Instrumented tests (compile-only) + +**Files:** +- Create: `app/src/androidTest/kotlin/com/draftlens/FloatingControlOverlayInstrumentedTest.kt` — attaches the view via `OverlayController` under an `ActivityScenario`/service context, asserts `WindowManager` view added then removed after `detach()`, asserts `setFrozen` toggles glyph resource, skips gracefully with `Assume.assumeTrue(Settings.canDrawOverlays(context))` when overlay permission is absent (always, on CI emulators). + +- [ ] **Step 1:** Write the test; mirror the header/skip style of `CaptureLifecycleInstrumentedTest`. +- [ ] **Step 2:** Commit `test: instrumented overlay attach/removal (compile-only)` + +### Task 8: Remaining JVM coverage + +**Files:** +- Test: `app/src/test/kotlin/com/draftlens/capture/ServiceStopAndFreezeTest.kt` (or extend `CaptureFreezeTest`) — repeated freeze commands (AlreadyFrozen keeps first pin), freeze with no frame (NoFrame), stale command after stop (`hasActivePipeline` false → NoPipeline), `terminateSession` while frozen preserves the pin, `reset` clears it, `resumeLive` while "hidden" (policy) then visible resumes publication (policy truth table via Task 1), resize during freeze does not clear the frozen flag (policy-level: `shouldPublish` stays false with frozen=true across dimension deltas — assert the policy takes no dimensions as input). +- Verify existing `CaptureFreezeTest`, `CaptureStateHolderTest`, `FrameSnapshotsTest` pass unchanged; add only what's missing. + +- [ ] **Step 1:** Add the missing cases; run the full JVM suite in CI (no local toolchain). +- [ ] **Step 2:** Commit `test: freeze/visibility/stop edge cases for Sprint 1C` + +### Task 9: Documentation + +**Files:** +- Create: `docs/sprints/SPRINT_1C_APP_ONLY_CAPTURE.md` — objective, branch/PR, what shipped, visibility lifecycle section, overlay section, honest overlay/capture isolation statement, verification matrix (CI run id, device rows marked NOT RUN), limitations. +- Modify: `docs/SCREEN_CAPTURE.md` — rewrite the single-app (Android 14+ chooser guidance) and visibility sections; remove "onCapturedContentVisibilityChanged is not handled"; document overlay isolation unknowns. +- Modify: `docs/TESTING_PHYSICAL_DEVICE.md` — add rows: overlay appearance in single-app capture, overlay appearance in full-display capture, tap-freeze in MLBB, edge snap, rotation, denial path, shade test. +- Modify: `README.md`, `tasks/todo.md`, `tasks/plan.md`, `docs/IMPLEMENTATION_REPORT.md` — Sprint 1C sections; CI run id, unit totals, APK digest after the green run (fill in during Task 10, second docs commit). + +- [ ] **Step 1:** Write all docs except CI numbers; commit `docs: Sprint 1C capture-visibility and overlay documentation` + +### Task 10: CI, PR, report + +**Files:** none (verification only). + +- [ ] **Step 1:** Push branch; note the CI run id from `gh run list --branch sprint-1c-app-only-capture`. +- [ ] **Step 2:** If red: diagnose systematically (superpowers:systematic-debugging), fix, re-push. +- [ ] **Step 3:** Harvest unit totals from the run, APK digest from the artifact (`gh run download` + `sha256sum`), instrumented compiled-vs-executed status; fill into docs; commit `docs: record Sprint 1C CI results` and push. +- [ ] **Step 4:** `gh pr create --base sprint-1b-hero-recognition --title "Sprint 1C: App-only capture & circular floating control"` with a body covering requirements 1–11 (unmerged dependency on #3 stated; device verification NOT RUN). +- [ ] **Step 5:** Link the session to the PR (openchamber), name the task; final report with SHA, run id, unit totals, digest, instrumentation status, device status, limitations. Do not merge. diff --git a/docs/superpowers/specs/2026-11-16-sprint-1c-app-only-capture-design.md b/docs/superpowers/specs/2026-11-16-sprint-1c-app-only-capture-design.md new file mode 100644 index 0000000..6ccc38e --- /dev/null +++ b/docs/superpowers/specs/2026-11-16-sprint-1c-app-only-capture-design.md @@ -0,0 +1,59 @@ +# Sprint 1C — App-Only Capture & Circular Floating Control (Design) + +**Date:** 2026-11-16 · **Branch:** `sprint-1c-app-only-capture` (stacked on `sprint-1b-hero-recognition`, PR #3) · **Spec:** maintainer Sprint 1C brief (in task tracker) + +## Intent + +Let the user freeze/resume the live MLBB capture without leaving the game. One small +circular overlay button; the notification remains only a Stop affordance; no recommendation +UI of any kind. + +## Key decisions + +1. **Capture chooser is never forced.** `createScreenCaptureIntent()` is unchanged. The + debug screen guides the user to pick *Single app / app window → Mobile Legends* + (Android 14+ partial capture). The full-display path stays supported and documented as + the fallback, with no promise of notification-free frames. The old guidance text that + told users to choose Entire screen is corrected. +2. **Visibility is a gate, not a frame source.** API 34+ + `MediaProjection.Callback.onCapturedContentVisibilityChanged()` is overridden. While the + selected window is hidden, images are acquired and closed but never converted or + published; the last valid published frame is preserved untouched. While visible, live + publication resumes automatically *unless a frame is pinned* (user intent = FROZEN). + A resume requested while hidden is safely remembered: it flips the intent LIVE, clears + the pin, and frames flow again when visibility returns — publication while hidden stays + gated regardless of the pin state. +3. **Freeze semantics are unchanged.** `CaptureFreeze` + `FrameSnapshots` keep their + contracts (independently owned copy, projection stays running, idempotent). The overlay + button tap runs the same `freezeCurrentFrame()` / `resumeLive()` calls as everything else. +4. **The floating control is a button, not a panel.** One 48–56dp circular `WindowManager` + view with `TYPE_APPLICATION_OVERLAY`, attached by `CaptureSessionService` (the existing + foreground mediaProjection service, so it is alive exactly while capture is running). + `SYSTEM_ALERT_WINDOW` is requested (ACTION_MANAGE_OVERLAY_PERMISSION) only when the user + taps **Enable floating control**; denied → capture + in-app Freeze/Resume unaffected, + explanation shown; the enable is retried on enable tap or re-request, never forced. +5. **Overlay drives off pure Kotlin seams.** Position storage/snapping/clamping and the + visibility/publication gate are pure classes with JVM unit tests. The `WindowManager` + shell stays thin (attach/detach/touch). No DataStore on the hot path; position persists + as normalized coordinates on drag end. +6. **Overlay is attached only during capture sessions** and always removed on stop, + terminate, revocation and resize-failure teardown paths. Rotation re-derives pixel + position from the normalized coordinates. +7. **No masking of capture content.** The overlay sets no flags to hide itself from capture; + whether it appears in single-app vs full-display captures is a physical-device finding, + documented honestly, never "proven" by JVM tests. +8. **Stacked delivery.** PR base = `sprint-1b-hero-recognition`; 1B history untouched; no + merge; physical-device acceptance is maintainer-run and marked NOT RUN. + +## Non-goals + +Everything in the spec §10: no draft/phase detection, no slot detection, no recognition +models, no scoring, no recommendation overlays, no game interaction, no accessibility service. + +## Verification + +CI (GitHub Actions) is the build of record (host has no Android toolchain): `test`, +`lint`, `assembleAndroidTest`, `assembleDebug` + APK artifact. Instrumentation tests are +compiled, not executed (no KVM emulator). The physical-device matrix for overlay and +single-app capture is written into `docs/TESTING_PHYSICAL_DEVICE.md` and the sprint doc, +marked NOT RUN. diff --git a/tasks/plan.md b/tasks/plan.md index d4c8c0a..94123fb 100644 --- a/tasks/plan.md +++ b/tasks/plan.md @@ -142,7 +142,7 @@ reference-loader ownership) and wire the freeze workflow through the capture arc No recommendation features, no overlays, no automation, PR stays draft and unmerged. ### Work items in order -- [x] Freeze a game frame from the notification (`CaptureFreeze`, `ACTION_FREEZE` / +- [x] Freeze a game frame from the notification (`CaptureFreeze`, `ACTION_FREEZE` (superseded: notification now issues Pause) / `ACTION_RESUME_LIVE`, `FramePipeline` freeze gate, independently owned snapshot via `FrameSnapshots`, FROZEN notification with Resume action, in-app Resume unchanged) - [x] Full-device-capture guidance; POST_NOTIFICATIONS handling; freeze actions in the @@ -184,3 +184,59 @@ No recommendation features, no overlays, no automation, PR stays draft and unmer See `docs/VISION_TESTING.md` for the exact CI runs, test totals and what was and was not executed. The authoring host remains aarch64, so the CI is the build of record. Physical-device verification of the freeze workflow is pending. + +## Sprint 1C — App-only capture & circular floating control +Objective: let the user freeze/resume the live MLBB capture without leaving the game — +prefer single-app (partial) capture, keep the last valid game frame when MLBB is hidden, +and add exactly one circular overlay button. No recommendation UI, no panels, no automation. +PR #4 stacked on PR #3 (`sprint-1b-hero-recognition`); nothing merges automatically; +Sprint 1B history intact. + +### Work items in order +- [x] Pure visibility policy + truth-table tests (hidden ⇒ nothing publishes; frozen wins over visible) +- [x] `FramePipeline` gate: images acquired/closed but never converted while hidden; + publish-time synchronized re-check; API-34 `onCapturedContentVisibilityChanged` in the + existing CAPTURING guard +- [x] Freeze-while-hidden / resume-while-hidden semantics: last valid frame preserved; + LIVE intent remembered across hidden periods; no second VirtualDisplay/projection ever +- [x] Overlay position math (normalized free-axis coords, edge snap) + JVM tests +- [x] Overlay enablement decision + DataStore persistence (position + enabled) +- [x] 52dp circular `TYPE_APPLICATION_OVERLAY` button: tap = Freeze/Resume (green/orange, + white glyph), 400 ms long-press drag, edge snap, rotation re-anchor, non-focusable +- [x] Service attach/detach only during capture, every teardown path removes it; + `ACTION_OVERLAY_UPDATE` re-sync; manifest declares `SYSTEM_ALERT_WINDOW`, requested + only from the enable toggle +- [x] Two-mode capture guidance in the debug screen + Vision Lab; denial path documented + (capture + in-app Freeze/Resume unaffected) +- [x] Instrumentation test for overlay attach/remove (compile-only in CI) +- [x] JVM sequence tests: hidden→freeze→resume→visible; explicit freeze blocks auto-resume +- [ ] CI green record, APK digest, device rows 33–43 NOT RUN (maintainer) +- [ ] PR #4 description + report; physical-device acceptance checklist is maintainer-run + +## Sprint 1C — Follow-up fixes (this revision) +Objective: make the floating control robust for app-only capture and slim the in-app UI to +permissions only; replace the notification Freeze action with Pause/Resume on the shared state. + +### Work items +- [x] Root-cause pass on the app-only overlay: no visibility↔overlay lifecycle coupling found; + fixes are the silent-tap feedback (toast on NoFrame) and a single `CapturePublishState` + derivation shared by notification and overlay (pin wins over pause for presentation) +- [x] Unified `CapturePublishState` (LIVE/PAUSED/FROZEN) with JVM-tested decision matrix +- [x] Notification actions: Stop + Pause (LIVE) / Stop + Resume (PAUSED/FROZEN); Pause maps to + the existing CaptureStatus.PAUSED gate; Resume clears pin + pause; combined + frozen-frame + session-state collector keeps notification and overlay in sync +- [x] UI slim-down: removed tutorial cards / in-game help texts; one compact Permissions card + (notification permission + overlay permission, status + grant/settings actions) +- [x] Tests: PublishStateMatrixTest, holder pause-preservation tests, sequence pause test, + instrumented overlay publication-flip attach test +- [ ] CI green record; physical-device rows 44–53 NOT RUN + +## Sprint 1C — single-app first-frame fix (this revision) +- [x] Diagnostics first: `CaptureDiagnostics` + per-stage callback/acquisition/gate logs +- [x] Service-owned `CaptureVisibilityPolicy` with UNKNOWN/VISIBLE/HIDDEN; early callbacks + preserved; UNKNOWN open; first frame promotes to VISIBLE; stale callbacks rejected +- [x] Service-owned `PendingResize`: early captured-content dimensions retained, applied to + both ImageReader and VirtualDisplay, idempotent; no session kill before VD creation +- [x] Pipeline no longer resets the gate; holder records frames published during STARTING +- [x] JVM tests: early callbacks, UNKNOWN semantics, promotions, pending-resize retention, + sequences; device A–E checklist added (NOT RUN) diff --git a/tasks/todo.md b/tasks/todo.md index f20bfcb..563ca8d 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -124,3 +124,30 @@ PASS** (device details not supplied). The MediaProjection lifecycle matrix is ** - [ ] Physical-device smoke of the freeze workflow; on-device accuracy evaluation; threshold/heuristic calibration (pending) - [ ] Physical-device verification of the landscape capture fix in MLBB (rows 23–32) (pending) + +## Sprint 1C: App-Only Capture & Circular Floating Control +- [x] Task 1: Pure visibility gate (`CaptureVisibilityPolicy`) + JVM truth table (`d68f7bc`) +- [x] Task 2: Pipeline visibility gate; `onCapturedContentVisibilityChanged` (API 34+) wired into the CAPTURING guard (`6eaceb9`) +- [x] Task 3: Overlay position math (normalized/free-axis, edge snap) + JVM tests (`ffc4f5e`) +- [x] Task 4: Overlay enablement decision + DataStore keys (`48e0e11`) +- [x] Task 5: 52dp circular overlay button (`FloatingControlView`/`OverlayController`), long-press drag, tap toggle (`f88cce4`) +- [x] Task 6: Service attach/detach on every teardown path, rotation re-anchor, enable card + two-mode capture guidance, manifest `SYSTEM_ALERT_WINDOW` (`494df62`) +- [x] Task 7: Instrumented overlay attach/remove test (compile-only) (`c6edb52`) +- [x] Task 8: Visibility/freeze/stop sequence tests (`94662b3`) +- [x] Task 9: Documentation (sprint doc, capture docs, device matrix rows 33–43) (this branch) +- [x] Task 10: CI green record (run 43784157138 at `f992d45`: 886 unit tests, 0 failures; APK sha256 `1d1f…`; stacked PR #4); review findings M1/M2 fixed +Verification runs in GitHub Actions because the authoring host has no Android toolchain (below). Physical-device rows 33–43 are **NOT RUN**. + +## Sprint 1C follow-up: notification Pause, permissions-only UI, overlay independence +- [x] Root-cause trace + CapturePublishState unification (`CapturePublishState.kt`) +- [x] Notification Freeze → Pause (LIVE → Pause, PAUSED/FROZEN → Resume); combined state collector +- [x] Overlay glyph/tap driven by the shared publish state; toast on NoFrame +- [x] Permissions-only UI (notification + overlay); tutorial and help text removed +- [x] Docs sync + device rows 44–53; tests added (JVM + compile-only instrumented) +- [x] Task 10: CI green record (run 43922482183 at `130ce6c`: 904 unit tests, 0 failures; APK sha256 `48b5…`); report + +## Sprint 1C single-app first-frame fix +- [x] Diagnostics before policy change (`CaptureDiagnostics`) +- [x] Explicit visibility state model + pending captured-content resize; startup ordering fixed +- [x] JVM regression tests + physical-device A–E checklist (NOT RUN) +- [x] Task 10: CI green record (run 43936524131 at `3c7b58c`: 926 unit tests, 0 failures; APK sha256 `ade9…`); report