Skip to content

macOS: keep foreground tracking coherent when browser context fails - #141

Closed
hawai-i wants to merge 1 commit into
ActivityWatch:masterfrom
wave-i:fix/macos-foreground-reconciliation
Closed

hawai-i wants to merge 1 commit into
ActivityWatch:masterfrom
wave-i:fix/macos-foreground-reconciliation

Conversation

@hawai-i

@hawai-i hawai-i commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Browser URL/title enrichment is optional context, but the macOS Swift watcher currently returns without emitting a foreground heartbeat when some Chrome/Safari ScriptingBridge lookups fail. Heartbeat merging then extends the previous application's event across the real browser interval.

This change:

  • emits foreground app/title evidence even when browser context acquisition fails;
  • centralizes activation and polling through one PID reconciliation path;
  • binds the tracked application, AX observer, focused window, and emitted app identity to the same PID;
  • repairs a missing/stale observer and rejects stale AX callbacks;
  • consumes the activated NSRunningApplication from the workspace notification;
  • safely removes the previous title notification and run-loop source.

Validation

A controlled Codex → Chrome → Finder → Chrome → Codex sequence was scored against WindowServer's top visible layer-0 owner. Browser context failure was forced at the existing pre-heartbeat failure point in both the before and fixed test builds.

Metric Before After
Overall foreground accuracy 41.9460% 99.8482%
Chrome accuracy 0.0000% 99.8799%
Boundaries within ±3 s 2/4 4/4
Unmatched time 60.811 s 0.159 s

A normal companion build also retained same-title Chrome page-a → page-b URL changes.

Automated Swift state tests cover:

  • PID change → observer rebuild;
  • same PID → focused-window refresh;
  • missing observer → repair;
  • stale callback → reject;
  • callback for the current PID → accept.

make build-swift, make test-swift, a real PyInstaller 6.11 package build, mypy, and git diff --check passed. The packaged arm64 helper preserves the macOS 12.0 minimum. The borrowed local Python environment reported 61 passing tests and two macOS config-path failures that reproduce unchanged on the base commit.

Related work

ActivityWatch/activitywatch#1411 grants the responsible outer app Apple Events permission, which lets browser context acquisition succeed. This PR is logically independent: foreground truth must still be emitted when optional context is unavailable.

This also corrects the mismatched title-notification cleanup and observer teardown discussed in #139, but this PR does not claim a separate multi-day leak validation.

@greptile-apps

greptile-apps Bot commented Aug 30, 2026

Copy link
Copy Markdown

Greptile Summary

The PR restructures the native macOS watcher around PID-based foreground reconciliation and preserves app/title heartbeats when optional browser enrichment fails.

  • Centralizes startup, activation, polling, AX observer repair, and stale-callback rejection around one tracked PID.
  • Adds fallback heartbeat delivery for failed Chrome and Safari context acquisition.
  • Adds isolated Swift state tests and wires them into the Darwin test target.
  • Updates the helper build to compile the new reconciliation source.

Confidence Score: 3/5

The PR should not merge until the Chrome fallback preserves incognito privacy when browser scripting is unavailable.

The new fallback fixes dropped foreground evidence but also sends AX-derived Chrome identity and titles when the failed browser lookup prevents the only incognito check; the Swift test target additionally does not exercise the supported deployment floor.

Files Needing Attention: aw_watcher_window/macos.swift, Makefile

Security Review

The Chrome context-failure fallback can expose incognito application and title data because ScriptingBridge is both the enrichment mechanism and the only incognito detector. How this was verified: Tracing the failed optional ScriptingBridge chain shows that AX-derived data reaches sendHeartbeat while incognito blanking exists only in the successful branch.

Important Files Changed

Filename Overview
aw_watcher_window/macos.swift Consolidates foreground reconciliation and improves recovery, but the browser-failure fallback bypasses Chrome incognito blanking.
aw_watcher_window/macos_state.swift Adds small, deterministic helpers for observer rebuild decisions and stale AX callback rejection.
tests/macos_state_tests.swift Covers PID changes, observer repair, and callback ownership for the extracted reconciliation helpers.
Makefile Wires the new Swift source and tests into builds, though the test compiler invocation does not preserve the production macOS 12 target.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Startup, activation, or poll] --> B[Select NSRunningApplication]
  B --> C{Tracked PID changed or observer missing?}
  C -->|Yes| D[Rebuild AX observer]
  C -->|No| E[Reuse observer]
  D --> F[Refresh focused AX window]
  E --> F
  F --> G[Read app and window title]
  G --> H{Supported browser?}
  H -->|Chrome or Safari| I{Browser context available?}
  I -->|Yes| J[Add URL and tab title]
  I -->|No| K[Keep AX app and title]
  H -->|Firefox| L[Read URL from AX tree]
  H -->|Other| M[Keep app and title]
  J --> N[Apply privacy filters]
  K --> N
  L --> N
  M --> N
  N --> O[Send heartbeat]
Loading

Reviews (1): Last reviewed commit: "fix(macos): preserve foreground events w..." | Re-trigger Greptile

Comment on lines +738 to 740
} else {
log("Failed to read Chrome context; emitting foreground heartbeat without URL")
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Fallback bypasses incognito blanking

If Chrome automation access is unavailable while Accessibility access remains granted, this fallback sends the AX-derived application and window title even for an incognito window, causing private browsing information to be stored in the currentwindow bucket. Incognito blanking occurs only inside the successful ScriptingBridge branch, with no equivalent guard before sendHeartbeat. How this was verified: The failed optional ScriptingBridge chain was traced to heartbeat delivery while the only incognito check remains inside its successful branch.

Knowledge Base Used:

Comment thread Makefile
fi

test-swift:
swiftc aw_watcher_window/macos_state.swift tests/macos_state_tests.swift -o /tmp/aw-watcher-window-macos-state-tests

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Tests omit deployment target

The new state-test command does not use the production helper's macOS 12 target, so it can pass with APIs available only on the runner's newer deployment target and provide false compatibility coverage.

Suggested change
swiftc aw_watcher_window/macos_state.swift tests/macos_state_tests.swift -o /tmp/aw-watcher-window-macos-state-tests
swiftc -target "$(shell uname -m)-apple-macosx$(MACOSX_DEPLOYMENT_TARGET)" aw_watcher_window/macos_state.swift tests/macos_state_tests.swift -o /tmp/aw-watcher-window-macos-state-tests

Knowledge Base Used:

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@ErikBjare

Copy link
Copy Markdown
Member

@TimeToBuildBob build on top of this to resolve conflict and fix Greptile concerns. Make sure to keep attribution.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Rebased onto current master (after #145) and opened #146.

Greptile P1: Chrome ScriptingBridge failure now keeps app identity but drops title and URL, since that lookup is the only incognito detector.

Greptile P2: make test-swift compiles with the macOS 12 deployment target.

Original work stays attributed to hawai-i (Co-Authored-By on the rebase commit). Leaving this PR open until #146 merges.

TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Sep 15, 2026
Adding macos_state.swift made macos.swift a non-script compilation unit.
Top-level assignments (formatter options, encoder strategy, start()) are
illegal outside main.swift. Restore the ActivityWatch#141 closure inits and @main
wrapper that the rebase dropped.

Git-Session-Id: ca3f0209-55b1-58ce-8820-82fb6f5bc8e0
TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Oct 6, 2026
Rebase of ActivityWatch#141 (hawai-i) onto current
master after ActivityWatch#145.

Keep PID-based foreground reconciliation and emit a heartbeat when
Chrome/Safari ScriptingBridge fails, so merging does not extend the
previous application across the real browser interval.

Preserve ActivityWatch#145 AX safety (axString/axElement) and observer teardown
(unregister focused-window notification, track observedApp).

Chrome context-failure fallback keeps app identity for tracking but
drops title and URL: ScriptingBridge is the only incognito detector
(Greptile P1 on ActivityWatch#141). Swift state tests now compile with the macOS
12 deployment target (Greptile P2).

Co-Authored-By: hawai-i <32040032+hawai-i@users.noreply.github.com>
Git-Session-Id: 9615c06b-2a11-592d-b1ad-299a3433e323
TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Oct 6, 2026
Adding macos_state.swift made macos.swift a non-script compilation unit.
Top-level assignments (formatter options, encoder strategy, start()) are
illegal outside main.swift. Restore the ActivityWatch#141 closure inits and @main
wrapper that the rebase dropped.

Git-Session-Id: ca3f0209-55b1-58ce-8820-82fb6f5bc8e0
TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Oct 14, 2026
Rebase of ActivityWatch#141 (hawai-i) onto current
master after ActivityWatch#145.

Keep PID-based foreground reconciliation and emit a heartbeat when
Chrome/Safari ScriptingBridge fails, so merging does not extend the
previous application across the real browser interval.

Preserve ActivityWatch#145 AX safety (axString/axElement) and observer teardown
(unregister focused-window notification, track observedApp).

Chrome context-failure fallback keeps app identity for tracking but
drops title and URL: ScriptingBridge is the only incognito detector
(Greptile P1 on ActivityWatch#141). Swift state tests now compile with the macOS
12 deployment target (Greptile P2).

Co-Authored-By: hawai-i <32040032+hawai-i@users.noreply.github.com>
Git-Session-Id: 9615c06b-2a11-592d-b1ad-299a3433e323
TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Oct 14, 2026
Adding macos_state.swift made macos.swift a non-script compilation unit.
Top-level assignments (formatter options, encoder strategy, start()) are
illegal outside main.swift. Restore the ActivityWatch#141 closure inits and @main
wrapper that the rebase dropped.

Git-Session-Id: ca3f0209-55b1-58ce-8820-82fb6f5bc8e0
@ErikBjare

Copy link
Copy Markdown
Member

@TimeToBuildBob Is this still needed? If so, fix it (conflict).

TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Nov 4, 2026
Rebase of ActivityWatch#141 (hawai-i) onto current
master after ActivityWatch#145.

Keep PID-based foreground reconciliation and emit a heartbeat when
Chrome/Safari ScriptingBridge fails, so merging does not extend the
previous application across the real browser interval.

Preserve ActivityWatch#145 AX safety (axString/axElement) and observer teardown
(unregister focused-window notification, track observedApp).

Chrome context-failure fallback keeps app identity for tracking but
drops title and URL: ScriptingBridge is the only incognito detector
(Greptile P1 on ActivityWatch#141). Swift state tests now compile with the macOS
12 deployment target (Greptile P2).

Co-Authored-By: hawai-i <32040032+hawai-i@users.noreply.github.com>
Git-Session-Id: 9615c06b-2a11-592d-b1ad-299a3433e323
TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Nov 4, 2026
Adding macos_state.swift made macos.swift a non-script compilation unit.
Top-level assignments (formatter options, encoder strategy, start()) are
illegal outside main.swift. Restore the ActivityWatch#141 closure inits and @main
wrapper that the rebase dropped.

Git-Session-Id: ca3f0209-55b1-58ce-8820-82fb6f5bc8e0
@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Yes—the browser-context fallback and PID/observer reconciliation are still needed; #172 fixes startup initialization, not those paths. The maintained replacement is #146 rather than this original branch.

I have already resolved that replacement's conflict with current master (25d8940): its new head is 1e85a37, and GitHub confirms it is mergeable. It retains hawai-i's attribution in the first commit, the Greptile privacy/deployment-target repairs, and master's queued heartbeat delivery and app-exclusion behavior. The rebase passed 198 Python tests plus 11 subtests locally; native Swift validation is still pending in macOS CI.

Please use #146 for review/merge. I am leaving this original PR open until the replacement lands, as previously agreed, rather than maintaining two competing versions of the same change.

ErikBjare pushed a commit that referenced this pull request Nov 4, 2026
* fix(macos): rebase foreground reconciliation onto master

Rebase of #141 (hawai-i) onto current
master after #145.

Keep PID-based foreground reconciliation and emit a heartbeat when
Chrome/Safari ScriptingBridge fails, so merging does not extend the
previous application across the real browser interval.

Preserve #145 AX safety (axString/axElement) and observer teardown
(unregister focused-window notification, track observedApp).

Chrome context-failure fallback keeps app identity for tracking but
drops title and URL: ScriptingBridge is the only incognito detector
(Greptile P1 on #141). Swift state tests now compile with the macOS
12 deployment target (Greptile P2).

Co-Authored-By: hawai-i <32040032+hawai-i@users.noreply.github.com>
Git-Session-Id: 9615c06b-2a11-592d-b1ad-299a3433e323

* fix(macos): restore @main entry so two-file swiftc compiles

Adding macos_state.swift made macos.swift a non-script compilation unit.
Top-level assignments (formatter options, encoder strategy, start()) are
illegal outside main.swift. Restore the #141 closure inits and @main
wrapper that the rebase dropped.

Git-Session-Id: ca3f0209-55b1-58ce-8820-82fb6f5bc8e0

* fix(macos): drop stale same-window title callbacks and Safari AX titles

PID-only AX guards accepted queued title-change events from a previous
window of the same app. Bind title callbacks to the focused window.

Safari ScriptingBridge failure kept the AX window title, unlike Chrome.
Use the same keep-app / drop-title-URL fallback so private pages cannot
leak via accessibility when context lookup fails.

Git-Session-Id: 9a7625ed-1cac-5d98-b0d9-013b84aae02d

* fix(macos): release unused AXObserver when notification add fails

rebuildObserver created an AXObserver then returned on
AXObserverAddNotification failure without adopting it, leaking the Mach
port. Adopt and tear down through the #139 path. Bind foreground
identity in the caller so a failed install still emits a heartbeat.

Git-Session-Id: 28375f79-1abe-5518-8f8c-75fcc067315a

* fix(macos): drop unused AXObserver via ARC and isolate swift tests

The add-failure path never installed the observer in the run loop, so
tearDownObserver's CFRunLoopRemoveSource was a no-op. Nil the managed
reference instead; CFRelease would double-free a Swift-ARC CF type.

Write the Swift state-test binary into a mktemp directory so make test
does not execute a predictable path in /tmp.

Git-Session-Id: 28375f79-1abe-5518-8f8c-75fcc067315a

* fix(macos): retry title-change observer after transient AX add failure

If AXObserverAddNotification for kAXTitleChangedNotification fails once,
keep the focused window but leave registration retryable so the next poll
can install it. Poll heartbeats already carry the current AX title; this
restores live title-change callbacks after a transient add failure.

Git-Session-Id: 2b8b1429-6ecf-575e-b812-491b60c68d9e

* fix(macos): preserve app exclusion in reconciled heartbeat path

Git-Session-Id: 857984da-6219-579c-b1f4-44a9cc88fa80

---------

Co-authored-by: hawai-i <32040032+hawai-i@users.noreply.github.com>
@ErikBjare ErikBjare closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants