Repository navigation
fix(windows): stop trusting recycled HWNDs in the capsule hit-test subclass - #1267
Open
lxcadoza993 wants to merge 1 commit into
Open
lxcadoza993 wants to merge 1 commit into
lxcadoza993 wants to merge 1 commit into
Conversation
…bclass
`install_capsule_hit_test_proc` keys its bookkeeping on the numeric HWND, but
Windows recycles HWND values. Once a capsule/card window is destroyed and its
handle value is handed to a new window, a lookup by that key still finds the
dead window's entry, and three things go wrong:
- the fast path reuses the stale entry and returns early, so the *new* window
never gets the subclass procedure installed -- hit-testing silently stops
working for it;
- `WM_NCHITTEST` hit-tests the live window against the dead window's geometry;
- message forwarding can call a `previous_proc` that belonged to another window.
The only reliable identity check is whether *our own* procedure is still
installed on the handle: a recycled HWND carries whatever procedure its new
owner installed. Every read of the map now proves that first, and an entry that
fails the check is dropped and re-installed instead of trusted.
Two supporting fixes while in here:
- `install_capsule_hit_test_proc` now refuses to record itself as the
predecessor. When the map has been wiped but the procedure is already on the
window, `SetWindowLongPtrW` returns our own address; storing it would make
every forwarded message recurse into this same procedure.
- `WM_NCDESTROY` only retires the entry this window owns.
Adds `reinstall_keeps_the_real_predecessor_and_never_self_wraps`, which uses a
real `HWND_MESSAGE` window because the identity check reads the live
`GWLP_WNDPROC` and cannot be mocked.
Note on running tests: `cargo test --lib` exits with `0xC0000139`
(STATUS_ENTRYPOINT_NOT_FOUND) on Windows before any test runs. The crate
imports `comctl32!TaskDialogIndirect`, which lives in Common-Controls v6, and
v6 is bound only through the `build.rs` `/manifestdependency` link arg -- the
cargo test harness links separately and does not get that manifest. Embedding a
v6 manifest in the test binary makes it run:
mt.exe -nologo -manifest cc6.manifest "-outputresource:<test-exe>;Open-Less#1"
With that, the new test passes (1 passed; 0 failed), as does `cargo check`
(0 errors) and `cargo build --release`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
install_capsule_hit_test_prockeeps its bookkeeping in aHashMap<isize, CapsuleHitTestEntry>keyed by the numeric HWND value. Windows recycles HWND values: after a capsule/card window is
destroyed, the same handle value can be handed to an unrelated new window. Because the map is keyed
by value, a lookup during the new window's lifetime still finds the destroyed window's entry, and
three separate things go wrong:
the new geometry into the stale entry, and returns
Ok(())without installing anything. The newwindow therefore never carries our subclass procedure, so it never receives
WM_NCHITTESTand theclick-through region stops being honoured.
WM_NCHITTESThit-tests the wrong geometry — the live window is tested against the deadwindow's control rect.
previous_proc.WM_NCDESTROYremoved the entry by key alone, so a recycled handle could retire a survivingwindow's entry, stranding it with our procedure installed and nothing recorded to forward to.
Why the fix looks like this
The only fact that reliably distinguishes "same window" from "recycled handle value" is whether our
own procedure is still installed on the handle: a recycled HWND carries whatever procedure its new
owner installed. So every read of the map now proves that first, and an entry that fails the check is
dropped and re-installed rather than trusted.
Two supporting fixes:
install_capsule_hit_test_procrefuses to record itself as the predecessor. When the map hasbeen wiped but the procedure is already on the window,
SetWindowLongPtrWreturns our own address;storing it would make every forwarded message recurse into this same procedure until the stack
overflows. In that case no entry is recorded and
DefWindowProcWtakes the traffic.WM_NCDESTROYonly retires the entry this window owns.I deliberately did not add a "creating thread id" check alongside this. Once we require that our
own procedure is on the handle, the thread id is redundant, and it would add a
GetWindowThreadProcessIdsyscall to the
WM_NCHITTESThover path, which runs per mouse move.Reachability
On the current
beta,desktop_resources::hide_capsule_windowcallswindow.destroy()when desktoplow-memory mode is active, so capsule windows genuinely are destroyed and recreated — the recycled-handle
path is reachable, not theoretical.
configure_capsule_hit_test/configure_card_hit_testare bothcalled on every style/geometry update.
Testing
New test
reinstall_keeps_the_real_predecessor_and_never_self_wrapsregisters a real window class,creates an
HWND_MESSAGEwindow, and installs the subclass three times. It asserts that:previous_procpointing at the real classprocedure rather than at ourselves;
(the self-install guard), so forwarding cannot recurse.
A real window is required here — the identity check reads the live
GWLP_WNDPROC, which no mock canstand in for.
Results:
1 passed; 0 failed.cargo check→ 0 errors, warning count unchanged.cargo build --release→ OK.Heads-up:
cargo test --libdoes not run on Windows out of the boxBefore any test executes, the test binary exits with
0xC0000139(STATUS_ENTRYPOINT_NOT_FOUND).The crate imports
comctl32!TaskDialogIndirect, which only exists in Common-Controls v6; v6 isbound solely through the
/manifestdependencylink arg thatbuild.rsemits vialink_windows_common_controls_v6_manifest_dependency(). The cargo test harness links separatelyand does not receive that manifest, so it loads against comctl32 v5 and fails at load time.
Workaround used to get the results above:
where
cc6.manifestdeclares theMicrosoft.Windows.Common-Controls6.0.0.0 dependency. This ispre-existing and independent of this change, but it means
cargo test --libcurrently gives the wholerepo zero coverage. Worth fixing separately (e.g. a
rustflagslink arg in.cargo/config.toml);happy to send that as its own PR if you want it.
Scope — please read before merging as a crash fix
This is a robustness / correctness fix. It is not a fix for the
0xC0000374heap-corruptioncrashes. I traced that separately from two local minidumps and it does not implicate this code: the
subclass path never produced an invalid pointer to free, and the crash build did not even destroy
capsule windows (
desktop_resources.rscame later, in6336823). I have not claimed a root cause forthe crashes anywhere in this PR and do not want this merged under that impression.
If it's useful I can also post the heap-failure decode from the two dumps (
FailureType8block_not_busyand 13freelists_corruption, both fromRtlpHeapHandleError) as a separate issue —that data is what #1227/#1261 asked for, but it stops at "the two crashes are different shapes of the
same free-side corruption", which is not yet enough to name a culprit.