Repository navigation
Manage the lifetime of oneMKL FFT descriptors - #759
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #759 +/- ##
==========================================
- Coverage 86.83% 85.99% -0.85%
==========================================
Files 55 55
Lines 5136 5112 -24
==========================================
- Hits 4460 4396 -64
- Misses 676 716 +40 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Every plan commits a oneMKL descriptor and nothing destroyed it, so each `fft`, `bfft` or `ifft` call leaked one, together with the device memory committing allocates for its workspace (31 MiB for a 64×32000×8 ComplexF32 array transformed along dimension 2). A loop that transforms repeatedly fills the device after a few hundred calls. Descriptors are now owned by a `Descriptor` object that plans and the inverse plans made from them share. Its finalizer queues the descriptor, like sparse handles, and queued descriptors are destroyed on a regular task when creating a descriptor and when memory is reclaimed. That doesn't require synchronizing: compute calls wait for their work and keep the descriptor alive while they run. Co-authored-by: Tim Besard <tim.besard@gmail.com>
Neither garbage collector sees the workspace that committing a descriptor allocates, so a loop that makes plans and allocates little else, e.g. one calling `fft!`, filled the device before any plan was collected. Account for the workspace in `_allocated_bytes`, which only exists to drive the pool's proactive GC, so that the GC collects plans as the device fills up. oneMKL reports the exact size after committing; before committing it only gives an estimate, which is used to decide whether to collect first. Co-authored-by: Wilkie Olin-Ammentorp <wolinamm@gmail.com>
A descriptor executes on the SYCL queue it was committed to, but plans only kept its raw handle, while the queue object, and the SYCL context and Level Zero queue under it, live in the creating task's local storage. Once that task was collected, using a plan it created crashed. Descriptors now keep their queue rooted in a global registry until they are destroyed. Referencing it from the descriptor would only cover executing plans: once a task and its plans are collected together, the queue can be finalized before the deferred destruction of their descriptors.
maleadt
force-pushed
the
fft-descriptor-main
branch
from
November 5, 2026 11:30
4092913 to
94705c9
Compare
maleadt
changed the title
Member
|
Thanks. Rebased and slightly reworked. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #758.
Every FFT plan commits a oneMKL DFT descriptor, and nothing destroyed it, so each
fft,bfftorifftcall leaked one, along with the device memory that committing allocates for the descriptor's workspace: 31 MiB for a 64×32000×8 ComplexF32 array transformed along dimension 2. On an Arc A770 (16 GB), the loop from the issue (fft!(x, 2); ifft!(x, 2)) fails after a few hundred plans withFFT failed (-1).Descriptorobject owns the oneMKL handle, and is shared by a plan and the inverse plans made from it. Its finalizer queues it, like sparse handles, and queued descriptors are destroyed on a regular task when creating a descriptor and when memory is reclaimed. This doesn't need to synchronize: compute calls wait for their work and keep the descriptor alive while they run._allocated_bytes, so the pool's existing proactive GC collects plans as the device fills up.On the A770, the issue's loop now completes 3000 plans in 33 s in place and 34 s out of place. Reusing a plan (
p * x) remains the fast path; reusing committed descriptors across plans, as CUDA.jl's cuFFT does with a handle cache, is left for a follow-up.