You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test(cuda.core): keep graph-pool tests off the device default pool - #3496
The device default mempool and the graph memory pool each reserve about twice the device memory of virtual address space for the lifetime of the process, and neither can be capped. A test that touches both needs two such reservations at once, which does not fit into a small virtual address space on a large-memory GPU (e.g. two ~180-190 GiB reservations on a 96 GB device vs. a 256 GiB address space).
Running each cuda_core test in its own process and probing both pools at session end, only two tests touch both:
graph/test_graph_memory_resource.py::test_graph_alloc (graph-capture params): DeviceMemoryResource(device) for the output buffer, plus GraphMemoryResource.
test_object_protocols.py::test_no_cross_type_equality: sample_buffer (Device.allocate) plus sample_alloc_node.
Both only need some device pool, so they now use a pool capped at POOL_SIZE:
test_graph_alloc creates a capped DeviceMemoryResource and releases the buffer and pool in a try/finally.
sample_buffer and sample_buffer_alt allocate from a new sample_memory_resource fixture (capped pool closed at teardown; falls back to the device's memory resource if pools are unsupported).
This deliberately departs from the cuda_core/tests/AGENTS.md rule against adding max_size to a call that passes no options, since that rule assumes the default pool already exists. AGENTS.md now documents the exception.
No actionable comments were generated in the recent review.
Summary
Summary by CodeRabbit
Tests
Updated memory-resource and object-protocol tests to use capped memory pools where supported, with a fallback to the device’s existing resource when pools are unavailable.
Added cleanup and device synchronization after pool use, and tested buffers of different sizes.
Documentation
Clarified testing guidance for cases that use both device default pools and graph memory pools.
The object protocol tests and graph allocation test now use device memory resources capped at POOL_SIZE. The graph allocation test also closes its device memory resource and synchronizes the device in a finally block.
A fixture uses the device resource when memory pools are unsupported. Otherwise, it creates a resource capped at POOL_SIZE, then closes it and synchronizes the device. The 64-byte and 1024-byte buffers allocate through this fixture on the default stream. Test guidance describes this capped-pool pattern.
test_graph_alloc caps its device memory resource at POOL_SIZE, uses a context-managed output buffer, then closes the resource and synchronizes the device in a finally block. The graph memory resource remains uncapped.
Change: Other
Merge Risk: Minimal · up to 80985
This PR caps the device memory pools used by two tests and releases them explicitly at the end of each test, so the tests no longer reserve address space for both the default and graph pools. The earlier cleanup concern is addressed. Only test code changes, so merge risk is minimal.
Security architecture risk: Minimal · up to 80985
The change uses capped, explicitly owned memory pools in tests without changing production interfaces or trust boundaries. No introduced or worsened architecture-level security risk was established.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The device default mempool and the graph memory pool each reserve about
twice the device memory of virtual address space for the lifetime of the
process, and neither can be capped (NVIDIA#2381). A test that touches both needs
two such reservations at once, which does not fit in a small virtual
address space with a large-memory GPU.
Measured per test (one pytest process each, probing both pools at session
end), only two tests touch both pools:
- graph/test_graph_memory_resource.py::test_graph_alloc used
DeviceMemoryResource(device), the default pool, for its output buffer.
- test_object_protocols.py::test_no_cross_type_equality combined
sample_buffer (Device.allocate, default pool) with sample_alloc_node.
Both only need some device pool next to the graph pool. Use a pool capped
at POOL_SIZE instead. The object-protocol buffer fixtures now share a
capped pool fixture that is closed at teardown; on devices without memory
pool support it falls back to the device's memory resource as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cuda.coreEverything related to the cuda.core moduletestImprovements or additions to tests
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow-up to #2381.
The device default mempool and the graph memory pool each reserve about twice the device memory of virtual address space for the lifetime of the process, and neither can be capped. A test that touches both needs two such reservations at once, which does not fit into a small virtual address space on a large-memory GPU (e.g. two ~180-190 GiB reservations on a 96 GB device vs. a 256 GiB address space).
Running each
cuda_coretest in its own process and probing both pools at session end, only two tests touch both:graph/test_graph_memory_resource.py::test_graph_alloc(graph-capture params):DeviceMemoryResource(device)for the output buffer, plusGraphMemoryResource.test_object_protocols.py::test_no_cross_type_equality:sample_buffer(Device.allocate) plussample_alloc_node.Both only need some device pool, so they now use a pool capped at
POOL_SIZE:test_graph_alloccreates a cappedDeviceMemoryResourceand releases the buffer and pool in atry/finally.sample_bufferandsample_buffer_altallocate from a newsample_memory_resourcefixture (capped pool closed at teardown; falls back to the device's memory resource if pools are unsupported).This deliberately departs from the
cuda_core/tests/AGENTS.mdrule against addingmax_sizeto a call that passes no options, since that rule assumes the default pool already exists. AGENTS.md now documents the exception.Checklist