Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions cuda_core/cuda/core/graph/_graph_definition.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,58 @@ class GraphDefinition:
Graph
An executable graph that can be launched on a stream.
"""
def clone(self) -> GraphDefinition:
"""Create an independent copy of this graph definition.

Nodes, edges, parameters, and nested child graphs are copied. Resources
referenced by nodes (buffers, kernels, events, and callbacks) are shared
and independently retained; their contents are not copied. Subsequent
graph mutations do not affect the other definition or its executables.
Comment on lines +198 to +201

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

important: Document which graphs clone() rejects.

The method help says it copies nodes, but cuGraphClone rejects otherwise valid graphs that contain allocation, free, or conditional nodes. The API guide lists these restrictions; the stub and runtime docstrings do not. Add the same limits to both docstrings so callers can check whether their graph is eligible before calling clone(). (docs.nvidia.com)


Returns
-------
GraphDefinition
An independently owned graph hierarchy. Use :meth:`find_node` on
the clone to obtain the counterparts of source nodes.

Notes
-----
CUDA rejects graphs containing memory allocation, memory free, or
conditional nodes.
Attached resources remain alive independently for each graph and are released
at the appropriate lifetime boundary.
Driver errors propagate as :class:`~cuda.core.CUDAError`.
Serialize this call with all CUDA access to the underlying graph.
See :ref:`cuda-core-graph-thread-safety`.
"""
def find_node(self, original: GraphNode) -> GraphNode:
"""Find a source node's counterpart in this cloned graph.

Parameters
----------
original : GraphNode
A valid node from the source graph that existed when this graph
was cloned. The corresponding cloned node must still exist.

Returns
-------
GraphNode
The corresponding node, with its operation-specific subclass.

Raises
------
ValueError
If ``original`` is the synthetic entry node.
RuntimeError
If either graph definition is invalid or ``original`` was destroyed.
CUDAError
If CUDA cannot find a counterpart for a valid source node.

Notes
-----
Lookup is local to this graph. Serialize this call with all CUDA
access to either hierarchy. See :ref:`cuda-core-graph-thread-safety`.
"""
def debug_dot_print(self, path: str, options: GraphDebugPrintOptions | None=None) -> None:
"""Write a GraphViz DOT representation of the graph to a file.

Expand Down
91 changes: 90 additions & 1 deletion cuda_core/cuda/core/graph/_graph_definition.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,17 @@ from libcpp.vector cimport vector

from cuda.bindings cimport cydriver

from cuda.core.graph._graph_node cimport GraphNode
from cuda.core.graph._graph_node cimport GraphNode, GN_check_valid
from cuda.core._rt cimport (
GraphHandle,
as_cu,
as_intptr,
as_py,
attach_rollback_failure,
create_graph_handle,
create_graph_node_handle,
graph_clone_attachments,
invalidate_root_graph_state,
)
from cuda.core._utils.cuda_utils cimport HANDLE_RETURN
IF CUDA_CORE_BUILD_MAJOR >= 13:
Expand Down Expand Up @@ -360,6 +363,92 @@ cdef class GraphDefinition:

return _instantiate_graph(self, options)

def clone(self) -> GraphDefinition:
"""Create an independent copy of this graph definition.

Nodes, edges, parameters, and nested child graphs are copied. Resources
referenced by nodes (buffers, kernels, events, and callbacks) are shared
and independently retained; their contents are not copied. Subsequent
graph mutations do not affect the other definition or its executables.

Returns
-------
GraphDefinition
An independently owned graph hierarchy. Use :meth:`find_node` on
the clone to obtain the counterparts of source nodes.

Notes
-----
CUDA rejects graphs containing memory allocation, memory free, or
conditional nodes.
Attached resources remain alive independently for each graph and are released
at the appropriate lifetime boundary.
Driver errors propagate as :class:`~cuda.core.CUDAError`.
Serialize this call with all CUDA access to the underlying graph.
See :ref:`cuda-core-graph-thread-safety`.
"""
GD_check_valid(self)
cdef GraphDefinition cloned = GraphDefinition.__new__(GraphDefinition)
cdef cydriver.CUgraph graph = NULL
cdef cydriver.CUresult result
try:
with nogil:
HANDLE_RETURN(cydriver.cuGraphClone(&graph, as_cu(self._h_graph)))
cloned._h_graph = create_graph_handle(graph)
HANDLE_RETURN(graph_clone_attachments(cloned._h_graph, self._h_graph))
except:
if graph != NULL:
# attempt roollback
with nogil:
result = cydriver.cuGraphDestroy(graph)
if result == cydriver.CUDA_SUCCESS:
if cloned._h_graph:
invalidate_root_graph_state(cloned._h_graph)
else:
attach_rollback_failure(
b"cuGraphDestroy", result,
b"failed while rolling back a cloned graph")
raise
return cloned

def find_node(self, GraphNode original not None: GraphNode) -> GraphNode:
"""Find a source node's counterpart in this cloned graph.

Parameters
----------
original : GraphNode
A valid node from the source graph that existed when this graph
was cloned. The corresponding cloned node must still exist.

Returns
-------
GraphNode
The corresponding node, with its operation-specific subclass.

Raises
------
ValueError
If ``original`` is the synthetic entry node.
RuntimeError
If either graph definition is invalid or ``original`` was destroyed.
CUDAError
If CUDA cannot find a counterpart for a valid source node.

Notes
-----
Lookup is local to this graph. Serialize this call with all CUDA
access to either hierarchy. See :ref:`cuda-core-graph-thread-safety`.
"""
GD_check_valid(self)
GN_check_valid(original)
if original._is_entry:
raise ValueError("The entry node has no cloned counterpart")
cdef cydriver.CUgraphNode node = NULL
with nogil:
HANDLE_RETURN(cydriver.cuGraphNodeFindInClone(
&node, as_cu(original._h_node), as_cu(self._h_graph)))
return GraphNode._create(self._h_graph, node)

def debug_dot_print(self, path: str, options: GraphDebugPrintOptions | None = None) -> None:
"""Write a GraphViz DOT representation of the graph to a file.

Expand Down
26 changes: 24 additions & 2 deletions cuda_core/docs/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ avoid recompiling identical source + options + target without writing the
CUDA graphs
-----------

.. currentmodule:: cuda.core

A CUDA graph captures a set of GPU operations and their dependencies,
allowing them to be defined once and launched repeatedly with minimal
CPU overhead. Graphs can be constructed in two ways:
Expand All @@ -154,6 +156,28 @@ CPU overhead. Graphs can be constructed in two ways:
edges. Both produce an executable :class:`~graph.Graph` that can be
launched on a :class:`Stream`.

:meth:`graph.GraphDefinition.clone` copies a definition and its nested child
graphs into an independently owned hierarchy. Node parameters and dependencies
are preserved. Buffers, kernels, events, and callback objects are shared and
retained independently; cloning does not copy their contents. Subsequent
mutations to either definition do not change the other definition or existing
executables.

Use :meth:`graph.GraphDefinition.find_node` on the clone to reconstruct a
source node's counterpart with its operation-specific subclass::

clone = definition.clone()
cloned_node = clone.find_node(original_node)
cloned_child = clone.find_node(original_child_node).child_graph
cloned_inner_node = cloned_child.find_node(original_inner_node)

Lookup is local to each graph and requires a valid source node that existed at
cloning time, and a corresponding node still present in the clone. The clone
does not retain its source hierarchy for lookup. CUDA does not support cloning
graphs containing allocation, free, or conditional nodes; driver errors
propagate. See :ref:`cuda-core-graph-thread-safety` for the synchronization
requirements of cloning and node lookup.

With CUDA >= 13.1, graph definitions and
executable graphs expose a read-only integer, ``id``. Nodes expose ``local_id``
(unique within their graph) and ``tools_id`` (for profiler correlation).
Expand All @@ -163,8 +187,6 @@ shares ownership of the underlying graph. IDs are ephemeral, driver-assigned.
``kernel.attributes.device_node_update_supported`` reports whether a kernel
supports device-side node updates. It requires CUDA >= 13.3.

.. currentmodule:: cuda.core

.. autosummary::
:toctree: generated/

Expand Down
21 changes: 19 additions & 2 deletions cuda_core/docs/source/concurrency.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
Concurrency and Thread Safety
=============================

``cuda.core`` allows concurrent reads of its objects from multiple host
threads, but concurrent *mutation* of the same object is **not supported** --
Subject to the CUDA driver limitations below, ``cuda.core`` allows concurrent
reads of its objects from multiple host threads. Concurrent *mutation* of the
same object is **not supported** --
for example, adding nodes to the same graph, or closing a resource while
another thread is using it. Whenever an object is shared across threads and at
least one of them may mutate it, the application is responsible for providing
Expand All @@ -26,3 +27,19 @@ safe. For example, modifying peer device access from one thread while another
thread accesses device memory affected by that change is unsafe, even though the
two threads use different objects. The application is responsible for
synchronizing operations that concurrently read and modify shared driver state.

.. _cuda-core-graph-thread-safety:

CUDA graph access
-----------------

Applications must serialize CUDA access to a shared graph hierarchy across host
threads, including read-only calls and calls through different views of that
hierarchy. :meth:`graph.GraphDefinition.clone` accesses the source hierarchy.
:meth:`graph.GraphDefinition.find_node` accesses both the source and the clone.
Hold the relevant lock or locks for the entire call. Independent graph
hierarchies can use separate locks.

See the CUDA Driver API's `Graph object thread safety
<https://docs.nvidia.com/cuda/cuda-driver-api/graphs-thread-safety.html>`_
documentation.
Loading
Loading