Skip to content

ck-libs/cache: CkTreeCacheManager, a process-shared tree-node cache with CkCache's protocol - #4013

Merged
lvkale merged 1 commit into
mainfrom
treecache-nodecache-main
Oct 5, 2026
Merged

lvkale merged 1 commit into
mainfrom
treecache-nodecache-main

Conversation

@lvkale

@lvkale lvkale commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Adds CkTreeCacheManager<Key, Traits> to ck-libs/cache, beside CkCacheManager. It keeps CkCache's request/reply protocol (requestData, recvData, chunk finish) but stores fetched tree nodes once per process instead of once per PE: one process-shared store owned by the rank-0 branch, lock-free on the fetch path (atomic child-pointer exchange and parked request lists, from paratreet2's TreeCacheCore.h, vendored here with a provenance block). The application supplies a small static Traits class that maps its node type onto the store. First client: ChaNGa's tree-node cache (N-BodyShop/changa#228), where a configure option selects this or CkCacheManager.

Changes to existing code: CkCache.h now always compiles its request/reply counters (a few integer increments per request). Before, they were compiled only under the application macro COSMO_STATS, which changed the class layout in the application but not in libmoduleCkCache (the registration stubs live in the library). CmiResetMaxMemory is skipped under CMK_RECONVERSE. Nothing else in charm uses the new class.

Validation:

  • Classic Charm++ (netlrts-darwin-arm8-smp, and mpi-linux-x86_64-smp on Purdue Anvil) and reconverse (Anvil, NCSA Delta, OLCF Frontier): ChaNGa teststep energy and restart tests pass, results identical to CkCache.
  • ChaNGa gravity, 80.6M particles, big steps 1-3 summed, new cache vs CkCache in the same tree (--enable-smpcache=no), 2 repeats:
    • classic mpi-smp on Anvil, 8 processes x 8 PEs per node: 0.79 at 4 nodes, 0.77 at 8 nodes (cache teardown per step 2.5-3.6 s -> 0.2-0.55 s).
    • reconverse: 0.82-0.95 across 17 layouts on Delta; 0.83-0.96 at 4-32 nodes on Frontier with the simulation's own parameters.

The same commit applies cleanly to reviewed-with-reconverse (branch treecache-nodecache, needs a rebase onto its current tip).

🤖 Generated with Claude Code

@lvkale

lvkale commented Sep 29, 2026

Copy link
Copy Markdown
Contributor Author

The same commit for reviewed-with-reconverse is #4014.

…ache's protocol

A nodegroup (one branch per process, shared lock-free by its PEs) that
offers CkCacheManager's request/reply protocol and member names for
caches whose entries are the nodes of a tree, backed by paratreet2's
TreeCacheCore over the application's own process-shared tree, reached
through a Traits parameter. Missing children become placeholder nodes;
requestors park on them; a reply (cache line) is built fully and then
published with one atomic child-pointer exchange, delivering the parked
requestors exactly once. Chunk teardown is process-wide (the location
managers of every PE of the node are counted). First client: ChaNGa's
node cache.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lvkale
lvkale force-pushed the treecache-nodecache-main branch from ff1a531 to da0c95a Compare October 5, 2026 14:30
@lvkale
lvkale merged commit e117631 into main Oct 5, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants