Repository navigation
Declare RR target pointers LIBINT2_RESTRICT - #449
Merged
Merged
Conversation
Generated RR functions take the target and the source integral sets as plain pointers, so every store to the target forces the compiler to reload source values (and inteval quantities) it already loaded. Targets never overlap their sources: allocate_mem assigns a vertex its block before releasing its children's blocks. Declare the target (the stack, for prerequisite functions) restrict, and check at code generation that no RR target overlaps any of its children. Adds the LIBINT2_RESTRICT macro to config.h. It expands to __restrict__ for GCC-compatible compilers because macOS <sys/cdefs.h> defines __restrict to nothing in C++.
Same reasoning as for the generated RR functions, which forward their arguments to these templates when LIBINT2_ENABLE_GENERIC_CODE is ON. With an unqualified target the inner loops reload the inteval quantities (oo2z, roz, PA, WP, ...) after every store to the target.
7 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #443. This hoists more loads out of generated kernels by telling the compiler that RR targets don't alias their sources.
What
Generated RR functions take
LIBINT2_REALTYPE* target, const LIBINT2_REALTYPE* src0, ...with no aliasing information. Each store totargettherefore forces the compiler to reload source values andintevalquantities it already loaded. In a scalar library with am ≤ 4, about 109k source reads in 1164 RR functions come after a store totarget.This PR:
LIBINT2_RESTRICTmacro toconfig.h;LIBINT2_RESTRICT. That covers 2214 RR functions and, throughparent_stack, 587_prereqfunctions;src/lib/libint/*.h)LIBINT2_RESTRICTas well;DirectedGraph::print_def) that no RR target overlaps any of its children.Why it's sound.
allocate_memassigns a vertex its stack block before it releases its children's blocks, so a target never overlaps a live source. The new check enforces this: it validates over 10k target/child pairs even for an am-2 library. Inside RR functions every store goes throughtarget; none of them usesstackscratch space orinteval->vstack. No_prereqfunction touchesinteval->stack.Pitfall. The macro must not expand to
__restrictwith GCC-compatible compilers, because macOS<sys/cdefs.h>defines__restrictto nothing in C++. A first attempt with__restrictproduced identical machine code.LIBINT2_RESTRICTuses__restrict__there,__restrictfor MSVC, and nothing otherwise. In a g-shell OS VRR kernel, load instructions drop from 907 to 430.Testing
Built and ran
check(compiler coretest, eritest, unit tests with 20M assertions, hf, hf++) for both master (a1c3e63) and this branch in two configurations:LIBINT2_ENABLE_GENERIC_CODE=ON,LIBINT2_OPT_AM=1;1and G12 to am 2. Here theOSVRR_xs_xs,OSVRR_xs_xs_derivandGenericGaussDerivtemplates are on the hot path.All tests pass in all four builds. Benchmark checksums are bit-identical between master and the branch.
Not covered: no config instantiates
OSVRR_sx_sx*,ITR_xs_xs,VRR_GTG_1d_xx_xx*orVRR_r12kg12_xs_xs. Syntax-checking all 12 template headers together gives the same 21 errors on master and on this branch, so the change adds none. Those errors are pre-existing, e.g. stray)inOSVRR_sx_sx_deriv.hand an undeclarednptsinVRR_GTG_1d_xx_xx_vec.h. Vectorized builds (LIBINT2_MAX_VECLEN > 1) were not tested.Timings
Apple M-series, Apple clang
-O3. Each case covers all unique shell sets. Libraries alternate within each round; the table shows the min over 3 rounds × 7 runs. Small cases (≤ 10 ms) vary by up to ~4% between rounds.The gains follow the share of time spent in RR kernels: 15–19% for quadruple-zeta and derivative ERIs, a few percent for small basis sets. Those are dominated by per-shell-set overhead.
Considered and dropped: top-level kernels. They re-read
inteval->stackbefore every HRR call. Using the existingstacklocal, with or without restrict, changed timings by at most 0.2% (5 rounds × 15 runs), so it is not part of this PR.