Skip to content

reapi: hash files and build a remote compile's input root - #4

Merged
tiny-ac merged 2 commits into
reapi/mvpfrom
reapi/digests
Oct 21, 2026
Merged

tiny-ac merged 2 commits into
reapi/mvpfrom
reapi/digests

Conversation

@tiny-ac

@tiny-ac tiny-ac commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Adds SHA-256 file digests, a cache of them, and the digest of a remote compile's input root, REAPI's name for the directory tree of every file the compile reads. Nothing calls them yet; a later PR sends them to a REAPI server.

DigestCache hashes a file again when its size, modification time, ctime or inode changes. It doesn't cache files changed in the last 2 s, since on a filesystem with coarse timestamps a same-size edit can leave the size, modification time and ctime unchanged. When full (100,000 entries), it drops every entry that isn't pinned. digest_and_pin pins an entry, save writes the pinned entries to a file, and load reads them back as pinned.

InputRoot::build lays out the files as REAPI's canonical Directory tree and returns the root's digest. It fails on conflicting or invalid paths.

Changes

  • src/reapi/digest.rs: of_bytes, of_file and DigestCache.
  • src/reapi/input_root.rs: InputRoot and InputRootFile.
  • Cargo.toml: reapi enables the existing optional sha2 dependency.

Cost

Without the reapi feature, nothing new is compiled. With it, only sha2 is added, which dist-client already uses.

Known limits

On Windows the cache checks a file's creation time instead of its ctime and inode, so an in-place edit that keeps the size and modification time goes unnoticed. sccache's preprocessor cache has the same limit.

Validation

  • fmt, and clippy with -D warnings and --features reapi, at each commit.
  • cargo test --features reapi --lib reapi::: 27 new tests, 13 for digests (one Unix-only, one Linux-only) and 14 for the input root.
  • cargo check --no-default-features --features reapi.

@tiny-ac
tiny-ac merged commit 0cd83f5 into reapi/mvp Oct 21, 2026
63 of 99 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant