Skip to content

[server] Detach flushed entries from previousEntry chains in KvPreWriteBuffer - #4245

Merged
platinumhamburg merged 3 commits into
apache:mainfrom
Kaixuan-Duan:fix-4243-kv-prewrite-previous-entry-leak
Sep 8, 2026
Merged

[server] Detach flushed entries from previousEntry chains in KvPreWriteBuffer#4245
platinumhamburg merged 3 commits into
apache:mainfrom
Kaixuan-Duan:fix-4243-kv-prewrite-previous-entry-leak

Conversation

@Kaixuan-Duan

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #4243

Brief change log

For each key that had entries flushed, cut the previousEntry chain of the newest still-buffered version at its first reference to a FLUSHED entry. Reachability is transitive, so this single cut makes the whole flushed suffix eligible for GC.

References to ACTIVE/PREPARED entries are never touched, and FLUSHED references are already invisible to truncation rollback (previousEntryInBuffer), so rollback semantics are preserved.

Tests

Two new cases in KvPreWriteBufferTest: chain detachment at the flushed boundary (and not growing back across flush cycles), and unchanged truncation rollback behavior.

API and Format

Documentation

Set<Key> flushedKeys = new HashSet<>();
for (KvEntry entry : preparedFlush.entries) {
if (flushedKeys.add(entry.getKey())) {
detachFromFlushedChain(kvEntryMap.get(entry.getKey()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we maintain a doubly linked version chain by adding nextEntry to KvEntry?
The current approach walks from the newest version to the flushed boundary for every affected key in each segment. With many buffered versions of a hot key, successive segments repeatedly scan the remaining chain. This amplifies traversal work on the flush hot path and extends the time spent holding kvLock.
A successor pointer would let completeFlush() directly clear entry.nextEntry.previousEntry and detach the flushed entry during the existing removal loop. This would eliminate the flushedKeys set, combine the two passes into one, and make cleanup O(1) per flushed entry, regardless of the remaining chain length.
The tradeoff seems manageable: one additional reference per entry and constant-time pointer maintenance during insertion and truncation. Insertion already has the previous entry available, and truncation can directly clear the retained predecessor’s nextEntry. We would need to maintain both directions consistently so truncated entries cannot remain reachable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. KvEntry now carries a nextEntry successor pointer, so completeFlush() clears entry.nextEntry.previousEntry inside the existing removal loop; the flushedKeys set and the chain walk are gone. doPut and truncateTo maintain the forward link in constant time on insertion and truncation, so cleanup is O(1) per flushed entry regardless of the remaining chain length. Tests now also assert the forward links stay consistent after flush and truncation.

@platinumhamburg platinumhamburg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1.

@platinumhamburg
platinumhamburg merged commit dbf908a into apache:main Sep 8, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[KV] Flushed entries remain reachable through previousEntry chains in KvPreWriteBuffer

2 participants