[server] Detach flushed entries from previousEntry chains in KvPreWriteBuffer - #4245
Conversation
| Set<Key> flushedKeys = new HashSet<>(); | ||
| for (KvEntry entry : preparedFlush.entries) { | ||
| if (flushedKeys.add(entry.getKey())) { | ||
| detachFromFlushedChain(kvEntryMap.get(entry.getKey())); |
There was a problem hiding this comment.
Could we maintain a doubly linked version chain by adding nextEntry to KvEntry?
The current approach walks from the newest version to the flushed boundary for every affected key in each segment. With many buffered versions of a hot key, successive segments repeatedly scan the remaining chain. This amplifies traversal work on the flush hot path and extends the time spent holding kvLock.
A successor pointer would let completeFlush() directly clear entry.nextEntry.previousEntry and detach the flushed entry during the existing removal loop. This would eliminate the flushedKeys set, combine the two passes into one, and make cleanup O(1) per flushed entry, regardless of the remaining chain length.
The tradeoff seems manageable: one additional reference per entry and constant-time pointer maintenance during insertion and truncation. Insertion already has the previous entry available, and truncation can directly clear the retained predecessor’s nextEntry. We would need to maintain both directions consistently so truncated entries cannot remain reachable.
There was a problem hiding this comment.
Good point. KvEntry now carries a nextEntry successor pointer, so completeFlush() clears entry.nextEntry.previousEntry inside the existing removal loop; the flushedKeys set and the chain walk are gone. doPut and truncateTo maintain the forward link in constant time on insertion and truncation, so cleanup is O(1) per flushed entry regardless of the remaining chain length. Tests now also assert the forward links stay consistent after flush and truncation.
Purpose
Linked issue: close #4243
Brief change log
For each key that had entries flushed, cut the
previousEntrychain of the newest still-buffered version at its first reference to a FLUSHED entry. Reachability is transitive, so this single cut makes the whole flushed suffix eligible for GC.References to ACTIVE/PREPARED entries are never touched, and FLUSHED references are already invisible to truncation rollback (
previousEntryInBuffer), so rollback semantics are preserved.Tests
Two new cases in
KvPreWriteBufferTest: chain detachment at the flushed boundary (and not growing back across flush cycles), and unchanged truncation rollback behavior.API and Format
Documentation