Repository navigation
Optimize substring copies in the Message.extractField hot path - #1340
Open
leonchen83 wants to merge 3 commits into
Open
leonchen83 wants to merge 3 commits into
leonchen83 wants to merge 3 commits into
Conversation
added 3 commits
October 10, 2026 18:34
Parsing allocated several Strings and arrays per field: a substring for tag conversion, a substring for the field value, and - for raw data fields - substring().getBytes() on every scan iteration to measure the encoded length. Fields are now parsed as a range (offset/length) of the raw message: - IntConverter.convert(String, offset, length) parses tags without allocating; convert(String) delegates to it. Error messages are unchanged. - The data field length scan uses the char count for string-equivalent charsets and a char-wise UTF-8 count otherwise, falling back to the original measurement for any other charset. - StringField materializes the value lazily on first access and then collapses to the value, releasing the raw message. equals/hashCode/ valueEquals work directly on the range using the String formulas.
FieldMap's typed getters (getBoolean/getChar/getChars/getInt/getDouble/ getDecimal) converted the materialized value String, paying a substring copy per call. The converters now accept a (String, offset, length) range and parse it in a single character scan: - IntConverter, CharConverter, BooleanConverter, CharArrayConverter: pure character checks, zero allocation (CharArrayConverter also drops the per-call regex compilation and the split copies). - DoubleConverter, DecimalConverter: fast path for plain numbers (single correctly-rounded division / BigDecimal.valueOf(unscaled, scale)); exponents, oversized values and invalid input fall back to the original implementation. convert(String) delegates to the range-based implementation, so results and error messages are unchanged. StringField exposes the conversion as to* methods, and FieldMap dispatches to them for string fields instead of calling getString().
hasValue() now returns true when the field carries a value.
Member
|
@leonchen83 thanks for the PR. 👍 |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1340 +/- ##
============================================
- Coverage 70.80% 70.19% -0.61%
- Complexity 2271 2306 +35
============================================
Files 161 161
Lines 9077 9241 +164
Branches 1192 1239 +47
============================================
+ Hits 6427 6487 +60
- Misses 2197 2265 +68
- Partials 453 489 +36 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Already done — I force-pushed the branch and the PR now contains only the 3 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
While profiling temporary object allocations with async-profiler, we found that
Message.extractFieldis one of the hottest allocation paths. Parsing a singlemessage copies the raw message data several times via
String.substring,String.getBytesandArrays.copyOf:IntConverter.convertStringFieldsubstring().getBytes()on every scaniteration to measure the encoded length, copying the whole span repeatedly
(O(n²) for large payloads)
hedger1-alloc-20260930-151550.html
Changes
This PR optimizes the substring allocations inside the hot method
extractField.Core idea: instead of copying the value eagerly, we keep the original String
plus the value's offset/length, and only perform the substring when the value
is actually needed.
StringField— keeps the raw message String together with offset andlength, and only does the substring when
getValue()is called, releasingthe reference to the raw message afterwards.
equals/hashCodecompare andhash the value range directly, without triggering a substring.
All converters gained
convert(String, offset, length)— type conversionreads the given range of the original String directly, so no substring is
needed before converting. The typed getters of
FieldMap(
getInt/getDouble/getDecimal, ...) go through this path and areallocation-free.
Message.extractField— buildsStringFieldinstances from the valuerange instead of substringing every field. The length measurement for data
fields (RawData etc.) no longer uses
substring().getBytes()either; theencoded byte length is computed directly over the original String.
Compatibility
Field values, error messages and parse positions are byte-for-byte identical to
the previous implementation, including the null-value contract covered by
MessageTest/FieldMapTest. Non-String fields and unknown charsets fall back to
the original code paths.
Result
async-profiler allocation profile after this change — the substring,
String.getBytesandArrays.copyOfframes inextractFieldare gone:hedger1-alloc-20260930-153207.html