Skip to content

Keep searchAfter cursors on their positions when sorting locally - #1646

Open
stefangutica wants to merge 4 commits into
developmentfrom
fix-local-sort-search-after
Open

stefangutica wants to merge 4 commits into
developmentfrom
fix-local-sort-search-after

Conversation

@stefangutica

@stefangutica stefangutica commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Reasoning

  • Some results are sorted locally after they come from elastic: sortElasticTransfers and sortElasticTransfersByTxsOrder for transfers, and reorderAccountSentTransactionsByNonce for account transactions and transfers.
  • Each item's searchAfter holds the elastic sort values of that item. After a local sort, the last item is not always the last elastic hit, so continuing with its searchAfter starts the next page from an earlier position, and items already returned come back again.
  • To avoid this, the local sorts were skipped when searchAfter was set, and withTxsOrder + miniBlockHash with searchAfter was rejected with a 400. As a result, the first page and the next ones were ordered differently.
  • The local sorts were always descending, so with order=asc the first page came back descending while elastic returned the next ones ascending.

Proposed Changes

  • Added SearchAfterUtils.sortKeepingSearchAfterPositions: it sorts the items and then puts each searchAfter back on its original position. When two items swap places, they also swap their searchAfter, so the last item always carries the cursor of the last elastic hit.
  • Used it for all three local sorts.
  • Local sorting now also applies to pages requested with searchAfter, and withTxsOrder + miniBlockHash + searchAfter is accepted instead of returning 400.
  • The transfers sort and the sent transactions nonce sort follow filter.order: ascending for order=asc, descending otherwise (same default as before). The withTxsOrder execution order is left as it is.
  • Note: a cursor now belongs to a position, not to the item shown there. Continuing from the last item's searchAfter (the normal flow) is exact; continuing from an item in the middle of a page resumes from that position in elastic order.
  • Added a disclaimer to the pagination section in docs/swagger.md: on endpoints that reorder the results, only the searchAfter of the last item is guaranteed to be consistent.

How to test

  • npx jest --config ./src/test/jest-unit-config.json src/test/unit/utils/search.after.utils.spec.ts src/test/unit/services/transfers.spec.ts src/test/unit/services/transactions.spec.ts
  • GET /accounts/:address/transfers?size=50, then request the next pages with the searchAfter of the last item. There should be no duplicated txHash across pages, and each page should be ordered the same way as the first one.
  • Same for GET /accounts/:address/transactions: sent transactions should be ordered by nonce on every page.
  • Repeat both with order=asc: every page should be ascending, including the first one.
  • GET /transfers?miniBlockHash=<hash>&withTxsOrder=true&searchAfter=<cursor> should return 200 instead of 400.
  • Open /docs and check the new disclaimer in the Pagination section.

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

k6 load testing comparison.
Base Commit Hash: 3d783b2
Target Commit Hash: f4d2b8c

Metric Base Target Diff
AvgMax9095AvgMax9095AvgMax9095
Transactions197.9332250.56131.98161.4187.241802.76188.61281.03-55.92% ✅-94.41% ✅+42.91% 🔴+74.11% 🔴
Tokens64.83389.48125.80223.9863.55486.11142.88166.12-1.97% ✅+24.81% 🔴+13.58% 🔴-25.83% ✅
Blocks89.231204.67204.59297.5282.761070.77156.93185.98-7.25% ✅-11.11% ✅-23.29% ✅-37.49% ✅
Accounts68.27713.89150.06226.2472.11663.79155.71180.78+5.62% 🔴-7.02% ✅+3.77% 🔴-20.10% ✅
Nodes1118.0334546.60171.25256.1264.14486.30143.06168.10-94.26% ✅-98.59% ✅-16.46% ✅-34.36% ✅
Mex64.72392.76124.72223.5464.48601.35143.25167.76-0.37% ✅+53.11% 🔴+14.85% 🔴-24.95% ✅
Pool64.44392.76123.65222.6163.29486.23142.45166.06-1.78% ✅+23.80% 🔴+15.21% 🔴-25.40% ✅
Test Run Duration60003.6060001.78

Legend: Avg - Average Response Time, Max - Maximum Response Time, 90 - 90th Percentile, 95 - 95th Percentile
All times are in milliseconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants