Skip to content

Chain Simulator tests fixes - #1630

Open
stefangutica wants to merge 9 commits into
developmentfrom
fix-cs-tests
Open

stefangutica wants to merge 9 commits into
developmentfrom
fix-cs-tests

Conversation

@stefangutica

@stefangutica stefangutica commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator

Reasoning

  • The chain simulator e2e tests failed intermittently, on every branch including development, and a different test failed each time.
  • Duplicate websocket messages: under a slow handshake, the reconnect loop in the websocket test sent a second CONNECT on the same connection. The server opened a second socket for it, so every message arrived twice and the exact-count assertions failed.
  • Transfers without action: the transaction pool warmer, enabled only in the e2e config, resolved every pending transaction and SCR each second. That included the ESDTTransfer SCR that delivers a newly issued token, which it looked up before the token was indexed. The miss was cached for an hour, so every later transfer of that token came back without action.
  • Hidden errors and unstable order:
    • the transaction helpers swallowed errors and returned placeholders, so failures surfaced in unrelated assertions;
    • the test files not listed in the sequencer ran in whatever order jest discovered them.

Proposed Changes

  • Websocket test:
    • retry a connection only after connect_error, subscribe once per client, and wait for the subscription acks;
    • count distinct messages only, since the broadcaster replays a round when it times out.
  • E2E config: disabled transactionPoolWarmer, as on mainnet and devnet (/pool is still read on request), and enabled fastWarm.
  • Issuance helpers: ESDT, NFT/SFT collection and Meta-ESDT issuance wait until the token is found through the same Elastic query the API runs (tokens/_search?q=_id:<id>). The Elastic URL is added to the test config (ELASTIC_URL).
  • Test data: prepare-test-data waits on the API's own view (tokens, collections, NFTs, shards, node ratings) instead of a fixed 30s sleep.
  • Error handling: sendTransaction, deploySc and getNonce rethrow instead of returning 'n/a' or 0, and deploySc fails when there is no SCDeploy event.
  • Ordering and sleeps: unlisted test files run in alphabetical order, and the fixed sleep in the accounts NFT test is replaced by polling.

How to test

  • Run the Chain simulator e2e tests workflow several times on this branch. A single green run proves little, since it passed intermittently before.
  • Locally: npm run start-chain-simulator, docker compose up -d, npm run start:mainnet:e2e, npm run prepare:test-data, then npm run test:cs-e2e.

@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

k6 load testing comparison.
Base Commit Hash: 3d783b2
Target Commit Hash: edaf822

Metric Base Target Diff
AvgMax9095AvgMax9095AvgMax9095
Tokens40.51153.8157.3560.0728.201176.5945.4575.53-30.39% ✅+664.98% 🔴-20.76% ✅+25.75% 🔴
Nodes62.7815725.8359.5861.3870.4913505.2566.3287.30+12.26% 🔴-14.12% ✅+11.31% 🔴+42.22% 🔴
Mex40.68239.8857.3560.0128.291729.9244.1973.96-30.45% ✅+621.17% 🔴-22.95% ✅+23.24% 🔴
Blocks48.971698.1959.2761.4237.601532.9761.57115.17-23.21% ✅-9.73% ✅+3.88% 🔴+87.51% 🔴
Accounts41.76928.0557.8360.1629.102954.0440.0159.05-30.32% ✅+218.30% 🔴-30.82% ✅-1.85% ✅
Pool40.93632.5657.6060.0628.14707.6045.7377.50-31.25% ✅+11.86% 🔴-20.61% ✅+29.04% 🔴
Transactions67.1814408.7359.9161.5052.188997.4854.92119.78-22.33% ✅-37.56% ✅-8.33% ✅+94.77% 🔴
Test Run Duration60005.3660003.15

Legend: Avg - Average Response Time, Max - Maximum Response Time, 90 - 90th Percentile, 95 - 95th Percentile
All times are in milliseconds.

stefangutica and others added 8 commits September 3, 2026 15:37
- websocket: retry a connection only after connect_error. calling connect()
  while the handshake is in flight sent a second CONNECT packet, the server
  opened a second socket and every message arrived twice
- websocket: subscribe once per client, wait for the subscription acks and
  count distinct messages, since the broadcaster replays rounds on timeout
- run the test files missing from the sequencer list in a fixed order
- rethrow errors from the transaction helpers instead of returning
  placeholders that fail later, unrelated assertions
- poll the api for bob's nft instead of a fixed sleep

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cs tests

the transaction pool warmer, enabled only in the e2e config, resolved the
action of every pending transaction and smart contract result each second,
including the ESDTTransfer that delivers an issued token, before the token
was indexed. the unresolved token properties were then cached for an hour,
so later transfers of that token came back without an action, depending on
whether a warmer tick caught that result in the pool.

the warmer is now off, as on mainnet and devnet, and the pool is still read
on request. the issuance helpers also wait until the token or collection can
be found through the same elastic query the api runs, before returning it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants