perf(yearn): query small parent flows on the blockTimestamp index (fix Envio 504s) - #389
Merged
Merged
Conversation
…ndex The cursor query used a blockNumber range with sinceTs=0 once a cursor existed. Envio only has single-column indexes and block numbers overlap across chains (katana 43.7M falls inside Base's range), so Postgres walked other chains' rows and the hosted gateway returned 504 after ~15s. Live benchmarks: katana deposits 504'd on most attempts; the same query with a blockTimestamp floor peaked at 0.43s. Store the cursor block's timestamp and use it as a top-level blockTimestamp floor. Legacy cursors without a timestamp resolve it once via a cheap blockNumber equality lookup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… filter A cursor from a vault that has since left the active set was excluded by the vaultAddress filter, so the lookup returned None and every run fell back to the unfloored query that 504s. Every event in a block shares its timestamp, so match on chain and block only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #388, which added retries. This PR fixes the root cause of the
Small parent flow monitor: Envio GraphQL request failed (HTTP Error 504: Gateway Timeout)alerts (4 of 24 hourly runs on Sep 26–27).Why
Once a stream has a cursor, the query filtered on a
blockNumberrange withsinceTs=0. The Envio schema only has single-column indexes, and block numbers overlap across chains (katana's position at 43.7M falls inside Base's block range). Postgres walks the globalblockNumberindex through other chains' rows, and the hosted gateway gives up at about 15s.alert_large_flowsfilters onblockTimestamp, which is comparable across chains, and never failed.Live benchmark against production Envio with production cursors (5 runs each):
blockNumberrange +_or)blockNumberrange without_orblockTimestampfloorThe retries from #388 don't help much when a query consistently takes more than 15s, as katana deposits currently do.
What changed
blockTimestamp >= <cursor block timestamp>now sits at the top level, next to the existing_or, so the cursor semantics stay the same.block_timestamp.EventCursorordering and equality are unchanged, because the new field is excluded from comparison.blockNumberequality query (about 0.15s, which is selective). They are upgraded the next time the cursor is saved. If no event is indexed at the cursor block, the monitor logs a warning and falls back to the old unfloored query.Tests
pytest: 1407 passed, 6 skipped.ruffis clean.mypyreports no new errors.monitor_flow_typefor all chains against production Envio, using a copy of the production state DB (no alerts sent, no state saved): every stream finished within 2.6s. Katana deposits took 0.14s.🤖 Generated with Claude Code