Skip to content

#2057 FetcherBolt: extract the crawl-delay policy - #2205

Merged
rzo1 merged 1 commit into
mainfrom
crawl-delay-policy
Oct 2, 2026
Merged

rzo1 merged 1 commit into
mainfrom
crawl-delay-policy

Conversation

@dpol1

@dpol1 dpol1 commented Sep 29, 2026

Copy link
Copy Markdown
Member

Moves the crawl-delay decision to CrawlDelayPolicy, with a table of tests.
FetcherThread applies the result.

Part of #2057.

@dpol1
dpol1 added this pull request to stack #2202 September 29, 2026 09:50
@dpol1 dpol1 added this to the 4.0.0 milestone Sep 29, 2026
@dpol1 dpol1 self-assigned this Sep 29, 2026
@dpol1 dpol1 added the fetcher label Sep 29, 2026
@dpol1
dpol1 requested a review from rzo1 September 29, 2026 09:55
@dpol1 dpol1 added the core label Sep 29, 2026
@dpol1
dpol1 requested review from jnioche, mvolikas and sigee September 29, 2026 09:55
@rzo1
rzo1 force-pushed the crawl-delay-policy branch 3 times, most recently from 6e9313a to 79c8542 Compare October 2, 2026 18:54
Base automatically changed from sitemap-discovery to main October 2, 2026 18:54
Signed-off-by: Davide Polato <dpol1@apache.org>
@rzo1
rzo1 force-pushed the crawl-delay-policy branch from 79c8542 to 123ad87 Compare October 2, 2026 18:54
@rzo1
rzo1 merged commit bef41bc into main Oct 2, 2026
1 check passed
@dpol1
dpol1 deleted the crawl-delay-policy branch October 2, 2026 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants