[kevinheneveld]

Summary

Adds a relevance gate to the two automatic-search paths so they stop queuing downloads that merely share a surname or a common word with the audiobook — e.g. a Patterson Hood concert recording matched to a James Patterson audiobook, or a different Dickens novel matched on the author's name. The gate judges whether the release names the book; it fails open when it cannot judge, and manual search keeps the old behavior so users browsing results still see everything.

Changes

Added

• AutomaticSearchRelevanceGate (Search/Filters/), a standalone class — deliberately not an ISearchResultFilter. The SearchResultFilterPipeline filters are tuned for metadata-page results and reject legitimate raw indexer releases ("The Lord of the Rings" → product-like "ring", "Man's Search for Meaning" → promotional "search"), so the automatic paths run this gate alone and never consult the pipeline.
• How a result is judged (Evaluate, returns a RelevanceVerdict with the reason and both scores for the log line):
  • Title on its own. The audiobook's main title (the part before a subtitle separator — :, ;, -, (, [) or its full title must reach 50% of its significant tokens in the result's title. The Artist field never supplies title evidence, and a leading artist credit in the title ("Frank Herbert - Dune") is stripped before the title is judged (unless that prefix is the book's title, as when Torznab infers Artist from a title-first release), so an author's name can never stand in for a missing title word: "Charles Dickens - Oliver Twist" is rejected for David Copperfield, and a long stored subtitle ("Good Omens: The Nice and Accurate Prophecies of Agnes Nutter, Witch") cannot dilute "Good Omens [M4B]".
  • Author corroboration for generic matches. A match that rests on a single shared title word additionally needs one credited author named in the result (half of that author's tokens, so "Dickens" alone vouches for "Charles Dickens"). That covers one-word titles ("Betrayed", "The Vision", "Stories", "Dune") and, equally, titles a subtitle padded out: "Dune: A Novel" is judged exactly like "Dune", so neither "Dune" by John Doe nor "The Shining: A Novel" by Stephen King can pass on the subtitle. A match on several title words stands on its own ("The Final Empire" for "Mistborn: The Final Empire"). Corroboration is judged per author, so any one of several credited contributors suffices, and the structured Artist field counts the same as an author embedded in the title (MyAnonamouse returns Title "Dune" / Artist "Frank Herbert"). A generic match with no known author fails open.
• SignificantTokens: the stop-word-filtered tokenizer the gate uses. Tokens are runs of Unicode letters/digits in any script (native-script titles such as "Война и мир" keep their words); text is NFC-composed first so precomposed and decomposed spellings produce identical tokens in both the title and the Artist field; and normalization follows IndexerQuerySanitizer, which builds the query the indexer answered — hyphens split ("Full-Cast" → full, cast) and apostrophes are dropped without splitting ("Ender's Game" → enders, game), so a release named after the sanitized query ("Enders Game") tokenizes like the stored title. Edition descriptors are handled by SignificantTokens.FromTitle at the segment level rather than as stop words: a separated segment that is nothing but a descriptor ("Dune: A Novel (Unabridged Edition)" → dune) and a trailing descriptor phrase inside a segment ("Dune Unabridged Edition" → dune) are omitted, but the first segment is always a title and a descriptor word inside a longer segment is kept, so "A Novel Obsession", "The Novel" and "The Last Edition" keep their real words. A stored main title or a credited author that is literally such a word ("Dune - Novel" by the author "Novel") is protected.

Changed

• AutomaticSearchService.ProcessAudiobookAsync and DownloadService.SearchAndDownloadAsync call relevanceGate.Apply(results, audiobook) after the raw search and before scoring. Rejections are logged with the reason (title_not_relevant / author_not_corroborated) and both title scores.
• SearchAndDownloadAsync moved into a DownloadService.SearchAndDownload.cs partial to keep DownloadService.cs under the 500-line cap.
• The gate is registered scoped in ListenarrWorkflowRegistration (and in the test ServiceCollectionBuilder).

Testing

• AutomaticSearchRelevanceGateTests (114 cases): the bug-report scenario (Patterson Hood vs Dog Diaries), both naming orders for David Copperfield, the metadata-filter regressions (Lord of the Rings, Man's Search for Meaning), the MyAnonamouse split title/artist shape, long stored subtitles (Good Omens, David Copperfield), same-author decoys rejected (Oliver Twist, Anna Karenina, Neverwhere, Speaker for the Dead), a generic title corroborated by any one of six credited contributors (Stories), apostrophe-omitted release names with straight and curly controls (Ender's Game), native-script titles and authors, NFC/NFD equivalence in both the title and the Artist field (Les Misérables, Saint-Exupéry), generic subtitles never bypassing the author check ("Dune: A Novel" vs "Dune" by John Doe / "The Shining: A Novel" by Stephen King, with the corroborated and no-known-author controls, and "The Final Empire" still passing for Mistborn), edition words that are real title words ("A Novel Obsession", "The Novel Cure", "The Last Edition" exact title-only releases kept; "The Novel" still rejecting an unrelated release), unseparated and Unicode-separated edition metadata never supplying identity ("Dune Unabridged Edition" vs "The Shining Unabridged Edition"), the Artist field and an identified author prefix never supplying a title word ("Novel Writer - Obsession" for "A Novel Obsession", with -/−/， separators), title-first releases whose inferred Artist is the title, descriptor-shaped author names and compound titles, MainTitle splitting, and the tokenizer's Unicode/apostrophe/hyphen/edition-descriptor behavior.
• DownloadService_AutomaticRelevanceTests (14 cases, end to end through SearchAndDownloadAsync with a strict ISearchResultFilter mock registered in the pipeline): exact matches reach the client submission; the off-topic release does not; and with a better-seeded same-author decoy present, the correct release is the one submitted (David Copperfield vs Oliver Twist, Война и мир vs Анна Каренина, NFD "Les Misérables", "Enders Game", "Good Omens [M4B]", and "Stories" with Artist = one of five credited authors); for "Dune: A Novel", the better-seeded "The Shining: A Novel" (Stephen King) and an uncorroborated "Dune" are passed over for "Dune" / Artist "Frank Herbert", and nothing is submitted when only those two are returned. The strict mock proves the metadata-page filters are never consulted on this path.
• AutomaticSearchProcessorRelevanceTests (2 cases, end to end through the sweep's RunCycleAsync to the download-service handoff, modeled on AutomaticSearchProcessorFallbackTests): the same two wrong-author releases never reach StartDownloadAsync (strict mock), and the corroborated release is the one handed over.
• Based on canary v1.5.2 (over the merged feat(search): configurable automatic-search sweep interval, per-book cooldown and book delay #714/fix(wanted): recover books with no quality profile — picker, explicit bulk-assign, failures named #605); dotnet format --verify-no-changes and the architecture tests are green. The round-6 edition-descriptor/artist-evidence design and its regression tests are @therobbiedavis's proposal from review, adopted as-is.

Notes

• Sits alongside Reject releases that name another book of the same series in automatic search #1039 (series-position rejection) and fix(search): indexer query building and result gating, three fixes in one PR #1019 (shared query builder) without overlapping them — they touch scoring/query building, this is the pre-scoring relevance pass.
• Not included (dropped from the original draft, unchanged rationale): the Newznab category-3030 restriction (canary no longer applies it; per-indexer category config is the right place) and a hard language filter (canary's SearchResultScorer already penalizes language mismatches).

🤖 Generated with Claude Code

[T4g1]

Solid work!

[kevinheneveld]

@T4g1 — I think both your points are addressed:

• Category 3030: went with option 2 — dropped the category layer from this PR entirely (force-pushed), so it's just the relevance + language filters now (which caught the music-album grabs in testing anyway). Filed Automatic search: use per-indexer configured categories instead of imposing one at query time #594 to track the proper per-indexer category work for when the indexer-side foundations are ready.
• ScoreSearchResults(audiobook, profile) redundancy: per your conclusion we're keeping it as-is for now (the profile-test endpoint needs the no-audiobook path), so no change there.

Mind giving it another look when you have a moment? Happy to rebase onto current canary if it's drifted.

[kevinheneveld]

Thanks @T4g1 — both addressed in 4f53cab4 (additive commit, no force-push):

• Single method with a null default — collapsed the two ShouldFilter overloads into one ShouldFilter(SearchResult result, Audiobook? audiobook = null). The context-free overload + default-interface method are gone; filters that don't need the audiobook just ignore the param, RelevanceFilter keys off it (null ⇒ fail open). The doc comment now describes the null/fail-open contract instead of explaining the migration.
• Exception filter — switched the context-aware filter pass to catch (Exception ex) when (ex is not (OperationCanceledException or OutOfMemoryException or StackOverflowException)). (Left the ~42 pre-existing identical catches elsewhere in the controller alone to keep this PR scoped — happy to do a sweep separately if you'd like.)

[T4g1]

lgtm! Let's rebase and check pipeline result before merge though

[kevinheneveld]

@T4g1 rebased onto canary (now v1.0.11) as requested — the CHANGELOG conflicts are resolved and both the relevance-filter commit and the dropped-3030-category follow-up replayed cleanly. Pipeline is green locally (Release build + 134 search/filter tests pass). Ready for the re-run whenever you are.

[therobbiedavis]

@T4g1 rebased onto canary (now v1.0.11) as requested — the CHANGELOG conflicts are resolved and both the relevance-filter commit and the dropped-3030-category follow-up replayed cleanly. Pipeline is green locally (Release build + 134 search/filter tests pass). Ready for the re-run whenever you are.

@kevinheneveld The changelog is no longer needed at all. Please update the PR description to match the template.

[kevinheneveld]

Rebased onto canary v1.2.1. This needed more than a replay — canary refactored the whole search subsystem since approval, so I re-homed the change: ISearchResultFilter now lives in Search/Contracts/, DI moved to ListenarrWorkflowRegistration, and the edited services relocated (Downloads/Submission/DownloadService, infrastructure/HostedServices/Search/AutomaticSearchService). The old LibraryController auto-pick edit is gone — that flow was extracted into DownloadService.SearchAndDownloadAsync during the refactor, which this PR already filters. Net behavior is identical to the approved version (RelevanceFilter applied before scoring in both auto-pick flows); the diff shape just changed, so a re-look is welcome. Full suite 1024/1024, format check clean. CI will need an Approve and run.

[kevinheneveld]

Rebased onto canary v1.3.17 (clean replay) and marked ready. Nothing changed in the diff since @T4g1's approval beyond the rebase; still applies as-is because the auto-pick paths run no filter pass on current canary (the pipeline only runs inside AsinEnricher). Release build clean, filter tests green. Will need an Approve and run for CI.

[therobbiedavis]

Since you folded in additional logic, there were some additional findings.

I found two regressions that prevent valid automatic downloads. The Patterson concert is correctly rejected, but the exact-match cases in the inline comments are rejected too. I’d request changes for those two points.

[kevinheneveld]

Thanks for the live comparison — both findings addressed in f5490b0 (rebased onto v1.3.22, one commit).

Pipeline scope (DownloadService:170): the relevance check is now a standalone AutomaticSearchRelevanceGate, and the automatic paths (SearchAndDownloadAsync and AutomaticSearchService.ProcessAudiobookAsync) no longer call SearchResultFilterPipeline at all, so the product-like/promotional filters never see raw indexer titles. Every change to ISearchResultFilter, the existing filters and the pipeline is reverted. J. R. R. Tolkien - The Lord of the Rings [M4B] and Viktor Frankl - Man's Search for Meaning [M4B] are covered as gate unit cases and end to end through SearchAndDownloadAsync (they reach the client submission; a strict ISearchResultFilter mock registered in the pipeline proves the metadata-page filters are not consulted on that path).

Author corroboration (RelevanceFilter:66): ComputeRelevance and RequiresAuthorCorroboration now take the result's Artist into account alongside Title, so the MyAnonamouse shape (Title = "Dune", Artist = "Frank Herbert") passes exactly like the Frank Herbert - Dune [M4B] control; both are regression-tested at unit and end-to-end level. The Patterson concert is still rejected, and the Betrayed/The Vision single-word cases still require corroboration (satisfied by either the title or the Artist field).

SearchAndDownloadAsync moved into a DownloadService.SearchAndDownload.cs partial to stay under the 500-line cap. Format check and the architecture tests are green locally.

[therobbiedavis]

The earlier metadata-filter and structured-artist issues are fixed. I found three remaining regressions in the new gate: valid releases can be rejected because of long subtitles or additional author credits, Unicode text and equivalent Unicode encodings, and apostrophe-omitted release names.

Two mixed-result cases also queue the wrong book after the correct release is filtered out: Oliver Twist for David Copperfield, and Anna Karenina for War and Peace. Both reproduce through the API and the natural scheduled worker. I'd request changes for the three inline points, with regression coverage for the variants described there.

Verified against separate live Linux containers for base d220aa8 and head f5490b0, each with a fresh database: 74 API outcomes and 10 scheduled-worker outcomes checked against submitted torrent payloads and persisted download records. Indexers and qBittorrent were controlled HTTP fixtures; no real media transfers or imports were performed.

[kevinheneveld]

All three threads addressed in aac85e4 (rebased onto canary v1.3.23, still one commit); details are on each inline comment. In short:

• Title judged on its own. Main title (before a subtitle separator) or full title must reach 50% in the result; author tokens no longer count toward the title, so same-author decoys (Oliver Twist, Anna Karenina, Neverwhere, Speaker for the Dead) are rejected and long stored subtitles cannot dilute an exact release. Generic single-word titles are corroborated per credited author, so any one of several contributors suffices.
• Unicode. Tokens are Unicode letters/digits in any script; NFC normalization first, so NFC/NFD spellings match in both the title and Artist.
• Apostrophes/hyphens normalized like IndexerQuerySanitizer (Ender's → enders, Full-Cast → full cast).

The mixed-result selection is now covered end to end: SearchAndDownloadAsync_SameAuthorDecoy_SubmitsTheCorrectRelease seeds the decoy at 900 seeders and the correct release at 5 and asserts the submitted title for each of your cases. The PR body is rewritten to describe the current design.

Gates locally: 60 gate unit cases + 13 end-to-end cases + architecture tests green (118/118 in the slice); the wider Application.Search / Application.Downloads slices pass apart from the 51 macOS-Arm64 filesystem guard failures; dotnet format --verify-no-changes clean.

[therobbiedavis]

The earlier findings are fixed. I found one remaining gap where adding a subtitle bypasses the author check and allows the wrong book to be queued. I’d request changes for the inline comment.

[kevinheneveld]

Round 5 addressed in a39d805 (rebased onto canary v1.5.2 over the merged #714/#605, one commit); details on the inline thread. In short: the author check now keys off how many title words the result shares, not how long the stored title is, so a subtitle cannot bypass it (Dune: A Novel ≡ Dune), while a multi-word match (The Final Empire) still stands on its own; and novel/unabridged/edition are stop words like audiobook/book. Both automatic paths are covered end to end, including a new sweep-level test modeled on AutomaticSearchProcessorFallbackTests. The PR body is updated.

Gates locally: gate 77 + SearchAndDownloadAsync 14 + sweep 2 + architecture tests green; the wider Application.Search / Application.Downloads / HostedServices.Search slices pass apart from the 51 macOS-Arm64 filesystem guard failures; dotnet format --verify-no-changes clean.

[therobbiedavis]

@kevinheneveld the subtitle issue is fixed, but the edition stop words are still removing actual title words. i’d request changes for the inline below. i added a possible fix, but feel free to take another approach.

[therobbiedavis]

@kevinheneveld

sorry for jumping in and adding commits to your branch. you'd given me the okay in discord to experiment and commit fixes if i found a way to resolve the findings. after several review rounds, i wanted to work through a concrete solution rather than keep sending findings back and forth.

compared with 4b447b1, the changes through 032e9df5 replace the separate relevance and fallback filters with one shared matching and selection path.

• the api, scheduled worker and fallback use the same rules. every candidate gets scored; automatic selection requires confirmed identity and a positive, non-rejected quality score.
• matching protects complete titles, respects stored subtitle relationships, and supports complete author credits, coauthors and initials. narrator credits and inferred artist fields cannot independently confirm a book. uncorroborated title-only results remain available for manual selection.
• candidate associations survive sorting and duplicate ids. changes to the candidate, book or profile prevent stale handoff. the minimum-seeder check also handles both Torrent and torrent.

the backend run passed 4,042 tests, with 270 skipped. the additional live comparison passed all 90 cases. all 37 intended downloads matched the client receipts and saved rows; the other 53 produced no download. another 47 evidence, boundary and mutation checks passed.

some examples from the pr discussion, checked against the live test results:

example	result
Patterson Hood concert returned for James Patterson's Dog Diaries	rejected; no download
David Copperfield with Charles Dickens credited before or after the title	accepted; the Oliver Twist same-author control was rejected
Dune by Frank Herbert for stored Dune: A Novel, with A Novel stored separately as the subtitle	accepted; without that stored subtitle relationship, the shortened title stayed manual
complete Tolkien initials written as J.R.R. or J. R. R.	accepted with the matching title

i also benchmarked both heads in fresh containers with identical limits. across three runs, median processing time increased from 2.6 to 6.6 ms at 100 results, and from 31.2 to 46.2 ms at 1,000, including the new freshness check. allocation at 1,000 results increased from 10.9 to 26.7 MB per search. that's an acceptable cost for the additional checks in this workflow.

the version, language-detection and torrent-removal differences in the wider comparison came from the canary rebase.

please review what i've added and let me know if you'd prefer a different approach.

[kevinheneveld]

@therobbiedavis Thanks for working this through end to end, and no apology needed. The shared identity-plus-quality path, the deferral to manual, and the freshness fingerprint are all better than the gate, and I'm happy to go with your design.

One concern from replaying my own download history (712 rows, all Prowlarr Torznab/Usenet, so the author has to come from the release name): the current policy auto-confirms 7 of 269 releases that actually imported, while rejecting 59 of the 60 that were wrong-book grabs. The precision is exactly what we want. The recall problem is "unexplained release text": series tags (Star Force, Bk 6), group tags ((NMR, -PoF), bitrates (64k), narrator names, and dotted scene names (Pierce.Brown-Red.Rising.03-Morning.Star.Unabr-64k.[2016], which never finds the title because dots are kept). Treating dots as spaces alone only gets to 9 of 269.

Would you be open to a middle tier where, once the complete title and a complete credited author are both found, release annotations of those shapes are explained rather than blocking? I can send the anonymized release-name set if useful. Also for the record, the round-6 artist-prefix strip in my gate had a bug your replacement fixes: Prowlarr infers Artist as the text before the first dash, so title-first releases with a series tag lost their title entirely.

[therobbiedavis]

@kevinheneveld thanks for replaying your history. 7 out of 269 makes it clear we're deferring too many valid releases. our tests covered the supported layouts, but missed a lot of normal release naming.

yes, i'm open to that middle tier. once the complete title and credited author are established, we should explain recognized series, group, bitrate and narrator annotations. conflicting book numbers or another work's title should still block the download.

please send the anonymized set, including the wrong-book grabs. i'd like to use it as a regression corpus and look at the incorrect download we're still confirming too. that gives us something concrete to test against instead of guessing at more naming rules.

[kevinheneveld]

@therobbiedavis Here is the set: https://gist.github.com/kevinheneveld/f158b2ee8a7e7a4def79321a8c7b8c94. 498 distinct release-name cases from Prowlarr-fronted Torznab and Usenet indexers, each with the library book the grab was attributed to, the outcome (imported, import-blocked, download-failed), and the verdicts from both your head (032e9df) and the old gate for reference. Authors came only from the release name, so all evidence is ReleaseNameDerived. HTML entities are unescaped; no IDs, paths or timestamps.

Two notes on labels. imported mixes manual and automatic grabs, since the grab history does not record which. import-blocked is mostly wrong-book grabs but not individually adjudicated. The one you still confirm there, Brandon Sanderson-The Fires of December [m4b], is actually the right book; that import was blocked by an unpacking failure, so it is a true positive. The real wrong-book grabs the gate let through and you reject include Critical Mass (Strieber for Clarke), Food of the Gods (McKenna for Clarke), and a Patterson release for a book titled "Thriller".

Happy to re-run the replay against any revision you push.

[therobbiedavis]

@kevinheneveld thanks for the set and the clarification on the labels. i've pushed a738d6d1.

the parser now handles supported bitrate, group, series and narrator annotations, plus dotted author credits and stored subtitle joins. complete title/author evidence is still required, inferred Artist isn't independent corroboration, and unknown or conflicting text stays manual. series positions and unlabeled narrators are checked against library metadata.

i replayed all 498 cases. Critical Mass, Food of the Gods and the Patterson/Thriller grab still reject for the wrong-book targets. Fires of December stays confirmed. the Godling Chronicles example also passes identity matching now; automatic download still depends on the normal quality and selection checks.

could you review the changes and rerun that revision against your full library metadata? if a valid release still defers unexpectedly, please include the case index and stored series, memberships and narrators. those fields weren't in the export and would help reproduce the decision.

[kevinheneveld]

@therobbiedavis Replayed d1d1fb9 against the live library with series, memberships and narrators supplied (BookMatchContext.From on the stored Audiobook, Artist as ReleaseNameDerived). 423 of the 498 corpus cases still have a download row; the rest were cleaned up since the export.

outcome	cases	confirmed, title+author only	confirmed, with library series/narrators
imported	120	27	39
import-blocked	26	3	5
download-failed	277	97	119

Weighted by grab count the imported figure is 141 of 254 rows, up from 7 of 269 at 032e9df. Library facts never flipped a confirm into a deferral. Precision holds: no wrong-book release is confirmed in any bucket, and Critical Mass, Food of the Gods and the Patterson/Thriller lure still reject. The five import-blocked cases now confirmed (Dark Tower II and III, Other Worlds Than These x2, Chain Reaction, Odyssey) are all the right book; their jobs failed on Unable to perform Copy to the NFS library, so they are true positives like Fires of December. On the other side, several imported cases that defer are wrong-book manual imports your policy is right to refuse: 4, 27, 35, 41, 46, 101, 115, 133.

Remaining deferrals on valid releases, by pattern, with case indexes and the span left Unknown:

1. Series name variants (library name differs by article, suffix or plural): 130 Wheel of Time #4 vs stored The Wheel of Time; 81 The First Law, Bk 2 vs The First Law Trilogy; 144 Presidential Agents, Bk 4 vs Presidential Agent; 31 Boxcar.Children.128 vs The Boxcar Children Mysteries.
2. Library series without a position (about 30 cases, 50 to 80): Janet.Benge-Christian.Heroes.Then.&.Now.03-Hudson.Taylor-Deep.in.the.Heart.of.China.Tim.Gregory-PoF. Author, title, stored subtitle, narrator and group all bind; the series span fails because the stored membership has a name but no number. A missing library position is not a conflicting one, so I would let it explain when the name matches.
3. Technical and group text outside the grammar: 98 dcrypt64k-Burns.2010.NMR (narrator surname + year); 39 [Snyder 2015]) 227.33 MB (size); 40 WEB-SE-2021-CRAViNGS iNT and 136 MP3-JD07, 121 {FerraBit} (scene suffixes); 132 (miok) [Audiobook] (Horror) where the group is recognized but genre parentheticals follow (also 85, 129).
4. Library metadata gaps on my side, correctly deferred: 39 and 84 coauthor missing from the stored authors; 131 series not stored; 123 title/subtitle stored swapped (Dark Tower I / The Gunslinger).

I added listenarr-live-release-corpus-reference-facts.json to the gist: stored series, series number, memberships and narrators for those 423 cases keyed by corpus index, plus the d1d1fb9 verdict with those facts. Focused Matching, relevance and scorer slices pass here (852 tests). Code looks good to me; the only design note is that the Groups set is a hardcoded list from my indexers, so a trailing-group rule may age better than the list.