Skip to content

spec(content-drive): Title / All Content scope selector for the search box - #37518

Open
KevinDavilaDotCMS wants to merge 1 commit into
mainfrom
issue-37479-content-drive-search-scope
Open

spec(content-drive): Title / All Content scope selector for the search box#37518
KevinDavilaDotCMS wants to merge 1 commit into
mainfrom
issue-37479-content-drive-search-scope

Conversation

@KevinDavilaDotCMS

Copy link
Copy Markdown
Contributor

Spec-Kit PR 1 of 2. Carries spec.md alone. Review it as a spec, not as code: is this the right problem, scoped right, with criteria a reviewer can tell "done" from? Approval here — not merge — unblocks /speckit-plan.

Refs #37479

What it specifies

A scope dropdown beside the Content Drive search input, with two options — Title and All Content — so an author who knows the name of what they want can say so, instead of getting back every document whose body or Story Block happens to mention the word. It is also the cheap query path: the all-content gate is +(catchall:<value>*^10 OR title_dotraw:*<value>*^2), an all-fields aggregate plus a leading wildcard, and in the drive a broad match multiplies DB hydration and permission filtering downstream, not just index time.

The four open decisions, settled

The issue marks these as needing a call before implementation. Settled by the issue owner on 2026-09-11; approving this PR is the record of that sign-off.

# Decision Settled as Why
1 Default scope All Content No-regression. Doing nothing gives you exactly today's results; the fast path is opt-in.
2 Title-mode fields Contentlet title only Keeps the promise the label makes and the query on one field. File assets stay findable by name in practice — dotCMS keeps title in step with the file name.
3 Stickiness URL only, per search Behaves like every other Content Drive filter. No new preference storage.
4 Sorting Unchanged Void as asked — see correction 1 below.

Premises checked against main — four corrections

These are the part most worth a second pair of eyes, because three of them shrink the work and the fourth adds a rule the issue's file list has no place for.

  1. Nothing sorts by score today. Decision 4 asks to confirm score-desc holds in Title mode. There is no score sort: DEFAULT_SORT is modDate:DESC (shared/constants.ts:51), the store sends it unconditionally (dot-content-drive.store.ts:157), and the server default is SORT_BY = "modDate" (AbstractDriveRequestForm.java:81). The only trace is a stale comment at dot-content-drive.store.ts:481 sitting above code that does nothing of the kind. → Sorting out of scope; comment corrected in passing.

  2. Folders and links are already matched on name only, in both scopes. They never reach Elasticsearch — loaded from the DB and narrowed in Java by substring (BrowserAPIImpl.java:3026 for folders, :2908-2913 for links). → The selector governs the contentlet clause alone; folder/link matching must not change.

  3. buildPureESQuery is unreachable for the drive under shipped configuration. doPureESQuery runs only when BROWSE_API_HEURISTIC_TYPE=PURE_ES; the default is HYBRID_SINGLE_CHUNKED_QUERY_ES (BrowserAPIImpl.java:701-709). → Explicitly out of scope rather than changed unverified.

  4. A scope stored as a filter would offer "Clear all" on an unfiltered drive. The issue routes the scope through filter state — which is what carries it into the URL — but stops there. That state also feeds the chip bar, and hasNonDefaultFilters (utils/functions.ts:334-355) counts every key but sharedAssets and languageId as a non-default filter, which is exactly what shows "Clear all" (dot-filter-bar.component.html:7). Picking All Content — the default — would light it up on a drive with nothing filtered. → FR-021: the scope counts as filter state only while it differs from the default, mirroring how the search term already deletes its own key when it goes empty.

Scope boundaries

  • In: the Content Drive search box, its filter/URL state, the drive's search request contract, and the title-scoped query.
  • Out: buildPureESQuery; the all-content strategy shared with the Search portlet and the Relationships dialog, which keeps serving them unmodified — Title is a sibling path, not a branch inside it; the Asset Picker, which reuses the same search box and must stay untouched unless it opts in; sorting; any DB, index-mapping or content-model change.

UI

Deliberately thin. The issue's ASCII diagram is the only design input — no mock image exists — so the spec fixes which components are on screen (input, scope dropdown beside it, placeholder that follows the scope) and stops there. No dimensions, spacing or layout rules.

What a reviewer should push back on

  • Decision 1 if you think Title-by-default is worth the behavior change for existing users.
  • Decision 2 if excluding fileName / metadata.name will bite in practice — the spec names the gap (a file whose title was edited away from its name) and calls widening it a follow-up rather than a silent change.
  • SC-003 carries no fixed latency threshold. No baseline exists yet; producing the dataset and the before/after comparison is part of the work and gets recorded on the issue.

🤖 Generated with Claude Code

…h box

Spec-Kit PR 1 for #37479. Carries spec.md alone.

Settles the four decisions the issue left open: All Content stays the
default, Title mode matches the contentlet title only, the scope lives in
the URL rather than a user preference, and sorting is untouched.

Four premises verified against main and corrected in the spec:

- Nothing sorts by score today — the default is modDate:desc on both
  sides, and the only trace of score sorting is a stale comment. Open
  decision 4 is void.
- Folders and links never reach Elasticsearch; they are already matched
  on name only, in both scopes, so the selector governs the contentlet
  clause alone.
- buildPureESQuery is unreachable under the shipped heuristic, so it
  stays out of scope.
- A scope written into the filters on every selection would light up
  "Clear all" on an unfiltered drive, because hasNonDefaultFilters counts
  every key but two. Hence FR-021: the scope counts as filter state only
  while it differs from the default.

Refs #37479

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

**Related GitHub Issue**: [#37479](https://github.com/dotCMS/core/issues/37479) — related history: [#36688](https://github.com/dotCMS/core/issues/36688) (replaced the broad `catchall:*kw*` wildcard with the current strategy), [#36814](https://github.com/dotCMS/core/issues/36814) (search performance at scale)

**Input**: User description: "Content Drive: add a Title / All Content scope selector to the search box. Default scope = All Content (no regression); Title mode matches strictly the contentlet title plus folder names, not fileName/metadata.name; scope persists only in the URL filters; sorting unchanged."

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is not all content, is all metadata, 2 different things, the all content im covering it on the PR i have open #37487

@zJaaal zJaaal Sep 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With metadata I mean fields, we might need a tooltip explaining this, because is a new feature


---

### User Story 3 - Everything that is not Content Drive is untouched (Priority: P3)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if the asset picker is using the same endpoint, if they are, we need to support it

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answering my own question, from the blast-radius sweep I did for #37487: yes, same endpoint.

The Asset Picker builds a DotContentDriveSearchRequest in with-asset-browse.feature.ts and calls DotContentDriveService.search(), the same POST /drive/search Content Drive uses. Those two are the only callers of that service, so the picker is the one surface that inherits whatever the server does with a request that never mentions the search scope.

The good news is your spec already carries the defence. The requirement that a request omitting the scope behaves exactly as today, plus the scenario pinning an omitted request against an explicit all-content one, covers the Asset Picker without touching a line of Asset Picker code, as long as the default is all-content and stays that way.

Two things I would still add. Name the Asset Picker explicitly in that requirement rather than leaving it as "other callers", so a later change to the default has to confront it instead of only re-running Content Drive's tests. And if the implementation ends up touching the shared query building underneath the endpoint rather than only the search-text branch, the blast radius widens past those two callers to the assets API, the old file browser, the legacy admin browser and a Velocity viewtool, which all reach the same listing by a different door. Staying inside the search-text branch keeps it at two.

Reply written by Claude on @zJaaal's behalf.

@zJaaal

zJaaal commented Sep 11, 2026

Copy link
Copy Markdown
Member

One naming ask, because our two specs are about to collide in prose and then in the same request object.

This one introduces a scope that says which fields a search reads. #37487 introduces one that says where in the site you are browsing: All, the site root, or System Host. Bare "scope" now names either, and a reader landing on either spec cannot tell which is meant. Worse, both end up as fields on the same Content Drive search request, so the ambiguity outlives the specs.

I have already renamed mine to spell browse scope everywhere, with a note in Key Entities pointing at yours so the two cannot be conflated. Could you do the matching half here: say search scope in the prose rather than plain "scope", and name the field searchScope on the wire?

Then the pair reads unambiguously wherever they meet: searchScope says how a search reads what is there, browseScope says where "there" is. Neither name is load-bearing on its own, but together they stop a future reader guessing.

Unrelated to the naming, and only because it touches the same sentence I was reading: your spec's "All Content" label is the one I flagged above as really meaning all fields. If that wording changes, the field values change with it, so it is worth settling before the name goes on the wire.

Comment written by Claude on @zJaaal's behalf.

@zJaaal

zJaaal commented Sep 11, 2026

Copy link
Copy Markdown
Member

One more on the contract, following the naming note above: I think the search scope belongs inside filters, next to text, rather than as another top-level field.

AbstractQueryFilters today is { text, filterFolders }, and filterFolders's own Javadoc says "when text is provided". Both members exist to qualify the text search, which is exactly what the search scope does: it says which fields text reads, and it means nothing without text. Putting it beside text makes them travel together, and a request carrying a scope with no text becomes visibly nonsense instead of a validation rule someone has to remember later. So filters.searchScope.

To be clear that this is not a general "scopes go in filters" rule: the browse scope in #37487 stays at the top level, for two reasons. It qualifies assetPath rather than text. And the filter bar's "Clear all" resets the filters object, so a browse scope living there would let clearing your filters navigate you out of System Host. A filter chip narrows what you see where you are; it must not be able to move you somewhere else.

Related, since it came up while looking at this: we also considered letting the path alone carry the browse scope, with no field at all, mirroring how the Content Drive URL now encodes it in one value. It does not survive contact with this endpoint, and for a reason that matters to your spec too: assetPath: "//site/" already means "the whole site at any depth", and the Asset Picker sends exactly that. Redefining it would silently change what the picker asks for. The general rule both our features are leaning on is the same one, that an existing payload has to keep meaning what it means today, and it constrains where new meanings can be expressed.

Worth flagging so nobody invokes consistency in either direction: filters is not currently the home for filters. contentTypes, baseTypes, language, workflow, status and userSearchable all sit at the top level and all of them genuinely are chips in the filter bar. Moving those six in is only part of a real cleanup though, so I would not treat it as the benchmark. The contract also lets a caller say one thing twice, archived: true and status: ["ARCHIVED"], which ContentDriveHelper then has to reconcile, and it carries live as a boolean when it actually selects which version you get rather than filtering anything. That is a contract PR of its own, worth doing while this endpoint still has exactly two callers, and not something either of our feature PRs should take on.

Comment written by Claude on @zJaaal's behalf.

@ihoffmann-dot ihoffmann-dot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mid: possible conflict with ADR-0018 (Database-First Search for Content Drive)

ADR-0018's routing table states explicitly:

Title → DB ∪ Index (contentlet.title column ∪ index records)

This union exists specifically to preserve read-your-writes: a just-saved/renamed item that the index hasn't caught up to yet must still be findable via the DB title column.

FR-008/FR-010 define Title scope purely as a narrower Elasticsearch query (drop catchall, drop the leading wildcard, gate on title). Nothing in the FR/SC list references consulting the contentlet.title DB column, and no acceptance scenario or success criterion covers "a document created/renamed just before search, not yet indexed, searched by title."

The ADR does note the DB ∪ Index union is gated on a separate title-persistence effort and may not be fully implemented in the current codebase either, so this may already be a pre-existing gap rather than something this feature introduces. It's worth an explicit call before /speckit-plan

No other findings. No other ADRs in platform-adrs were found that this spec contradicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[TASK] Content Drive: add a Title / All Content scope selector to the search box

3 participants