Repository navigation
feat(sources): linked PDFs, categories, subcollections, Zotero groups and Library naming - #858
Open
yaacovcorcos wants to merge 1 commit into
Conversation
Contributor
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Baseline: unavailable · PR result: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
… and Library naming Reference-file import attaches the PDFs a BibTeX, RIS or EndNote file links to and reports every link it leaves out. The managed bibliography keeps listing text it set aside after a failed write and cleans up only its own partial copies. A background pass adds study design and topics, both filterable by people and agents. Collections nest one level. Zotero import, refresh and send reach group libraries, including with a key that reads only groups. The place is called the Library in every user-facing string. The managed bibliography files no longer carry abstracts. The OpenAlex key from Accounts is sent on every OpenAlex request and Google Books is a new research key. scient_sources_read_text refuses unknown parameters and returns smaller pages of text without cutting a page. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
yaacovcorcos
force-pushed
the
claude/sources-intake-groups-20261111
branch
from
November 11, 2026 17:23
3983869 to
0517e05
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
.bib, RIS or EndNote file dropped the PDFs the file pointed to, so a move from another reference manager arrived without its papers.references.biborreferences.json, the text it had set aside was reported once and then forgotten, and a failed copy could leave a partial file behind.references.bibandreferences.jsoncarried every abstract, which made them large and noisy in diffs although no document uses abstracts.scient_sources_read_textcalls passed a parameter that does not exist (page,pages,offset) and it was dropped without a word, so the agent read the wrong pages. Large reads overflowed the provider's inline result size, and a page longer than the limit was cut with the rest lost.Resulting behavior
Linked PDFs from reference files. BibTeX
file, RISL1/L4and EndNotepdf-urlslinks are resolved when the file is staged. A PDF is attached when it is a real PDF inside the project or the reference file's folder (never Scient's own folder or a filesystem root), up to 20 per entry. Every link that is not attached is listed in the import report with the reason. Agents getpdfsAttachedand the reasons in thescient_sources_import_fileresult; no new tool input.Managed bibliography. A failed write no longer loses the kept text: each sync lists
references.{bib,json}.kept-<id>copies found on disk until the person deletes them. The copy fallback writes to a private name and removes only a file it created itself, so it cannot delete another writer's file. The two managed files no longer contain abstracts; an explicit export still does. The first sync after the upgrade rewrites an untouched file quietly (no "edited" report, no kept copy); a file edited by hand is still reported as edited. Reading hand edits back only adds entries, so a missing abstract in the file never clears one in the Library.Study design and topics. A background pass fills study design (PubMed publication types, then MeSH design headings) and topics (arXiv categories, OpenAlex top topics) into
.scient/sources/categories.json, apart from the records, 50 sources per request, resumable, with a 6-hour retry after an outage. The filter menu gains Study design and Topic with counts; source details show the categories. Agents filter bystudyDesignandtopicinscient_sources_listand see the counts inscient_sources_collections.Nested collections (one level). Collections can have subcollections; a parent's filter and count include them. Agents can create a set of themes with parents in one bulk edit, undone as one step. Zotero import keeps one level and reports deeper collections it folded into their level-1 ancestor.
Zotero group libraries. The import picker lists the groups Zotero on this computer syncs, or on zotero.org the groups the key can read. A group item is linked as
groups/<groupId>with its own source key, so it never collides with an item of the user's that has the same key. Refresh reads each source from the library it came from. Send to Zotero can target a group when the key may write there and the group lets the user edit. A key that reads groups but not My Library works: the picker opens on a group and leaves My Library out. Agents passlibraryto the existing Zotero tools.One name: the Library. The panel, tab, launcher, back buttons, toasts, error and empty states, the built-in plugin name and the tool error now say "Library"; one item in it is still "a source". Identifiers, storage paths, the plugin ID
scient.sources, tool names and the new-tab shortcut (S) are unchanged. Past release notes keep their wording.Research keys. The OpenAlex key from Accounts is sent on every OpenAlex request, including the background category pass and the Library's own routes. Google Books is a new research key in Accounts (Research group), sent as
keytowww.googleapis.comonly. If Google refuses the key (400 or 403, for example a key without the Books API turned on), the request is repeated once without it. A row turned Off is not sent; background work reads the keys again at each pass. All reads stay inresearchKeys.ts(accounts seam manifest updated).Reading text.
scient_sources_read_textrefuses unknown parameters and names the accepted ones in the error (Tool.Stricton this tool only; the advertised schema is unchanged). The default read is 12,000 characters and the maximum 30,000 (were 20,000 and 60,000). A page is never cut: one longer than the limit comes back whole and alone, andnextPagecontinues after it.Verification
On the combined branch at
9df221e983(this PR's tree), macOS, Node 26.12.1:vp fmt --checkandvp lint --report-unused-disable-directiveson the changed files,knip:check,brand:check,git diff --check,scripts/scient-seam-check.mjs --base origin/main: pass.vp run typecheckinpackages/scient-sources,packages/scient-citations,apps/server,apps/web: 0 errors. (packages/contractsandpackages/client-runtimewere typechecked at42d69a1014, before the last five fixes, which do not touch them.)apps/servertest files (76 tests: Sources tool registration and handlers, write authorization, reference-check handlers, full text, managed bibliography, research keys, categories, ISBN lookup, paper queue, and the three tool-list budget tests),apps/webaccountsView.test.tsandsurfaces.test.ts(57),packages/scient-citationsmanagedBibliography.test.ts(4),packages/scient-sourcesbibliography.test.ts(44): pass.42d69a1014: the six Zotero test files (45 tests) andapps/webSourceDetails.test.tsxandsourcesWorkflow.test.ts(34): pass.40b265ff70):packages/scient-sources(238 tests),packages/contracts, allapps/serverSources tests (240) and allapps/webSources unit tests (106): pass.apps/serverandapps/websuites, web browser tests,pnpm run buildand the desktop smoke test. This machine was under memory pressure, so only focused files were run; CI covers the rest. No test file asserts one of the renamed strings (searched)."<doi>"[aid]search and the OpenAlexpmid:andopenalex:filters. Not checked live: the local Zotero group list, Better BibTeX group library IDs, zotero.org group reads and writes, and keyed OpenAlex or Google Books requests.Documentation impact
Updated:
docs/user/sources.md,docs/user/accounts.md,docs/internals/scient-sources.md, and the place name indocs/README.mdanddocs/user/{markdown,extensions,latex,thread-sidebar,pdf-reader,file-previews,telemetry}.md.Model and harness
Built by Claude Opus 5.5 (Claude agent in Scient), reviewed by Codex gpt-6.1-sol, review fixes and integration by Claude in Factory Droid (Scient).