Repository navigation
feat(sources): the personal Library - #881
yaacovcorcos wants to merge 42 commits into
Conversation
One Library per computer for all of a researcher's sources; projects keep their citation keys and project notes and cite into it. Existing projects move in once, with a sealed copy of their evidence kept in the project. Includes the agent-trust work of #823, M2 (linked PDFs, categories, Zotero groups, styles) and M3 (store, merges, possible duplicates, bulk edits in parts, the move, packages, recovery), reviewed and gated together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Thread transfer impact✅ Thread transfer remains within every enforced ceiling.
Other scenariosAll 64 scenario measurements match All 64 scenario measurements
Baseline: Scenario and decoded snapshot size10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.
Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed. |
Conflict in .github/workflows/ci.yml (Windows Overleaf job): kept main's accounts and browser tests and the Google sign-in smoke test, then the Sources file tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict: the measured agent context budget file. Main's values, then the Library's three measured values on the merged tree. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
This PR changes the test suite by +1064 test cases (browser +152, unit +530, server +382). New browser test files: Keep tests that would fail for a real reason, and test logic below the UI where it can be. See Testing. |
Main: one conflict, the measured agent context budget file (main's values, then the Library's three as measured on the merged tree). Sources improvement program, part 1 (library branch 666642d..4987854): - an interrupted merge or Undo is finished before the next change; - after an Undo, a key never links to the other paper and both papers' project copies are their Library records; - Unlink puts the saved copies back and holds; - the Library's rules are tested under random use, killed processes and a full disk. Details per part: reviews/sources-improve-20261115/INDEX.md in the workspace. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- The crash and random rule tests timed out on the CI runners (about eight times slower there). Their default runs are now a smoke check; the full sweeps stay as local runs. - Typing in the Library search no longer waits for the list to filter. Library branch 4987854..b272e10. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…l fixes; merge main (220f377) Brings the library branch's tree at 9aba20b onto the PR branch: - typed LaTeX citations offer the document's .bib entries and the Library's sources - a copy a merge took away is not in the Trash; Empty Trash keeps a merge's Undo - Move to Trash in This project rows, Add to project after adding by identifier, Try again for a failed move, Check again for Zotero, Show versions - the list's diagnostics are sparse; typing in the search is not held by the list - main merged; measured agent context sizes updated Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… values after a merge Undo Brings the library branch's tree at dcdd131 onto the PR branch: - Recent changes: one list of bulk edits, merges, imports and the Trash, each with who made it and one action (Undo, Show sources, Restore) - a bulk Undo offered later leaves a tag or a type that was changed on the paper since, and remembers what it left - after a merge is undone, a value a project's saved details had and the Library record lacks is kept in the project instead of dropped - merged copies are deleted before their paper; agents see the panel's Trash - the Library names its computer when it is not this one - the typed citation list updates when a .bib read finishes - the move and the full-text store use the shared safe file writer Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…for agents Brings the library branch's tree at bdf680b onto the PR branch: - Back up the Library: a dated folder with every file the Library keeps, a manifest with each file's hash, and a check that says what changed, is gone or does not belong; runs in the background with progress in the panel - a merge Undo keeps displaced values per holder: the project's saved details and each review, one copy per Undo - an agent can read Recent changes through scient_sources_list Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings the library branch's tree at e626a2e onto the PR branch: - Restore from a backup: the backup is checked and copied beside the Library, and takes the Library's place the next time Scient starts; Cancel until then. A Library that held anything is kept as a folder beside the restored one - a backup leaves files out by exact name only, checks its snapshot inside the lock, and says when a project had not finished moving into the Library - a backup's progress line no longer stops after a failed status read - the agent wording for Recent changes says to undo only unfinished parts Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…rd, a kept Move back Brings the library branch's tree at f794298 onto the PR branch: - a restore that stops half way cannot be shadowed by a new, empty Library - the restored Library records what it replaced; Recent changes lists each restore with the kept Library's folder - a 'Move back into this project' chosen after the backup is kept - a refused second request is said beside the copy that still waits; the panel drops a status answer for a project it has left - a backup file name a restore will not write is named in the answer Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings the library branch's tree at 1328dc8 onto the PR branch: - Add from another Library: adds the sources a backup, or the Library a restore replaced, holds and this Library lacks, each whole; nothing that is in the Library is changed, and what both hold is counted and left as it is here - Recent changes lists each join with Undo, and offers Add its sources on a restore whose replaced Library is kept - a join that stops is settled before a later one adds anything; the same folder can be added from again Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings the library branch's tree at c0d00d3 onto the PR branch: - a source is added only when nothing here could be it, checked again as it is added; two copies rebuilt from one project source are recognised - Undo leaves a source a project uses, one filed in another collection, and one whose PDF was highlighted since - a merge the other Library undid stays undone here; each source is filed as it arrives; a stopped join is settled and its sources filed - a restore that has not finished keeps every project out of the Library until the next start - the guide no longer suggests restoring to make two worked-on Libraries equal Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings main's 25 commits (the guest room work and the 0.6.29 release notes) under the Sources stack. No conflicts; the agent context budget is unchanged. The tree is the library branch's at 03ff2b1. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ond review fixes Brings main's T3 alignment through 7f6e403 under the Sources stack. One conflict: the measured agent context budget, measured again on the merged tree (the tool list is 121,072 bytes; main's is 91,677). Also brings the library branch's fixes after the second review of adding from another Library (c4d3453): a kill-safe order of adding, an Undo that keeps whatever it cannot show is untouched, collections a source could not be put in named in the result, and no write to the Library while a restore is unfinished. The tree is the library branch's at a3d5edb. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings the library branch's tree at 03e9136 onto the PR branch: - Show differences on an addition in Recent changes: the papers both Libraries have where the other has something this one lacks, with both sides - three actions, one at a time, each an ordinary edit: add the other note under your own, attach one PDF, add a look-alike paper separately - an action is refused when either paper changed since the list was shown Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…found Same tree as claude/sources-library-server-20261111 at f5ae10e. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Same tree as claude/sources-library-server-20261111 at 639c367. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…eview of Put it back Same tree as claude/sources-library-server-20261111 at 91b4d1d. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Same tree as claude/sources-library-server-20261111 at e277a3a. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Same tree as claude/sources-library-server-20261111 at 48039ce. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nd the join Same tree as claude/sources-library-server-20261111 at 6bf2d9c. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Same tree as claude/sources-library-server-20261111 at ea4b20b. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… one import Same tree as c6df1ff on claude/sources-library-server-20261111. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…n versions of a review; main merged Same tree as 4b0321f on claude/sources-library-server-20261111. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…e; the weekly retraction watch; main merged Same tree as a82aa56 on claude/sources-library-server-20261111. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same tree as bc4fd8f on claude/sources-library-server-20261111. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… night Same tree as d9805b6 on claude/sources-library-server-20261111. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same tree as ca30c75 on claude/sources-library-server-20261111. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Library menus regrouped, one result slot and Activity instead of stacked cards, sort headings and cards, source page facts rail and filing row, silent note saves, every bibliography key kept, incomplete author lists, journal icons from the article page, My Library wording, university connect in the header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter, star beside the menu; no Main note in the university picker Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pandoc citation groups and LaTeX citation commands yield each cited key with the locator written for it; a locator's first page is read as a printed page label. Shared by the citation navigation and the list of a source's uses. Co-Authored-By: Claude Fable <noreply@anthropic.com>
Quote in my document no longer cites the first of several sources that share one PDF (the chapters of a book carry the book's PDF). The source the PDF was opened from wins, and gives no key when it has none; opened another way, one keyed source is used and several are offered as a choice. The citation's page is the PDF's printed page label when every quoted page has one, also when quoting from a source's highlight list, and the PDF page number otherwise. Co-Authored-By: Claude Fable <noreply@anthropic.com>
Clicking a Markdown citation (Control/Command-click while writing) now goes as far as the citation says: a citation ending a quote made by Quote in my document reveals that highlight; a page locator opens the source's PDF at the page with that printed label (the page number when the PDF has no labels); no page opens the source's details. Several Library sources in one citation are offered in a menu instead of opening the first. A source without a PDF opens its details and says so. The PDF reader gains a page reveal next to the passage reveal. Co-Authored-By: Claude Fable <noreply@anthropic.com>
The guide promised that Control/Command-click opens a citation's source while writing; LaTeX had no such handler. In Source, Control/Command-click inside a \cite-family command opens the key under the click, or every key of the command; biblatex multicites give each key its own note. In Visual, it opens the clicked citation (the clicked key when citations are numbered), and a plain click does when the editor cannot be edited. Both pass the text quoted just before the command, so a quote made by Quote in my document reveals its highlight. Co-Authored-By: Claude Fable <noreply@anthropic.com>
A % comment between a citation command and its arguments, or inside a key list, hid the citation from the key rename or turned the comment into a key. Arguments are now read from text whose comments are masked to spaces (offsets kept), and only the key bytes are rewritten. Found by the Codex review of the citations work (finding 6).
A source's page lists every citation of it in the project's Markdown and LaTeX documents, by document, with line, locator and the sentence around it; an occurrence opens the document at that line. Only in-text citations count: prose in Markdown, \cite-family commands outside comments and verbatim in LaTeX; \nocite is reported apart. A key counts when it is one of the project's keys for the source; when a document's own bibliography defines it, the entry must name the same work (DOI, PMID, arXiv), or it is reported as naming another work. Anything that could not be read makes the result incomplete, and the page then never says the source is not cited. Agents get the same list through scient_sources_get's new uses input (+82 bytes of tool text, mcpToolListJsonBytes 122041). Co-Authored-By: Claude Fable <noreply@anthropic.com>
Opening a citation now asks the server which work each key names in the document's effective bibliography (its own files, the managed references.bib, and for LaTeX the including document's): another work is not opened and Scient says why; an entry that cannot be compared opens with a note. Retained keys (alsoCitedAs) open their source. A repeated quotation is narrowed by each PDF's printed page labels, never by physical page numbers; unreadable or ambiguous labels keep the repeated-passage notice. LaTeX Source clicks ignore verbatim text and read arguments past comments. Fixes Codex review findings 1, 2, 5 and 7.
A built-in compare-sources skill compares the sources a researcher names on their question and writes an editable Markdown document: columns from the question, each cell reported, derived (with its working) or interpretation, cited with the printed page when seen and the PDF page beside it, empty cells as Not reported, Could not assess or Not checked, possible shared studies flagged, and what was read stated. It puts a finding into another document only when asked. Compare in the Library's selection bar (2+ sources) puts the request in the chat draft and never sends. Labelled experimental: there is no evaluation of its output yet. Co-Authored-By: Claude Fable <noreply@anthropic.com>
The user guide and internals describe quoting at the printed page and the choice between sources sharing a PDF, opening a citation's page or passage in Markdown and LaTeX, Used in this project with its rules and limits, and Compare. Co-Authored-By: Claude Fable <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he newest annotation reading wins In the Visual LaTeX editor that cannot be edited, the citation button became focusable but Enter did nothing; it now opens the citation like a click. Overlapping reloads of a PDF's annotations keep only the newest answer, so a quote cites the current source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What Changed
Scient's Sources become a personal Library: one store per computer for all of a researcher's sources. Projects keep their own citation keys and project notes and cite into the Library. This PR carries the whole Sources stack, squashed onto current
main, and supersedes #823 (its agent-trust work is included).Why
The owner's goal is for Scient to replace a researcher's reference manager. Per-project stores could not do that. The design, the scale gate, and two independent first-principles reviews (FP-1, FP-2) are in the Sources roadmap folder. Every step was reviewed by Codex until clean; the integrated result was reviewed again.
Documentation Impact
Documentation impact: Updated —
docs/user/sources.md,docs/internals/scient-sources.md, and the systematic-review skill text.Verification
mainmerged, macOS Apple Silicon only): install, format, lint, typecheck, build, desktop smoke, branding, diff check and context budget all pass.mainfails the same way: FX skills, Git paths, Antigravity casing, runtime cleanup, file watching, SyncTeX setup, image-dialog pan.latexVisualStabilityscroll drift appeared once, then passed 40/40 on this branch and 40/40 onmainat CPU throttle 6 and 8. It is the known flaky family; no tolerance or retry was changed.main's 6 newest commits were merged: the budget, skills, analytics and process-tree tests pass, and the server typecheck passes.Added since the first version (2026-11-15)
Hardening found and fixed four defects, each with a test that fails without its fix:
Then the first experience fixes (2026-11-15, later the same day):
\cite{and the natbib and biblatex forms (multicites included), in Visual and in Source: the entries of the document's own.bibfiles, then this project's Library sources, by key, author, year or title. If the document does not loadreferences.bibyet, the list says which line to add.Then, after an independent first-principles review of the whole area:
.scient/library/displaced/).Then backup and restore:
reviews/sources-improve-20261115/DECISIONS.md): the list no longer hashes the whole backup on every page, an edit made during an addition is noticed, and wording that was untrue was corrected.The rules these rest on, each with its trade-off, are one page for the owner:
reviews/sources-improve-20261115/RULES-FOR-OWNER.md(local).Then the move-in from Zotero, first step:
reviews/sources-improve-20261115/design/F1-zotero.md): today's import drops at least one bibliographic field on 11% to 100% of references, most often the place of publication; the loss is concentrated in books, reports, software and humanities material..bib, RIS, EndNote XML and CSL-JSON carry these details in and out.pnpm check:zotero-schemacompares the two.Codex reviewed every part and challenged each design before it was built; each finding is fixed or declined with a reason. Limits that remain are stated in
docs/internals/scient-sources.md("Rules the stores keep", the notes on unmerged sources, and "Library restore").Agent Context
Needs
agent-context-increase. The tool list is 121,450 bytes, against 91,677 onmain(+29,773: +29,227 from the first version, +168 for the one input that lets an agent read Recent changes, +275 for the Zotero step: three source types, a source's further details in the update input, and the refresh choice, +103 for going on with a large Zotero import; measured again after each step). Codex measured the change tool by tool on both trees:addToLibrary,sourceId): +497.mainin name, description and input schema.referencesskill. Earlier experiments support each of those instructions.UI Changes
Screens and every user flow (75 screens, 48 flows, light/dark, wide/narrow) are in the local review pack the owner has (
reviews/sources-roadmap-20261110/review-pack/index.html). The owner's test app runs this build.Checklist
🤖 Generated with Claude Code