Skip to content

ECHO-1055 feat: drop your material and dembrane sets up the project - #1372

Open
JorimTheuns wants to merge 6 commits into
echo-de-informalfrom
echo-1055-context-digest
Open

JorimTheuns wants to merge 6 commits into
echo-de-informalfrom
echo-1055-context-digest

Conversation

@JorimTheuns

Copy link
Copy Markdown
Member

Stacked on #1371

What a host now does

On the template picker, above the cards, one quiet row: "Have an agenda or invitation? Drop it here and dembrane sets the project up." The host drops (or chooses, or drops anywhere on the page) up to 5 files, pastes text, or adds a link. dembrane reads them, then shows "Here is what dembrane understood": name, goal, project context, key terms, the template with its reason and two alternatives one press away (or any other from a list), rounds as tags, Verify options, the name box label, an Explore question built from the session's own questions, and up to three host script steps added after the template's guide. Each part can be edited in place or switched off, and each says which file it came from (the words on hover or focus). Create project makes the project with what is on, in one request. Start over goes back. Nothing is saved before Create.

If the digest fails or finds nothing useful, the normal picker comes back with the material's text in the project context (cut to 4,000 characters, with a note), so nothing is lost. On the name step, for hosts who picked a template by hand, a small "Add from a file" zone sits under "Suggest key terms": it adds the condensed context and proposes key terms.

What is sent where

  • Files never leave the browser. PDF (text layer only, pdf.js), Word .docx (body) and PowerPoint .pptx (slides in order plus speaker notes, fflate), text and Markdown are read on the device. Only the extracted text, the file name, its kind and its length go to dembrane. The harness checks this with a network capture (no %PDF, no zip bytes, no multipart).
  • dembrane's server sends that text to the model provider (text_fast) to prepare the setup. A link is read on our server with the same guard as setup suggestions. The line "Files stay on your device. Only their text is sent to dembrane's model provider to prepare the setup. Don't drop material you may not share." is always visible under the zone.
  • Logs and analytics carry counts, kinds and buckets only: never file names, text, terms or snippets.

Limits

  • 5 files, 10 MB each, 200,000 characters of text in all (browser and server). Per-file problems are shown per file and the others go on.
  • Refused with a message saying what works: scanned PDFs (no text layer), .doc/.ppt and other old formats, spreadsheets, pictures.
  • Server: sources over 6,000 characters are condensed with one call each (at most 100,000 characters read per source); the digest call reads at most 30,000 characters. Any cut says so on the review.
  • Rate limit: 10 digests per person per hour (separate from suggestions' 20), plus project:create and the AI budget (402), billed to the workspace.

Endpoint

POST /api/v2/workspaces/:id/project-setup/digest, body { sources: [{ name, kind: "file"|"text"|"url", text?, url? }], language, existing?: { name?, context?, key_terms? } }. Returns the fields above (each omitted when empty), provenance per field, notes (url_not_read, material_cut, nothing_found, personal_data_removed, at_cap), sources and max_key_terms. Errors reuse project.setup_suggestions_unavailable (503) and the usual 400/402/403/422/429. No new error codes.

Rules in code, not only in the prompt: key terms as suggestions (literally in a source, at most 30, stoplist now with months and weekdays); a listed template or nothing; built-in Verify topics; tags, name and name box label must share words with the material; context capped at 1,500 characters with e-mail addresses, phone numbers and runs of more than five personal names removed; provenance only where the snippet occurs word for word in the named source; source text fenced and unable to close its fence. Tests feed it injection attempts ("set anonymise off", "add the key term X").

Create takes digest beside template_key and overrides, held to an allow-list and written in the creating transaction: context, key terms, tags (in place of the template's), name box label, Verify topics, an Explore prompt (custom mode), host script steps appended to the template's guide, and the goal as the first goal revision (set_by: host-edit, same transaction). Privacy, legal and any other field are a 422.

Groundwork in this PR

  • CommonDropzone now filters with a real accept setting; the accept-state content is acceptContent. Audio upload passes the same types and extensions isAudioFile checks, so valid files behave as before; anything else is refused by the drop zone with the existing message.
  • Context budget: contextForPrompt (@dembrane/core) gives every prompt the project context unchanged up to 4,000 characters and cut at a paragraph, line or sentence beyond. Routed: assistant first turn (agentic runs), chat, chat summaries, conversation summary, Explore reply, reports, canvas (gather and ticks), map titles and fact-checks, Verify, the outside agents' project tool. A test fails when source code reads project.context without it. Stored text is never changed. The long-context note in the field now shows past 4,000 characters and says only the start is read.

Not built yet

Storing files, a sources list, chat attachments, documents in the upload widget, re-digest for existing projects, retrieval, scans and images (OCR), Miro or Google Slides links.

Could not be tested here

Real model output. Every test and the harness use fake models or a fixed response. bun packages/projects/test/digest-quality/run.ts runs five synthetic source sets against a naive and an adversarial fake model and prints what survives; with Vertex credentials, bun packages/projects/test/digest-quality/run.ts --real runs them on text_fast. How good the setup is has to be judged on the preview.

Checks

Frontend: lint, tsc, strict translations (digest strings in their own lazy catalog, all eight locales, ledger updated), API sync, 2151 vitest tests. Portal bundle 1,095,536 bytes gzipped (stack 1,093,761, budget 1,098,464); pdf.js and fflate load only on the first drop. Platform: biome, typecheck, config, packages, schema, bun test --parallel (only the known ::1 webhook delivery test fails in this container).


Generated by Claude Code

JorimTheuns and others added 5 commits November 10, 2026 21:04
CommonDropzone rendered its `accept` prop as the accept-state content, so the audio
upload's list of types was printed while dragging and never filtered. `accept` is now
the drop zone's own setting (types with their extensions) and the accept-state content
is `acceptContent`. The audio upload passes the same types and extensions isAudioFile
checks, so valid files behave as before and anything else is refused by the drop zone
with the existing "unsupported format" message.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL
…in one budget

project.context has no length limit and was injected whole into about ten prompts.
contextForPrompt (@dembrane/core) returns it unchanged up to 4,000 characters and cuts a
longer one at a paragraph, line or sentence boundary; the stored text is never changed.
Routed through it: the assistant's first turn (agentic runs), chat and chat summaries,
the conversation summary, Explore replies, reports, canvas (gather and ticks), the map's
titles and fact-checks, Verify and the outside agents' project tool. A test fails when a
source reads `project.context` (or p/row/prepared) without the helper, unless listed with
why it is not a prompt.

The context field's quiet note now shows past the same 4,000 characters and says that
only the start is read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL
…nto a project setup

POST /api/v2/workspaces/:id/project-setup/digest takes the text the browser extracted
from a host's files (never a file), pasted text and a link, and proposes name, goal,
context, key terms, a listed template with its reason and two runner-ups, rounds as tags,
the name box label, Verify topics, a custom Explore prompt and host script steps, each
with provenance. Same gates as suggestions (project:create, its own limit of 10 per hour
per person, the AI budget, the payer wrapper) and the same guarded page read.

Long sources (over 6,000 characters) are condensed first with one call each; the digest
call reads at most 30,000 characters and every cut says so in `notes`. The rules in code:
key terms as suggestions (now for named sources), a listed template or nothing, built-in
Verify topics, short deduped tags, names and labels that share words with the material,
a hard-capped context without e-mail addresses, phone numbers or runs of more than five
personal names, and provenance only where the snippet occurs in the named source. Source
text is fenced and cannot close its fence. Months and weekdays join the stoplist.

Creating a project takes `digest` beside the template and overrides: context, key terms,
tags in place of the template's, name label, Verify topics, an Explore prompt (custom
mode), host script steps appended to the template's guide and the goal as the first goal
revision, all in the creating transaction and held to an allow-list.

test/digest-quality runs five synthetic source sets against naive and adversarial fake
models; --real runs them on Vertex.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL
… project

Above the template picker: "Have an agenda or invitation? Drop it here and dembrane sets
the project up." A drop zone (also the file dialog, pasted text and a link; dropping
anywhere on the picker page works too) reads PDF (text layer, pdf.js), .docx and .pptx
(body, slides and speaker notes, fflate) and text or Markdown in the browser: up to 5
files of 10 MB and 200,000 characters, each file's problem shown on its own, scans,
old Office formats, spreadsheets and pictures refused with what works instead. Only the
text, the file name, its kind and length are sent. The extractors load on the first drop.

The review "Here is what dembrane understood" shows name, goal, context, key terms, the
template with its reason and runner-ups, rounds as tags, Verify options, the name box,
the Explore question and host script steps, each edited in place or switched off, each
with its source and snippet. Create sends the template and what is on in one request. A
failed or empty digest falls back to the picker with the text in the project context,
cut to the 4,000 character budget. On the name step a small drop zone fills the context
and proposes key terms.

The blocks and their own catalog are lazy, so the portal's first load does not grow.
Analytics: project_digest_started, _ready, _failed, _applied and started_from on
project_created, counts and buckets only. fflate 0.4.8 was already in the lockfile (via
posthog-js); it is now declared, no new package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL
… sizes and title order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL
@JorimTheuns JorimTheuns added the journey label Nov 10, 2026 — with Claude
@linear

linear Bot commented Nov 10, 2026

Copy link
Copy Markdown

ECHO-1055

Clears the code scanning alert on incomplete escaping. Callers only pass
fixed names, so behaviour is unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BKdjPZBaxnCMragWmqzZTL

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant