Distribution: extension packages + release workflow, portable MCP configs, a softer coaching directive - #24
Merged
Merged
Conversation
app.ts held every route, its helpers and the middleware in one 87 KB file.
It is now the app shell (CORS, body caps, tracing, auth, LLM rate limits,
GET /, route mounting, error handler: 243 lines), with:
- http.ts: what the middleware and the routes share (env type, tenant
policy, rate limiting, the prompt cap, skill-observation writes)
- routes/{score,coach,assist,wiki,prompts,metrics,teams,onboard}.ts: one
Hono sub-app per area, mounted after the middleware so every route still
gets auth, body caps, tracing and rate limits
Code moved verbatim (apart from `app.<verb>` → `<router>.<verb>` and
`export` on the shared helpers), so behaviour is unchanged: the 38
Postgres integration tests, which drive every route and cross-check the
GET / catalog, pass unchanged. apps/api/README.md documents the layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wiki-bootstrap-job.ts built its own GoogleGenAI client, so the rich bootstrap (the biggest fan-out of LLM calls in the API) bypassed the retry policy and never appeared in Langfuse traces. It now calls gemini.generateText, which goes through the same tracedGenerate/withRetry path as the request handlers: 429/5xx are retried with backoff, timeouts are still never retried (90 s per call, as before). generateText does not apply the 4 000-char cap meant for short structured answers, because folder narratives are several thousand words; a new test pins both properties. Removed as dead code: extractLearning (gemini.ts), EXTRACT_MODEL (gemma-4-31b-it) and the extract-prompt module in packages/scoring with its tests. They served the Stop hook that was removed in PR #5; nothing has imported them since. The README's "Gemma for async learning extraction" claim goes with them; every call uses gemini-3-flash-preview. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecutable Next 16.3 generates AGENTS.md and CLAUDE.md in apps/dashboard on every `next dev` unless `agentRules: false` is set; they showed up as untracked files for anyone running the dashboard. apps/mcp-server/bin/cli.mjs has a shebang and is the package's `bin`, but was committed as 100644 (npm fixed the mode on install, leaving a dirty tree). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…utes/ The README said gemma-4-31b-it handled "async learning extraction" and "diff narration, rich bootstrap". Learning extraction was never wired up (removed in the previous commits) and /diff and the bootstrap use gemini-3-flash-preview. Also points the route table at the new layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l that runs in CI The scorer at the centre of the product (one Gemini call) had never been measured: the eval harness existed but needs a key, so nothing ran. This adds the part that can run anywhere. packages/scoring/src/heuristic-score.mjs scores a prompt on the five rubric dimensions with transparent rules over surface features (file paths, identifiers, numbers with units, constraint and output phrasing, meta instructions aimed at the scorer). It is deterministic, has no imports and no Node APIs, so the API's offline mode and the landing page can both use it. It is a floor for the model, not a substitute. The eval gains: - an ordering metric: for every pair of prompts whose expected bands don't overlap, does the scorer rank the better one higher (ties count as wrong); - `--scorer heuristic` (free, no key, no --yes); - eval/holdout.json: 16 prompts with bands, written before the rule set was frozen and never used to tune it (golden.json was visible while writing); - baseline.test.ts: runs the rule-based scorer over both sets in `npm test` and fails if it drops below what it measured when frozen. Measured (heuristic-v1): golden (seen): 29/30 overall in band, 47/49 dims, 152/152 pairs ordered holdout (unseen): 14/16 overall in band, 18/22 dims, 38/38 pairs ordered The known misses are listed in eval/README.md. The Gemini scorer still has not been run on either set; that needs a key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing ran without a Gemini key: the API refused to boot, so a stranger could not see the coach work, and nobody could try the stack before getting a key. TRAILHEAD_LLM=offline runs the same API with no model: - llm.ts: every LLM call the routes make goes through it and picks its provider per call (llm-mode.ts): gemini.ts by default, offline-llm.ts in offline mode. gemini.ts now creates its client on first use, so importing it needs no key. - offline-llm.ts: the rule-based scorer from packages/scoring for /score and /coach; keyword rules for topics; '' for the LLM-written rewrites, tips, acknowledgements and summaries (the /coach renderers already fall back to their static templates); a template /diff narrative naming the biggest gap; an /improve that asks the static question per weak dimension and appends the answers. POST /onboard/repo/full answers 503 llm_unavailable. - It says so everywhere: GET / reports "llm", /score and /coach return "scorer": "heuristic", coaching text ends with a note, the score card in the extensions shows "Rule-based score", and the API warns at startup. - It never switches on by itself: with no key and no TRAILHEAD_LLM the API still refuses to start (now naming both ways to fix it), so a deploy that lost its key fails loudly instead of quietly scoring with rules. docker-compose.yml no longer requires the key up front, so `TRAILHEAD_LLM=offline docker compose up` works with no .env at all. SELFHOSTING.md documents what offline mode does and does not do, including that the confirming re-score for library promotion is the same rules. Tests: llm-mode and offline-llm unit tests (with fetch stubbed to fail), and an integration test against real Postgres that scores, coaches, promotes, diffs and improves in offline mode and asserts no request-path Gemini call was made (verified: pointing /score back at gemini.ts makes it fail). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…DEMO_TEAM) The seeded demo team's secret (trailhead_demo_acme_2026) is in this repo, and schema.sql always creates the team. On a server reachable from a network, anyone could write to its wiki and spend the operator's Gemini quota through it; the only bound was the per-team and per-IP rate limits, and there was no way to turn it off. TRAILHEAD_DEMO_TEAM=on|off controls it. The default stays on for a local setup, but setting TRAILHEAD_ADMIN_TOKEN (the sign of a networked deploy) turns it off unless TRAILHEAD_DEMO_TEAM=on says otherwise. Off, the demo secret gets 401 with reason demo_team_disabled and a message saying what to do; the team row stays, so turning it back on just works. GET / reports `demo_team`. Integration test: default on; off once an admin token is set (other teams unaffected); an explicit value wins in both directions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…demo up`
The seed wrote a wiki and four library prompts but no prompt activity
("Full §11 seed ... deferred"), and compose never ran it, so a fresh stack
showed an empty skill-arc chart and zeros on the team page.
packages/db/seed.mjs now also writes, for three made-up users, 48 scored
prompts each spread over the last six days with scores that drift upwards
(720 skill_observations, deterministic: fixed-seed PRNG), plus 18 captures
with outcomes. It stays idempotent: the synthetic rows are replaced on every
run, which also moves the window up to "now" (the dashboard shows the last
24 h and the team page the last 7 days). The header says plainly that all of
it is synthetic, and the dashboard's demo-team card no longer calls the
fictional Acme Fintech "the team's actual repo conventions".
docker-compose.yml gains a one-shot `seed` service under the `demo`
profile, started after the API is healthy:
TRAILHEAD_LLM=offline docker compose --profile demo up --build
Verified with that command on a fresh volume: seed exits 0, /team/metrics
reports 720 observations, 3 active users, avg 5.2; the dashboard's
/skill-arc renders 24 hour-buckets and /team shows non-zero cards (headless
Chromium). Running the seed twice leaves the same counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI built the API image but never ran the stack, so "docker compose up" could break (schema mount, healthcheck, env wiring) without a red build. The docker job now brings the stack up the way a stranger would, with no key (TRAILHEAD_LLM=offline), runs the demo seed, and checks: GET / is ok and offline, a team registers, a wiki write lands, /score answers with the rule-based scorer, and the seeded demo team has its activity. Logs are printed on failure and the stack is always torn down (down -v). The same commands were run locally against a compose stack before committing (all four jq checks true). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and gitignores what it creates `init` wrote absolute paths into the conventionally shared project configs: TRAILHEAD_TEAM_FILE=/home/<you>/repo/.trailhead-team and the path to the server inside your LearnLoop clone. A teammate who pulled the repo got an MCP config pointing at someone else's home directory. - TRAILHEAD_TEAM_FILE is now the relative `.trailhead-team`. The server looks for a relative team file in its working directory, then each parent up to the git root (findTeamFile), so it works when a host starts it in a subdirectory and never reads a sentinel from another repo above the root. With --user-scope this also stops one absolute path from binding every project to the same team. - The server entry still has to point at the clone (the package is not on npm), so a config `init` creates is added to .gitignore and `init` says why; a config that already existed (it may hold the team's other servers) is left alone, with a warning. Verified: in a fresh git repo, `cli.mjs init` against a local API writes `TRAILHEAD_TEAM_FILE: ".trailhead-team"` and gitignores .mcp.json; the MCP server started from src/api/ with that env answered a `coach` call over stdio. New tests for both behaviours; the CLI smoke tests now expect the relative path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…verriding them `init` writes the directive into the user's CLAUDE.md and copilot-instructions.md, and the tool descriptions say the same. Both told the agent that the tools "REPLACE native Read/Grep/Glob", that coach is "MANDATORY before answering ANY code task" (an extra LLM round-trip on every message), to save conventions "silently, never ask", and to start a rich bootstrap on its own whenever the wiki looked empty — which uploads the repo's source files to the LLM without the user having asked. The directive (154 → 50 lines) and the tool descriptions now say: - coach once at the start of each new code task, not on follow-ups; it never blocks work (the proceed/round_token/skip_reveal protocol is kept); - wiki_lookup gives the team's conventions for an area, then read the code as usual; - wiki_save when the user states a team rule, and say in one line that it was saved; - wiki_bootstrap only when the user asks; if the wiki looks empty, offer it, and say that rich mode sends source files to the team's API. A test pins the directive's shape (no REPLACE/INSTEAD OF/MANDATORY, no unasked bootstrap, ≤ 60 lines). Verified over stdio: tools/list and resources/read serve the new text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed release Neither extension could be installed without cloning and building: the browser extension had no icons and no package, the VS Code .vsix shipped its sources, tests, .vscode/ and a sourcemap (no .vscodeignore), and nothing produced either file in CI. - Browser extension: icons/icon.svg (the LearnLoop rings on the brand ink) rendered to 16/32/48/128 PNGs and declared in the manifest; the build copies them into dist/. `npm run package` makes a production build and learnloop-browser-ext-<version>.zip with a dependency-free zip writer (node:zlib deflate + crc32, fixed timestamps so the same dist zips to the same bytes). bundle-load.test checks every file the manifest references is in dist/. - VS Code extension: .vscodeignore (sources, tests, maps out: 22 files / 42 KB → 8 files / 18 KB), a marketplace icon, LICENSE in the package, displayName "LearnLoop", and `npm run package` (vsce 3.9.2 via npx). - Both at 0.1.0 (lockfile entries patched to match). - ci.yml: a `package` job builds both and uploads them as the `extensions` artifact on every run. - release.yml: on a v* tag, checks the tag matches both manifests, tests, packages, and attaches both files to the release for that tag (creating it as a draft if needed). Nothing is published to any store. Verified: the zip passes `unzip -t`; unzipped and loaded with `chrome --load-extension`, Chromium started the extension's service worker and its popup rendered. `vsce ls` lists only package.json, README, LICENSE, media/ and dist/extension.js. The release workflow's version check accepts v0.1.0 and rejects a mismatched tag (run locally with jq); the workflow itself runs only when a tag is pushed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
.vsixshipped its sources, tests,.vscode/and a sourcemap (there was no.vscodeignore).initwrote machine-specific paths into the shared MCP configs..mcp.jsonand.vscode/mcp.jsongotTRAILHEAD_TEAM_FILE=/home/<you>/…/.trailhead-teamplus the path into your LearnLoop clone. A teammate who pulled the repo got a config pointing at someone else's home directory.initwrites it into the user'sCLAUDE.md, and the tool descriptions say the same thing. Both told the agent that:coachis "MANDATORY before answering ANY code task" (an extra LLM round-trip on every message);What changed
Portable MCP configs (
5f5b864).TRAILHEAD_TEAM_FILEis the relative.trailhead-team. The server looks for it from its working directory up to the git root (findTeamFile) and never above.initcreates is added to.gitignore, andinitsays why. A config that already existed is left alone, with a warning.Coaching directive (
10b7bbd). It goes from 154 to 50 lines, and the tool descriptions are rewritten to match:coachonce per new code task;wiki_lookup, then read the code as usual;wiki_save, and say that it saved;wiki_bootstraponly when asked: if the wiki looks empty, offer it, and say that rich mode sends source files to the API.The
proceed/round_token/skip_revealprotocol is unchanged.Extension packaging (
5ba1abe).npm run packagemakes a production build pluslearnloop-browser-ext-<version>.zip, using a dependency-free zip writer..vscodeignore: 22 files / 42 KB → 8 files / 18 KB. Adds an icon, a LICENSE inside the package, the display name "LearnLoop", andnpm run package.packagejob that uploads both files as theextensionsartifact.release.yml: on av*tag it checks that the tag matches both manifests, runs the tests, packages, and attaches both files to that tag's release (creating it as a draft if needed). Nothing is published to any store.Verification
npm test(all workspaces),npm run typecheck,npm run lintdist/)npm ciwith the patched lockfilecli.mjs initin a fresh git repo, against a local offline APITRAILHEAD_TEAM_FILE: ".trailhead-team"and gitignores.mcp.json. The MCP server, started fromsrc/api/with that env, answered acoachcall over stdio.tools/list+resources/readover stdionpm --workspace=@trailhead/browser-ext run packageunzip -t. Unzipped and loaded withchrome --load-extension, Chromium started the extension's service worker and the popup rendered.npm --workspace=apps/vscode-ext run package/vsce lsrelease.ymlversion check, run locally withjqAfter merging (for the orchestrator)
v0.1.0on main (git tag -a v0.1.0 -m "LearnLoop 0.1.0" && git push origin v0.1.0).release.ymlthen creates a draft release with both files; publish it after a look. Or rungh release createfirst, and the workflow attaches to it.publisheridtrailheadis a placeholder.🤖 Generated with Claude Code