feat(core): fall back to the CoDev mirror for ripgrep downloads

CHANGELOG.md

@@ -9,6 +9,14 @@ file is the human-readable summary of what changed and why.
 
 ### Core
 
+- **ripgrep downloads work without GitHub access** — when `rg` is not
+  installed, CoDev Code fetches it on first use from GitHub releases, which
+  corporate networks commonly block. The download now falls back to the CoDev
+  site's own mirror of the official binaries, and each attempt is bounded to
+  30 seconds so a proxy that swallows traffic can't stall first-time file
+  search for minutes. GitHub remains the primary source; nothing changes on
+  networks where it is reachable.
+
 - **`@` file mentions work on Windows** — on Windows the `@` popup is powered
   by a ripgrep-based fallback (the native fuzzy finder is disabled there), and
   that fallback could end up permanently empty with no error shown: if

DIVERGENCES.md

@@ -167,7 +167,7 @@ from the squash message first, or use a plain merge.
 | `packages/tui/src/component/prompt/autocomplete.tsx`, `packages/app/src/components/prompt-input.tsx`, `packages/opencode/src/cli/cmd/run/footer.prompt.tsx`, `packages/core/src/filesystem/search.ts`, `packages/tui/test/fixture/tui-sdk.ts`, `packages/opencode/test/cli/run/footer.view.test.tsx`, `packages/core/test/filesystem/search.test.ts` | `@` mention autocomplete lists file/directory results before references/agents/MCP resources; TUI and run-CLI menus cap the leading files so up to 3 non-file matches stay in the first visible page; web app group rank reordered to recent → file → reference → agent → resource (extracted into fork-only `prompt-input/at-options.ts`). TUI and run-CLI file resources track menu visibility so opening on an empty query fetches (previously nothing refetched until the query changed); core's ripgrep-fallback `find` returns its first entries for an empty query to mirror fff. Tests: tui-sdk fixture's `FetchHandler` gains an optional `method` arg, `renderFooter` accepts `findFiles`/`agents` overrides + a mention-menu ordering test, core search tests cover the fallback's empty query (fork-only test files: `packages/tui/test/prompt-autocomplete.test.tsx`, `packages/app/test-browser/prompt-at-options.test.ts`, `packages/app/src/components/prompt-input/at-options{,.test}.ts`) | Files are the dominant `@` use case; they should rank first and appear on a bare `@`. | Keep our ordering/reserve logic, resource source tracking, empty-query branch, fixture/test additions; take upstream's surrounding changes. |
 | `packages/slack/.gitignore`, `sdks/vscode/.gitignore` | Added the missing trailing newline (content unchanged) | Upstream's copies end without a newline; POSIX text files should end with one, and tools that concatenate or append to these files misbehave without it. | Whitespace-only: keep the trailing newline; take upstream's entries. |
 | `packages/tui/src/config/keybind.ts`, `packages/tui/src/component/prompt/index.tsx`, `packages/tui/src/app.tsx` | Permission-mode indicator + shift+tab cycle. keybind.ts: added `permission_mode_cycle: keybind("shift+tab", ...)` + its `permission.mode` CommandMap entry; `agent_cycle_reverse` default `"shift+tab"` → `"none"` (tab still cycles agents forward; previous-agent stays rebindable). prompt/index.tsx: the footer badge next to the agent name always renders in prompt mode — gray `manual` (`theme.textMuted`) in normal mode, red `auto` (`theme.error`) in auto mode (upstream only showed gray `auto`); the idle footer's shortcut hints gain a `shift+tab modes` text (via a `modeShortcut` `useCommandShortcut("permission.mode")` memo) left of `tab agents`, in the same no-usage `Match` arm. app.tsx: `"permission.mode"` added to `appBindingCommands` so the binding is gathered. | The permission mode is always visible (manual/auto) with auto in red as a danger signal, and shift+tab toggles it without opening the palette. | Keep the `permission_mode_cycle` definition + mapping, the `agent_cycle_reverse` `"none"` default, the badge `Show` block, the `modeShortcut` memo + hint text, and the `appBindingCommands` entry; take upstream's surrounding changes. If upstream reworks the auto badge or rebinds shift+tab, re-apply on top and re-audit the shift+tab conflict. |
-| `packages/core/src/filesystem/search.ts`, `packages/core/src/ripgrep/binary.ts`, `packages/core/src/filesystem.ts`, `packages/core/src/tool/read-filesystem.ts`, `packages/core/test/filesystem/search.test.ts`, `packages/core/test/location-filesystem.test.ts` | Windows file-search fallback robustness. search.ts: the background ripgrep index walk no longer dies silently (`Effect.orDie` in a forked fiber) — each attempt resets the accumulated file/directory state, failures log a warning ("file index scan failed, file search may be empty"), and the walk retries forever on a 1s exponential backoff capped at 30s spacing, so installing rg mid-session self-heals; directory entries are suffixed and detected with `"/"` instead of `path.sep` (both ripgrep and fff layers), and fallback `find` queries normalize `\` to `/` and collapse doubled slashes (directory tab-expansion appends `/` to an already-`/`-suffixed entry). binary.ts: `Effect.cached` on ripgrep binary resolution (which memoizes failure exits for the process lifetime) replaced with a success-only cache behind a semaphore, so a failed rg download (offline, blocked proxy) retries on the next use instead of bricking search until restart. filesystem.ts `list` + read-filesystem.ts `list`: directory suffix `path.sep` → `"/"`; `fs.list` also `/`-normalizes `path.relative` output. Tests: search.test.ts pins `"/"`-suffixed directories, adds query-normalization coverage, and a stub-Ripgrep scan-retry test asserting the partial first walk is discarded; location-filesystem.test.ts pins `"src/"`. | On Windows fff is disabled by default (upstream `e4300e9b74`), so the `@` mention popup and file search depend entirely on the ripgrep fallback; a missing `rg.exe` plus one failed download left the index permanently empty with no user-visible signal (the CI-workflows row's choco preinstall papered over the same swallowed failure). `\`-suffixed directory entries also broke `endsWith("/")` directory-mime detection in the run CLI and poisoned drill-down queries with backslashes that can never match the `/`-normalized index. | Keep our retry/reset scan, success-only binary cache, `"/"` suffixes, and query normalization; take upstream's surrounding changes. If upstream reworks the fallback or re-enables fff on win32, preserve the invariants: entry paths are `/`-normalized with `/`-suffixed directories on every platform, index-scan failures are logged and retried, and a failed binary download is never cached. |
+| `packages/core/src/filesystem/search.ts`, `packages/core/src/ripgrep/binary.ts`, `packages/core/src/filesystem.ts`, `packages/core/src/tool/read-filesystem.ts`, `packages/core/test/filesystem/search.test.ts`, `packages/core/test/location-filesystem.test.ts`, `packages/core/test/ripgrep-binary.test.ts`, `packages/core/test/preload.ts` | Windows file-search fallback robustness. search.ts: the background ripgrep index walk no longer dies silently (`Effect.orDie` in a forked fiber) — each attempt resets the accumulated file/directory state, failures log a warning ("file index scan failed, file search may be empty"), and the walk retries forever on a 1s exponential backoff capped at 30s spacing, so installing rg mid-session self-heals; directory entries are suffixed and detected with `"/"` instead of `path.sep` (both ripgrep and fff layers), and fallback `find` queries normalize `\` to `/` and collapse doubled slashes (directory tab-expansion appends `/` to an already-`/`-suffixed entry). binary.ts: `Effect.cached` on ripgrep binary resolution (which memoizes failure exits for the process lifetime) replaced with a success-only cache behind a semaphore, so a failed rg download (offline, blocked proxy) retries on the next use instead of bricking search until restart; the download tail tries GitHub first (each attempt bounded to 30s) and falls back to the CoDev landing-page mirror (`https://netmind.viettel.vn/codev/docs/code/downloads/rg-<version>-<node platform>-<node arch>[.exe]`, a raw executable — no archive/extract — staged in a scoped temp dir then copied into place), via `Effect.catchCause` so a proxy block page (200 + HTML) that dies in `extract` as a defect still reaches the mirror. filesystem.ts `list` + read-filesystem.ts `list`: directory suffix `path.sep` → `"/"`; `fs.list` also `/`-normalizes `path.relative` output. Tests: search.test.ts pins `"/"`-suffixed directories, adds query-normalization coverage, and a stub-Ripgrep scan-retry test asserting the partial first walk is discarded; location-filesystem.test.ts pins `"src/"`; ripgrep-binary.test.ts adds mocked-HTTP coverage of the fallback order, success caching, and failure non-caching; preload.ts redirects `XDG_CACHE_HOME` to a fresh temp dir so no test can read or clobber the developer's real `~/.cache/codev`. | On Windows fff is disabled by default (upstream `e4300e9b74`), so the `@` mention popup and file search depend entirely on the ripgrep fallback; a missing `rg.exe` plus one failed download left the index permanently empty with no user-visible signal (the CI-workflows row's choco preinstall papered over the same swallowed failure). `\`-suffixed directory entries also broke `endsWith("/")` directory-mime detection in the run CLI and poisoned drill-down queries with backslashes that can never match the `/`-normalized index. | Keep our retry/reset scan, success-only binary cache, `"/"` suffixes, and query normalization; take upstream's surrounding changes. If upstream reworks the fallback or re-enables fff on win32, preserve the invariants: entry paths are `/`-normalized with `/`-suffixed directories on every platform, index-scan failures are logged and retried, a failed binary download is never cached, and the GitHub download keeps its CoDev-mirror fallback — the mirror filename scheme must match what codev-landing-page hosts under `public/docs/code/downloads` (and codevhub's `RG_VERSION` pin in its `src/lib/ripgrep.ts`); bump all three together when changing `VERSION`. |
 | `packages/tui/src/context/permission.tsx`, `packages/tui/src/app.tsx`, `packages/tui/src/ui/dialog-confirm.tsx`, `packages/tui/src/ui/toast.tsx` | Auto-approve consent gate. permission.tsx: mode always starts `"normal"` (upstream initialized it from `args.auto`); `toggle()` replaced by `set(mode)` plus a per-run `consented` flag + `consent()`. app.tsx: shared `enableAutoApprove()` (title in the `AUTO_APPROVE_TITLE` module const, body from an `autoApproveMessage` accessor, and a shorter one-line `autoApproveToast` accessor for the toast, which only has a few seconds on screen; both redden the word "dangerous" with `theme.error` and are passed unevaluated because renderables can only be created during render) shows a `DialogConfirm` opened on cancel and, on confirm, a warning toast under the same title, then sets the mode; both entry points funnel through it — an `onMount` when `args.auto` is set, and the `permission.mode` command (shift+tab / palette), which enables via the gate and disables directly. Consent is asked once per run; later toggles are silent. dialog-confirm.tsx: optional `defaultChoice` prop + 5th `DialogConfirm.show` arg selecting the initially-active button (omitted keeps upstream's confirm-first default); `message` widened to `JSX.Element \| (() => JSX.Element)` and called at the render site. toast.tsx: same `message` widening, resolved by a `message()` helper in `Toast()` (fork-only test file: `packages/tui/test/auto-approve-consent.view.test.tsx`). | Auto-approve is dangerous, so `--auto` (and its hidden aliases) and the first shift+tab into auto require explicit confirmation; declining or dismissing the dialog stays in manual mode, opening on cancel keeps a stray return from granting it, and the risk word is red like the `auto` badge. | Keep the consent flag + gate flow, the `toggle()` removal, the `defaultChoice` prop, and the accessor-aware `message` on both dialog and toast; take upstream's surrounding changes. If upstream adds new writers of the permission mode, route their enable path through `enableAutoApprove()`. |
 
 ## Known rebrand gaps

packages/core/src/ripgrep/binary.ts

@@ -12,6 +12,13 @@ import { which } from "../util/which"
 
 export namespace RipgrepBinary {
   const VERSION = "15.1.0"
+  // Fallback for networks where GitHub is blocked: the CoDev landing page
+  // mirrors the official executables for this VERSION (raw, not archived) at
+  // rg-<version>-<platform>-<arch>[.exe]. Bumping VERSION requires hosting the
+  // new binaries there first; codevhub pins the same version in lib/ripgrep.ts.
+  // Base64 (like codev-gateway's const.ts) to keep the host out of casual
+  // greps, not as a secret.
+  const MIRROR = atob("aHR0cHM6Ly9uZXRtaW5kLnZpZXR0ZWwudm4vY29kZXYvZG9jcy9jb2RlL2Rvd25sb2Fkcw==")
   const PLATFORM = {
     "arm64-darwin": { platform: "aarch64-apple-darwin", extension: "tar.gz" },
     "arm64-linux": { platform: "aarch64-unknown-linux-gnu", extension: "tar.gz" },
@@ -93,6 +100,21 @@ export namespace RipgrepBinary {
       // broken until restart. Cache successful resolution only.
       const resolved = { filepath: undefined as string | undefined }
       const semaphore = Semaphore.makeUnsafe(1)
+      const download = Effect.fnUntraced(function* (url: string) {
+        yield* Effect.logInfo("downloading ripgrep", { url })
+        const bytes = yield* HttpClientRequest.get(url).pipe(
+          http.execute,
+          Effect.flatMap((response) => response.arrayBuffer),
+          // Bounded per attempt: a proxy that blackholes instead of refusing
+          // would otherwise sit on OS-level TCP timeouts for minutes before
+          // the fallback source is even tried.
+          Effect.timeout("30 seconds"),
+          Effect.mapError((cause) => (cause instanceof Error ? cause : new Error(String(cause)))),
+        )
+        if (bytes.byteLength === 0) return yield* Effect.fail(new Error(`failed to download ripgrep from ${url}`))
+        return new Uint8Array(bytes)
+      })
+
       const resolve = Effect.gen(function* () {
         const system = yield* Effect.sync(() => which(process.platform === "win32" ? "rg.exe" : "rg"))
         if (system && (yield* fs.isFile(system).pipe(Effect.orDie))) return system
@@ -104,22 +126,43 @@ export namespace RipgrepBinary {
         const config = PLATFORM[platformKey]
         if (!config) throw new Error(`unsupported platform for ripgrep: ${platformKey}`)
 
-        const filename = `ripgrep-${VERSION}-${config.platform}.${config.extension}`
-        const url = `https://github.com/BurntSushi/ripgrep/releases/download/${VERSION}/${filename}`
-        const archive = path.join(Global.Path.bin, filename)
-
-        yield* Effect.logInfo("downloading ripgrep", { url })
         yield* fs.ensureDir(Global.Path.bin).pipe(Effect.orDie)
-        const bytes = yield* HttpClientRequest.get(url).pipe(
-          http.execute,
-          Effect.flatMap((response) => response.arrayBuffer),
-          Effect.mapError((cause) => (cause instanceof Error ? cause : new Error(String(cause)))),
-        )
-        if (bytes.byteLength === 0) throw new Error(`failed to download ripgrep from ${url}`)
 
-        yield* fs.writeWithDirs(archive, new Uint8Array(bytes))
-        yield* extract(archive, config, target)
-        yield* fs.remove(archive, { force: true }).pipe(Effect.ignore)
+        const fromGithub = Effect.gen(function* () {
+          const filename = `ripgrep-${VERSION}-${config.platform}.${config.extension}`
+          const archive = path.join(Global.Path.bin, filename)
+          const bytes = yield* download(`https://github.com/BurntSushi/ripgrep/releases/download/${VERSION}/${filename}`)
+          yield* fs.writeWithDirs(archive, bytes)
+          yield* extract(archive, config, target)
+          yield* fs.remove(archive, { force: true }).pipe(Effect.ignore)
+        })
+
+        // The mirror hosts the raw executable (no archive), named with node's
+        // platform/arch vocabulary — see codev-landing-page
+        // public/docs/code/downloads. Staged into a scoped temp dir first: the
+        // cache check above is bare existence, so partial bytes written to
+        // `target` directly would permanently satisfy it.
+        const fromMirror = Effect.gen(function* () {
+          const bytes = yield* download(
+            `${MIRROR}/rg-${VERSION}-${process.platform}-${process.arch}${process.platform === "win32" ? ".exe" : ""}`,
+          )
+          const dir = yield* fs.makeTempDirectoryScoped({ directory: Global.Path.bin, prefix: "ripgrep-" })
+          const staged = path.join(dir, path.basename(target))
+          yield* fs.writeWithDirs(staged, bytes)
+          yield* fs.copyFile(staged, target)
+          if (process.platform !== "win32") yield* fs.chmod(target, 0o755)
+        }).pipe(Effect.scoped)
+
+        // catchCause, not catch: a proxy serving a block page returns 200 with
+        // HTML, which passes filterStatusOk and dies in extract as a defect —
+        // that case must reach the mirror too.
+        yield* fromGithub.pipe(
+          Effect.catchCause((cause) =>
+            Effect.logWarning("github ripgrep download failed, trying the CoDev mirror", { cause }).pipe(
+              Effect.flatMap(() => fromMirror),
+            ),
+          ),
+        )
         return target
       })
       return Service.of({

packages/core/test/preload.ts

@@ -1,5 +1,13 @@
 import path from "path"
+import fs from "fs"
+import os from "os"
 
 process.env.CODEV_DB = ":memory:"
 process.env.CODEV_MODELS_PATH = path.join(import.meta.dir, "plugin", "fixtures", "models-dev.json")
 process.env.CODEV_DISABLE_MODELS_FETCH = "true"
+// Redirect the XDG cache (Global.Path.cache / Global.Path.bin) into a fresh
+// temp dir so tests never read or clobber the developer's real ~/.cache/codev
+// — ripgrep-binary.test.ts writes fake bytes at the rg target path. Must
+// happen here: xdg-basedir snapshots env when first imported, and bun runs
+// the whole suite in one process.
+process.env.XDG_CACHE_HOME = fs.mkdtempSync(path.join(os.tmpdir(), "codev-test-cache-"))