Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
190 lines (138 loc) · 15.4 KB

ARCHITECTURE.md

File metadata and controls

190 lines (138 loc) · 15.4 KB
Raw

Architecture — agent-code-cli-java

A from-scratch reimplementation of a coding-agent harness (in the spirit of Claude Code) on Java 25, built to understand the moving parts of an agentic CLI rather than to ship a product. The same harness runs against the Anthropic API or a local Ollama server, so the whole loop can be developed and tested offline, without spending tokens.

A harness here means everything around the model: the agent loop, the tool layer and its permission gate, conversation history and context compaction, streaming, and the terminal UI. The LLM is the engine; this repo is the chassis.

A sibling port — agent-code-cli-rust — implements the same design in Rust; see Rust sibling for the comparison.

What this is (and isn't)

  • Is: a small, fully-tested agent runtime — agent loop, sealed tool layer with a permission gate, context window management, dual LLM backends, and a JLine REPL.
  • Isn't: a production service, an SDK, or a Claude Code drop-in. There is no auth, no telemetry, no multi-user state. Scope is a single local workspace.
  • Why it exists: to build the harness, not just use one — and to exercise Java 25 (records, sealed types, structured concurrency, AOT startup) on a real, concurrent, I/O-heavy problem.

High-level architecture

                 ┌─────────────────────────────────────────────────┐
   stdin / CLI ─▶ │  App (Picocli)  ─▶  REPL (JLine)  |  one-shot     │
                 └───────────────────────┬─────────────────────────┘
                                         │ prompt
                                         ▼
                              ┌────────────────────────┐   summarize when
              History  ◀──────│       AgentLoop         │◀── ≥75% of ctx ── Compactor
            (Turn deque)      │ gather → act → verify → │
                              │        repeat           │
                              └─────┬─────────────┬─────┘
                          prompt    │             │  tool_use
                                    ▼             ▼
                        ┌────────────────┐   ┌────────────────────────────────┐
                        │   LlmGateway   │   │   ToolRegistry  +  Gate          │
                        │   (interface)  │   │   (PermissionMode)               │
                        ├────────────────┤   ├────────────────────────────────┤
                        │ Anthropic (SDK,│   │ Read · Write · Bash · Glob · Grep│
                        │   SSE stream)  │   │ executed in parallel via         │
                        │ Ollama  (HTTP, │   │ StructuredTaskScope, then gated  │
                        │  NDJSON stream)│   │ by tool.readOnly() + mode        │
                        └────────────────┘   └────────────────────────────────┘

Entry point: com.caiocv18.agentcode.App (Picocli). Build: Gradle 9.5.1 (Kotlin DSL), Java 25 with --enable-preview.

The agent loop

AgentLoop implements the classic cycle — gather context → call the model → run the tools it asks for → feed results back → repeat — until the model returns a terminal stop reason or maxIterations is hit.

  1. Append the user turn to History; if the estimated context is ≥ 75% of maxTokens, Compactor summarizes the oldest turns first.
  2. Send the prompt to the selected LlmGateway; stream text deltas to the UI as they arrive.
  3. If the reply's stop reason is TOOL_USE, dispatch every requested tool in parallel via StructuredTaskScope, gating each call through Gate before execution.
  4. Append tool results as a new turn and loop.
  5. Terminate into a ChatResult — a sealed type with Done, Truncated, or Refused, so the caller switch is exhaustive on the model's Stop reason.

Turn and ChatResult are sealed interfaces with immutable record cases; conversation state is therefore pattern-matchable and copy-on-construct.

Component map

Package Type Responsibility
(root) AgentLoop The loop: model call ↔ parallel tool dispatch ↔ history/compaction, with ActivityListener lifecycle callbacks
(root) App Picocli entry point; wires components and picks one-shot vs interactive path
(root) ChatResult (sealed) Done / Truncated / Refused loop outcome
llm/ LlmGateway (interface) Backend contract: Prompt/Reply/ToolDef records + Stop enum; streaming via a text-delta Consumer
llm/ AnthropicLlmGateway Adapts Anthropic Java SDK 2.34.0; accumulates streamed tool-input JSON; maps SDK stop reasons
llm/ OllamaLlmGateway HttpClient against Ollama 0.24+; parses NDJSON; extracts tool_calls by hand (no SDK)
session/ History, Turn (sealed) Ordered turns; User/Assistant records carry tool results/uses
context/ Compactor Token estimate (≈4 chars/token), summarizes old turns past the threshold, keeps the latest turn
tools/ Tool, ToolRegistry LinkedHashMap registry; dispatch ToolUse → ToolResult, wrapping exceptions as ToolResult.error()
tools/ ReadFileTool, WriteFileTool, BashTool, GlobTool, GrepTool The five built-ins; each declares readOnly() and a JSON input schema
tools/ffm/ Ripgrep Subprocess bridge to rg --json behind a ProcessFactory seam (see note on FFM)
permissions/ Permission, Gate, PermissionMode Allow/deny per call based on mode and tool.readOnly()
tui/ ReplEngine, ChatPrompt, StreamingRenderer, ToolRenderer, theme/, markdown/ JLine REPL, incremental streaming output, tool-call tree, 12-color theme (with PT-BR aliases), markdown highlighting

Key design decisions

Backend behind one interface (LlmGateway), two implementations. Anthropic and Ollama speak different protocols — SSE with event:/data: framing vs. one JSON object per line (NDJSON) — and Ollama needs no API key. Hiding both behind a single Prompt → Reply contract means AgentLoop never knows which backend it's talking to, the whole loop is developable offline against a local model, and tests stub HTTP with WireMock instead of hitting the network. Default backend is local Ollama (gemma2:2b); Anthropic is opt-in.

Tools as a closed set, dispatched by a registry. Each tool implements a small Tool interface, declares readOnly(), and ships its own JSON-Schema input. ToolRegistry is the single dispatch point and the single place exceptions become ToolResult.error() — so a misbehaving tool degrades to an error message the model can read, never an uncaught throw that kills the loop.

A three-mode permission gate, keyed on readOnly(). PermissionMode is DEFAULT (prompt before any write/exec), AUTO_ACCEPT (run everything — batch/dev use), and PLAN (read-only; the agent may inspect and propose but not mutate). The gate reads tool.readOnly(), so adding a read-only tool needs no gate changes. Path-mutating tools additionally normalize-and-startsWith-check every path against the workspace root, rejecting ../ escapes and absolute paths.

Parallel tool calls via Structured Concurrency. When the model requests several tools in one turn, they run concurrently inside a StructuredTaskScope (Java 25 preview): the scope joins all subtasks and propagates cancellation/timeout as a unit — no orphaned threads, no manual executor lifecycle. Single-tool turns take a plain synchronous path.

Context compaction instead of unbounded history. The model's window is finite, so Compactor watches a cheap char-based token estimate and, past 75% of maxTokens, replaces the oldest turns with a text summary while preserving the system prompt and the most recent turn. Summaries are prose, not structured JSON, to stay legible when re-fed.

Testability designed in via SAM seams. Subprocess-touching code (BashTool, Ripgrep) sits behind single-method factory seams (ProcessRunner/ProcessFactory) so tests inject deterministic fakes — fast/slow/timeout — without bytecode instrumentation. This kept the coverage gate (below) achievable without mocking the JDK.

A 95% JaCoCo gate as a forcing function. jacocoTestCoverageVerification fails the build below 95% across all six counters (instruction, branch, line, complexity, method, class). Coverage stops being a vanity number and becomes part of "green"; in practice it pushed better seams and small testable helpers rather than test theater. Genuinely untestable terminal side-effect classes (App, SpinnerLoop, ChromeActivityListener) are explicitly excluded rather than faked.

Java 25 in practice

Honest split between what the code uses and what was evaluated while reading the JEPs. (The README and design notes discuss the second group; the source does not wire it — calling it "used" would be a stretch.)

JEP / feature Status Where / why
395 Records ✅ Used ~17 record types (Prompt, Reply, ToolUse, ToolResult, Turn.*, ChatResult.*, …) — immutable data with copy-on-construct
409 Sealed types ✅ Used Turn, ChatResult — exhaustive switch, compiler catches a new case left unhandled
453 Structured Concurrency (preview) ✅ Used StructuredTaskScope in AgentLoop for parallel tool dispatch; requires --enable-preview
483 / 514 / 515 AOT ✅ Used (scripts) aot-record/create/run scripts produce an AOT cache; cold start measured < 100 ms (hyperfine)
506 Scoped Values ⬜ Evaluated Considered for session propagation; the MVP uses constructor injection + activity callbacks instead
442 FFM ⬜ Evaluated Ripgrep calls rg --json as a subprocess (simple, deterministic, testable); a real FFM Linker binding is deferred
519 Compact Headers · 521 Gen. Shenandoah · 502 Stable Values ⬜ Evaluated Discussed in the design notes; not configured/wired in the MVP build

Testing

  • JUnit 5 + AssertJ, 50 test files.
  • Mockito 5.22 loaded as a -javaagent (required on JDK 25 — see below).
  • WireMock stubs the LLM HTTP layer, so the full loop is tested end-to-end with deterministic replies and no network.
  • JaCoCo 0.8.13 gate at ≥95% on six counters. Final coverage: instruction ≈97%, branch ≈96%, line ≈96%, complexity ≈96%, method ≈98%, class 100%.
  • ./gradlew build compiles (--enable-preview), tests, reports, and verifies coverage; anything under threshold breaks the build. Every roadmap step landed green; all commits are SSH-signed.

What broke (and how it was fixed)

The interesting part of building on a brand-new JDK is the friction. Each of these was a real blocker:

  • Mockito couldn't mock on JDK 25. ByteBuddy in Mockito 5.14 didn't understand class-file format 69, and JDK 21+ restricts dynamic agent loading. Fix: upgrade to Mockito 5.22 and attach it as an explicit -javaagent in the test task.
  • Anthropic SDK builders fail .build() on "optional" fields. The Kotlin builders run checkRequired even for fields the consumer treats as optional (citations, cacheCreation). Fix: in gateway tests, mock Message directly (Mockito 5.22 + agent) rather than constructing it via the builder; public constants like StopReason.END_TURN stay usable.
  • **/*.java didn't match top-level files. Java's glob requires at least one / after **, so a.java slipped through. Fix: a compileGlob helper rewrites **/<rest> to {**/,}<rest> (zero-or-one separator).
  • Grep tests hung forever. The ripgrep subprocess left stderr open and the pipe filled and blocked. Fix: redirect stderr in the default factory and introduce the ProcessFactory seam, with Ripgrep.alwaysUnavailable() as the test default.
  • JLine fell back to a "dumb terminal" under gradle run. JavaExec hands the REPL a closed stdin, so no real TTY. Fix: wire standardInput = System.in, add a --prompt one-shot flag for pipes, and prefer the installDist launcher which inherits a real TTY.
  • gemma2:2b rejected tool calls with HTTP 400. Some Ollama models have no function-calling support and 400 when a tools field is present. Fix: a --tools flag (off by default); for real tool-calling use a capable model (llama3.2:3b).
  • Ollama crashed on a 2 GB GPU. The runner tried to place a ~966 MiB compute graph on a GTX 750 Ti and died (exit code 2). Fix: force CPU with num_gpu=0 (and CUDA_VISIBLE_DEVICES=-1), which is plenty since LLM latency, not the client, is the bottleneck.

Running it

# Build, test, and enforce coverage
./gradlew build

# Interactive REPL against a local Ollama model (default backend)
./gradlew run --args="--model llama3.2:3b --tools"

# One-shot (pipe-friendly), CPU-only
./gradlew run --args="--prompt 'list the build files' --num-gpu 0"

# Cold-start optimization (AOT cache)
scripts/aot-record.sh && scripts/aot-create.sh && scripts/aot-run.sh

Requirements: JDK 25 (Temurin), Gradle 9.5.1, and either an ANTHROPIC_API_KEY or a running Ollama 0.24+ (default http://host.docker.internal:11434, which is the WSL2 host). Flags: --model, --ollama-url, --num-gpu, --prompt, --tools, --permission-mode, --accent, --no-color.

Rust sibling

agent-code-cli-rust is the same harness in Rust — useful as a controlled comparison of the same design under two runtimes.

Aspect Java 25 Rust
Backend Anthropic SDK or local Ollama Local Ollama (trait leaves the door open for others)
Concurrency StructuredTaskScope (preview) tokio::JoinSet
ripgrep subprocess (rg --json) in-process crates (grep, globset, ignore)
Tools Read · Write · Bash · Glob · Grep + an edit tool with a colored diff
Distribution JVM + classpath (AOT cache ~<100 ms start) single static binary (~5–10 MB, strip + LTO + panic=abort, ~10 ms start)
Tests / coverage JUnit5 + Mockito + WireMock, JaCoCo ≥95% tokio + wiremock + insta, llvm-cov ≥90%
Logging Logback tracing (structured, RUST_LOG)

Limitations & non-goals

  • No real tokenizer — the compactor uses a 4-chars/token heuristic, which is approximate.
  • FFM binding deferred — ripgrep is a subprocess today; a native Linker binding is future work.
  • Some Java 25 features are evaluated, not wired (see the JEP table) — kept honest on purpose.
  • Single local workspace, no concurrency across sessions, no persistence beyond the process.
  • Preview features mean the build pins --enable-preview; not for production use.