diff --git a/intend.md b/intend.md new file mode 100644 index 0000000..797b0ac --- /dev/null +++ b/intend.md @@ -0,0 +1,26 @@ +# Intent: Agent Skills Component + +SDLC stage 1 of 4: **Intent** -> spec.md -> plan.md -> validation.md + +## Project Goal + +The goal of this project is to extend the existing mini coding agent by implementing an agent skills component. This component will allow the agent to identify available skills and use their instructions when relevant to a user's prompt or when a specific skill is needed during execution. + +## Problem + +The current mini coding agent includes several components for handling user requests and performing coding tasks, but it does not have an agent skills component. Without this functionality, the agent cannot automatically recognize and use predefined skills to assist with specific tasks. + +## Proposed Solution + +We plan to develop an agent skills component that integrates with the existing mini coding agent. The component will: + +- Discover available skills stored in folders containing a `skill.md` file. +- Check whether a user's prompt is relevant to an available skill. +- Load and execute the appropriate skill instructions when needed. +- Log user prompts, similarity checks, and executed skills to track the agent's behavior. + +## Expected Outcome + +The completed component should allow the mini coding agent to recognize and use relevant skills during execution. We will demonstrate this functionality by using the skill created in Lab 5 to attempt the tasks from Lab 4. + +The logging functionality will also provide a record of how the agent identifies and uses skills, helping us verify that the component is working as intended. \ No newline at end of file diff --git a/mini_coding_agent.py b/mini_coding_agent.py index 1bf2403..cc3a9db 100644 --- a/mini_coding_agent.py +++ b/mini_coding_agent.py @@ -9,6 +9,7 @@ import uuid from datetime import datetime, timezone from pathlib import Path +from skill_logger import log_event DOC_NAMES = ("AGENTS.md", "README.md", "pyproject.toml", "package.json") @@ -447,6 +448,13 @@ def ask(self, user_message): if not memory["task"]: memory["task"] = clip(user_message.strip(), 300) self.record({"role": "user", "content": user_message, "created_at": now()}) + log_event( + self.root / ".mini-coding-agent" / "skill_events.jsonl", + "prompt_received", + self.session["id"], + prompt=user_message, + ) + tool_steps = 0 attempts = 0 diff --git a/plan.md b/plan.md new file mode 100644 index 0000000..edd03b0 --- /dev/null +++ b/plan.md @@ -0,0 +1,26 @@ +# Plan: Agent Skills Component + +SDLC stage 3 of 4: intent.md -> spec.md -> **Plan** -> validation.md + +## Design Decisions +- Same file format as Claude Code to keep consistency. +- Read CLAUDE.md for persistent-memory strategy. The skill relies on the grid in CLAUDE.md but the agent only reads README.md/AGENTS.md +- Execute = inject instructions into the prompt so when the model executes the skill, the skill dictates what it writes and which tools it uses, then is cleared after each request. + +## Tasks + +| # | Task | Files | Done when | +|---|------|-------|-----------| +| 1 | Skill parsing + discovery | `skills.py` | discovery tests pass | +| 2 | Tokenizer, cosine, trigger and explicit matching | `skills.py` | similarity tests pass | +| 3 | Registry in `MiniAgent`, catalog in prefix, `use_skill` tool + validation | `mini_coding_agent.py` | original tests still pass | +| 4 | `select_skill` / `activate_skill` / skill block in `prompt()` / cleanup in `ask()` | `mini_coding_agent.py` | injection + crash tests pass | +| 5 | Log the 4 events | `mini_coding_agent.py`, `skill_logger.py` | order/field tests pass | +| 6 | CLI flags, `/skills`, welcome row, child agents, `.claude/skills`, `CLAUDE.md`, `--num-ctx` | `mini_coding_agent.py` | `--help` shows flags | +| 7 | Import Lab 5 skill + add triggers; extra skills `pr-review`, `python-tdd` as distractors | `skills/`, `lab4_template/` | skill tests pass | +| 8 | Lab 4 template (CLAUDE.md, README, code-reviewer, skill; no code) + runner (scripted / `--ollama`) | `lab4_template/`, `run_lab4_with_skill.py`, `demo/` | `python main.py` exits 0 | +| 9 | Validation report | `validation.md` | filled with results | + +## Risks +- A small model may ignore skill rules -> the skill's rules are short and explicit; the transcript shows which were followed. +- Follow-ups with no domain words ("run main.py and fix it") miss the skill -> name the domain, use `/skill vacuum-rl`, or rely on the model calling `use_skill` from the catalog. \ No newline at end of file diff --git a/skill_logger.py b/skill_logger.py new file mode 100644 index 0000000..9138c55 --- /dev/null +++ b/skill_logger.py @@ -0,0 +1,20 @@ + +import json +from datetime import datetime, timezone +from pathlib import Path + + +def log_event(log_path, event_type, session_id, **details): + """Append a timestamped skill event to a JSONL log file.""" + path = Path(log_path) + path.parent.mkdir(parents=True, exist_ok=True) + + entry = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "session_id": session_id, + "event": event_type, + **details, + } + + with path.open("a", encoding="utf-8") as file: + file.write(json.dumps(entry) + "\n") diff --git a/spec.md b/spec.md new file mode 100644 index 0000000..fa8b73d --- /dev/null +++ b/spec.md @@ -0,0 +1,53 @@ +# Spec: Agent Skills Component + +SDLC stage 2 of 4: intent.md -> **Spec** -> plan.md -> validation.md + +Lists requirements, skill file format, and the log events. + +## Definitions +- **Skill**: a folder containing SKILL.md +- **Skill root**: a folder whose sub-folders are skills (e.g. `.claude/skills/`). +- **Active skill**: the skill whose instructions are injected into the model prompt for the current request. + +## Skill File Format +name: vacuum-rl # optional, defaults to folder name +description: What it does and when to use it. # optional, defaults to first paragraph +tools: Skill, Read, Write # optional, Claude Code field - ignored by the mini agent +triggers: [vacuum robot, q-learning] # optional, added for this agent; `keywords:` is an alias + +## Instructions the agent executes +Front matter supports `key: value`, `key: [a, b]` and block lists (`key:` then `- item`). Unknown keys are ignored. + +## Functional Requirements +| ID | Requirement | +|----|-------------| +| FR1 | **Discovery.** At start-up the agent loads skills from, in priority order: `/skills`, `/skill`, `/.claude/skills`, `/.mini-coding-agent/skills`, any `--skills-dir`, then the skills bundled next to `mini_coding_agent.py` (CLI only; `--no-bundled-skills` disables). On a name clash the earlier root wins. | +| FR2 | **Catalog.** If at least one skill exists, the prompt prefix lists `- name: description` under `Skills:` and the tool `use_skill(name)` is offered. With no skills, prefix and tools are unchanged. | +| FR3 | **Similarity check.** For every user prompt each skill gets a score in [0, 1]; the best skill is selected if `score >= threshold` (default 0.25, `--skill-threshold`). First matching rule wins: (a) explicit `/skill `, `$`, "use the skill" -> 1.0; (b) a trigger phrase occurs in the prompt as whole words, ignoring case and `_ - . /` -> 0.9; (c) else cosine similarity between prompt term counts and the skill profile (name x3, triggers x2, description x1) after lower-casing, stop-word removal and light stemming. | +| FR4 | **Execution (prompt-triggered).** The selected skill becomes active: its rendered instructions (name, description, source path, supporting files, body) are inserted after the cached prefix and memory and before the transcript, in every model call of that request. | +| FR5 | **Execution (agent-triggered).** The model may call `use_skill(name)` at any step; the skill becomes active for the rest of the request and its body is returned as the tool result. Unknown/empty names fail validation with the list of available skills. | +| FR6 | The active skill is cleared when the request ends (also on exceptions) and on `/reset`. Memory gets the note `skill activated ()`. | +| FR7 | **Logging.** Events are appended as JSON lines to `/.mini-coding-agent/skill_events.jsonl`, each with `timestamp`, `session_id`, `event`. | +| FR8 | Delegated child agents use the same skill roots and threshold, and log with `depth` > 0. | +| FR9 | REPL `/skills` lists skills and the log path; the welcome screen shows skill names. | +| FR10 | **Project memory.** `CLAUDE.md` is read into the workspace context with AGENTS.md/README.md, so the Lab 4 grid definition reaches the model. | +| FR11 | `--num-ctx` sets Ollama's context window (only sent when given). | + +## Non Functional Requrements +- NFR1: No new dependencies, no network calls in the similarity check; deterministic. +- NFR2: Backward compatible: all original tests pass unchanged. +- NFR3: The stable prompt prefix stays cache-friendly; the per-request skill block comes after it. +- NFR4: Injected skill body capped at 6000 chars; `use_skill` output at 8000. + +## Log Events +| event | when | extra fields | +|-------|------|--------------| +| `prompt_received` | first thing in `ask()` | `prompt` | +| `similarity_check` | right after | `prompt`, `threshold`, `candidates[]` (`skill`, `score`, `method`, `matched_terms`, `matched_triggers`, best first), `selected` (name or null), `reason`, `depth` | +| `skill_executed` | a skill becomes active | `skill`, `path`, `trigger` (`explicit`/`trigger_phrase`/`cosine`/`tool`), `score`, `instructions_chars`, `depth` | +| `skill_completed` | request ends with an active skill | `skill`, `tool_calls[]` (`name`, `ok`), `status` (`final`/`stopped`/`error`), `final` (clipped), `depth` | + +Order per request: `prompt_received` -> `similarity_check` -> (`skill_executed` -> `skill_completed`)? + +## Acceptance Tests (`test_skills.py`) +Discovery incl. `.claude/skills` (FR1), catalog / no-skill behaviour (FR2), Lab 4 prompts -> `vacuum-rl`, TDD prompts -> `python-tdd`, review prompts -> `pr-review`, unrelated -> none, explicit invocation, threshold (FR3), injection order (FR4), `use_skill` mid-run and unknown name (FR5), cleanup on crash (FR6), event order/fields (FR7), CLAUDE.md in prompt (FR10), num_ctx (FR11), Lab 5 skill body unchanged, Lab 4 end-to-end run. \ No newline at end of file diff --git a/tests/test_mini_coding_agent.py b/tests/test_mini_coding_agent.py index 5c62693..5582e80 100644 --- a/tests/test_mini_coding_agent.py +++ b/tests/test_mini_coding_agent.py @@ -397,3 +397,21 @@ def fake_urlopen(request, timeout): assert captured["body"]["raw"] is False assert captured["body"]["think"] is False assert captured["body"]["options"]["num_predict"] == 42 + + +def test_prompt_logging(tmp_path): + agent = build_agent(tmp_path, ["Hello!"]) + + agent.ask("Test prompt logging") + + log_path = tmp_path / ".mini-coding-agent" / "skill_events.jsonl" + + assert log_path.exists() + + with log_path.open("r", encoding="utf-8") as file: + entry = json.loads(file.readline()) + + assert entry["event"] == "prompt_received" + assert entry["prompt"] == "Test prompt logging" + assert entry["session_id"] == agent.session["id"] + assert "timestamp" in entry