Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .claude/skills/check-vacuum/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
name: check-vacuum
description: Check the Lab 5 notebook and use focused specialists to review the environment, requirements, and code features.
disable-model-invocation: true
---

# Check Vacuum

1. Work from the CS440-Lab5 repository root. Run `python scripts/check_project.py`.
2. Report errors and unfinished TODOs. A passing check does not prove that Q-learning works.
3. Invoke `environment-specialist` with `context/environment.md` and only the environment cell from `q_learning.ipynb`. Ask it to check movement, rewards, reset, and terminal handling.
4. Invoke `requirements-specialist` with `context/requirements.md` and the check summary. Ask it to identify missing Lab 5 deliverables.
5. Invoke `configuration-reviewer` with `.claude/settings.json`, this skill, and the three specialist definitions. Supply the checker output and only relevant learning-function excerpts if reviewing Q-learning.
6. Combine the specialists' short findings. Use actual subagent calls and report any invocation failure.
7. Return: checks run, hyperparameters, findings, and remaining work. Do not change files unless requested or invent training results.
18 changes: 18 additions & 0 deletions .claude/skills/compare-hyperparameters/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
name: compare-hyperparameters
description: Sweep alpha, epsilon, and gamma over several seeds and summarize the trade-offs, to support the hyperparameter explanation the lab asks for.
disable-model-invocation: true
argument-hint: "[--alpha ...] [--epsilon ...] [--gamma ...] [--seeds N]"
---

# Compare Hyperparameters

1. Work from the CS440-Lab5 repository root. Run `python scripts/sweep_hyperparameters.py $ARGUMENTS`. With no arguments it sweeps alpha and epsilon around the notebook's current values over 5 seeds. Training seeds and evaluation seeds are disjoint from the notebook's own seeds.
2. Add `--out sweep_results.json` if the user wants the table saved. Otherwise do not write files. The script never changes the notebook or `results.json`.
3. Summarize the table: success rate (mean and range across seeds), average evaluation reward and steps, and late-training traps for each setting.
4. Explain the trade-offs only as far as the numbers show them:
- alpha: how fast values move versus how noisy they are.
- epsilon: exploration of rarely visited cells versus wasted or risky steps while training.
- gamma: how strongly the +10 goal outweighs the per-step cost.
5. Say when differences fall within the seed-to-seed range and cannot be told apart. Do not call any setting "best" or "optimal" on that basis alone. The sweep covers a small grid and a small number of seeds.
6. Return: the settings tried, the table, what the evidence supports, and a suggested wording for the README's hyperparameter explanation. Do not edit the README or the notebook parameters unless asked, and do not invent results.
14 changes: 14 additions & 0 deletions .claude/skills/verify-results/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
name: verify-results
description: Sanity-check results.json for consistency and compare the learned policy with a model-based optimum, then review the claims made about it.
disable-model-invocation: true
---

# Verify Results

1. Work from the CS440-Lab5 repository root. If `results.json` is missing, say so and stop; do not create or invent results. Tell the user to run the notebook's training and recording cells.
2. Run `python scripts/verify_results.py`. It checks that outcome counts add up, that reward matches steps, and that evaluation is not better than the 6-step best case. It also compares each learned action with a value-iteration solution. That solution is used only for verification and is never given to the learner.
3. Report any `INCONSISTENT` lines first. Policy differences from optimal are findings, not failures. Note which cells differ and by how much, and that rarely visited cells are expected to be less accurate.
4. Read the Results section of `README.md` and check each number and claim against `results.json` and the script output. Flag any claim the output does not support, such as "optimal", "best", or an explanation of why a cell's action was chosen.
5. Note that one seed shows only that a configuration worked in that run, and that the policy is evaluated from the start cell only.
6. Return: the recorded timestamp, consistency result, cells that differ from optimal, README claims that need correcting, and what remains unverified. Do not edit files unless asked, and do not rerun training.
88 changes: 86 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ The model backend is currently based on Ollama.


 
## Six Core Components
## Seven Components

<a href="https://magazine.sebastianraschka.com/p/components-of-a-coding-agent">
<img alt="Six core components of a coding agent" src="https://sebastianraschka.com/images/github/mini-coding-agent/six-components.webp" width="500px">
</a>

This coding harness is organized around six practical building blocks:
This coding harness is organized around seven components:

1. **Live repo context**
The agent collects stable workspace facts upfront, such as repo layout, instructions, and git state.
Expand All @@ -47,6 +47,90 @@ This coding harness is organized around six practical building blocks:
The runtime keeps both a full durable transcript and a smaller working memory so sessions can be resumed while preserving important state via working memory.
6. **Delegation and bounded subagents**
Scoped subtasks can be delegated to helper agents that inherit enough context to help (but operate within limits).
7. **Agent skills** discover skill folders, select an applicable skill, add its instructions to the model prompt, and log the decision.

## CS440 Project 2

| Path | Purpose |
| --- | --- |
| `mini_coding_agent.py` | Agent loop, skill discovery and matching, `use_skill` tool, and logs |
| `.claude/skills/` | Original Lab 5 skills |
| `intent/` | SDLC intent, specification, and implementation plan |
| `q-learning-9b/` | Vacuum robot demo, notebook, tests, and saved results |
| `tests/` | Agent and skill regression tests |

The agent searches the target repository root for one-level skill folders under
`skills/`, `skill/`, and `.claude/skills/`. Each folder contains `SKILL.md`
or `skill.md`. The registry reads the name, description, and optional keywords.
It scores the prompt against each skill's metadata using word-count cosine
similarity and selects the best score at or above 0.25. A `/skill-name` prompt
or a model call to `use_skill` selects by exact name. The skill body is
included in subsequent model prompts for that turn. The model follows it with
the agent's existing tools and approval rules.

### Vacuum robot demonstration

The `q-learning-9b/` folder inside this repository contains a copy of CS440 Lab 5
with its original `check-vacuum` skill and simulator. The demo workspace contains
`learning_todos.py`, `q_learning.ipynb`, and `q_learning_demo.py` alongside
the simulator and tests. From the repository root, install the demo dependencies
and verify the included implementation:

```powershell
python -m pip install -r q-learning-9b/requirements.txt
Push-Location .\q-learning-9b
python q_learning_demo.py verify
Pop-Location
```

To open the included demo workspace in the agent, start Ollama per the
[requirements](#requirements) and run this from the repository root:

```powershell
python mini_coding_agent.py --cwd q-learning-9b --approval ask --max-steps 15 --max-new-tokens 2048
```

To start from TODOs, run `python q_learning_demo.py prepare` inside a fresh
disposable demo checkout that has the notebook and helper but no
`learning_todos.py`. Preparation generates the learner starter, clears
notebook outputs, and removes any existing `results.json` from that checkout.
Launch the agent from the repository root and pass that checkout's path to
`--cwd`. At the agent prompt, enter `/skills` to verify discovery. Then try:

> Check the Lab 5 notebook for the vacuum robot and use focused specialists to review the environment, requirements, and code features. To solve the Lab 4 vacuum robot problem, implement the five Q-learning TODOs in learning_todos.py. Read the environment and tests for the expected interfaces. Use only observations from the simulator. Then run python q_learning_demo.py verify and report the results. Keep the simulator and tests intact.

Review the tool arguments when the agent requests approval. Approve the
checker command, learner-file edits, and verification command if they match the
task. The verification command copies the learner into the notebook,
runs the checker and Q-learning tests, then trains and evaluates if those
checks succeed. A syntax checker pass can still contain unfinished TODOs.
The `verify` command writes fresh `results.json` only after successful training
and evaluation. To trigger the skill explicitly, prefix the prompt with
`/check-vacuum`.

To display the skill log from a run in the included demo workspace, open a
second terminal at the repository root:

```powershell
Get-Content -LiteralPath '.\q-learning-9b\.mini-coding-agent\skills.log' |
ForEach-Object { $_ | ConvertFrom-Json } |
Where-Object { $_.depth -eq 0 } |
ConvertTo-Json -Depth 6
```

Show the `prompt`, `similarity_check`, and `skill_executed` records for the
same session. The execution record means the instructions were activated.
Use the tool calls, the source diff, and the test or training output to show
what the agent completed and what remains.
Logs store prompt text locally. Delegated calls have their own session IDs and
`depth: 1` records.

To run the automated tests from the agent repository:

```powershell
python -m pip install -e . pytest
python -m pytest -q
```

&nbsp;
## Requirements
Expand Down
12 changes: 12 additions & 0 deletions intent/intent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Coding Agent Skills

## Repository
This project extends the upstream lightweight coding agent with a seventh component for skills.

## Objective
The code agent must be able to identify a folder skill (such as .claude\skills\pr-review) and trigger a skill.md file by executing the instructions when an appropriate prompt is triggered or when the code agent needs a specific agent skill during execution.
Each skill has a description which the agent will use to identify the right skill to use.
The new component must log the prompt, the similarity check, and the executed skill.

Demonstrate the component by attempting the CS440 Lab 4 vacuum robot problem
with the original Lab 5 `check-vacuum` skill and a Q-learning starter.
Loading