Backend for Calibrate, a framework for evaluating AI agents which let you move from slow, manual testing to a fast, automated, and repeatable testing process for your entire agent stack.
Install dependencies using uv:
uv sync --frozenFor local development with tests:
uv sync --frozen --group devCreate src/.env.
cp src/.env.example src/.env
See ENV.md for what each variable means.
Start the development server:
cd src
uv run uvicorn main:app --reloadThe app will be available at: http://localhost:8000
API documentation: http://localhost:8000/docs
Related repositories:
Reference docs:
After cloning the repo, enable the project's git hooks so the pre-commit test runner fires on commits to main:
git config core.hooksPath .githooksEvery contributor needs to run it once.
For local development, TablePlus is a handy GUI for opening and inspecting the SQLite database.
From the repository root (not src/):
pytest testsThis work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
