DOJO is an open, community-driven platform for continuous adversarial evaluation of clinical AI, led by MIT Critical Data. It tests a health AI system as a system, across three connected planes: the data that shapes a model, the model itself, and the clinical workflow in which people rely on it. Findings stay linked to the model, data, evaluator, environment and re-test that produced them, so a claim about readiness can be traced and repeated.
Website: criticaldata.github.io/dojo
| Paper | Venue | Links |
|---|---|---|
| Distributed Open Justice Oversight (DOJO): A Community-Driven, Modality-Agnostic Platform for Adversarial Evaluation of Health AI | SSRN preprint, 2026 | paper |
| Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems | MICCAI 2026 MedAgent Workshop (LNCS) | paper (open access) · arXiv · code |
| ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs | arXiv preprint, 2026 | arXiv · code |
| Towards a Deterministic Math Solver for Clinical Language Models | NeurIPS 2026 Global South CV Workshop (accepted) | arXiv · code |
- Leo Anthony Celi, Principal Investigator (lceli@mit.edu)
- Sebastián Cajas Ordóñez (sebasmos@mit.edu)
- Yehudhah Rodriguez (yehudhah.rodriguez@yachaytech.edu.ec)
Collaboration, contributions and institutional questions: open an issue.
Repository layout: index.html, styles.css, script.js and assets/ are the site; scripts/ holds the QA checks run before publishing (links, bibliography, team entries); docs/ holds working notes. Licensed under the MIT License.
The site is static (HTML, CSS, JavaScript) and holds no clinical data, credentials, API keys or live evaluation services; never commit those here. It describes an evaluation concept and a contribution model, not a regulatory approval, a universal safety claim or a patient-outcome benefit.