LLM Alpha Mining

English | 简体中文

A quantitative research framework that turns LLM proposals into constrained factor definitions, point-in-time signals, and reproducible research artifacts.

The central question is how to make automated factor discovery inspectable: which hypothesis was proposed, what data could it see, why was it accepted, and can its signal be reproduced? Language models propose and review candidates; deterministic code controls expression validity, risk vetoes, data alignment, evaluation, and provenance.

This repository contains the reusable framework developed during a quantitative research project. It includes an offline example, 251 automated tests, and English and Chinese documentation. Research data and individual factor submissions are kept outside the repository. This project uses language models for research assistance; it does not train a foundation model.

Start here

Use Python 3.12 or 3.13. Installation downloads dependencies. The example runs offline with no API key.

git clone https://github.com/ZebinZ/llm-alpha-mining.git
cd llm-alpha-mining
python -m venv .venv
source .venv/bin/activate
python -m pip install '.[test]'
alpha-demo --output outputs/demo
alpha-demo --output outputs/demo --verify
python -m pytest -q

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1. Each demo run needs a new output directory; --verify checks an existing run.

The example generates 100 business days for 40 fictional stocks plus a fictional index. Scripted role responses propose three formulas; one receives a risk veto and two reach the real factor engine. It exercises missing observations, rolling-window warm-up, a changing trading universe, and exclusion of non-stock instruments.

Expected outcome: two accepted factors, zero network calls, and PASS from artifact verification. The output includes signal Parquet files, descriptive diagnostics, the call ledger, report.json, and a SHA-256 manifest.json. These are software checks on synthetic data, not evidence of investment performance.

What the framework does

flowchart LR
    A[Research hypotheses] --> B[Structured LLM proposals]
    B --> C[DSL validation and role review]
    C --> D[Point-in-time factor computation]
    D --> E[Temporal validation and cost evaluation]
    E --> F[Selection and reproducible artifacts]

Loading

Capability	Implementation	Evidence to inspect
Structured proposal, critic, risk, and arbiter roles	Mining protocols and LLM calls	Structured LLM tests
Formula allowlists and restrictions on future information	DSL interpreter	DSL tests
Separation of instrument identity, available history, and signal-time universe	Factor engine	Factor contract tests
Labels, temporal splits, and factor evaluation	Research evaluation	Label and validation tests, evaluation tests
Portfolio constraints, turnover, costs, and execution timing	Backtest engine	Backtest tests
Frozen candidate families and multiple-testing corrections	Significance evaluation	Significance tests
Budgets, restricted feedback, persistent state, and checkpoint recovery	Campaign orchestration	Campaign tests, recovery tests

The example covers proposal review, signal computation, descriptive diagnostics, and artifact verification. Formal backtesting and campaign orchestration are tested separately. Connecting a real model and data source requires an integration runner; the example does not start a live mining campaign.

Repository layout

src/llm_alpha_mining/
  mining/       # Candidate protocols, LLM roles, DSL, state, and campaigns
  research/     # Data contracts, factors, evaluation, portfolios, and backtests
  demo/         # Reproducible example with synthetic data and scripted responses
tests/          # Tests named by behavior and research component
docs/           # Paired English and Chinese guides

The current tree contains one maintained implementation. Historical research runners, vendor-specific panel repair tools, unused model-training branches, caches, and submission archives are excluded. Serialized schema and operator version identifiers remain explicit so old record identities retain their meaning.

Read further

• Architecture and research decisions: responsibilities, point-in-time semantics, missing data, and recovery.
• Reproducibility and integration: running the example, using a model provider, bringing your own data, and development checks.
• Research scope and evidence: what this project demonstrates, what the private research established, and what remains unverified.

The source is publicly readable. An open-source license has not yet been specified; see publication scope.