Statistical arbitrage research and automated trading system - from cointegration screening through live execution.
dynabeta is a pipeline for mean-reversion pairs trading built around cointegration, Kalman-filter dynamic hedge ratios, and Ornstein-Uhlenbeck spread modelling. The system spans the full lifecycle: data ingestion, pair selection, signal generation, backtesting, and eventual automated live trading via broker APIs.
Current phase: Research complete; live infrastructure built and paper-ready
Act I - Classical stat arb (static β)
| Module | Status | Description |
|---|---|---|
| Data ingestion | ✅ | Yahoo Finance → SQLite, train/test split, validation |
| I(1) / ADF checks | ✅ | Per-ticker stationarity classification |
| Engle-Granger screening | ✅ | Pairwise cointegration tests, ranked by p-value |
| Johansen confirmation | ✅ | Direction-independent trace test; eigenvector gives cointegrating vector |
| OU parameter estimation | ✅ | κ, μ, σ via discrete OLS; half-life filter (5–60 days) |
| Signal generation | ✅ | Rolling z-score (2x half-life window); entry ±2, exit 0.5, stop 3 (config-driven) |
| Backtesting engine | ✅ | 5bps per-side costs; 1-bar signal lag; in-sample and OOS metrics |
| Walk-forward backtest | ✅ | Expanding-window Q/R re-fit every ~63 days; no global in-sample fit |
Act II - Kalman extension (dynamic β)
| Module | Status | Description |
|---|---|---|
| Kalman dynamic hedge ratio | ✅ | State-space β̂_t with EM-estimated Q and R noise hyperparameters |
| Comparative analysis | ✅ | Act I vs Act II on same OOS period; combined Sharpe -0.96 → +2.74 |
System build
| Module | Status | Description |
|---|---|---|
| C++ compute core | ✅ | pybind11/Eigen Kalman + backtest inner loop; exact parity, ~12x faster backtest |
| Broker abstraction | ✅ | BrokerAdapter ABC; DryRunAdapter (default), AlpacaAdapter (paper), WebullAdapter (skeleton) |
| Daily pipeline | ✅ | Netted dollar-neutral sizing, risk layer, reconciliation, Discord alerts, cron |
| Risk layer | ✅ | HALT kill switch, stale-data/daily-loss/drawdown halts, per-order size cap |
| Test suite + CI | ✅ | 15 pytest tests (parity, causality, netting, costs); GitHub Actions |
| Paper trading campaign | 🔲 | 6–8 week unattended run (see EXTERNAL_CHECKLIST.md) |
| Live capital | 🔲 | Requires Webull verification + go/no-go (see EXTERNAL_CHECKLIST.md) |
See dynabeta-roadmap.md for the full phased plan and EXTERNAL_CHECKLIST.md for remaining manual/external steps.
dynabeta/
├── src/
│ ├── config.py # StrategyConfig (thresholds) + tradeable PAIRS
│ ├── data_loader.py # Ingestion, splitting, validation, plotting
│ ├── cointegration.py # ADF I(1) checks, Engle-Granger + Johansen tests
│ ├── ou.py # OU parameter estimation, half-life filter
│ ├── signal.py # Z-score construction, entry/exit/stop logic
│ ├── backtest.py # Single-pair backtest engine, cost model, metrics
│ ├── kalman.py # Dynamic hedge ratio via Kalman filter, Act I vs II
│ ├── walkforward.py # Walk-forward Kalman (periodic Q/R re-fit)
│ ├── plots.py # Beta drift visualisations
│ ├── broker.py # BrokerAdapter ABC + DryRunAdapter
│ ├── alpaca.py # AlpacaAdapter (paper trading)
│ ├── webull.py # WebullAdapter skeleton (live, not yet wired)
│ ├── risk.py # Risk layer: HALT, halts, size caps
│ ├── reconcile.py # Broker vs internal-state reconciliation
│ ├── alert.py # Discord webhook notifications
│ ├── state.py # SQLite position + trade log (data/state.db)
│ ├── logging_config.py # Console + structured JSON logging
│ └── pipeline.py # Daily execution: signals → net → risk → submit
├── cpp/ # C++ compute core (pybind11 + Eigen)
│ ├── kalman.cpp/.h # Kalman filter
│ ├── backtest.cpp/.h # Backtest inner loop
│ ├── bindings.cpp # pybind11 module
│ └── CMakeLists.txt # Build (-O3 -march=native, Eigen via FetchContent)
├── scripts/run_pipeline.sh # Cron wrapper (4:15 PM ET, Mon-Fri)
├── .github/workflows/ci.yml # Build extension + run pytest on push/PR
├── data/prices.db # Reference prices (tables: train, test)
├── test.py # pytest suite (15 tests)
├── pyproject.toml # scikit-build-core build backend
├── EXTERNAL_CHECKLIST.md # Remaining manual/external steps
├── dynabeta-roadmap.md # Full 7-phase system plan
└── stat_arb_plan.html # Research phase detail (Act I & II)
# Clone and set up
git clone <repo-url> && cd dynabeta
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Build the C++ compute core (optional; tests skip it if absent)
pip install . --no-build-isolation
cp build*/dynabeta_cpp*.so . # or build via cmake; see cpp/CMakeLists.txt
# Download data and persist train/test splits
python -m src.data_loader
# Research pipeline
python -m src.cointegration # I(1) + Engle-Granger + Johansen screening
python -m src.ou # OU half-life filter
python -m src.kalman # Act I vs Act II comparison
python -m src.walkforward # walk-forward Kalman backtest
# Live pipeline (dry-run by default) and tests
python -m src.pipeline
python test.py # or: pytest test.py
# Dashboard (control room): API on :8000, UI on :5173
uvicorn src.api:app --port 8000 # backend
cd frontend && npm install && npm run dev # frontend -> http://localhost:5173Copy .env.example conventions: set ALPACA_API_KEY, ALPACA_SECRET_KEY,
and DISCORD_WEBHOOK_URL in a local .env (gitignored).
- Downloads adjusted close prices for 15 US tech sector tickers from Yahoo Finance:
- Semiconductors: NVDA, AMD, INTC, AVGO, QCOM, MU
- Big Tech / Cloud: MSFT, GOOGL, META, AMZN, AAPL
- Enterprise SaaS: CRM, NOW, ADBE, ORCL
- Time range: 2015-01-01 to today (dynamic), with a train/test split at 2023-12-31 (in-sample: 2015–2023; OOS: 2024–present)
- Persists to
data/prices.db(SQLite, tables:trainandtest) - Validates shape, date range, NaN counts, and trading-day gap distribution
For each ticker, runs Augmented Dickey-Fuller (ADF) tests on both levels and first differences. A series is classified as I(1) when it has a unit root in levels (p > 0.05) but is stationary after differencing (p < 0.05).
For all unordered pairs, estimates OLS in both directions (AB and BA) and runs Engle-Granger cointegration tests on the residuals. Results are sorted by ascending p-value to surface the strongest candidate pairs.
Johansen trace test confirms EG pairs direction-independently; the leading eigenvector gives the cointegrating vector. 7 pairs survived both tests across 4 tradeable after OU half-life filtering (34–38 days).
Act I result: In-sample combined Sharpe 0.74 (2015–2023). OOS Sharpe -0.96 (2024–present) - static beta drifts, motivating the Kalman filter in Act II.
Models the hedge ratio as a latent state that evolves bar by bar. Q (beta drift noise) and R (observation noise) are estimated via EM on training data. At inference time the filter runs online with no look-ahead - each beta estimate uses only prices up to that bar.
Act II result: OOS combined Sharpe improves from -0.96 (static beta) to +2.74 (Kalman dynamic beta). Per-pair breakdown (2024-present):
| Pair | Act I Sharpe | Act II Sharpe |
|---|---|---|
| CRM-ADBE | -0.16 | +1.88 |
| NOW-AMD | -1.07 | +1.44 |
| AMD-MSFT | -0.84 | +1.68 |
| AMZN-CRM | -0.26 | +1.66 |
| Combined | -0.96 | +2.74 |
A stricter, fully out-of-sample test: Q/R are re-estimated on an expanding window every ~63 trading days and the scalar Kalman filter runs forward day by day, so no beta ever uses future data. Over the full 2015–present period, combined Sharpe is +3.27 (per-pair 1.75–2.13).
The Kalman filter and backtest inner loop are ported to C++ (pybind11 + Eigen), validated against the Python reference in CI: Kalman parity to ~1e-16, backtest parity exact (0.0). The C++ Kalman is ~3000x faster than pykalman's EM-fit filter call; the backtest loop is ~12x faster than the pandas version.
Current research stack (all pinned in requirements.txt):
pandas,numpy- data handlingyfinance- market data ingestionstatsmodels- ADF, OLS, Engle-Granger, Johansenscipy- OU MLE fitting (scipy.optimize.minimizeon OU log-likelihood)pykalman- Kalman filter reference implementation (EM noise estimation)matplotlib- visualisationsqlite3- database I/O (stdlib, no extra install)pybind11+Eigen+scikit-build-core- C++ compute corealpaca-py- paper/live execution;requests- Discord alertspython-dotenv- credential loading;python-json-logger- structured logspytest- test suite
Planned additions:
- Webull Python SDK (
webull-python-sdk-core) - primary live execution (Phase 6)
Phase 1 - Research Core (current, detail in stat_arb_plan.html):
- Act I (classical stat arb): Johansen confirmation → OU fitting (κ, μ, σ; half-life filter 5–60 days) → config-driven z-score signals → backtest (5bps per side, dollar-neutral) with a walk-forward variant
- Act II (Kalman extension): Dynamic β̂_t via state-space model with EM-estimated Q/R → comparative analysis vs static β, regime study over the 2022 rate cycle
Full system phases (detail in dynabeta-roadmap.md):
- Admin - Webull API access, paper environment, capital rules
- C++ Compute Core - pybind11/Eigen port of Kalman + backtest inner loop; benchmark table
- Trading Infrastructure -
BrokerAdapterinterface,WebullAdapter(live),AlpacaAdapter(paper),DryRunAdapter(default); daily cron pipeline; SQLite state + reconciliation - Paper Trading Campaign - 6–8 weeks unattended; ops journal; go/no-go criteria
- Live Capital - 2–3 pairs at minimal size; second sleeve (long-only ETF trend)
- Portfolio Packaging - public repo, writeup, benchmark numbers, resume deliverables
Applied from live-phase onward:
- Dry-run is the default; live submission requires an explicit flag
- The risk layer can only reduce or block orders - never enlarge
- Reconciliation mismatches or data-quality failures halt trading
- Daily loss limit and max drawdown halt enforced in code
- No strategy change goes live without passing backtest + parity suite in CI
- Real capital is an amount whose total loss changes nothing about your life
Proprietary - not yet licensed for redistribution.