Skip to content
coldfinityPublic

About

Statistical arbitrage research and automated trading system - from cointegration screening through live execution.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

dynabeta

Statistical arbitrage research and automated trading system - from cointegration screening through live execution.

Overview

dynabeta is a pipeline for mean-reversion pairs trading built around cointegration, Kalman-filter dynamic hedge ratios, and Ornstein-Uhlenbeck spread modelling. The system spans the full lifecycle: data ingestion, pair selection, signal generation, backtesting, and eventual automated live trading via broker APIs.

Project Status

Current phase: Research complete; live infrastructure built and paper-ready

Act I - Classical stat arb (static β)

Module Status Description
Data ingestion ✅ Yahoo Finance → SQLite, train/test split, validation
I(1) / ADF checks ✅ Per-ticker stationarity classification
Engle-Granger screening ✅ Pairwise cointegration tests, ranked by p-value
Johansen confirmation ✅ Direction-independent trace test; eigenvector gives cointegrating vector
OU parameter estimation ✅ κ, μ, σ via discrete OLS; half-life filter (5–60 days)
Signal generation ✅ Rolling z-score (2x half-life window); entry ±2, exit 0.5, stop 3 (config-driven)
Backtesting engine ✅ 5bps per-side costs; 1-bar signal lag; in-sample and OOS metrics
Walk-forward backtest ✅ Expanding-window Q/R re-fit every ~63 days; no global in-sample fit

Act II - Kalman extension (dynamic β)

Module Status Description
Kalman dynamic hedge ratio ✅ State-space β̂_t with EM-estimated Q and R noise hyperparameters
Comparative analysis ✅ Act I vs Act II on same OOS period; combined Sharpe -0.96 → +2.74

System build

Module Status Description
C++ compute core ✅ pybind11/Eigen Kalman + backtest inner loop; exact parity, ~12x faster backtest
Broker abstraction ✅ BrokerAdapter ABC; DryRunAdapter (default), AlpacaAdapter (paper), WebullAdapter (skeleton)
Daily pipeline ✅ Netted dollar-neutral sizing, risk layer, reconciliation, Discord alerts, cron
Risk layer ✅ HALT kill switch, stale-data/daily-loss/drawdown halts, per-order size cap
Test suite + CI ✅ 15 pytest tests (parity, causality, netting, costs); GitHub Actions
Paper trading campaign 🔲 6–8 week unattended run (see EXTERNAL_CHECKLIST.md)
Live capital 🔲 Requires Webull verification + go/no-go (see EXTERNAL_CHECKLIST.md)

See dynabeta-roadmap.md for the full phased plan and EXTERNAL_CHECKLIST.md for remaining manual/external steps.

Architecture

dynabeta/
├── src/
│   ├── config.py           # StrategyConfig (thresholds) + tradeable PAIRS
│   ├── data_loader.py      # Ingestion, splitting, validation, plotting
│   ├── cointegration.py    # ADF I(1) checks, Engle-Granger + Johansen tests
│   ├── ou.py               # OU parameter estimation, half-life filter
│   ├── signal.py           # Z-score construction, entry/exit/stop logic
│   ├── backtest.py         # Single-pair backtest engine, cost model, metrics
│   ├── kalman.py           # Dynamic hedge ratio via Kalman filter, Act I vs II
│   ├── walkforward.py      # Walk-forward Kalman (periodic Q/R re-fit)
│   ├── plots.py            # Beta drift visualisations
│   ├── broker.py           # BrokerAdapter ABC + DryRunAdapter
│   ├── alpaca.py           # AlpacaAdapter (paper trading)
│   ├── webull.py           # WebullAdapter skeleton (live, not yet wired)
│   ├── risk.py             # Risk layer: HALT, halts, size caps
│   ├── reconcile.py        # Broker vs internal-state reconciliation
│   ├── alert.py            # Discord webhook notifications
│   ├── state.py            # SQLite position + trade log (data/state.db)
│   ├── logging_config.py   # Console + structured JSON logging
│   └── pipeline.py         # Daily execution: signals → net → risk → submit
├── cpp/                     # C++ compute core (pybind11 + Eigen)
│   ├── kalman.cpp/.h        # Kalman filter
│   ├── backtest.cpp/.h      # Backtest inner loop
│   ├── bindings.cpp         # pybind11 module
│   └── CMakeLists.txt       # Build (-O3 -march=native, Eigen via FetchContent)
├── scripts/run_pipeline.sh  # Cron wrapper (4:15 PM ET, Mon-Fri)
├── .github/workflows/ci.yml # Build extension + run pytest on push/PR
├── data/prices.db           # Reference prices (tables: train, test)
├── test.py                  # pytest suite (15 tests)
├── pyproject.toml           # scikit-build-core build backend
├── EXTERNAL_CHECKLIST.md    # Remaining manual/external steps
├── dynabeta-roadmap.md      # Full 7-phase system plan
└── stat_arb_plan.html       # Research phase detail (Act I & II)

Quick Start

# Clone and set up
git clone <repo-url> && cd dynabeta
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Build the C++ compute core (optional; tests skip it if absent)
pip install . --no-build-isolation
cp build*/dynabeta_cpp*.so .   # or build via cmake; see cpp/CMakeLists.txt

# Download data and persist train/test splits
python -m src.data_loader

# Research pipeline
python -m src.cointegration      # I(1) + Engle-Granger + Johansen screening
python -m src.ou                 # OU half-life filter
python -m src.kalman             # Act I vs Act II comparison
python -m src.walkforward        # walk-forward Kalman backtest

# Live pipeline (dry-run by default) and tests
python -m src.pipeline
python test.py                   # or: pytest test.py

# Dashboard (control room): API on :8000, UI on :5173
uvicorn src.api:app --port 8000          # backend
cd frontend && npm install && npm run dev # frontend -> http://localhost:5173

Copy .env.example conventions: set ALPACA_API_KEY, ALPACA_SECRET_KEY, and DISCORD_WEBHOOK_URL in a local .env (gitignored).

Current Capabilities

Data Layer (src/data_loader.py)

  • Downloads adjusted close prices for 15 US tech sector tickers from Yahoo Finance:
    • Semiconductors: NVDA, AMD, INTC, AVGO, QCOM, MU
    • Big Tech / Cloud: MSFT, GOOGL, META, AMZN, AAPL
    • Enterprise SaaS: CRM, NOW, ADBE, ORCL
  • Time range: 2015-01-01 to today (dynamic), with a train/test split at 2023-12-31 (in-sample: 2015–2023; OOS: 2024–present)
  • Persists to data/prices.db (SQLite, tables: train and test)
  • Validates shape, date range, NaN counts, and trading-day gap distribution

Cointegration Screening (src/cointegration.py)

For each ticker, runs Augmented Dickey-Fuller (ADF) tests on both levels and first differences. A series is classified as I(1) when it has a unit root in levels (p > 0.05) but is stationary after differencing (p < 0.05).

For all unordered pairs, estimates OLS in both directions (AB and BA) and runs Engle-Granger cointegration tests on the residuals. Results are sorted by ascending p-value to surface the strongest candidate pairs.

Johansen trace test confirms EG pairs direction-independently; the leading eigenvector gives the cointegrating vector. 7 pairs survived both tests across 4 tradeable after OU half-life filtering (34–38 days).

Act I result: In-sample combined Sharpe 0.74 (2015–2023). OOS Sharpe -0.96 (2024–present) - static beta drifts, motivating the Kalman filter in Act II.

Kalman Filter (src/kalman.py)

Models the hedge ratio as a latent state that evolves bar by bar. Q (beta drift noise) and R (observation noise) are estimated via EM on training data. At inference time the filter runs online with no look-ahead - each beta estimate uses only prices up to that bar.

Act II result: OOS combined Sharpe improves from -0.96 (static beta) to +2.74 (Kalman dynamic beta). Per-pair breakdown (2024-present):

Pair Act I Sharpe Act II Sharpe
CRM-ADBE -0.16 +1.88
NOW-AMD -1.07 +1.44
AMD-MSFT -0.84 +1.68
AMZN-CRM -0.26 +1.66
Combined -0.96 +2.74

Walk-forward (src/walkforward.py)

A stricter, fully out-of-sample test: Q/R are re-estimated on an expanding window every ~63 trading days and the scalar Kalman filter runs forward day by day, so no beta ever uses future data. Over the full 2015–present period, combined Sharpe is +3.27 (per-pair 1.75–2.13).

C++ compute core (cpp/)

The Kalman filter and backtest inner loop are ported to C++ (pybind11 + Eigen), validated against the Python reference in CI: Kalman parity to ~1e-16, backtest parity exact (0.0). The C++ Kalman is ~3000x faster than pykalman's EM-fit filter call; the backtest loop is ~12x faster than the pandas version.

Dependencies

Current research stack (all pinned in requirements.txt):

  • pandas, numpy - data handling
  • yfinance - market data ingestion
  • statsmodels - ADF, OLS, Engle-Granger, Johansen
  • scipy - OU MLE fitting (scipy.optimize.minimize on OU log-likelihood)
  • pykalman - Kalman filter reference implementation (EM noise estimation)
  • matplotlib - visualisation
  • sqlite3 - database I/O (stdlib, no extra install)
  • pybind11 + Eigen + scikit-build-core - C++ compute core
  • alpaca-py - paper/live execution; requests - Discord alerts
  • python-dotenv - credential loading; python-json-logger - structured logs
  • pytest - test suite

Planned additions:

  • Webull Python SDK (webull-python-sdk-core) - primary live execution (Phase 6)

Roadmap

Phase 1 - Research Core (current, detail in stat_arb_plan.html):

  • Act I (classical stat arb): Johansen confirmation → OU fitting (κ, μ, σ; half-life filter 5–60 days) → config-driven z-score signals → backtest (5bps per side, dollar-neutral) with a walk-forward variant
  • Act II (Kalman extension): Dynamic β̂_t via state-space model with EM-estimated Q/R → comparative analysis vs static β, regime study over the 2022 rate cycle

Full system phases (detail in dynabeta-roadmap.md):

  1. Admin - Webull API access, paper environment, capital rules
  2. C++ Compute Core - pybind11/Eigen port of Kalman + backtest inner loop; benchmark table
  3. Trading Infrastructure - BrokerAdapter interface, WebullAdapter (live), AlpacaAdapter (paper), DryRunAdapter (default); daily cron pipeline; SQLite state + reconciliation
  4. Paper Trading Campaign - 6–8 weeks unattended; ops journal; go/no-go criteria
  5. Live Capital - 2–3 pairs at minimal size; second sleeve (long-only ETF trend)
  6. Portfolio Packaging - public repo, writeup, benchmark numbers, resume deliverables

Standing Risk Rules

Applied from live-phase onward:

  1. Dry-run is the default; live submission requires an explicit flag
  2. The risk layer can only reduce or block orders - never enlarge
  3. Reconciliation mismatches or data-quality failures halt trading
  4. Daily loss limit and max drawdown halt enforced in code
  5. No strategy change goes live without passing backtest + parity suite in CI
  6. Real capital is an amount whose total loss changes nothing about your life

License

Proprietary - not yet licensed for redistribution.

About

Statistical arbitrage research and automated trading system - from cointegration screening through live execution.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages