Skip to content

fix(backtest): fall back to day freq when minute-level benchmark data is missing - #2349

Open
Pujitha Paladugu (pujitha24) wants to merge 1 commit into
microsoft:mainfrom
pujitha24:auto/issue-1854
Open

fix(backtest): fall back to day freq when minute-level benchmark data is missing#2349
Pujitha Paladugu (pujitha24) wants to merge 1 commit into
microsoft:mainfrom
pujitha24:auto/issue-1854

Conversation

@pujitha24

Copy link
Copy Markdown

Description

get_higher_eq_freq_feature() in qlib/utils/resam.py retries with freq="1min" when
the originally requested minute-level frequency (e.g. 30min) has no data. That retry
was not wrapped in a try/except, unlike the sibling day/week/month branch a few lines
above it, which already falls back from day to 1min on failure. This change wraps
the 1min retry in a try/except (ValueError, KeyError) and falls back to day,
mirroring the existing pattern.

Motivation and Context

Fixes ValueError: can't find a freq from [Freq(30min)] that can resample to 1min!
when backtesting with time_per_step="30min" and a benchmark (e.g. CSI300) that only
has daily data available.

PortfolioMetrics._cal_benchmark() (qlib/backtest/report.py) calls
get_higher_eq_freq_feature() with freq="30min" to fetch benchmark returns. Since the
benchmark only has daily data, the initial D.features(freq="30min") call raises, and
the function falls into the NORM_FREQ_MINUTE branch, which retries with
freq="1min" unguarded. When 1min data doesn't exist either (true for a daily-only
benchmark), the raw ValueError from deep inside
FileCalendarStorage._freq_file (qlib/data/storage/file_storage.py) propagates
uncaught, producing the confusing message in the issue instead of falling back to
day, which does have data for a daily-only benchmark.

Report: #1854

How Has This Been Tested?

  • Added a targeted regression test:
    tests/misc/test_utils.py::TestGetHigherEqFreqFeature::test_minute_freq_falls_back_to_day_when_no_minute_data.
    It mocks qlib.data.data.D.features to raise ValueError for freq in
    ("30min", "1min") and succeed for freq="day", then asserts
    get_higher_eq_freq_feature falls back to "day".
    Verified the test reproduces the reported bug: with only the test added (fix
    reverted), it fails with the exact error from the issue,
    ValueError: can't find a freq from [] that can resample to 1min!; with the fix
    applied, it passes.
  • Ran python -m pytest tests/misc/test_utils.py -v (3 passed).
  • Ran black . -l 120 --check --diff, flake8 --ignore=E501,F541,E266,E402,W503,E731,E203 --per-file-ignores="__init__.py:F401,F403" qlib/utils/resam.py, and pylint (repo's
    configured disable list) on qlib/utils/resam.py — all clean (pylint 10.00/10).
  • Did not run a live end-to-end backtest with real CSI300/30min data (no such dataset
    available in this environment); the targeted unit test reproduces the exact failure
    mode and fixes it, which I believe is sufficient to demonstrate correctness for this
    specific defect (a missing try/except around one fallback call).
  • Note: the most recent Test qlib from source run against main failed in 5s with
    "This run likely failed because of a workflow file issue" (unrelated to source code);
    the last full run on main completed successfully.

Screenshots of Test Results (if appropriate):

  1. Pipeline test: not run (no local dataset available).
  2. Your own tests: python -m pytest tests/misc/test_utils.py -v → 3 passed.

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

Fixes #1854

… is missing

Motivation:
get_higher_eq_freq_feature() in qlib/utils/resam.py retries with
freq="1min" when the originally requested minute-level frequency
(e.g. 30min) has no data. That retry was not wrapped in a
try/except, unlike the sibling day/week/month branch a few lines
above it, which already falls back from day to 1min on failure. When
1min data also doesn't exist (true for a benchmark like CSI300 that
only has daily data), the raw ValueError from deep inside
FileCalendarStorage._freq_file propagates uncaught as:
"ValueError: can't find a freq from [Freq(30min)] that can resample
to 1min!" -- a message that is confusing because the user never
asked for 1min data. This crashes PortfolioMetrics._cal_benchmark()
during backtests that use time_per_step="30min" with a daily-only
benchmark.

Approach:
Wrap the 1min retry in the NORM_FREQ_MINUTE branch in a
try/except (ValueError, KeyError) and fall back to freq="day" on
failure, mirroring the existing day/week/month branch's fallback
pattern.

Validation:
Added tests/misc/test_utils.py::TestGetHigherEqFreqFeature::test_minute_freq_falls_back_to_day_when_no_minute_data,
which mocks qlib.data.data.D.features to raise ValueError for freq
in ("30min", "1min") and succeed for freq="day". Verified it fails
with the exact reported error before this fix and passes after.
Ran `python -m pytest tests/misc/test_utils.py -v` (3 passed). Ran
`black . -l 120 --check --diff`, the repo's configured flake8
command, and the repo's configured pylint command against
qlib/utils/resam.py -- all clean (pylint 10.00/10). Did not run a
live end-to-end backtest with real CSI300/30min data (no such
dataset available in this environment); the targeted unit test
reproduces the exact failure mode and fixes it.

Report: microsoft#1854
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An error when I do 30min trade.

1 participant