fix(backtest): fall back to day freq when minute-level benchmark data is missing - #2349
Open
Pujitha Paladugu (pujitha24) wants to merge 1 commit into
Open
fix(backtest): fall back to day freq when minute-level benchmark data is missing#2349Pujitha Paladugu (pujitha24) wants to merge 1 commit into
Pujitha Paladugu (pujitha24) wants to merge 1 commit into
Conversation
… is missing
Motivation:
get_higher_eq_freq_feature() in qlib/utils/resam.py retries with
freq="1min" when the originally requested minute-level frequency
(e.g. 30min) has no data. That retry was not wrapped in a
try/except, unlike the sibling day/week/month branch a few lines
above it, which already falls back from day to 1min on failure. When
1min data also doesn't exist (true for a benchmark like CSI300 that
only has daily data), the raw ValueError from deep inside
FileCalendarStorage._freq_file propagates uncaught as:
"ValueError: can't find a freq from [Freq(30min)] that can resample
to 1min!" -- a message that is confusing because the user never
asked for 1min data. This crashes PortfolioMetrics._cal_benchmark()
during backtests that use time_per_step="30min" with a daily-only
benchmark.
Approach:
Wrap the 1min retry in the NORM_FREQ_MINUTE branch in a
try/except (ValueError, KeyError) and fall back to freq="day" on
failure, mirroring the existing day/week/month branch's fallback
pattern.
Validation:
Added tests/misc/test_utils.py::TestGetHigherEqFreqFeature::test_minute_freq_falls_back_to_day_when_no_minute_data,
which mocks qlib.data.data.D.features to raise ValueError for freq
in ("30min", "1min") and succeed for freq="day". Verified it fails
with the exact reported error before this fix and passes after.
Ran `python -m pytest tests/misc/test_utils.py -v` (3 passed). Ran
`black . -l 120 --check --diff`, the repo's configured flake8
command, and the repo's configured pylint command against
qlib/utils/resam.py -- all clean (pylint 10.00/10). Did not run a
live end-to-end backtest with real CSI300/30min data (no such
dataset available in this environment); the targeted unit test
reproduces the exact failure mode and fixes it.
Report: microsoft#1854
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
get_higher_eq_freq_feature()inqlib/utils/resam.pyretries withfreq="1min"whenthe originally requested minute-level frequency (e.g.
30min) has no data. That retrywas not wrapped in a
try/except, unlike the sibling day/week/month branch a few linesabove it, which already falls back from
dayto1minon failure. This change wrapsthe
1minretry in atry/except (ValueError, KeyError)and falls back today,mirroring the existing pattern.
Motivation and Context
Fixes
ValueError: can't find a freq from [Freq(30min)] that can resample to 1min!when backtesting with
time_per_step="30min"and a benchmark (e.g. CSI300) that onlyhas daily data available.
PortfolioMetrics._cal_benchmark()(qlib/backtest/report.py) callsget_higher_eq_freq_feature()withfreq="30min"to fetch benchmark returns. Since thebenchmark only has daily data, the initial
D.features(freq="30min")call raises, andthe function falls into the
NORM_FREQ_MINUTEbranch, which retries withfreq="1min"unguarded. When 1min data doesn't exist either (true for a daily-onlybenchmark), the raw
ValueErrorfrom deep insideFileCalendarStorage._freq_file(qlib/data/storage/file_storage.py) propagatesuncaught, producing the confusing message in the issue instead of falling back to
day, which does have data for a daily-only benchmark.Report: #1854
How Has This Been Tested?
tests/misc/test_utils.py::TestGetHigherEqFreqFeature::test_minute_freq_falls_back_to_day_when_no_minute_data.It mocks
qlib.data.data.D.featuresto raiseValueErrorforfreqin("30min", "1min")and succeed forfreq="day", then assertsget_higher_eq_freq_featurefalls back to"day".Verified the test reproduces the reported bug: with only the test added (fix
reverted), it fails with the exact error from the issue,
ValueError: can't find a freq from [] that can resample to 1min!; with the fixapplied, it passes.
python -m pytest tests/misc/test_utils.py -v(3 passed).black . -l 120 --check --diff,flake8 --ignore=E501,F541,E266,E402,W503,E731,E203 --per-file-ignores="__init__.py:F401,F403" qlib/utils/resam.py, andpylint(repo'sconfigured disable list) on
qlib/utils/resam.py— all clean (pylint 10.00/10).available in this environment); the targeted unit test reproduces the exact failure
mode and fixes it, which I believe is sufficient to demonstrate correctness for this
specific defect (a missing
try/exceptaround one fallback call).Test qlib from sourcerun againstmainfailed in 5s with"This run likely failed because of a workflow file issue" (unrelated to source code);
the last full run on
maincompleted successfully.Screenshots of Test Results (if appropriate):
python -m pytest tests/misc/test_utils.py -v→ 3 passed.Types of changes
Fixes #1854