Pin optuna to latest version 5.0.0 by pyup-bot · Pull Request #138 · sugatoray/stackoverflow

pyup-bot · 2026-09-07T09:24:10Z

This PR pins optuna to the latest release 5.0.0.

Changelog

5.0.0rc1

Highlights

Massive default sampler algorithm enhancement

&lt;img width=&quot;1198&quot; height=&quot;558&quot; alt=&quot;Screenshot 2026-08-03 14 18 27&quot; src=&quot;https://github.com/user-attachments/assets/e563b7e2-5f01-4d1d-9332-b46e34ccf25e&quot; /&gt;

[Multivariate TPE](https://tech.preferred.jp/en/blog/multivariate-tpe-makes-optuna-even-more-powerful/) with a constant liar strategy and an enhanced bandwidth computation ([Watanabe 2023](https://arxiv.org/abs/2304.11127)) has become the default algorithm for single-objective optimization. Multi-Objective TPE has been adopted as the new default sampler for multi-objective optimization, replacing NSGA-II. We conducted comprehensive benchmarking, carefully selected default options, and modified implementation details to maximize optimization performance.

Conditional PED-ANOVA (KDD 2026)

Our paper, [Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical &amp; Dynamic Search Spaces](https://arxiv.org/abs/2601.20800), has been accepted to the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026)!

Optuna v5.0 now supports this feature and adopts it as the default algorithm for hyperparameter importance.

python
import optuna

Starting in Optuna v5.0, the default hyperparameter importance algorithm
supports conditional search spaces like the one below:
def objective(trial):
 classifier_name = trial.suggest_categorical(&quot;classifier&quot;, [&quot;SVC&quot;, &quot;RandomForest&quot;])
 if classifier_name == &quot;SVC&quot;:
     svc_c = trial.suggest_float(&quot;svc_c&quot;, 1e-10, 1e10, log=True)
     classifier_obj = sklearn.svm.SVC(C=svc_c, gamma=&quot;auto&quot;)
 else:
     rf_max_depth = trial.suggest_int(&quot;rf_max_depth&quot;, 2, 32, log=True)
     classifier_obj = sklearn.ensemble.RandomForestClassifier(
         max_depth=rf_max_depth, n_estimators=10
     )
 return …

study = optuna.create_study()
study.optimize(objective)

optuna.importance.get_param_importances(study)


New APIs for Constrained Optimization

Starting in Optuna v5.0, the interface for constrained optimization has changed:

- `trial.set_constraint()` is added to set constraint values.
- `trial.constraints` is added to get constraints.

Along with these new apis, `constraints_func` argument on samplers is now deprecated.

python
import optuna

def objective(trial):
 trial.set_constraint(“c0”, c0)
 trial.set_constraint(“c1”, c1)
 …

The constraints_func argument is now deprecated.
sampler = optuna.samplers.TPESampler()
study = optuna.create_study(sampler=samplera)
study.optimize(objective)
print(study.best_trial.constraints)


Breaking Changes

- Remove `optuna.multi_objective` module (6686)
- Remove deprecated integration wrappers for AllenNLP, Chainer, and MXNet (6693)
- Stabilize GPSampler (6715)
- Enable `constant_liar` by default (6738)
- Support categorical parameters in QMCSampler (6742, thanks saivedant169!)
- Enable multivariate by default in TPESampler (6746)
- Fix: remove deprecated positional-arg compatibility from study and trial APIs (6747, thanks yen-0!)
- Make `PedAnovaImportanceEvaluator` the default importance evaluator (6748)
- Deprecate `constraints_func` (6773)
- Change to treat cases without constraints as feasible (6774)
- Normalize trial timestamps to UTC in RDBStorage and JournalStorage (6776)
- Remove `axis_order` argument from `plot_pareto_front` (6781)
- Remove `system_attrs` from `StudySummary` (6782)
- Remove `categorical_distance_func` from `TPESampler` (6783)

New Features

- Implement qLogEI acquisition function (6640)
- Add support for conditional search spaces in PED-ANOVA (6682)
- Add conditional gp regressor (6721)
- Add mutation to ga (6724, thanks hrntsm!)
- Use MOTPE-like split in PED-ANOVA for multi-objective studies when `target` is `None` (6728)
- Add `constraints` property to `Trial` (6736)
- Add `set_constraint` method to `Trial` (6754)

Enhancements

- Fix QMCSampler fallback to independent sampling in distributed setups (6638, thanks Rishabh-git10!)
- Speed up `BruteForceSampler` by avoiding full tree build based on tree size check (6646)
- Speed up `BruteForceSampler` by candidates caching (6650)
- Split `BruteForceSampler` refactoring [3/3] (6657)
- Refactor PED-ANOVA (6681)
- Add lazy tree node to `BruteForceSampler` for speedup (6705)
- Raise `ValueError` in `PedAnovaImportanceEvaluator` for multi-objective studies without `target` (6716)
- Fix for the GPSampler OMP issue (6753)
- Make `TPESampler` the default sampler for multi-objective optimization (6766)

Bug Fixes

- Handle insufficient trials without raising `ValueError` in importances (6720)
- Fix a concurrency issue when setting study attributes (6751)

Documentation

- Add ablation study tutorial using `BruteForceSampler` (6652)
- Simplify the document for deprecated CmaEsSampler options (6694)
- Move deprecated TPESampler options to the end (6696)
- Fix typos in docstrings and duplicated author names in SPXCrossover citation (6698, thanks Divyansh-ag14!)
- Fix wording in specify params tutorial (6711, thanks Ryo2611!)
- Add a documentation about constrained `TPESampler` (6712)
- Enhance `AutoSampler` citation path (6714)
- Update sampler table (6717)
- docs: clarify `n_warmup_steps` boundary in `MedianPruner` and `PercentilePruner` (6733, thanks vin0san!)
- Fix trial report doc (6735)
- chore: consolidate instruction for external dependency (6749)
- Update sampler table for constrained optimization (6775)

Tests

- Refactoring importance tests (6725)
- Add importance evaluator test cases to optuna.testing (6765)

Code Fixes

- Refactor batched distribution classes to reduce branch duplication (6689)
- Reduce redundancy in `test_brute_force.py` (6706)
- Replace SciPy with Torch in `gp.py` (6710)
- Fix and refactor `params` validation in PED-ANOVA (6729)
- Show all study names upon load/create study failure (6750)
- Changed type of study&#x27;s direction into Literal | StudyDirection (6762, thanks yen-0!)
- Fix unused mypy ignores in `optuna/visualization/matplotlib/_rank.py` (6767)
- Avoid unnecessary trial values in storage tests (6769)
- Follow-up 6762: Fix mypy errors (6778)

Continuous Integration

- Specify workflow permissions explicitly (6687)
- Capture Windows stdout (6699)
- Fix logging tests broken by pytest 9.1.0 caplog behavior change (6719)
- Hotfix for Sphinx CI (6726)
- Fix mypy errors with latest NumPy stubs (6730)
- Skip flaky gRPC journal storage tests (6770)

Other

- Bump up to version number v5.0.0.dev (6685)
- Add `attestations: false` to fix release workflow (6690)
- Update README with Optuna 4.9.0 release news (6701)
- Add CODEOWNERS file (6718)
- Bump the version up to `5.0.0rc1` (6784)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

Alnusjaponica, Divyansh-ag14, Rishabh-git10, Ryo2611, c-bata, gen740, himkt, hrntsm, kAIto47802, nabenabe0928, not522, porink0424, saivedant169, sawa3030, vin0san, y0z, yen-0

4.9.0

This is the release note of [v4.9.0](https://github.com/optuna/optuna/milestone/74?closed=1).

Highlights

Enhance Multi-Objective Constrained Parallel Optimization in GPSampler

sawa3030 introduces parallelization enhancements to `GPSampler`, leveraging the **Kriging Believer** approach for constrained and multi-objective optimization (6481). This improvement allows for more efficient exploration when multiple trials are running concurrently.

&lt;img width=&quot;1400&quot; height=&quot;616&quot; alt=&quot;1_0k7Hd3Ipp1IKAI_o-l8DOQ&quot; src=&quot;https://github.com/user-attachments/assets/d979aea6-df98-42ec-b76c-f7c139b2e67e&quot; /&gt;


The GP surrogate is updated by assigning temporary objective function values to the running trials.

For more technical details and benchmarks, please check out our blog post: [Improving Optuna’s GPSampler Parallelization by Considering Running Trials](https://medium.com/optuna/improving-optunas-gpsampler-parallelization-by-considering-running-trials-10d42aeb5d49).

Deprecate Several Features

The following features are deprecated in v4.9.0 and scheduled for removal in v6.0.0.

**optuna**

* **Several arguments in `TPESampler`** (6635)
 * `prior_weight`, `consider_magic_clip`, `consider_endpoints`, `gamma`, `weights`, `hyperopt_parameters`: These internal parameters are being deprecated to simplify the interface, as the default settings are optimal for most use cases.
 * `warn_independent_sampling`: Deprecated because `TPESampler` now robustly supports both independent and joint sampling, making this warning obsolete.
 * `categorical_distance_func`: This advanced feature will be migrated to OptunaHub in the future.
* **`x0` and `sigma0` options in `CmaEsSampler`** (6624)
 * These options have been deprecated because they require a deep understanding of `CmaEsSampler`&#x27;s internals to be configured effectively.
* **`optuna.terminator` module** (6668)
 * This feature will be migrated to OptunaHub in the future.
* **`RetryFailedTrialCallback`** (6670)
 * This class has been renamed to `RetryHeartbeatStaleTrialCallback` to better reflect its behavior and avoid confusion with general trial retries (6085).
* **`optuna.integration` module**
 * The `optuna.integration` module currently acts as a shortcut to the external `optuna_integration` package for backward compatibility. Please import directly from the `optuna_integration` package going forward.

**optuna-integration**

* `PyCmaSampler`: Please use Optuna&#x27;s native `CmaEsSampler` instead.
* `CometCallback`: This feature will be migrated to OptunaHub in the future.
* `MLflowCallback`: This feature will be migrated to OptunaHub in the future.
* `TensorBoardCallback`: This feature will be migrated to OptunaHub in the future.
* `TrackioCallback`: This feature will be migrated to OptunaHub in the future.
* `WeightsAndBiasesCallback`: This class has already been migrated to OptunaHub.


Breaking Changes

- Deprecate `PyCmaSampler` (https://github.com/optuna/optuna-integration/pull/276)
- Add deprecation message for `CometCallback` (https://github.com/optuna/optuna-integration/pull/280)
- Add deprecation message for MLflowCallback (https://github.com/optuna/optuna-integration/pull/281)
- Add deprecation message for TensorBoardCallback (https://github.com/optuna/optuna-integration/pull/282)
- Add deprecation message for `WeightsAndBiasesCallback` (https://github.com/optuna/optuna-integration/pull/283)
- Add deprecation message for TrackioCallback (https://github.com/optuna/optuna-integration/pull/284)
- Fix importance computation for single-distributed params (6500)
- Make `QMCSampler` stateless (6616)
- Deprecate `x0` and `sigma0` options in CmaEsSampler (6624)
- Deprecate a set of `TPESampler` arguments (6635)
- Deprecate legacy imports from optuna.integration (6667)
- Deprecate `optuna.terminator` module (6668)
- Rename `RetryFailedTrialCallback` to `RetryHeartbeatStaleTrialCallback` (6670)

Enhancements

- Enhance Multi-Objective Constrained Parallel Optimization in GPSampler (6481)
- Remove `prior_mu` from `compute_sigmas` (6574)
- Fix sampling bias in `BruteForceSampler` (6627, thanks Rishabh-git10!)
- Refactor `BruteForceSampler` (6645)
- Speed up `BruteForceSampler` by using any instead of count (6647)
- Fix the return type of `BruteForceSampler` (6648)
- Split `BruteForceSampler` refactoring [2/3] (6656)
- fix: collect all infeasible values in error message instead of early return (6661, thanks AshutoshDevpura!)

Bug Fixes

- Fix categorical `None` handling in slice plots (6621)
- fix: handle JSONDecodeError in TPESampler._get_params to avoid race condition (6628, thanks AshutoshDevpura!)
- Split `BruteForceSampler` refactoring [2/3] (6656)

Documentation

- Update LightGBM links (6511, thanks jameslamb!)
- Improve doc for `best_trial`/`best_trials` in constrained optimization (6522)
- Update CmaEsSampler docstring regarding categorical support (6625)
- Fix RST note directive typo in `QMCSampler` docstring (6631, thanks RudrenduPaul!)
- Enhance Generative Engine Optimization (GEO) of many objective and constraint handling (6639)
- Add NSGA-III to FAQ about constraint (6641)
- Update `BruteForceSampler` and `GridSampler` information in docs (6651)
- Enhance FAQ about killing trials (6653)

Examples

- Explicitly install tensorboard in the tensorboard example CI (https://github.com/optuna/optuna-examples/pull/356)
- Remove examples which uses deprecated integration modules (https://github.com/optuna/optuna-examples/pull/357)

Code Fixes

- Use `broadcast_object_list` instead of a custom method (https://github.com/optuna/optuna-integration/pull/274)
- Correct return type annotations for `Axes` in visualization functions (6504, thanks kvr06-ai!)
- Fix type checking in trial folder (6510, thanks sateeshkumarb!)
- Use TYPE_CHECKING in importance/_mean_decrease_impurity.py (6514, thanks saivedant169!)
- Use TYPE_CHECKING in importance/_ped_anova/evaluator.py (6515, thanks saivedant169!)
- Use TYPE_CHECKING in samplers/_cmaes.py (6516, thanks saivedant169!)
- Use TYPE_CHECKING in samplers/_base.py (6517, thanks saivedant169!)
- Use TYPE_CHECKING in pruners/_wilcoxon.py (6518, thanks saivedant169!)
- Use TYPE_CHECKING in pruners/_percentile.py (6520, thanks saivedant169!)
- Use TYPE_CHECKING in nsgaii/_constraints_evaluation.py (6521, thanks saivedant169!)
- Use TYPE_CHECKING in nsgaii/_elite_population_selection_strategy.py (6523, thanks saivedant169!)
- Use TYPE_CHECKING in nsgaii/_after_trial_strategy.py (6524, thanks saivedant169!)
- Use TYPE_CHECKING in nsgaii/_child_generation_strategy.py (6526, thanks saivedant169!)
- Move type-only imports in `optuna.samplers._partial_fixed` to TYPE_CHECKING (6527, thanks t7r0n!)
- Fix TC006 cast annotation in optuna._gp.acqf (6528, thanks t7r0n!)
- Fix TC006 cast annotation in optuna.trial._frozen (6529, thanks t7r0n!)
- Fix TC006 cast annotation in tests.test_distributions (6530, thanks t7r0n!)
- Fix TC006 cast annotation in optuna.study._multi_objective (6531, thanks t7r0n!)
- Fix TC006 cast annotation in optuna.terminator.improvement.emmr (6532, thanks t7r0n!)
- Fix TC006 cast annotation in optuna.terminator.erroreval (6533, thanks t7r0n!)
- Fix TC006 cast annotation in optuna.study.study (6534, thanks t7r0n!)
- Move type-only Callable import in optuna.testing.threading behind TYPE_CHECKING (6535, thanks t7r0n!)
- Move type-only TracebackType import in optuna.testing.tempfile_pool behind TYPE_CHECKING (6536, thanks t7r0n!)
- Move type-only stdlib imports in optuna.testing.storages behind TYPE_CHECKING (6537, thanks t7r0n!)
- test: move type-only imports in optuna.testing.trials behind TYPE_CHECKING (6538, thanks t7r0n!)
- test: move type-only imports in optuna.testing.pytest_samplers behind TYPE_CHECKING (6539, thanks t7r0n!)
- test: move type-only imports in optuna.testing.pytest_storages behind TYPE_CHECKING (6540, thanks t7r0n!)
- test: move type-only imports in optuna.testing.samplers behind TYPE_CHECKING (6541, thanks t7r0n!)
- Quote cast type expression in `tests/test_distributions.py` (6543, thanks t7r0n!)
- Move type-only `datetime` import in `tests/trial_tests/test_trial.py` (6544, thanks t7r0n!)
- Move type-only imports to `TYPE_CHECKING` in `samplers/_tpe/sampler.py` (6545, thanks yasumorishima!)
- Move type-only imports to `TYPE_CHECKING` in `nsgaii/_crossover.py` (6546, thanks yasumorishima!)
- Remove dead code: unused functions, methods, and variables (6547, thanks duriantaco!)
- Fix typing import issues in scott_parzen_estimator (6548, thanks rpathade!)
- fix(types): use TYPE_CHECKING for imports in samplers/_cmaes.py (6550, thanks Aliipou!)
- fix(types): use TYPE_CHECKING for BaseDistribution import in _transform.py (6552, thanks Aliipou!)
- refactor: move BaseDistribution import to TYPE_CHECKING in search_space/group_decomposed.py (6553, thanks Aliipou!)
- Use `TYPE_CHECKING` for imports in `importance/_ped_anova/scott_parzen_estimator.py` (6554, thanks Aliipou!)
- fix(types): use TYPE_CHECKING for imports in samplers/_brute_force.py (6555, thanks Aliipou!)
- Move type-only imports to `TYPE_CHECKING` in `samplers/_nsgaiii/_elite_population_selection_strategy.py` (6557, thanks Aliipou!)
- Move type-only imports to `TYPE_CHECKING` in `samplers/_nsgaiii/_sampler.py` (6558, thanks Aliipou!)
- Move `Study` import to `TYPE_CHECKING` in `_timeline.py` (6559, thanks rpathade!)
- Use TYPE_CHECKING in optuna/samplers/_ga/_base.py (6560, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `nsgaii/_crossovers/_base.py` (6561, thanks saivedant169!)
- Use `TYPE_CHECKING` in `nsgaii/_crossovers/_blxalpha.py` (6562, thanks saivedant169!)
- Use `TYPE_CHECKING` in `nsgaii/_crossovers/_uniform.py` (6563, thanks saivedant169!)
- Move imports to `TYPE_CHECKING` in `tests/test_multi_objective` (6564, thanks acabellom!)
- Use `TYPE_CHECKING` in `nsgaii/_sampler.py` (6565, thanks saivedant169!)
- Use `TYPE_CHECKING` in `optuna.samplers._qmc` (6566, thanks hnshah!)
- Use `TYPE_CHECKING` in `visualization/_rank.py` (6567, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `visualization/_parallel_coordinate.py` (6568, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `visualization/_slice.py` (6569, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `visualization/_intermediate_values.py` (6570, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `visualization/_hypervolume_history.py` (6571, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `visualization/_contour.py` (6572, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_edf.py` (6575, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_optimization_history.py` (6576, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_hypervolume_history.py` (6577, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_pareto_front.py` (6578, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_terminator_improvement.py` (6579, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_rank.py` (6580, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/_slice.py` (6581, thanks saivedant169!)
- Fixed spelling/typo in error message (6583, thanks craigulliott!)
- Use `TYPE_CHECKING` in `visualization/_parallel_coordinate.py` (6584, thanks saivedant169!)
- Use `TYPE_CHECKING` in `optuna/visualization/matplotlib/_utils.py` (6585, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `optuna/visualization/matplotlib/_edf.py` (6586, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `optuna/visualization/matplotlib/_contour.py` (6587, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `optuna/visualization/matplotlib/_pareto_front.py` (6588, thanks nightcityblade!)
- Use `TYPE_CHECKING` in `optuna/visualization/matplotlib/_param_importances.py` (6589, thanks nightcityblade!)
- Fix missing blank line after `TYPE_CHECKING` in `visualization/_intermediate_values.py` (6591, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/matplotlib/_optimization_history.py` (6592, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/matplotlib/_parallel_coordinate.py` (6593, thanks saivedant169!)
- Use `TYPE_CHECKING` in `visualization/matplotlib/_rank.py` (6594, thanks saivedant169!)
- Address integration type checking in `optuna.integration.__init__.py` (6597)
- Remove unused type ignore comment (6607)
- fix: move FrozenTrial import into TYPE_CHECKING block in storages/_callbacks.py (6609, thanks satishkc7!)
- fix: move type-only imports to TYPE_CHECKING in StudySummary (6610, thanks Zelys-DFKH!)
- Enhance the error message of `QMCSampler` (6614)
- Enhance the warning message for `TPESampler` with `multivariate=True` (6618)
- Simplify some GP processing and update staled comments (6642)
- Enhance LSP experience by adding explicit references for optuna modules (6671)
- Remove unused variable in `optuna.storages._rdb.storage.py` (6672)

Continuous Integration

- Exclude Python 3.9 from fastaiv2 CI (https://github.com/optuna/optuna-integration/pull/273)
- Pinned every action for pypi-publish to a commit SHA (https://github.com/optuna/optuna-integration/pull/285)
- Restore TestPyPI repository URL for scheduled publishes (https://github.com/optuna/optuna-integration/pull/286)
- Pinned every action for pypi-publish to a commit SHA (6675)
- Restore TestPyPI repository URL for scheduled publishes (6677)

Other

- Bump the version up to v4.9.0.dev (https://github.com/optuna/optuna-integration/pull/268)
- Bump up version (https://github.com/optuna/optuna-integration/pull/288)
- Bump the version up to v4.9.0.dev (6508)
- Update news (6513)
- Add a PEP810 note to lazy import module (6636)
- Add some notes to GPSampler (6637)
- Remove unused intersphinx mappings (6643)
- Clarify policy on low-quality LLM-generated PRs (6665)
- Migrate to uv from pip (6669)
- Remove deprecated integration modules from README (6673)
- Modify the TODO note of `GPSampler` (6680)
- Remove the links to Optuna v5 roadmap and feedback survey from README (6683)
- Bump up to version number v4.9.0 (6684)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

Aliipou, Alnusjaponica, AshutoshDevpura, Rishabh-git10, RudrenduPaul, Zelys-DFKH, acabellom, c-bata, craigulliott, duriantaco, gen740, hnshah, jameslamb, kAIto47802, kvr06-ai, nabenabe0928, nightcityblade, not522, rpathade, saivedant169, sateeshkumarb, satishkc7, sawa3030, t7r0n, y0z, yasumorishima

4.8.0

This is the release note of [v4.8.0](https://github.com/optuna/optuna/milestone/73?closed=1).

Highlights

Support for Constant Liar Strategy to GPSampler

A constant liar strategy for efficient parallelization has been introduced to GPSampler by sawa3030. The figures (left: v4.7.0, right: v4.8.0) show that the overlap of search points has decreased, and a wider variety of solutions are being explored. The experiment uses `n_jobs = 10` and `n_trials = 100`. Currently, this feature supports single-objective and unconstrained optimization. Further extensions are coming in v4.9.0.

4.7.0

This is the release note of [v4.7.0](https://github.com/optuna/optuna/milestone/72?closed=1).

Highlights

Two New Multi-Objective Samplers Added to OptunaHub!

&lt;img width=&quot;1487&quot; height=&quot;946&quot; alt=&quot;hype-sampler&quot; src=&quot;https://github.com/user-attachments/assets/717752e7-4f55-4519-a407-70b1e0502052&quot; /&gt;

hrntsm introduces two new multi-objective samplers—SPEA-II (Strength Pareto Evolutionary Algorithm 2) and HypE (Hypervolume Estimation Algorithm)—to OptunaHub. SPEA-II is an improved multi-objective evolutionary algorithm that differs from NSGA-II in its selection mechanism. HypE is a fast, hypervolume-based evolutionary algorithm designed for many-objective optimization problems. Please refer to the following pages for more details:

* SPEA-II: https://hub.optuna.org/samplers/speaii/
* HypE: https://hub.optuna.org/samplers/hype/

`PedAnovaImportanceEvaluator` Now Supports Local Hyperparameter Importance Computation

The `target_quantile` and `region_quantile` arguments have been introduced to [`PedAnovaImportanceEvaluator`](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.importance.PedAnovaImportanceEvaluator.html). This change allows you to investigate local hyperparameter importance rather than the global one with `region_quantile &lt; 1.0`. See [the original paper](https://www.ijcai.org/proceedings/2023/488) for the technical details.

Enhancements

- Introduce stacklevel-aware custom warnings (6293)
- Cache distributions to skip consistency check (6301)
- Add warnings when `JournalStorage` lock acquisition is delayed (6361)
- Add support for local HPI in PED-ANOVA (6362)

Bug Fixes

- Fix log PDF of discrete trunc log-norm distribution for `TPESampler` (6258)
- Fix coefficient in PED-ANOVA (6358)
- Fix GPSampler crash when default torch device is CUDA (6397, thanks Quant-Quasar!)

Documentation

- Add `SECURITY.md` (6317)
- Add a note for future dev of exclusive HV (6318)
- Update GPSampler documentation to include D-BE optimization details (6347, thanks Kaichi-Irie!)
- Revert PR 6354 to enable `-W` option on Sphinx build (6373)

Examples

- Disable scheduled runs for PyTorch and visualization temporarily (https://github.com/optuna/optuna-examples/pull/337)
- Fix skorch example: Replace unavailable OpenML MNIST (https://github.com/optuna/optuna-examples/pull/338, thanks sotagg!)
- Pin `minio` version to `&lt;=7.2.18` to fix CI &amp; stop daily CI running (https://github.com/optuna/optuna-examples/pull/339)
- fix spark example (https://github.com/optuna/optuna-examples/pull/342, thanks fritshermans!)
- Pin scikit-learn to &lt; 1.6.0 for lightgbm (https://github.com/optuna/optuna-examples/pull/343)
- Remove python 3.9 due to EOL Status (https://github.com/optuna/optuna-examples/pull/344, thanks ParagEkbote!)
- Add IPython as a dependency of fastai example (https://github.com/optuna/optuna-examples/pull/347)

Tests

- Fix TC006 violation in tests/visualization_tests/test_utils.py (6387, thanks jiayusu!)

Code Fixes

- Replace `.format()` with f-string in `_setup_studies` (6326, thanks haitham404!)
- Update `_upload.py` for `TYPE_CHECKING` (6327, thanks satyarth7srivastava!)
- Replace .format() with f-string in progress_bar.py (6328, thanks Nayil97!)
- Use f-strings in `optuna/samplers/_cmaes.py` (6331, thanks swativdusane!)
- Replace `.format()` with f-string in `_parallel_coordinate.py` (6333, thanks satyarth7srivastava!)
- Refactor/fstring storage rdb (6336, thanks gadmin7!)
- Migrate to ruff from black/blackdoc/isort/flake8 (6341)
- Replace `.format` with f-strings in `optuna/importance/_base` (6342, thanks VihaanMotwani!)
- updated `_terminator_improvement.py` for `TYPE_CHECKING` (6343, thanks satyarth7srivastava!)
- Replace `.format` with f-string in `_param_importances.py` (6345, thanks Harshadev-24!)
- Replace .format() with f-strings in several modules (6348, thanks varundevr!)
- Replace more .format() calls with f-strings (6349, thanks varundevr!)
- Replace `.format` with f-string in `tutorial/20_recipes/004_cli.py` (6350, thanks RektPunk!)
- Replace `.format` with f-string in `optuna/study/_optimize.py` (6351, thanks RektPunk!)
- Format `optuna/` files with Ruff (6352)
- Refactor: Use f-string in 001_rdb.py (6356, thanks sotagg!)
- Format `tests/` and `tutorials/` files with Ruff (6360)
- Remove redundant `_color_supported()` check (6363)
- Add `StorageTestCase` class in `optuna.testing` package (6369)
- Change string formatting in `optuna/pruners/_hyperband.py` (6370, thanks eleannapapaio!)
- Refactor `test_study.py` to use f-string instead of `.format()` (6372, thanks nepersoned!)
- Fix mypy error for np.select (6374)
- Change string formatting for `_successive_halving.py` (6375, thanks spenam!)
- Fix type annotations for `optuna/trial/_frozen.py` (6377, thanks spenam!)
- fix type annotations for `optuna/study/study.py` (6378, thanks spenam!)
- Fix type annotations for `tests/study_tests/test_study.py` (6379, thanks spenam!)
- Move type-only imports to `TYPE_CHECKING` in `test_visualizations.py` (6380, thanks Sip4818!)
- Move type-only imports to `TYPE_CHECKING` in `_constrained_optimization.py` (6381, thanks Sip4818!)
- Replace format with f-string (6383, thanks varundevr!)
- Move type-only imports to `TYPE_CHECKING` in `_multi_objective.py` (6385, thanks Sip4818!)
- Move `FrozenTrial` import under `TYPE_CHECKING` for `_study_summary.py` file (6386, thanks Sip4818!)
- Move type-check imports to `TYPE_CHECKING` in `optuna/terminator/callback.py` (6388, thanks Sip4818!)
- Using f-string instead of `.format()` (6389, thanks Lakshman142!)
- Use f-strings in `optuna/_experimental.py` (6390, thanks Rohan0497!)
- Fix invalid `StorageTestCase` scenarios involving trial state and values (6391)
- Move `type-hint` import inside `Type-Checking` block in `optuna\terminator\erroreval.py` (6395, thanks Sip4818!)
- Move `type-check` imports to `TYPE_CHECKING` in `optuna\terminator\improvement\emmr.py` (6396, thanks Sip4818!)
- Move typing-only imports under `TYPE_CHECKING` in `matplotlib/_slice.py` (6399, thanks kapishyadav!)
- Use `logger.warning` instead of `optuna_warn` for lock-acquisition delay notifications (6400)
- Update string formatting in `optuna/samplers/_grid.py` (6401, thanks kapishyadav!)
- Updating `storages/_in_memory.py` to use f-strings (6404, thanks jrings!)
- Move `type-hint` imports into `type-checking` block in `optuna\terminator\improvement\evaluator.py` (6405, thanks Sip4818!)
- Move `type-hint` imports into `type-checking` block in `median_erroreval.py` (6408, thanks Sip4818!)
- Replace `.format()` with f-string in `_rank.py` (6409, thanks jwalith!)
- Replace .format() with f-string in test_hyperband.py (6411, thanks Banjiola!)
- Replace .format() with f-string in _fixed.py (6412, thanks VedantMadane!)

Continuous Integration

- Fix for CI (https://github.com/optuna/optuna-integration/pull/258)
- Add ipython as a dependency of fastai (https://github.com/optuna/optuna-integration/pull/261)
- Remove FastAIV2PruningCallback from docs to resolve warning (https://github.com/optuna/optuna-integration/pull/264)
- Dispose SQLAlchemy&#x27;s engine in `storage_tests/test_with_server.py` (6330)
- Dispose SQLAlchemy&#x27;s engine in `storage_tests/test_cached_storage.py` (6337)
- Dispose SQLAlchemy&#x27;s engine in `storage_tests/rdb_tests/test_storage.py` (6338)
- Support Python 3.14 (6339)
- Temporarily disable `-W` option on Sphinx build (6354)

Other

- Bump up version (https://github.com/optuna/optuna-integration/pull/256)
- Bump up version number to 4.7.0 (https://github.com/optuna/optuna-integration/pull/262)
- Bump up to the version number `v4.7.0.dev` (6325)
- Update the news section on README (6335)
- Remove Codecov usage (6340)
- Remove `formats.sh` and tidy up `CONTRIBUTING.md` (6353)
- Remove `asv` and the speed benchmark workflow (6393)
- Bump up version to v4.7.0 (6413)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

Alnusjaponica, Banjiola, Harshadev-24, HideakiImamura, Kaichi-Irie, Lakshman142, Nayil97, ParagEkbote, Quant-Quasar, RektPunk, Rohan0497, Sip4818, VedantMadane, VihaanMotwani, c-bata, eleannapapaio, fritshermans, fusawa-yugo, gadmin7, gen740, haitham404, jiayusu, jrings, jwalith, kAIto47802, kapishyadav, nabenabe0928, nepersoned, not522, nzw0301, satyarth7srivastava, sawa3030, sotagg, spenam, swativdusane, toshihikoyanase, varundevr, y0z

4.6.0

This is the release note of [v4.6.0](https://github.com/optuna/optuna/milestone/71?closed=1).

Highlights

Optuna Dashboard LLM Integration

[Optuna Dashboard](https://github.com/optuna/optuna-dashboard) is a web-based tool that helps you easily explore and visualize your Optuna optimization history. The latest release, v0.20.0, introduces LLM integration, enabling the natural language-based Trial filtering and automatic Plotly chart generation. Please refer to the release blog for more details.

&lt;img width=&quot;900&quot; alt=&quot;image55&quot; src=&quot;https://github.com/user-attachments/assets/a2a2d6d2-e782-48ad-a4f7-3a33ca8599eb&quot; /&gt;

Further Speed Enhancements for `GPSampler`

`GPSampler` becomes significantly faster owing to parallelized multi-start acquisition function optimization via PyTorch batching, and to optimized NumPy operations.

&lt;img width=&quot;900&quot; alt=&quot;image15&quot; src=&quot;https://github.com/user-attachments/assets/d72c1fc7-f9c5-4dac-baf7-a88167bbcd27&quot; /&gt;

Full Support for Multi-objective and Constrained Optimization in AutoSampler

We have fully implemented sampler selection rules for multi-objective and constrained optimization in [AutoSampler](https://hub.optuna.org/samplers/auto_sampler/). For more details, please see our blog post, [&quot;AutoSampler: Full Support for Multi-Objective &amp; Constrained Optimization.&quot;](https://medium.com/optuna/autosampler-full-support-for-multi-objective-constrained-optimization-c1c4fc957ba2)

&lt;img width=&quot;900&quot; alt=&quot;optuna-blog-autosampler-multi-constrained&quot; src=&quot;https://github.com/user-attachments/assets/75c768b1-8d22-464b-90e8-b44700a044e8&quot; /&gt;

Additions of Robust Bayesian Optimization Packages

Robust Bayesian optimization methods have been added to OptunaHub. Robust Bayesian optimization enables suggesting more robust parameters against input perturbations. This is especially helpful for Sim2Real transfer scenarios.

&lt;img width=&quot;600&quot; alt=&quot;image25&quot; src=&quot;https://github.com/user-attachments/assets/9ad6de92-1c84-4716-a06e-c21ffcdb0233&quot; /&gt;

Breaking Changes

- Drop Python 3.8 &amp; Support Python 3.13 (https://github.com/optuna/optuna-integration/pull/253)
- Change `TrialState.__repr__` and `TrialState.__str__` (6281, thanks ktns!)
- Drop Python 3.8 (6302)

Enhancements

- Use iterator for lazy evaluation in journal storage’s `read_logs` (6144)
- Cache pair-wise distances to speed up `GPSampler` (6244)
- Speed up LogEI implementation (6248)
- Speed up EHVI by optimizing tensor operation order (6257)
- Use the decremental approach in the hypervolume contribution calculation (6264)
- Use cached trials in `TPESampler`&#x27;s `sample_relative` (6265)
- Remove `find_or_raise_by_id` in `_set_trial_value_without_commit` (6266)
- Speed up `GPSampler` by Batching Acquisition Function Evaluations (6268, thanks Kaichi-Irie!)
- Use cached study direction and trial for `_CachedStorage`&#x27;s `get_best_trial` (6270)
- Add upsert in `_set_trial_attr_without_commit` for PostgreSQL (6282, thanks jaikumarm!)
- Add `states` argument to `_read_trials_from_remote_storage` (6288)
- Use cached trials for intersection search space calculation (6291)
- Replace `np.linalg.inv` with `np.linalg.cholesky` to speed up `GPSampler` for `numpy&gt;=2.0.0` (6296)

Bug Fixes

- Skip trial validation on copy_study (6249)
- Fix incremental update algorithm in `_CachedStorage`&#x27;s `_read_trials_from_remote_storage` (6310)
- Add safety guard for exhaustive search (6321)

Documentation

- Add `AutoSampler` to the sampler comparison table in the API reference (6260, thanks Kaichi-Irie!)
- Update the `GPSampler` document to reflect support for constrained multi-objective optimization (6262)
- Add a link to the metric TPE paper in the `TPESampler` document (6263)
- Update announcement (6285)
- Update the table of Samplers in docs (6287, thanks fusawa-yugo!)
- Fix the table of samplers in the docs (6290)

Examples

- Add example of OpenTelemetry in Optuna Dashboard (https://github.com/optuna/optuna-examples/pull/330)
- [hotfix] Fix transformers example by adding the version constraint on transformers (https://github.com/optuna/optuna-examples/pull/332)
- Drop Python 3.8 (https://github.com/optuna/optuna-examples/pull/334)
- Remove Version Constraint for Transformers (https://github.com/optuna/optuna-examples/pull/335, thanks ParagEkbote!)

Tests

- Add unit tests for batched L-BFGS-B (6274, thanks Kaichi-Irie!)

Code Fixes

- Update target version of black from Python 3.8 to 3.9 (https://github.com/optuna/optuna-integration/pull/254)
- Move `fit_kernel_params` to `GPRegressor` (6243)
- Modify `TYPE_CHECKING` in `_brute_force.py` (6259, thanks Kaichi-Irie!)
- Move SciPy to the lazy import section in `_gp/scipy_blas_thread_patch.py` (6269, thanks Kaichi-Irie!)
- Make the interface of `batched_lbfgsb` module compatible with `scipy.optimize` (6273, thanks Kaichi-Irie!)
- Fix type checking in `optuna.study._frozen.py` (6275, thanks GabrielRomaoG!)
- Move typing-only imports under `TYPE_CHECKING` in `optuna.importance.__init__` (6278, thanks euangoodbrand!)
- Move typing-only imports under TYPE_CHECKING in `FanovaImportanceEvaluator` (6279, thanks euangoodbrand!)
- Move typing-only imports under `TYPE_CHECKING` in `/study/_optimize.py` (6280, thanks euangoodbrand!)
- Use `TYPE_CHECKING` in `optuna/pruners/_nop.py` (6297, thanks AddyM!)
- Use `TYPE_CHECKING` in `optuna/samplers/_random.py` (6298, thanks AddyM!)
- Speed up squared distance computation (6300)
- Refactor emmr (6304)
- Fix string format of `optuna/distributions.py` (6306)
- Fix string format of `tests/samplers_tests/tpe_tests/test_truncnorm.py` (6307)
- Update black target to Python 3.9 (6308)
- Fix string format for `optuna/study/study.py` (6309, thanks unKnownNG!)
- Changed the old `.format` code to the new f string format in the `test_journal.py` (6312, thanks Zrahay!)
- Update string formatting in `visualization/_pareto_front.py` (6314, thanks dross20!)
- Use f-string in `001_first.py` (6315, thanks satyarth7srivastava!)
- Use f-strings in `_intermediate_values.py` (6316, thanks nihalsiddiqui7!)
- Refactor `.format` to f-string in `_percentile.py` (6323, thanks Jongwan93!)

Continuous Integration

- Update `sklearn.py` to fix mypy checks (https://github.com/optuna/optuna-integration/pull/249)
- Fix CI (https://github.com/optuna/optuna-integration/pull/250)
- Fix fragile `test_parallel_optimize_with_sleep` (6241)
- Fix type checking in GP for CI (6276)
- Fix CI (6284)
- Limit the blackdoc version (6289)
- Migrate `.coveragerc` to `pyproject.toml` (6292, thanks ParagEkbote!)
- Explicitly close DB connections when discarding SQLAlchemy&#x27;s `Engine` (6303)

Other

- Bump up the version number to `4.6.0.dev` (https://github.com/optuna/optuna-integration/pull/245)
- Add `__version__` to init (https://github.com/optuna/optuna-integration/pull/247)
- Migrate `.coveragerc` to `pyproject.toml` (https://github.com/optuna/optuna-integration/pull/252, thanks ParagEkbote!)
- Bump up version (https://github.com/optuna/optuna-integration/pull/255)
- Bump up version to v4.6.0.dev (6252)
- Update NEWS section (6319)
- Bump up to version number 4.6.0 (6324)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

AddyM, GabrielRomaoG, Jongwan93, Kaichi-Irie, ParagEkbote, Zrahay, c-bata, contramundum53, dross20, euangoodbrand, fusawa-yugo, gen740, jaikumarm, kAIto47802, ktns, nabenabe0928, nihalsiddiqui7, not522, satyarth7srivastava, sawa3030, toshihikoyanase, unKnownNG, y0z

4.5.0

This is the release note of [v4.5.0](https://github.com/optuna/optuna/milestone/70?closed=1).

Highlights

`GPSampler` for constrained multi-objective optimization
`GPSampler` is now able to handle multiple objective and constraints simultaneously using the newly introduced constrained LogEHVI acquisition function.

The figures below show the difference between `GPSampler` (LogEHVI, unconstrained) vs `GPSampler` (constrained LogEHVI, new feature). The 3-dimensional version of the C2DTLZ2 benchmark problem we used is a problem where some areas of the Pareto front of the original DTLZ2 problem are made infeasible by constraints. Therefore, even if constraints are not taken into account, it is possible to obtain the Pareto front. Experimental results show that both LogEHVI and constrained LogEHVI can approximate the Pareto front, but the latter has significantly fewer infeasible solutions, demonstrating its efficiency.

4.4

|:--:|:--:|
|&lt;img width=&quot;320&quot; height=&quot;240&quot; alt=&quot;Log EHVI&quot; src=&quot;https://github.com/user-attachments/assets/f8f302c3-4f43-4afd-8033-3c631985861b&quot; /&gt;|&lt;img width=&quot;320&quot; height=&quot;240&quot; alt=&quot;Constrained LogEHVI&quot; src=&quot;https://github.com/user-attachments/assets/8bb70c11-b9c7-412a-9367-d6d832855a73&quot; /&gt;|


Significant speedup of `TPESampler`

`TPESampler` is significantly (about 5x as listed in the table below) faster! It enables a larger number of trials in each study. The speedup was achieved through a series of enhancements in constant factors.

The following table shows the speed comparison of `TPESampler` between v4.4.0 and v4.5.0. The experiments were conducted using `multivariate=True` on a search space with 3 continuous parameters and 3 numerical discrete parameters. Each row shows the runtime for each number of objectives and each column shows each number of trials to be evaluated. Each runtime is shown along with the standard error over 3 random seeds. The numbers in parentheses represent the speedup factor in comparison to v4.4.0. For example, (5.1x) means the runtime of v4.5.0 is 5.1 times faster than that of v4.4.0.

|`n_objectives`/`n_trials`|500|1000|1500|2000|
|:--:|:--:|:--:|:--:|:--:|

4.4.0

This is the release note of [v4.4.0](https://github.com/optuna/optuna/milestone/69?closed=1).

Highlights

In addition to new features, bug fixes, and improvements in documentation and testing, version 4.4 introduces a new tool called the [Optuna MCP Server](https://github.com/optuna/optuna-mcp).

Optuna MCP Server
The Optuna MCP server can be accessed by any MCP client via uv — for instance, with Claude Desktop, simply add the following configuration to your MCP server settings file. Of course, other LLM clients like VSCode or Cline can also be used similarly. You can also access it via Docker. If you want to persist the results, you can use the — storage option. For details, please refer to the [repository](https://github.com/optuna/optuna-mcp).


{
&quot;mcpServers&quot;: {
 … (Other MCP Servers&#x27; settings)
 &quot;Optuna&quot;: {
   &quot;command&quot;: &quot;uvx&quot;,
   &quot;args&quot;: [
     &quot;optuna-mcp&quot;
   ]
 }
}
}


![image3](https://github.com/user-attachments/assets/231165f7-5df6-4e11-8c20-c89f1da7af26)

Gaussian Process-Based Multi-objective Optimization

Optuna’s GPSampler, introduced in version 3.6, offers superior speed and performance compared to existing Bayesian optimization frameworks, particularly when handling objective functions with discrete variables. In Optuna v4.4, we have extended this GPSampler to support multi-objective optimization problems. The applications of multi-objective optimization are broad, and the new multi-objective capabilities introduced in this GPSampler are expected to find applications in fields such as material design, experimental design problems, and high-cost hyperparameter optimization.

GPSampler can be easily integrated into your program and performs well against the existing BoTorchSampler. We encourage you to try it out with your multi-objective optimization problems.

python
sampler = optuna.samplers.GPSampler()
study = optuna.create_study(directions=[&quot;minimize&quot;, &quot;minimize&quot;], sampler=sampler)


![image2](https://github.com/user-attachments/assets/24586f21-fd9c-4d8c-86db-2507f3a2c12f)

New Features in OptunaHub

During the development period of Optuna v4.4, several new features were also introduced to OptunaHub, the feature-sharing platform for Optuna:

- A [sampler utilizing Google Vizier](https://hub.optuna.org/samplers/vizier/) is now newly available.
- The [CMA-ES-based sampler with restart strategy](https://hub.optuna.org/samplers/restart_cmaes/), which was previously part of the Optuna core, has been migrated to OptunaHub, making it simpler and more user-friendly.
- A [benchmark problem solving aircraft design](https://hub.optuna.org/benchmarks/hpa/) as a black-box optimization task has been added, further enhancing the convenience of algorithm development using OptunaHub.
- A visualization feature has also been added, allowing users to see [how the acquisition function of the default TPESampler evolves as trials progress](https://hub.optuna.org/visualization/tpe_acquisition_visualizer/).
- A novel mutation operation has been added to the [MOEA/D evolutionary computation algorithm](https://hub.optuna.org/samplers/moead/) for multi-objective optimization.

| Vizier sampler performance  |
| --- |
| ![image1](https://github.com/user-attachments/assets/0c2f42b5-b85a-48f3-9d20-cbb1efa1204c) |

| TPE acquisition visualizer |
| --- |
| ![image4](https://github.com/user-attachments/assets/d3a92bc6-f5cb-486a-9fcf-f2f5748e6180) |

Breaking Changes

- Update `consider_prior` Behavior and Remove Support for `False` (6007)
- Remove `restart_strategy` and `inc_popsize` to simplify `CmaEsSampler` (6025)
- Make all arguments of `TPESampler` keyword-only (6041)

New Features

- Add a module to preprocess solutions for hypervolume improvement calculation (6039)
- Add `AcquisitionFuncParams` for LogEHVI (6052)
- Support Multi-Objective Optimization `GPSampler` (6069)
- Add `n_recent_trials` to `plot_timeline` (6110, thanks msdsm!)

Enhancements

- Adapt `TYPE_CHECKING` of `samplers/_gp/sampler.py` (6059)
- Avoid deepcopy in `_tell_with_warning` (6079)
- Add `_compute_3d` for hypervolume computation (6112, thanks shmurai!)
- Improve performance of `plot_hypervolume_history` (6115, thanks shmurai!)
- add deprecated/removed version specification to calls of `convert_positional_args` (6117, thanks shmurai!)
- Optimize `Study.best_trial` performance by avoiding unnecessary deep copy (6119, thanks msdsm!)
- Refactor and speed up HV3D (6124)
- Add `assume_pareto` for hv calculation in `_calculate_weights_below_for_multi_objective` (6129)

Bug Fixes

- Update vsbx (6033, thanks hrntsm!)
- Fix `request.values` in `OptunaStorageProxyService` (6044, thanks hitsgub!)
- Fix a bug in distributed optimization using NSGA-II/III (6066, thanks leevers!)
- Fix: fetch all trials in `BruteForceSampler` for `HyperbandPruner` (6107)

Documentation

- Add Pycma Example (https://github.com/optuna/optuna-integration/pull/226, thanks ParagEkbote!)
- Add SHAP Example (https://github.com/optuna/optuna-integration/pull/227, thanks ParagEkbote!)
- Document Behavior of `optuna.pruners.MedianPruner` and `optuna.pruners.PatientPruner` (6055, thanks ParagEkbote!)
- Change the link of tutorial docs of optunahub (6063, thanks fusawa-yugo!)
- Update the documentation string of `GPSampler` (6081)
- Add a warning about the combination of gRPC Proxy and Journal Storage (6097)
- Cosmetic fix to the terminator documents (6100)
- Note in docstring that heartbeat mechanism is experimental (6111, thanks lan496!)
- Update docstrings of `_get_best_trial` to follow coding conventions (6122)

Examples

- Add Example for Comet (https://github.com/optuna/optuna-examples/pull/305, thanks ParagEkbote!)
- Adding an OpenML example (https://github.com/optuna/optuna-examples/pull/310, thanks SubhadityaMukherjee!)
- Add workflow dispatch (https://github.com/optuna/optuna-examples/pull/311)
- Update PyTorch Checkpoint Example using tempfile (https://github.com/optuna/optuna-examples/pull/313, thanks ParagEkbote!)
- [hotfix] Fix Dask-ML example by adding the version constraint on numpy (https://github.com/optuna/optuna-examples/pull/315)
- Setup Pre-Commit (https://github.com/optuna/optuna-examples/pull/316, thanks ParagEkbote!)
- Remove Python 3.9 from haiku CI (https://github.com/optuna/optuna-examples/pull/318)
- Add a transformers example (https://github.com/optuna/optuna-examples/pull/322, thanks ParagEkbote!)
- Add transformer item to `RAEDME.md` (https://github.com/optuna/optuna-examples/pull/323)
- Remove version constraints of `tensorflow` and `numpy` (https://github.com/optuna/optuna-examples/pull/324)
- Update pre-commit hooks (https://github.com/optuna/optuna-examples/pull/326, thanks lan496!)
- Add preferential optimization picture (https://github.com/optuna/optuna-examples/pull/327, thanks milkcoffeen!)

Tests

- Add float precision tests for storages (6040)
- Refactor `test_base_gasampler.py` (6104)
- chore: run tests for importance only with in-memory (6109)
- Improve test cases for `n_recent_trials` of `plot_timeline` (follow-up 6110) (6116)
- Performance optimization for `test_study.py` by removing redundancy (6120)

Code Fixes

- Optional mypy check (6028)
- Update Type-Checking for `optuna/_experimental.py` (6045, thanks ParagEkbote!)
- Update Type-Checking for `optuna/importance/_base.py` (6046, thanks ParagEkbote!)
- Update Type-Checking for `optuna/_convert_positional_args.py` (6050, thanks ParagEkbote!)
- Update Type-Checking for `optuna/_deprecated.py` (6051, thanks ParagEkbote!)
- Update Type-Checking for `optuna/_gp/gp.py` (6053, thanks ParagEkbote!)
- Add validate `eta` in sbx (6056, thanks hrntsm!)
- Remove `CmaEsAttrKeys` and `_attr_keys` for Simplification (6068)
- Replace `np.isnan` with `math.isnan` (6080)
- Refactor warning handling of `_tell_with_warning` (6082)
- Implement Type-Checking for `optuna/distributions.py` (6086, thanks AdrianStrymer!)
- Update TYPE_CHECKING for `optuna/_gp/gp.py` (6090, thanks Samarthi!)
- Support mypy 1.16.0 (6102)
- Emit `ExperimentalWarning` if heartbeat is enabled (6106, thanks lan496!)
- Simplify tuple return in `optuna/visualization/_terminator_improvement.py` (6139, thanks Prashantdhaka23!)
- Refactor return standardization in `optim_mixed.py` (6140, thanks Ajay-Satish-01!)
- Simplify tuple return in `test_trial.py` (6141, thanks saishreyakumar!)
- Refactor return statement style in `optuna/storages/_rdb/models.py` for consistency among the codebase (6143, thanks Shubham05122002!)

Continuous Integration

- Add type ignore in `wandb` (https://github.com/optuna/optuna-integration/pull/228)
- Fix to prevent daily `checks-optional` CI on the fork repositories (6103)
- Fix CI (6137)
- [hotfix] Add version constraint on `blackdoc` (6145)

Other

- Bump up version number to v4.4.0.dev (https://github.com/optuna/optuna-integration/pull/220)
- Add pre commit config (https://github.com/optuna/optuna-integration/pull/231, thanks milkcoffeen!)
- Bump up version number to 4.4.0 (https://github.com/optuna/optuna-integration/pull/236)
- Bump up version to 4.4.0dev (6038)
- Update the news section in README.md (6049)
- Fix README for v5 roadmap (6094)
- Update pre-commit hooks (6108, thanks lan496!)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

AdrianStrymer, Ajay-Satish-01, Alnusjaponica, Copilot, HideakiImamura, ParagEkbote, Prashantdhaka23, Samarthi, Shubham05122002, SubhadityaMukherjee, c-bata, contramundum53, copilot-pull-request-reviewer[bot], fusawa-yugo, gen740, himkt, hitsgub, hrntsm, kAIto47802, lan496, leevers, milkcoffeen, msdsm, nabenabe0928, not522, nzw0301, saishreyakumar, sawa3030, shmurai, toshihikoyanase, y0z

4.3.0

This is the release note of [v4.3.0](https://github.com/optuna/optuna/milestone/66?closed=1).

Highlights

This has various bug fixes and improvements to the documentation and more.

Breaking Changes

- [fix] lgbm 4.6.0 compatibility (https://github.com/optuna/optuna-integration/pull/207, thanks ffineis!)

Enhancements

- Accept custom objective in `LightGBMTuner` (https://github.com/optuna/optuna-integration/pull/203, thanks sawa3030!)
- Improve time complexity of `IntersectionSearchSpace` (5982, thanks GittyHarsha!)
- Add `_prev_waiting_trial_number` in `InMemoryStorage` to improve the efficiency of `_pop_waiting_trial_id` (5993, thanks sawa3030!)
- Add arguments of versions to `convert_positional_args` (6009, thanks fusawa-yugo!)
- Add `wait_server_ready` method in GrpcStorageProxy (6010, thanks hitsgub!)
- Remove warning messages for Matplotlib-based `plot_contour` and `plot_rank` (6011)
- Fix type checking in `optuna._callbacks.py` (6030)
- Enhance `SBXCrossover` (6008, thanks hrntsm!)

Bug Fixes

- Convert storage into `InMemoryStorage` before copying to the local (https://github.com/optuna/optuna-integration/pull/213)
- Fix contour plot of `matplotlib` (5892, thanks fusawa-yugo!)
- Fix threading lock logic (5922)
- Use `_LazyImport` for grpcio package (5954)
- Prevent Lock Blocking by Adding Timeout to `JournalStorage` (5971, thanks sawa3030!)
- Fix a minor bug in GPSampler for objective that returns `inf` (5995)
- Fix a bug that a gRPC server doesn&#x27;t work with JournalStorage (6004, thanks fusawa-yugo!)
- Fix `_pop_waiting_trial_id` for finished trial (6012)
- Resolve the issue where `BruteForceSampler` fails to suggest all combinations (5893)

Documentation

- Follow recent changes in `optuna/optuna`&#x27;s document sphinx config (https://github.com/optuna/optuna-integration/pull/197)
- Fix links to external modules (https://github.com/optuna/optuna-integration/pull/198)
- Update `CONTRIBUTING.md` (https://github.com/optuna/optuna-integration/pull/200, thanks sawa3030!)
- Update comment in `.readthedocs.yml` (5976)
- Add comments on the reproducibility of `HyperBandPruner` (6018)

Examples

- [hotfix] Add the version constraint on `dask` (https://github.com/optuna/optuna-examples/pull/296)
- [hotfix] Add the version constraint on  `dask` for `dask-ml` (https://github.com/optuna/optuna-examples/pull/297)
- Extends execution span of `hiplot` and `sklearn` (https://github.com/optuna/optuna-examples/pull/298, thanks fusawa-yugo!)
- Apply black to fix CI (https://github.com/optuna/optuna-examples/pull/300)
- Bump up to 3.12 for CI (https://github.com/optuna/optuna-examples/pull/301)
- [hotfix] Add the version constraint on `lightgbm` (https://github.com/optuna/optuna-examples/pull/302)
- Fix Skorch Example (https://github.com/optuna/optuna-examples/pull/303, thanks ParagEkbote!)
- Add version constraint for tensorflow-related CI (https://github.com/optuna/optuna-examples/pull/304)
- Temporarily skip Python 3.9 in fastai example (https://github.com/optuna/optuna-examples/pull/308)
- Run the `skorch` example in the CI (https://github.com/optuna/optuna-examples/pull/309)
- Fix `fastai` Example (https://github.com/optuna/optuna-examples/pull/312)

Tests

- Use `JournalStorage` in `test_cli.py` (5990, thanks sawa3030!)

Code Fixes

- Add `BaseGASampler` (5864)
- Fix comments in `pyproject.toml` (5972)
- Remove `FirstTrialOnlyRandomSampler` (5973, thanks mehakmander11!)
- Remove `_check_and_set_param_distribution` (5975, thanks siddydutta!)
- Remove `testing/distributions.py` (5977, thanks mehakmander11!)
- Remove `_StudyInfo`&#x27;s `param_distribution` in `_cached_storage.py` (5978, thanks tarunprabhu11!)
- Introduce `UpdateFinishedTrialError` to raise an error when attempting to modify a finished trial (6001, thanks sawa3030!)
- Deprecate `consider_prior` in `TPESampler` (6005, thanks sawa3030!)
- Improve Code Readability by Following PEP8 Standards (6006, thanks sawa3030!)
- Made error message for `create_study`&#x27;s direction easier to understand `optuna.study` (6021, thanks sinano1107!)

Continuous Integration

- Hotfix ci (https://github.com/optuna/optuna-integration/pull/199)
- Add flake8 in CI (https://github.com/optuna/optuna-integration/pull/201, thanks sawa3030!)
- Remove test cases that uses `UnsupportedDistribution` (https://github.com/optuna/optuna-integration/pull/208)
- Fix a mypy error when using `numpy&gt;=2.2.4` (https://github.com/optuna/optuna-integration/pull/212)
- Fix a bug of `lightgbm` tuner for Python 3.8 users (https://github.com/optuna/optuna-integration/pull/214)
- Add a version constraint on `xgboost` (https://github.com/optuna/optuna-integration/pull/217)
- Run (https://github.com/optuna/optuna-integration/pull/218)
- Ensure gRPC server readiness before proceeding to prevent test failures (5938, thanks sawa3030!)
- Apply black to fix CI (5952)
- Add `workflow_dispatch` trigger to all the CI (6019)
- Fix CI (6026)

Other

- Bump up version number to 4.3.0.dev (https://github.com/optuna/optuna-integration/pull/192)
- Bump the version up to v4.2.1 (https://github.com/optuna/optuna-integration/pull/195)
- Set repository url (https://github.com/optuna/optuna-integration/pull/196, thanks ktns!)
- Bump up version number to v4.3.0 (https://github.com/optuna/optuna-integration/pull/221)
- Bump the version up to v4.3.0.dev (5927)
- Add the article to the news section (5928)
- Update news section for 4.2.0 release (5934)
- Update News (5936)
- Update README with the new blog entry (5980)
- Add `GPSampler` blog to the announcement (6014)
- Add grpc blog to README (6020)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

Alnusjaponica, GittyHarsha, HideakiImamura, ParagEkbote, c-bata, contramundum53, ffineis, fusawa-yugo, gen740, hitsgub, hrntsm, kAIto47802, ktns, mehakmander11, nabenabe0928, not522, nzw0301, porink0424, sawa3030, siddydutta, sinano1107, tarunprabhu11, toshihikoyanase, y0z

4.2.1

This is the release note of v4.2.1. This release includes a bug fix addressing an issue where Optuna was unable to import if an older version of the grpcio package was installed.

Bug

- [backport] Use `_LazyImport` for grpcio package (5965)

Other

- Bump up version number to v4.2.1 (5964)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.
c-bata HideakiImamura nabenabe0928

4.2.0

This is the release note of [v4.2.0](https://github.com/optuna/optuna/releases/tag/v4.2.0). In conjunction with the Optuna release, OptunaHub 0.2.0 is released. Please refer to [the release note of OptunaHub 0.2.0](https://github.com/optuna/optunahub/releases/tag/v0.2.0) for more details.

Highlights of this release include:

- 🚀gRPC Storage Proxy for Scalable Hyperparameter Optimization
- 🤖 SMAC3: Support for New State-of-the-art Optimization Algorithm by AutoML.org (automl)
- 📁 OptunaHub Now Supports Benchmark Functions
- 🧑‍💻 Gaussian Process-Based Bayesian Optimization with Inequality Constraints
- 🧑‍💻 c-TPE: Support Constrained TPESampler

Highlights

gRPC Storage Proxy for Scalable Hyperparameter Optimization


The gRPC storage proxy is a feature designed to support large-scale distributed optimization. As shown in the diagram below, gRPC storage proxy sits between the optimization workers and the database server, proxying the calls of Optuna’s storage APIs.

&lt;img width=&quot;1241&quot; alt=&quot;grpc-proxy&quot; src=&quot;https://github.com/user-attachments/assets/00220bc6-75da-4ee3-b9ff-a2b3440ac207&quot; /&gt;


In large-scale distributed optimization settings where hundreds to thousands of workers are operating, placing a gRPC storage proxy for every few tens can significantly reduce the load on the RDB server which would otherwise be a single point of failure. The gRPC storage proxy enables sharing the cache about Optuna studies and trials, which can further mitigate load. Please refer to [the official documentation](https://optuna.readthedocs.io/en/latest/reference/generated/optuna.storages.GrpcStorageProxy.html#optuna.storages.GrpcStorageProxy) for further details on how to utilize gRPC storage proxy.

SMAC3: Random Forest-Based Bayesian Optimization Developed by AutoML.org
[SMAC3](https://github.com/automl/SMAC3) is a hyperparameter optimization framework developed by [AutoML.org](http://automl.org/), one of the most influential AutoML research groups. The Optuna-compatible SMAC3 sampler is now available thanks to the contribution to OptunaHub by Difan Deng (dengdifan), one of the core members of AutoML.org. We can now use the method widely used in AutoML research and real-world applications from Optuna.

python
pip install optunahub smac
import optuna
import optunahub
from optuna.distributions import FloatDistribution

def objective(trial: optuna.Trial) -&gt; float:
 x = trial.suggest_float(&quot;x&quot;, -10, 10)
 y = trial.suggest_float(&quot;y&quot;, -10, 10)
 return x**2 + y**2

smac_mod = optunahub.load_module(&quot;samplers/smac_sampler&quot;)
n_trials = 100
sampler = smac_mod.SMACSampler(
 {&quot;x&quot;: FloatDistribution(-10, 10), &quot;y&quot;: FloatDistribution(-10, 10)},
 n_trials=n_trials,
)
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=n_trials)


Please refer to https://hub.optuna.org/samplers/smac_sampler/ for more details.

OptunaHub Now Supports Benchmark Functions
Benchmarking the performance of optimization algorithms is an essential process indispensable to the research and development of algorithms. The newly added OptunaHub Benchmarks in the latest version v0.2.0 of [optunahub](https://hub.optuna.org/) is a new feature for Optuna users to conduct benchmarks conveniently.

python
pip install optunahub&gt;=4.2.0 scipy torch
import optuna
import optunahub

bbob_mod = optunahub.load_module(&quot;benchmarks/bbob&quot;)
smac_mod = optunahub.load_module(&quot;samplers/smac_sampler&quot;)
sphere2d = bbob_mod.Problem(function_id=1, dimension=2)

n_trials = 100
studies = []
for study_name, sampler in [
 (&quot;random&quot;, optuna.samplers.RandomSampler(seed=1)),
 (&quot;tpe&quot;, optuna.samplers.TPESampler(seed=1)),
 (&quot;cmaes&quot;, optuna.samplers.CmaEsSampler(seed=1)),
 (&quot;smac&quot;, smac_mod.SMACSampler(sphere2d.search_space, n_trials, seed=1)),
]:
 study = optuna.create_study(directions=sphere2d.directions,
     sampler=sampler, study_name=study_name)
 study.optimize(sphere2d, n_trials=n_trials)
 studies.append(study)

optuna.visualization.plot_optimization_history(studies).show()


In the above sample code, we compare and display the performance of the four kinds of samplers using a two-dimensional Sphere function, which is part of a group of benchmark functions widely used in the black-box optimization research community known as [Blackbox Optimization Benchmarking (BBOB)](https://hub.optuna.org/benchmarks/bbob/).

&lt;img width=&quot;1250&quot; alt=&quot;bbob&quot; src=&quot;https://github.com/user-attachments/assets/89b09090-dd37-4679-9fa8-af987f138dcc&quot; /&gt;


Gaussian Process-Based Bayesian Optimization with Inequality Constraints
We worked on its extension and adapted `GPSampler` to constrained optimization in Optuna v4.2.0 since Gaussian process-based Bayesian optimization is a very popular method in various research fields such as aircraft engineering and materials science. We show the basic usage below.

python
pip install optuna&gt;=4.2.0 scipy torch
import numpy as np
import optuna

def objective(trial: optuna.Trial) -&gt; float:
 x = trial.suggest_float(&quot;x&quot;, 0.0, 2 * np.pi)
 y = trial.suggest_float(&quot;y&quot;, 0.0, 2 * np.pi)
 c = float(np.sin(x) * np.sin(y) + 0.95)
 trial.set_user_attr(&quot;c&quot;, c)
 return float(np.sin(x) + y)

def constraints(trial: optuna.trial.FrozenTrial) -&gt; tuple[float]:
 return (trial.user_attrs[&quot;c&quot;],)

sampler = optuna.samplers.GPSampler(constraints_func=constraints)
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=50)


Please try out `GPSampler` for constrained optimization especially when only a small number of trials are available!

c-TPE: Support Constrained TPESampler

![c-TPE](https://github.com/user-attachments/assets/03bfbbed-063e-4781-9589-88408189a3be)

Although Optuna has supported constrained optimization for `TPESampler`, which is the default Optuna sampler, since v3.0.0, its algorithm design and performance comparison have not been verified academically. OptunaHub now supports [c-TPE](https://arxiv.org/abs/2211.14411), which is another constrained optimization method for `TPESampler`. Importantly, the algorithm design and its performance comparison are publicly reviewed to be accepted to IJCAI, a top-tier AI international conference. Please refer to https://hub.optuna.org/samplers/ctpe/ for details.

New Features

- Enable `GPSampler` to support constraint functions (5715)
- Update output format options in CLI to include the `value` choice (5822, thanks iamarunbrahma!)
- Add gRPC storage proxy server and client (5852)

Enhancements

- Introduce client-side cache in `GrpcStorageProxy` (5872)

Bug Fixes

- Fix CI (https://github.com/optuna/optuna-integration/pull/185)
- Fix ticks in Matplotlib contour plot (5778, thanks sulan!)
- Adding check in `cli.py` to handle an empty database (5828, thanks willdavidson05!)
- Avoid the input validation fail in Wilcoxon signed ranked test for Scipy 1.15 (5912)
- Fix the default sampler of `load_study` function (5924)

Documentation

- Update OptunaHub example in README (5763)
- Update `distributions.rst` to list deprecated distribution classes (5764)
- Remove deprecation comment for `step` in `IntLogUniformDistribution` (5767)
- Update requirements for OptunaHub in README (5768)
- Use inline code rather than italic for `step` (5769)
- Add notes to `ask_and_tell` tutorial - batch optimization recommendations (5817, thanks SimonPop!)
- Fix the explanation of returned values of `get_trial_params` (5820)
- Introduce `sphinx-notfound-page` for better 404 page (5898)
- Follow-up 5872: Update the docstring of `run_grpc_proxy_server` (5914)
- Modify doc-string of gRPC-related modules (5916)

Examples

- Adapt docker recipes to Python 3.11 (https://github.com/optuna/optuna-examples/pull/292)
- Add version constraint for `wandb` (https://github.com/optuna/optuna-examples/pull/293)

Pin optuna to latest version 5.0.0

b295250

Navigation Menu

Pin optuna to latest version 5.0.0 - #138

Pin optuna to latest version 5.0.0#138
pyup-bot wants to merge 1 commit into
masterfrom
pyup-pin-optuna-5.0.0

pyup-bot commented Sep 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

pyup-bot commented Sep 7, 2026

5.0.0rc1

4.9.0

4.8.0

4.7.0

4.6.0

4.5.0

4.4

4.4.0

4.3.0

4.2.1

4.2.0

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant