Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Substantial slowdowns with scipy 1.15 #2668

Open
Balandat opened this issue Jan 4, 2025 · 0 comments
Open

[Bug] Substantial slowdowns with scipy 1.15 #2668

Balandat opened this issue Jan 4, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@Balandat
Copy link
Contributor

Balandat commented Jan 4, 2025

🐛 Bug

I have observed substantial slowdowns (at least 4X in some cases) in the tutorial runs (both for smoke test and regular execution) when using scipy 1.15.

It's not clear what causes this, but I was unable to reproduce this on my M1 Mac so it seems to be tied to something in the CI setup - possibly something with underlying numerical linalg libraries.

Runtimes in Nightly on 01/03/25 (using scipy 1.14.1):

Run python scripts/run_tutorials.py -p "$(pwd)"
Running tutorial(s) in standard mode.
This may take a long time...
Running tutorial GIBBON_for_efficient_batch_entropy_search.ipynb.
Running tutorial GIBBON_for_efficient_batch_entropy_search.ipynb took 19.36 seconds. Memory usage started at 46.421875 MB and the maximum was 1113.890625 MB.
Running tutorial Multi_objective_multi_fidelity_BO.ipynb.
Running tutorial Multi_objective_multi_fidelity_BO.ipynb took 149.58 seconds. Memory usage started at 46.421875 MB and the maximum was 777.238285 MB.
Running tutorial batch_mode_cross_validation.ipynb.
Running tutorial batch_mode_cross_validation.ipynb took 5.26 seconds. Memory usage started at 46.421875 MB and the maximum was 543.8984375 MB.
Running tutorial baxus.ipynb.
Running tutorial baxus.ipynb took 978.33 seconds. Memory usage started at 46.421875 MB and the maximum was 3127.25390625 MB.
Running tutorial bo_with_warped_gp.ipynb.
Running tutorial bo_with_warped_gp.ipynb took 237.89 seconds. Memory usage started at 46.421875 MB and the maximum was 556.265625 MB.
Running tutorial bope.ipynb.
Running tutorial bope.ipynb took 52 seconds. Memory usage started at 46.421875 MB and the maximum was 606.328125 MB.

Runtimes in Nightly on 01/04/25 (using scipy 1.15):

Run python scripts/run_tutorials.py -p "$(pwd)"
Running tutorial(s) in standard mode.
This may take a long time...
Running tutorial GIBBON_for_efficient_batch_entropy_search.ipynb.
Running tutorial GIBBON_for_efficient_batch_entropy_search.ipynb took 39.89 seconds. Memory usage started at 46.37109375 MB and the maximum was 1115.0 MB.
Running tutorial Multi_objective_multi_fidelity_BO.ipynb.
Running tutorial Multi_objective_multi_fidelity_BO.ipynb took 483.36 seconds. Memory usage started at 46.37109375 MB and the maximum was 772.97265625 MB.
Running tutorial batch_mode_cross_validation.ipynb.
Running tutorial batch_mode_cross_validation.ipynb took 6.00 seconds. Memory usage started at 46.37109375 MB and the maximum was 557.734375 MB.
Running tutorial baxus.ipynb.
Tutorial baxus.ipynb exceeded the maximum runtime of 30 minutes.
Running tutorial bo_with_warped_gp.ipynb.
Tutorial bo_with_warped_gp.ipynb exceeded the maximum runtime of 30 minutes.
Running tutorial bope.ipynb.
Running tutorial bope.ipynb took 219.32 seconds. Memory usage started at 46.37109375 MB and the maximum was 655.171875 MB.
@Balandat Balandat added the bug Something isn't working label Jan 4, 2025
Balandat added a commit to Balandat/botorch that referenced this issue Jan 4, 2025
There appear to be substantial slowdowns with the optimization (presumably both for fitting and candidate generation) with scipy 1.15: pytorch#2668
This pins the scipy version to <1.15 for now to avoid these slowdowns and terrible user experiences (at least on the `main` branch) until we've figured out the root cause of these slowdowns.
facebook-github-bot pushed a commit that referenced this issue Jan 4, 2025
Summary:
There appear to be substantial slowdowns with the optimization (presumably both for fitting and candidate generation) with scipy 1.15: #2668

This pins the scipy version to <1.15 for now to avoid these slowdowns and terrible user experiences (at least on the `main` branch) until we've figured out the root cause of these slowdowns.

Pull Request resolved: #2669

Reviewed By: saitcakmak

Differential Revision: D67826874

Pulled By: Balandat

fbshipit-source-id: 9562cb8ea8f25cc51165931fa4335191e6e4b34c
jduerholt added a commit to experimental-design/bofire that referenced this issue Jan 15, 2025
BoTorch is slowed down massively by scipy 1.15: pytorch/botorch#2668. We should fix it.
bertiqwerty pushed a commit to experimental-design/bofire that referenced this issue Jan 15, 2025
BoTorch is slowed down massively by scipy 1.15: pytorch/botorch#2668. We should fix it.
dlinzner-bcs pushed a commit to experimental-design/bofire that referenced this issue Jan 20, 2025
BoTorch is slowed down massively by scipy 1.15: pytorch/botorch#2668. We should fix it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant