Update Acquisition.optimize to use mixed optimizer if discrete parameters have high cardinality #3253
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ters have high cardinality Summary: `optimize_acqf_discrete_local_search` is not suitable for parameters with high cardinality, since it generates the neighbors by changing the value of one parameter at a time and trying out all possible values for that parameter. This can quickly get very expensive with high cardinality parameters. This diff updates the dispatch logic to utilize the mixed alternating optimizer in cases with high cardinality discrete parameters. The optimizer will internally use continuous relaxation for high cardinality parameters while using nearest neighbor search for the low cardinality discrete parameters. Reviewed By: bernardbeckerman Differential Revision: D68357373
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D68357373 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3253 +/- ##
==========================================
- Coverage 95.87% 95.87% -0.01%
==========================================
Files 513 513
Lines 51712 51717 +5
==========================================
+ Hits 49581 49585 +4
- Misses 2131 2132 +1 ☔ View full report in Codecov by Sentry. |
saitcakmak
added a commit
to saitcakmak/Ax
that referenced
this pull request
Jan 22, 2025
…ters have high cardinality (facebook#3253) Summary: Pull Request resolved: facebook#3253 `optimize_acqf_discrete_local_search` is not suitable for parameters with high cardinality, since it generates the neighbors by changing the value of one parameter at a time and trying out all possible values for that parameter. This can quickly get very expensive with high cardinality parameters. This diff updates the dispatch logic to utilize the mixed alternating optimizer in cases with high cardinality discrete parameters. The optimizer will internally use continuous relaxation for high cardinality parameters while using nearest neighbor search for the low cardinality discrete parameters. Reviewed By: bernardbeckerman, Balandat Differential Revision: D68357373
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
optimize_acqf_discrete_local_searchis not suitable for parameters with high cardinality, since it generates the neighbors by changing the value of one parameter at a time and trying out all possible values for that parameter. This can quickly get very expensive with high cardinality parameters.This diff updates the dispatch logic to utilize the mixed alternating optimizer in cases with high cardinality discrete parameters. The optimizer will internally use continuous relaxation for high cardinality parameters while using nearest neighbor search for the low cardinality discrete parameters.
Reviewed By: bernardbeckerman
Differential Revision: D68357373