Skip to content

modernize hyperparameter search example#960

Open
OrestisLomis wants to merge 2 commits intomasterfrom
modernize-hyperparam
Open

modernize hyperparameter search example#960
OrestisLomis wants to merge 2 commits intomasterfrom
modernize-hyperparam

Conversation

@OrestisLomis
Copy link
Copy Markdown
Contributor

Two points:

  • Not sure if the CPM_<solver>(model) pattern should indeed be updated to cp.SolverLookup.get(<solver>, model), but it made sense to me to update it
  • Also in the 4 years since this example was added CPMpy/OR-Tools has gotten way faster which led to the difference between best and worse being neglible, so I decided to up the number of queens a bit.

@OrestisLomis
Copy link
Copy Markdown
Contributor Author

I guess a third, and more important point is: should the whole example not be updated to use the ParameterTuner class?

Copy link
Copy Markdown
Collaborator

@IgnaceBleukx IgnaceBleukx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would increase the number of queens a little more to 75.
Best to exclude this example from the test_examples (there is an exclude list in that file somewhere).

We should also point to our actual hyperparameter tuning tool in the docstring of this file.
E.g.,

CPMpy also includes a fully-fledged hyperparameter tuner, located in `cpmpy.tools.tune_solver`.

Or actually, shouldn't we just rework this example to use the built-in tuner? I feel that makes more sense as an example on how to use the library.

model = nqueens(n=25)
model = nqueens(n=50)
# flatten once upfront, reduces overhead of multiple solves
model = flatten_model(model)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is from when flatten was the only transformation for ortools...
I would actually just remove this. The overhead should be negligable, and we don't count it in the runtime anyway (only solve time there)

from cpmpy.solvers import param_combinations
from cpmpy.transformations.flatten_model import flatten_model

def main():
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give model as arg?
Also function name could be more informative. Like "tune_parameters" or something.

@OrestisLomis
Copy link
Copy Markdown
Contributor Author

Indeed I agree it makes more sense to update the whole example to reflect how the ParameterTuner class should be used. Please have a look at the issue I opened regarding this class, because I feel like it's actually a bit outdated/undersupported #963.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants