Runner

The Runner is used to automatically do closed-loop optimization. NeptuneRunner can connect to the Neptune experiment tracker

class summit.run.Runner(strategy: summit.strategies.base.Strategy, experiment: summit.experiment.Experiment, num_initial_experiments=None, max_iterations=100, batch_size=1, f_tol=1e-05, max_same=None, max_restarts=0, **kwargs)[source]

Run a closed-loop strategy and experiment cycle

Parameters

  • strategy (Strategy) – The summit strategy to be used. Note this should be an object (i.e., you need to call the strategy and then pass it). This allows you to add any transforms, options in advance.

  • experiment (Experiment) – The experiment or benchmark class to use for running experiments

  • max_iterations (int, optional) – The maximum number of iterations to run. By default this is None.

  • num_initial_experiments (int, optional) – Number of initial experiments to run, if different than batch size. Default is to start with batch size.

  • batch_size (int, optional) – The number experiments to request at each call of strategy.suggest_experiments.

  • f_tol (float, optional) – How much difference between successive best objective values will be tolerated before stopping. This is generally useful for nonglobal algorithms like Nelder-Mead. Default is None.

  • max_same (int, optional) – The number of allowed iterations where the objectives don’t improve by more than f_tol. Default is None.

  • max_restarts (int, optional) – Number of restarts if max_same where is violated. Default is 0.

Examples

>>> from summit import *
>>> benchmark = SnarBenchmark()
>>> strategy = Random(benchmark.domain)
>>> r = Runner(strategy=strategy, experiment=benchmark, max_iterations=10)
>>> # Turn progress bar on by setting to True below
>>> r.run(progress_bar=False)
run(**kwargs)[source]

Run the closed loop experiment cycle

Parameters

  • prev_res (DataSet, optional) – Previous results to initialize the optimization

  • save_freq (int, optional) – The frequency with which to checkpoint the state of the optimization. Defaults to None.

  • save_at_end (bool, optional) – Save the state of the optimization at the end of a run, even if it is stopped early. Default is False.

  • save_dir (str, optional) – The directory to save checkpoints locally. Defaults to not saving locally.

class summit.run.NeptuneRunner(strategy: summit.strategies.base.Strategy, experiment: summit.experiment.Experiment, neptune_project: str, neptune_experiment_name: str, neptune_tags: Optional[list] = None, neptune_description: Optional[str] = None, neptune_files: Optional[list] = None, hypervolume_ref=None, **kwargs)[source]

Run a closed-loop strategy and experiment cycle with logging to Neptune

Parameters

  • strategy (Strategy) – The summit strategy to be used. Note this should be an object (i.e., you need to call the strategy and then pass it). This allows you to add any transforms, options in advance.

  • experiment (Experiment) – The experiment or benchmark class to use for running experiments

  • neptune_project (str) – The name of the Neptune project to log data to

  • neptune_experiment_name (str) – A name for the neptune experiment

  • netpune_description (str, optional) – A description of the neptune experiment

  • files (list, optional) – A list of filenames to save to Neptune

  • max_iterations (int, optional) – The maximum number of iterations to run. By default this is 100.

  • batch_size (int, optional) – The number experiments to request at each call of strategy.suggest_experiments. Default is 1.

  • f_tol (float, optional) – How much difference between successive best objective values will be tolerated before stopping. This is generally useful for nonglobal algorithms like Nelder-Mead. Default is None.

  • max_same (int, optional) – The number of iterations where the objectives don’t improve by more than f_tol. Default is max_iterations.

  • max_restarts (int, optional) – Number of restarts if f_tol is violated. Default is 0.

  • hypervolume_ref (array-like, optional) – The reference for the hypervolume calculation if it is a multiobjective problem. Should be an array of length the number of objectives. Default is at the origin.

run(**kwargs)[source]

Run the closed loop experiment cycle

Parameters

  • save_freq (int, optional) – The frequency with which to checkpoint the state of the optimization. Defaults to None.

  • save_at_end (bool, optional) – Save the state of the optimization at the end of a run, even if it is stopped early. Default is False.

  • save_dir (str, optional) – The directory to save checkpoints locally. Defaults to ~/.summit/runner.