tuning package

Model Tuning

ModelTuneResult

class autompc.tuning.model_tuner.ModelTuneResult(inc_cfg, cfgs, inc_cfgs, costs, inc_costs)

The ModelTuneResult contains information about a tuning process.

inc_cfg

The final tuned configuration.

cfgs

List of configurations. The configuration evaluated at each tuning iteration.

costs

The cost (evaluation score) observed at each iteration of tuning iteration.

inc_cfgs

The incumbent (best found so far) configuration at each tuning iteration.

inc_costs

The incumbent score at each tuning iteration.

ModelTuner

class autompc.tuning.ModelTuner(system, evaluator)

The ModelTuner class is used for tuning system ID models based on prediction accuracy.

__init__(system, evaluator)

Parameters:

• system (System) – System for which models will be tuned

• evaluator (ModelEvaluator) – This evaluator object will be used to asses model configurations

add_model_factory(model_factory, cs=None)

Add a model factory which is an option for tuning. Multiple model factories can be added and the tuner will select between them.

Parameters:

• model_factory (ModelFactory) –

• cs (ConfigurationSpace) – Configuration space for model factory. This only needs to be passed if the configuration space is customized, otherwise it will be derived from the model_factory.

run(rng, n_iters=10)

Run tuning process

Parameters:

• rng (numpy.random.Generator) – Random number generator used for tuning

• n_iters (int) – Number of tuning iterations to run

Returns:

• model (Model) – Resulting model

• tune_result (ModelTuneResult) – Additional information from tuning process

Pipeline Tuning

PipelineTuneResult

class autompc.tuning.pipeline_tuner.PipelineTuneResult(inc_cfg, cfgs, inc_cfgs, costs, inc_costs, truedyn_costs, inc_truedyn_costs, surr_trajs, truedyn_trajs, surr_tune_result)

PipelineTuneREsult contains information about a tuning process.

inc_cfg

The final tuned configuration.

cfgs

List of configurations. The configuration evaluated at each tuning iteration.

inc_cfgs

The incumbent (best found so far) configuration at each tuning iteration.

costs

The cost observed at each iteration of tuning iteration.

inc_costs

The incumbent score at each tuning iteration.

truedyn_costs

The true dynamics cost observed at each iteration of tuning iteration. None if true dynamics was not provided.

inc_truedyn_costs

The incumbent true dynamics cost observed at each iteration of tuning iteration. None if true dynamics was not provided.

surr_trajs

The trajectory simulated at each tuning iteration with respect to the surrogate model.

truedyn_trajs

The trajectory simulated at each tuning iteration with respect to the true dynamics. None if true dynamics are not provided.

surr_tune_result

The ModelTuneResult from tuning the surrogate model, for modes “autotune” and “autoselect”. None for other tuning modes.

PipelineTuner

class autompc.tuning.PipelineTuner(surrogate_mode='defaultcfg', surrogate_factory=None, surrogate_split=None, surrogate_cfg=None, surrogate_evaluator=None, surrogate_tune_holdout=0.25, surrogate_tune_metric='rmse')

This class tunes SysID+MPC pipelines.

__init__(surrogate_mode='defaultcfg', surrogate_factory=None, surrogate_split=None, surrogate_cfg=None, surrogate_evaluator=None, surrogate_tune_holdout=0.25, surrogate_tune_metric='rmse')

Parameters:

• surrogate_mode (string) – Mode for selecting surrogate model. One of the following: “defaultcfg” - use the surrogate factories default configuration, “fixedcfg” - use the surrogate configuration passed by surrogate_cfg, “autotune” - automatically tune hyperparameters of surrogate, “autoselect” - automatically tune and select surrogate model, “pretrain” - Use an already trained surrogate which is passed when the tuning process is run.

• surrogate_factory (ModelFactory) – Factory for creating surrogate model. Required for all modes except for autoselect.

• surrogate_split (float) – Proportion of data to use for surrogate training. Required for all modes except “pretrain”

• surrogate_cfg (Configuration) – Surrogate model config, required for “fixedcfg” mode

• surrogate_evaluator (ModelEvaluator) – Evaluator to use for surrogate tuning, used for “autoselect” and “autotune” modes. If not passed, will use HoldoutEvaluator with default arguments.

• surrogate_tune_holdout (float) – Proportion of data to hold out for surrogate tuning. Used for “autotune” and “autoselect” modes.

• surrogate_tune_metric (string) – Model metric to use for surrogate tuning. Used for “autotune” and “autoselect” modes. See documentation of ModelEvaluator for more details.

run(pipeline, task, trajs, n_iters, rng, surrogate=None, truedyn=None, surrogate_tune_iters=100, special_debug=False)

Run tuning.

Parameters:

• pipeline (Pipeline) – Pipeline to tune.

• task (Task) – Task specification to tune for

• trajs (List of Trajectory) – Trajectory training set.

• n_iters (int) – Number of tuning iterations

• rng (numpy.random.Generator) – RNG to use for tuning.

• surrogate (Model) – Surrogate model to use for tuning. Used for “pretrain” mode.

• truedyn (obs, ctrl -> obs) – True dynamics function. If passed, the true dynamics cost will be evaluated for each iteration in addition to the surrogate cost. However, this information will not be used for tuning.

• surrogate_tune_iters (int) – Number of iterations to use for surrogate tuning. Used for “autotune” and “autoselect” modes. Default is 100

Returns:

• controller (Controller) – Final tuned controller.

• tune_result (PipelineTuneResult) – Additional tuning information.