lir.config.experiment_strategies module

class lir.config.experiment_strategies.ExperimentStrategyConfigParser[source]

Bases: ConfigParser, ABC

Base class for an experiment strategy configuration parser.

data_config() → tuple[ContextAwareDict, list[Hyperparameter]][source]

Prepare the data provider and data strategy from the configuration.

The (hyper)parameters to vary for the data provider and data strategy are also parsed.

Returns:: Data configuration and associated hyperparameters.
Return type:: tuple[ContextAwareDict, list[Hyperparameter]]

abstractmethod get_experiment(name: str) → Experiment[source]

Get an experiment by name.

Parameters:: name (str) – Experiment name.
Returns:: Experiment instance configured by this strategy.
Return type:: Experiment

lrsystem_config() → tuple[ContextAwareDict, list[Hyperparameter]][source]

Parse the LR system section including hyperparameters.

The baseline configuration is provided along with the specified parameters to vary (the defined hyperparameters).

Returns:: LR system configuration and associated hyperparameters.
Return type:: tuple[ContextAwareDict, list[Hyperparameter]]

output_list() → Sequence[Aggregation][source]

Initialize corresponding aggregation classes based on the output section.

The initialized aggregation classes are returned as a sequence, to be iterated over in a later stage.

Returns:: Aggregation objects used to collect experiment outputs.
Return type:: Sequence[Aggregation]

parse(config: ContextAwareDict, output_dir: Path) → Experiment[source]

Parse the experiment section of the configuration.

Parameters:

config (ContextAwareDict) – Experiment strategy configuration.
output_dir (Path) – Base output directory for this experiment.

Returns:

Parsed experiment.

Return type:

Experiment

primary_metric() → Callable[source]

Parse the primary_metric field.

Returns:: Metric function used for optimisation.
Return type:: Callable

class lir.config.experiment_strategies.GridStrategy[source]

Bases: ExperimentStrategyConfigParser

Prepare Experiment consisting of multiple runs using configuration values.

get_experiment(name: str) → Experiment[source]

Get experiment for a grid search strategy.

Parameters:: name (str) – Experiment name.
Returns:: Predefined experiment with all parameter combinations.
Return type:: Experiment

class lir.config.experiment_strategies.OptunaStrategy[source]

Bases: ExperimentStrategyConfigParser

Prepare Experiment for optimizing configuration parameters.

get_experiment(name: str) → Experiment[source]

Get experiment for an Optuna optimisation strategy.

Parameters:: name (str) – Experiment name.
Returns:: Optuna-backed experiment.
Return type:: Experiment

class lir.config.experiment_strategies.SingleRunStrategy[source]

Bases: ExperimentStrategyConfigParser

Prepare Experiment consisting of a single run using configuration values.

get_experiment(name: str) → Experiment[source]

Get an experiment for a single run.

Parameters:: name (str) – Experiment name.
Returns:: Predefined single-run experiment.
Return type:: Experiment

lir.config.experiment_strategies.create_configs_from_hyperparameters(baseline_config: ContextAwareDict, parameters: list[Hyperparameter]) → list[tuple[ContextAwareDict, dict[str, Any]]][source]

Create configurations for all combinations of hyperparameter options.

Generates a Cartesian product of all hyperparameter options and creates a configuration for each combination by substituting the values into the baseline configuration.

This is used for both dataparameters and lrsystem hyperparameters in grid search.

Parameters:

baseline_config (ContextAwareDict) – Baseline configuration to augment.
parameters (list[Hyperparameter]) – Hyperparameters to vary.

Returns:

Augmented configurations and applied substitutions.

Return type:

list[tuple[ContextAwareDict, dict[str, Any]]]

lir.config.experiment_strategies.parse_experiment_strategy(config: ContextAwareDict, output_path: Path) → Experiment[source]

Instantiate the corresponding experiment strategy class, e.g. for a single or grid run.

A corresponding Experiment class is returned.

Parameters:

config (ContextAwareDict) – Experiment strategy configuration.
output_path (Path) – Output path for experiment artefacts.

Returns:

Parsed experiment strategy instance.

Return type:

Experiment

lir.config.experiment_strategies.parse_experiments(cfg: ContextAwareDict, output_path: Path) → Mapping[str, Experiment][source]

Extract which Experiment to run as dictated in the configuration.

Parameters:

cfg (ContextAwareDict) – Configuration section describing experiments.
output_path (Path) – Filesystem path to the results directory.

Returns:

Mapping from experiment name to parsed experiment.

Return type:

Mapping[str, Experiment]