lir.experiments package

class lir.experiments.Experiment(name: str, outputs: Sequence[Aggregation], output_path: Path)[source]

Bases: ABC

Representation of an experiment pipeline run for each provided LR system.

Parameters:

  • name (str) – Name used to identify this object in outputs and logs.

  • outputs (Sequence[Aggregation]) – Output aggregation definitions executed after each run.

  • output_path (Path) – Path where generated outputs are written.

run() → None[source]

Run experiment the configured experiment(s).

This method ensures that all outputs are properly closed after the experiment run.

class lir.experiments.OptunaExperiment(name: str, data_config: ContextAwareDict, outputs: Sequence[Aggregation], output_path: Path, baseline_config: ContextAwareDict, hyperparameters: list[Hyperparameter], n_trials: int, metric_function: Callable[[LLRData], float])[source]

Bases: Experiment

Representation of an experiment run for each provided LR system.

Parameters:

  • name (str) – Name used to identify this object in outputs and logs.

  • data_config (ContextAwareDict) – Data configuration used to construct datasets for runs.

  • outputs (Sequence[Aggregation]) – Output aggregation definitions executed after each run.

  • output_path (Path) – Path where generated outputs are written.

  • baseline_config (ContextAwareDict) – Baseline configuration to be tuned during optimisation.

  • hyperparameters (list[Hyperparameter]) – Hyperparameters varied during optimisation.

  • n_trials (int) – Number of optimisation trials to execute.

  • metric_function (Callable[[LLRData], float]) – Value passed via metric_function.

class lir.experiments.PredefinedExperiment(name: str, data_configs: list[tuple[ContextAwareDict, dict[str, Any]]], outputs: Sequence[Aggregation], output_path: Path, lrsystem_configs: list[tuple[ContextAwareDict, dict[str, Any]]])[source]

Bases: Experiment

Representation of an experiment run for each provided LR system.

Parameters:

  • name (str) – Name used to identify this object in outputs and logs.

  • data_configs (list[tuple[ContextAwareDict, dict[str, Any]]]) – Data configurations evaluated by this experiment.

  • outputs (Sequence[Aggregation]) – Output aggregation definitions executed after each run.

  • output_path (Path) – Path where generated outputs are written.

  • lrsystem_configs (list[tuple[ContextAwareDict, dict[str, Any]]]) – LR-system configurations evaluated by this experiment.

Submodules