lir.plotting package

Submodules

lir.plotting.expected_calibration_error module

Empirical Cross Entrpy (ECE).

The discrimination and calibration of the LRs reported by some systems can also be measured separately. The empirical cross entropy (ECE) plot is a graphical way of doing this.

The ECE is the average of -P(Hp) * log2(P(Hp|LRi)) for all LRi when Hp is true, and -P(Hd) * log2(P(Hd|LRi)) for all LRi when Hd is true.

See: [-] D. Ramos, Forensic evidence evaluation using automatic speaker recognition

systems. Ph.D. Thesis. Universidad Autonoma de Madrid.

[-] Bernard Robertson, G.A. Vignaux and Charles Berger, Interpreting Evidence:

Evaluating Forensic Science in the Courtroom, 2nd edition, 2016, pp. 96-97.

lir.plotting.expected_calibration_error.calculate_ece(lrs: ndarray, y: ndarray, priors: ndarray) → ndarray

Calculate empirical cross-entropy (ECE) of a set of LRs and corresponding ground-truth labels.

An entropy is calculated for each element of priors.

Parameters:

  • lrs – an array of LRs

  • y – an array of ground-truth labels of the LRs (values 0 for Hd or 1 for Hp); must be of the same length as lrs.

  • priors – an array of prior probabilities of the samples being drawn from class 1 (values in range [0..1])

Returns:

an array of entropy values of the same length as priors

lir.plotting.expected_calibration_error.plot_ece(llrdata: LLRData, log_prior_odds_range: tuple[float, float]=(-3, 3), ax: Any = <module 'matplotlib.pyplot' from '/home/runner/work/lir/lir/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py'>, show_pav: bool = True, ylim: str = 'neutral') → None

Generate an ECE plot for a set of LRs and corresponding ground-truth labels.

The x-axis indicates the log prior odds of a sample being drawn from class 1; the y-axis shows the entropy for (1) a non-informative system (dotted line), (2) the set of LR values (line), and (3) the set of LR values after PAV-transformation (Pool Adjacent Violators, dashed line).

Parameters:

  • llrdata – the LLR data containing LLRs and labels.

  • log_prior_odds_range – the range of prior odds (tuple of two values, indicating both ends of the range on the x-axis)

  • ax – the matplotlib axis to plot on

  • show_pav – whether to show the PAV-transformed LRs in the plot

  • ylim –

    the y-axis limits. Valid values are: - ‘neutral’: starts at 0, and ends automatically. In practice, this means that the upper limit is set slightly

    above the maximum of the ‘non-informative’ reference.

    • ’zoomed’: starts at 0 and ends slightly (10%) above the maximum ECE value of the LRs. This may cut off part

      of the ‘non-informative’ reference line.

Module contents

class lir.plotting.Canvas(ax: Axes)

Bases: object

Representation of an empty canvas, to be used in plotting multiple visualizations.

lir.plotting.axes(savefig: PathLike | None = None, show: bool | None = None) → Iterator[Canvas]

Create a plotting context.

Example

```py with axes() as ax:

ax.pav(lrs, y)

```

lir.plotting.llr_interval(llrdata: LLRData, ax: Axes = <module 'matplotlib.pyplot' from '/home/runner/work/lir/lir/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py'>) → None

Plot the lr’s on the x-as, with the relative interval score on the y-as.

Parameters:

  • llrdata (LLRData) – The LLRData object containing the likelihood ratios and interval scores.

  • ax (axes to plot figure to)

lir.plotting.lr_histogram(llrdata: LLRData, bins: int = 20, weighted: bool = True, ax: Axes = <module 'matplotlib.pyplot' from '/home/runner/work/lir/lir/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py'>) → None

Plot the 10log lrs.

Parameters:

  • llrs (the likelihood ratios)

  • y (a numpy array of labels (0 or 1))

  • bins (number of bins to divide scores into)

  • weighted (if y-axis should be weighted for frequency within each class)

  • ax (axes to plot figure to)

lir.plotting.pav(llrdata: LLRData, add_misleading: int = 0, show_scatter: bool = True, ax: Axes = <module 'matplotlib.pyplot' from '/home/runner/work/lir/lir/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py'>) → None

Generate a plot of pre-calibrated versus post-calibrated LRs using Pool Adjacent Violators (PAV).

Parameters:

  • llrs (numpy array of floats) – Likelihood ratios before PAV transform

  • y (numpy array) – Labels corresponding to lrs (0 for Hd and 1 for Hp)

  • add_misleading (int) – number of misleading evidence points to add on both sides (default: 0)

  • show_scatter (boolean) – If True, show individual LRs (default: True)

  • ax (pyplot axes object) – defaults to matplotlib.pyplot

  • ----------

lir.plotting.savefig(path: str) → _GeneratorContextManager[Canvas]

Create a plotting context, write plot when closed.

Example

```py with savefig(filename) as ax:

ax.pav(lrs, y)

```

A call to savefig(path) is identical to axes(savefig=path).

Parameters:

path (str) – write a PNG image to this path

lir.plotting.score_distribution(scores: ndarray, y: ndarray, bins: int = 20, weighted: bool = True, ax: Axes | None = None) → None

Plot the distributions of scores calculated by the (fitted) lr_system.

If weighted is True, the y-axis represents the probability density within the class, and inf is the fraction of instances. Otherwise, the y-axis shows the number of instances.

Parameters:

  • scores (scores of (fitted) lr_system (1d-array))

  • y (a numpy array of labels (0 or 1, 1d-array of same length as scores))

  • bins (number of bins to divide scores into)

  • weighted (if y-axis should be the probability density within each class,) – instead of counts

  • ax (axes to plot figure to)

lir.plotting.show() → _GeneratorContextManager[Canvas]

Create a plotting context, show plot when closed.

Example

```py with show() as ax:

ax.pav(lrs, y)

```

A call to show() is identical to axes(show=True).

lir.plotting.tippett(llrdata: LLRData, plot_type: int = 1, ax: Axes = <module 'matplotlib.pyplot' from '/home/runner/work/lir/lir/.venv/lib/python3.12/site-packages/matplotlib/pyplot.py'>) → None

Plot empirical cumulative distribution functions of same-source and different-sources lrs.

Parameters:

  • llrs (the likelihood ratios)

  • y (a numpy array of labels (0 or 1))

  • plot_type (an integer, must be either 1 or 2.) – In type 1 both curves show proportion of lrs greater than or equal to the x-axis value, while in type 2 the curve for same-source shows the proportion of lrs smaller than or equal to the x-axis value.

  • ax (axes to plot figure to)