Panoptica

Submodules

panoptica.instance_approximator module

Approximate instances from semantic segmentations via connected components.

class panoptica.instance_approximator.ConnectedComponentsInstanceApproximator(cca_backend: CCABackend | None = None)

Bases: InstanceApproximator

Instance approximator using connected components algorithm for panoptic segmentation evaluation.

cca_backend

The connected components algorithm backend.

Type:

CCABackend

__init__(self, cca_backend

CCABackend) -> None: Initialize the ConnectedComponentsInstanceApproximator.

_approximate_instances(self, semantic_pair

SemanticPair, **kwargs) -> UnmatchedInstancePair: Approximate instances using the connected components algorithm.

Example: >>> cca_approximator = ConnectedComponentsInstanceApproximator(cca_backend=CCABackend.cc3d) >>> semantic_pair = SemanticPair(…) >>> result = cca_approximator.approximate_instances(semantic_pair)

_abc_impl = <_abc._abc_data object>
_approximate_instances(semantic_pair: SemanticPair, **kwargs) → UnmatchedInstancePair

Approximate instances using the connected components algorithm.

Parameters:

  • semantic_pair (SemanticPair) – The semantic pair to be approximated.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance approximation.

Return type:

UnmatchedInstancePair

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

class panoptica.instance_approximator.InstanceApproximator

Bases: SupportsConfig

Abstract base class for instance approximation algorithms in panoptic segmentation evaluation.

None
_approximate_instances(self, semantic_pair

SemanticPair, **kwargs) -> UnmatchedInstancePair | MatchedInstancePair: Abstract method to be implemented by subclasses for instance approximation.

approximate_instances(self, semantic_pair

SemanticPair, **kwargs) -> UnmatchedInstancePair | MatchedInstancePair: Perform instance approximation on the given SemanticPair.

Raises:

ValueError – If there are negative values in the semantic maps, which is not allowed.

Example: >>> class CustomInstanceApproximator(InstanceApproximator): … def _approximate_instances(self, semantic_pair: SemanticPair, **kwargs) -> UnmatchedInstancePair | MatchedInstancePair: … # Implementation of instance approximation algorithm … pass … >>> approximator = CustomInstanceApproximator() >>> semantic_pair = SemanticPair(…) >>> result = approximator.approximate_instances(semantic_pair)

_abc_impl = <_abc._abc_data object>
abstract _approximate_instances(semantic_pair: SemanticPair, label_group: LabelGroup | None = None, **kwargs) → UnmatchedInstancePair | MatchedInstancePair

Abstract method to be implemented by subclasses for instance approximation.

Parameters:

  • semantic_pair (SemanticPair) – The semantic pair to be approximated.

  • label_group (LabelGroup | None, optional) – Information about the label group being processed. Defaults to None.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance approximation.

Return type:

UnmatchedInstancePair | MatchedInstancePair

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

approximate_instances(semantic_pair: SemanticPair, verbose: bool = False, label_group: LabelGroup | None = None, **kwargs) → UnmatchedInstancePair | MatchedInstancePair

Perform instance approximation on the given SemanticPair.

Parameters:

  • semantic_pair (SemanticPair) – The semantic pair to be approximated.

  • label_group (LabelGroup | None, optional) – Information about the label group being processed. Defaults to None.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance approximation.

Return type:

UnmatchedInstancePair | MatchedInstancePair

Raises:

ValueError – If there are negative values in the semantic maps, which is not allowed.

class panoptica.instance_approximator.OneHotConnectedComponentsInstanceApproximator(cca_backend: CCABackend | None = None)

Bases: InstanceApproximator

Instance approximator that first applies one-hot encoding to the prediction and reference arrays, then runs connected components on each channel and merges the results.

_abc_impl = <_abc._abc_data object>
_approximate_instances(semantic_pair: SemanticPair, label_group: LabelGroup | None = None) → UnmatchedInstancePair

Abstract method to be implemented by subclasses for instance approximation.

Parameters:

  • semantic_pair (SemanticPair) – The semantic pair to be approximated.

  • label_group (LabelGroup | None, optional) – Information about the label group being processed. Defaults to None.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance approximation.

Return type:

UnmatchedInstancePair | MatchedInstancePair

_one_hot(arr)
classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

panoptica.instance_evaluator module

Per-instance metric evaluation for matched instance pairs.

class panoptica.instance_evaluator._InstanceEvaluation(metrics: dict[~panoptica.metrics.metrics.Metric, float] = <factory>, voxel_count_ref: int = 0, volume_ref: float = 0.0)

Bases: object

Result of evaluating a single matched reference instance.

metrics

Per-metric scores keyed by Metric. Empty dict signals no overlap (the instance is filtered out by the caller’s if not metrics check).

Type:

dict[panoptica.metrics.metrics.Metric, float]

voxel_count_ref

Raw voxel count of the reference instance (np.count_nonzero of the cropped reference mask).

Type:

int

volume_ref

Physical volume of the reference instance, computed as voxel_count_ref * prod(voxelspacing).

Type:

float

metrics: dict[Metric, float]
volume_ref: float = 0.0
voxel_count_ref: int = 0
panoptica.instance_evaluator._evaluate_instance(reference_arr: ndarray, prediction_arr: ndarray, ref_idx: int, eval_metrics: list[Metric], voxelspacing: tuple[float, ...] | None = None, processing_pair_orig_shape: tuple[int, ...] | None = None, n_ref_labels: int | None = None, instance_slice: tuple[slice, ...] | None = None) → _InstanceEvaluation

Evaluate a single instance.

Parameters:

  • ref_labels (np.ndarray) – Reference instance segmentation mask.

  • pred_labels (np.ndarray) – Predicted instance segmentation mask.

  • ref_idx (int) – The label of the current instance.

  • iou_threshold (float) – The IoU threshold for considering a match.

Returns:

Per-metric scores, raw voxel count of the reference instance, and physical volume (voxel count * prod(voxelspacing)). If the reference label is absent from reference_arr (voxel_count_ref == 0), the result has empty metrics and zero count/volume. If the reference is present but the prediction has no voxels for this label, the result has empty metrics but still carries the reference’s true voxel count and volume, so the caller can record the ref as unmatched with its actual size.

Return type:

_InstanceEvaluation

panoptica.instance_evaluator._extended_voxelspacing(voxelspacing: tuple[float, ...], ndim: int)

Match a voxelspacing to a spatial array, padding non-spatial (label) axes.

Flattened one-hot arrays gain leading label axes when reshaped to (num_labels + 1, *spatial_shape); spatial metrics need a spacing entry per axis, so unit spacing is prepended for those extra dimensions.

panoptica.instance_evaluator._union_instance_slice(ref_slices: list, pred_slices: list, label: int, shape: tuple[int, ...], px_pad: int = 2) → tuple[slice, ...] | None

Padded union of a label’s reference and prediction bounding boxes.

ref_slices / pred_slices are scipy.ndimage.find_objects outputs (indexed by label - 1, None where the label is absent). Returns a per-axis slice tuple that bounds the instance in both arrays plus px_pad voxels (so the downstream _get_paired_crop reproduces exactly the crop it would compute on the full array), or None when the label is absent from both.

panoptica.instance_evaluator.evaluate_matched_instance(matched_instance_pair: MatchedInstancePair, eval_metrics: list[Metric] | None = None, decision_metric: Metric | None = Metric.IOU, decision_threshold: float | None = None, voxelspacing: tuple[float, ...] | None = None, processing_pair_orig_shape: tuple[int, ...] | None = None, n_ref_labels: int | None = None, speed_toggles: PanopticaSpeedToggles | None = None, **kwargs) → EvaluateInstancePair

Evaluate a given MatchedInstancePair given metrics and decision threshold.

Parameters:

  • processing_pair (MatchedInstancePair) – The matched instance pair containing original labels.

  • labelmap (Instance_Label_Map) – The instance label map obtained from instance matching.

Returns:

Evaluated pair of instances

Return type:

EvaluateInstancePair

panoptica.instance_matcher module

Algorithms for matching predicted instances to reference instances.

class panoptica.instance_matcher.InstanceMatchingAlgorithm

Bases: SupportsConfig

Abstract base class for instance matching algorithms in panoptic segmentation evaluation.

_match_instances(self, unmatched_instance_pair

UnmatchedInstancePair, context: MatchingContext = None, **kwargs) -> InstanceLabelMap: Abstract method to be implemented by subclasses for instance matching.

match_instances(self, unmatched_instance_pair

UnmatchedInstancePair, **kwargs) -> MatchedInstancePair: Perform instance matching on the given UnmatchedInstancePair.

Example: >>> class CustomInstanceMatcher(InstanceMatchingAlgorithm): … def _match_instances(self, unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext = None, **kwargs) -> InstanceLabelMap: … # Implementation of instance matching algorithm … pass … >>> matcher = CustomInstanceMatcher() >>> unmatched_instance_pair = UnmatchedInstancePair(…) >>> result = matcher.match_instances(unmatched_instance_pair)

_abc_impl = <_abc._abc_data object>
_calculate_matching_metric_pairs(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None, matching_metric: Metric) → list[tuple[float, tuple[int, int]]]

Calculate matching metric pairs based on context.

Parameters:

  • unmatched_instance_pair – The unmatched instance pair.

  • context – The matching context. If None, defaults to non-part group behavior.

  • matching_metric – The metric to use for matching.

Returns:

List of (matching_score, (ref_label, pred_label)) tuples.

abstract _match_instances(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None = None, **kwargs) → InstanceLabelMap

Abstract method to be implemented by subclasses for instance matching.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • context (Optional[MatchingContext]) – Context information for matching. If None, a default context will be created.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

InstanceLabelMap

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

match_instances(unmatched_instance_pair: UnmatchedInstancePair, label_group=None, n_ref_labels=None, processing_pair_orig_shape=None, **kwargs) → MatchedInstancePair

Perform instance matching on the given UnmatchedInstancePair.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • label_group – The label group object for this group.

  • n_ref_labels – Number of reference labels.

  • processing_pair_orig_shape – Original shape of the processing pair.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

MatchedInstancePair

class panoptica.instance_matcher.MatchingContext(label_group: LabelGroup | None = None, n_ref_labels: int | None = None, processing_pair_orig_shape: tuple | None = None)

Bases: object

Encapsulates context information needed for matching operations.

property is_part_group: bool

Check if this context represents a part group.

label_group: LabelGroup | None = None
n_ref_labels: int | None = None
processing_pair_orig_shape: tuple | None = None
class panoptica.instance_matcher.MaxBipartiteMatching(matching_metric: Metric = Metric.IOU, matching_threshold: float = 0.5, strict_threshold: bool = False)

Bases: ThresholdBasedMatching

Instance matching algorithm that performs optimal one-to-one matching based on maximum bipartite graph matching.

This implementation maximizes the global matching score between predictions and references.

_abc_impl = <_abc._abc_data object>
_create_cost_matrix(ref_labels: list[int], pred_labels: list[int], mm_pairs: list[tuple[float, tuple[int, int]]], matching_threshold: float) → ndarray

Create cost matrix for bipartite matching.

_match_instances(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None = None, *, matching_threshold: float, **kwargs) → InstanceLabelMap

Perform optimal instance matching based on maximum bipartite graph matching.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • context (Optional[MatchingContext]) – The matching context.

  • matching_threshold (float) – The threshold for matching instances.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

InstanceLabelMap

_solve_bipartite_matching(cost_matrix: ndarray, ref_labels: list[int], pred_labels: list[int]) → InstanceLabelMap

Solve the bipartite matching problem and return labelmap.

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

class panoptica.instance_matcher.MaximizeMergeMatching(matching_metric: Metric = Metric.IOU, matching_threshold: float = 0.5, strict_threshold: bool = False)

Bases: ThresholdBasedMatching

Instance matching algorithm that performs many-to-one matching based on metric. Will merge if combined instance metric is greater than individual one. Only matches if at least a single instance exceeds the threshold.

matching_metric

The metric to be used for matching.

Type:

Metric

matching_threshold

The threshold for matching instances.

Type:

float

_abc_impl = <_abc._abc_data object>
_match_instances(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None = None, *, matching_threshold: float, **kwargs) → InstanceLabelMap

Perform many-to-one instance matching based on metric values.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • context (Optional[MatchingContext]) – The matching context.

  • matching_threshold (float) – The threshold for matching instances.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

InstanceLabelMap

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

new_combination_score(pred_labels: list[int], new_pred_label: int, ref_label: int, unmatched_instance_pair: UnmatchedInstancePair)
class panoptica.instance_matcher.NaiveThresholdMatching(matching_metric: Metric = Metric.IOU, matching_threshold: float = 0.5, allow_many_to_one: bool = False, strict_threshold: bool = False)

Bases: ThresholdBasedMatching

Instance matching algorithm that performs threshold-based matching.

matching_metric

The metric used for matching.

Type:

Metric

matching_threshold

The threshold for matching instances.

Type:

float

allow_many_to_one

Whether to allow many-to-one matching.

Type:

bool

_abc_impl = <_abc._abc_data object>
_match_instances(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None = None, *, matching_threshold: float, **kwargs) → InstanceLabelMap

Perform threshold-based instance matching.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • context (Optional[MatchingContext]) – The matching context.

  • matching_threshold (float) – The threshold for matching instances.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

InstanceLabelMap

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

class panoptica.instance_matcher.ThresholdBasedMatching(matching_metric: Metric = Metric.IOU, matching_threshold: float = 0.5, strict_threshold: bool = False)

Bases: InstanceMatchingAlgorithm

Base class for matchers that rely on a metric and a cutoff threshold.

_abc_impl = <_abc._abc_data object>
abstract _match_instances(unmatched_instance_pair: UnmatchedInstancePair, context: MatchingContext | None = None, *, matching_threshold: float, **kwargs) → InstanceLabelMap

Abstract method to be implemented by subclasses for instance matching.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • context (Optional[MatchingContext]) – Context information for matching. If None, a default context will be created.

  • matching_threshold (float) – The threshold to use for matching instances in this operation.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

InstanceLabelMap

classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

match_instances(unmatched_instance_pair: UnmatchedInstancePair, label_group=None, n_ref_labels=None, processing_pair_orig_shape=None, *, matching_threshold: float | None = None, **kwargs) → MatchedInstancePair

Perform instance matching on the given UnmatchedInstancePair.

Parameters:

  • unmatched_instance_pair (UnmatchedInstancePair) – The unmatched instance pair to be matched.

  • label_group – The label group object for this group.

  • n_ref_labels – Number of reference labels.

  • processing_pair_orig_shape – Original shape of the processing pair.

  • **kwargs – Additional keyword arguments.

Returns:

The result of the instance matching.

Return type:

MatchedInstancePair

panoptica.instance_matcher.map_instance_labels(processing_pair: UnmatchedInstancePair, labelmap: InstanceLabelMap) → MatchedInstancePair

Map instance labels based on the provided labelmap and create a MatchedInstancePair.

Parameters:

  • processing_pair (UnmatchedInstancePair) – The unmatched instance pair containing original labels.

  • labelmap (InstanceLabelMap) – The instance label map obtained from instance matching.

Returns:

The result of mapping instance labels.

Return type:

MatchedInstancePair

panoptica.panoptica_aggregator module

Thread-safe aggregation of per-subject evaluation results into a TSV/JSONL file.

class panoptica.panoptica_aggregator.Panoptica_Aggregator(panoptica_evaluator: Panoptica_Evaluator, output_file: Path | str, log_times: bool = False, continue_file: bool = True, file_type: Literal['tsv', 'jsonl'] = 'jsonl', output_individual_instance_metrics: bool = False, is_autc: bool = False, threshold_step_size: float | None = None)

Bases: object

Aggregator that manages evaluations and saves resulting metrics per sample.

This class interfaces with the Panoptica_Evaluator to perform evaluations, store results, and manage file outputs for statistical analysis.

__exist_handler()

Handles cleanup upon program exit by removing the temporary output buffer file.

evaluate(prediction_arr: ndarray, reference_arr: ndarray, subject_name: str, voxelspacing: tuple[float, ...] | None = None, **kwargs)

Evaluates a single case using the provided prediction and reference arrays.

Parameters:

  • prediction_arr (np.ndarray) – The array containing the predicted segmentation.

  • reference_arr (np.ndarray) – The array containing the ground truth segmentation.

  • subject_name (str) – A unique name for the sample being evaluated. If none is provided, a name will be generated based on the count.

Raises:

ValueError – If the subject name has already been evaluated or is in process.

property evaluation_metrics
make_statistic() → Panoptica_Statistic

Generates statistics from the aggregated evaluation results.

Returns:

The statistics object containing the results.

Return type:

Panoptica_Statistic

property panoptica_evaluator
panoptica.panoptica_aggregator._append_buffer_entries(file: str | Path, entries: list[str]) → None

Appends subject names to the buffer file. NOT THREAD SAFE BY ITSELF.

panoptica.panoptica_aggregator._load_buffer_entries(file: str | Path) → list[str]

Loads buffer file entries (one subject name per row).

NOT THREAD SAFE BY ITSELF. The buffer file is the aggregator’s scratch file for tracking which subjects are in-flight or already done — it is not the user-visible output file.

Raises:

ValueError – If the buffer file contains duplicate entries.

panoptica.panoptica_evaluator module

Top-level Panoptica_Evaluator orchestrating the approximation -> matching -> evaluation pipeline.

class panoptica.panoptica_evaluator.Panoptica_Evaluator(expected_input: InputType = InputType.MATCHED_INSTANCE, instance_approximator: InstanceApproximator | None = None, instance_matcher: InstanceMatchingAlgorithm | None = None, edge_case_handler: EdgeCaseHandler | None = None, segmentation_class_groups: SegmentationClassGroups | None = None, instance_metrics: list[Metric] | None = None, global_metrics: list[Metric] | None = None, decision_metric: Metric | None = None, decision_threshold: float | None = None, per_region_evaluation: bool = False, save_group_times: bool = False, log_intermediate_steps: bool = False, log_times: bool = False, verbose: bool = False, speed_toggles: PanopticaSpeedToggles | None = None)

Bases: SupportsConfig

_evaluate_group(group_name: str, label_group: LabelGroup, processing_pair, decision_threshold: float | None = None, matching_threshold: float | None = None, result_all: bool = True, verbose: bool | None = None, log_intermediate_steps: bool = False, log_times: bool | None = None, save_group_times: bool = False, preprocess_time: float = 0.0, **kwargs) → PanopticaResult
_get_dummy_result() → PanopticaResult

Helper method to generate a blank evaluation for extracting dynamic metric keys.

_preprocess_input(prediction_arr: str | Path | ndarray | torch.Tensor | nib.nifti1.Nifti1Image | sitk.Image, reference_arr: str | Path | ndarray | torch.Tensor | nib.nifti1.Nifti1Image | sitk.Image, voxelspacing: tuple[float, ...] | None = None) → tuple[MatchedInstancePair | UnmatchedInstancePair | SemanticPair, dict]

Handles data ingestion, sanity checking, and initial validation.

_resolve_skip_groups(skip_groups: list[str] | None) → set[str]

Validate skip_groups and return it as a set of group names to skip.

Unknown names (e.g. typos) are dropped with a warning rather than raising, so a stale skip list never silently turns into “skip everything” or an error.

_set_instance_approximator(instance_approximator: InstanceApproximator)
_set_instance_matcher(matcher: InstanceMatchingAlgorithm)
classmethod _yaml_repr(node) → dict

Abstract method for representing the class in YAML.

Parameters:

node – The object instance to represent in YAML.

Returns:

A dictionary representation of the class.

Return type:

dict

evaluate(**kwargs)
evaluate_autc(**kwargs)
static generate_thresholds(step_size: float) → list[float]

Return AUTC threshold steps within the inclusive range [step_size, 1].

get_autc_metric_keys(threshold_step_size: float) → list[str]

Must produce keys in exactly the same order as PanopticaAUTCResult.to_dict().

get_resulting_metric_keys(output_individual_instance_metrics: bool = False) → list[str]
property resulting_metric_keys: list[str]
property segmentation_class_groups_names: list[str]
set_log_group_times(should_save: bool)

panoptica.panoptica_result module

The PanopticaResult container and its lazy, dependency-aware metric computation.

class panoptica.panoptica_result.PanopticaAUTCResult(threshold_results: dict[float, PanopticaResult])

Bases: object

Holds dict mapping thresholds across a range to PanopticaResult objects. Computes Area Under The Threshold Curve (AUTC) for metrics from different thresholds. Pads the left boundary to x=0.0 using nearest-neighbor so the integral always covers the full [0.0, 1.0] range regardless of step_size.

get_autc(metric_name: str) → float

Computes Area Under the Threshold Curve

NaN / uncomputable values are treated as 0.0.

Raises:

  • ValueError – if fewer than two thresholds were stored.

  • AttributeError – if metric_name is not a valid PanopticaResult field.

get_result_at_threshold(threshold: float) → PanopticaResult

Return the PanopticaResult that was evaluated at threshold.

Uses np.isclose for float-safe comparison.

Raises:

ValueError – if no result was stored for that threshold.

property threshold_results: dict[float, PanopticaResult]
property thresholds: list[float]
to_dict(output_individual_instance_metrics: bool = False) → dict[str, float]

Flat dictionary containing AUTC metrics AND individual threshold metrics.

AUTC is only computed for continuous ratio metrics that have a bounded domain from 0 to 1.

class panoptica.panoptica_result.PanopticaResult(reference_arr: ndarray, prediction_arr: ndarray, n_pred_instances: int, n_ref_instances: int, tp: int, list_metrics: dict[Metric, list[float]], edge_case_handler: EdgeCaseHandler, global_metrics: list[Metric] | None = None, processing_pair_orig_shape: tuple[int, int] | None = None, n_ref_labels: int | None = None, label_group: LabelGroup | None = None, intermediate_steps_data: IntermediateStepsData | None = None, computation_time: float | None = None, phase_times: dict[str, float] | None = None, instance_voxel_count_matched_ref: list[int] | None = None, instance_volume_matched_ref: list[float] | None = None, instance_voxel_count_unmatched_ref: list[int] | None = None, instance_volume_unmatched_ref: list[float] | None = None, **kwargs)

Bases: object

ROW_KEY_TO_MASTER_KEY: dict[str, str] = {'volume': 'instance_volume_ref', 'voxel_count': 'instance_voxel_count_ref'}
_add_metric(name_id: str, metric_type: MetricType, calc_func: Callable | None, long_name: str | None = None, default_value=None, was_calculated: bool = False, lower_bound: float | None = None, upper_bound: float | None = None)

Adds a new metric to the evaluation metrics.

Parameters:

  • name_id (str) – The unique identifier for the metric.

  • metric_type (MetricType) – The type of the metric.

  • calc_func (Callable | None) – The function to calculate the metric.

  • long_name (str | None) – A longer, descriptive name for the metric.

  • default_value – The default value for the metric.

  • was_calculated (bool) – Indicates if the metric has been calculated.

Returns:

The default value of the metric.

_calc(k, v)

Attempts to get the value of a metric and captures any exceptions.

Parameters:

  • k – The metric key.

  • v – The metric value.

Returns:

A tuple indicating success or failure and the corresponding value or exception.

_calc_global_bin_metric(metric: Metric, prediction_arr, reference_arr, do_binarize: bool = True)

Calculates a global binary metric based on predictions and references. For multi-channel data (LabelPartGroup), computes metrics per channel and averages.

Parameters:

  • metric (Metric) – The metric to compute.

  • prediction_arr – The predicted values.

  • reference_arr – The ground truth values.

  • do_binarize (bool) – Whether to binarize the input arrays. Defaults to True.

Returns:

The calculated metric value or mean of channel metrics for multi-channel data.

Raises:

MetricCouldNotBeComputedException – If the specified metric is not set.

_calc_metric(metric_name: str, supress_error: bool = False)

Calculates a specific metric by its name.

Parameters:

  • metric_name (str) – The name of the metric to calculate.

  • supress_error (bool) – If true, suppresses errors during calculation.

Returns:

The calculated metric value or raises an exception if it cannot be computed.

Raises:

MetricCouldNotBeComputedException – If the metric cannot be found.

_register_instance_metrics() → None

Register the per-instance sq/sq_std/pq metrics from _INSTANCE_METRIC_SPECS.

For each metric this adds sq_{suffix} (mean of the per-instance scores) and sq_{suffix}_std (their std); unit-interval overlap metrics additionally get pq_{suffix} = sq * rq. Lambda default args bind the loop variables so each closure captures its own metric/sq-key.

property autc_metrics: list[str]
calculate_all(print_errors: bool = False, phase_timer: PhaseTimer | None = None)

Calculates all possible metrics that can be derived.

Parameters:

  • print_errors (bool, optional) – If true, will print every metric that could not be computed and its reason. Defaults to False.

  • phase_timer (PhaseTimer | None, optional) – If provided, each metric’s evaluation duration is recorded under f"metric_{name}" in phase_timer.times.

property evaluation_metrics
get_channel_metrics(metric_name: str)

Returns the metrics for each channel when using LabelPartGroup.

Parameters:

metric_name (str) – Name of the metric (lowercase)

Returns:

Dictionary of metric values per channel or None if not computed with LabelPartGroup

get_list_metric(metric: Metric, mode: MetricMode)

Retrieves a list of metrics based on the given metric type and mode.

Parameters:

  • metric (Metric) – The metric to retrieve.

  • mode (MetricMode) – The mode of the metric.

Returns:

The corresponding list of metrics.

Raises:

MetricCouldNotBeComputedException – If the metric cannot be found.

classmethod normalize_row_to_master_schema(row: dict) → dict

Rewrite a reference_instances row dict so its keys match the master schema.

recall_by_volume(thresholds: list[float], volume: str = 'volume') → dict[str, float]

Instance detection recall stratified by reference-instance volume.

Single-sample counterpart of Panoptica_Statistic.recall_by_volume: bins this result’s reference instances by size and reports the matched fraction per bin — useful to see whether small instances are detected as well as large ones. Matched (TP) references count as detected, unmatched (FN) references as missed.

Parameters:

  • thresholds – User-supplied volume bin edges (e.g. [160, 271, 451]); n thresholds give n + 1 bins rec_q0 .. rec_qn.

  • volume – Which size to bin on — "volume" (physical volume, default) or "voxel_count" (raw voxel counts).

Returns:

{"rec_q0": ..., ...}, one entry per bin (nan for an empty bin).

Return type:

dict[str, float]

to_dict(output_individual_instance_metrics: bool = False) → dict

Converts the metrics to a dictionary format.

When output_individual_instance_metrics is False, returns a flat dict of subject-level (master) metrics.

When True, returns the same master dict augmented with one extra key:

  • "reference_instances": list of per-instance dicts, one per reference instance. Matched instances appear first, followed by unmatched (FN) instances. Each row carries "is_matched" (1 or 0) — this flag is row-only and is not mirrored on master. Rows also carry "voxel_count", "volume", and, for matched rows, the per-instance segmentation-quality metrics (sq_iou, sq_dsc, …). The _ref suffix is dropped inside rows because all rows live under reference_instances already; file backends translate them back to master keys via normalize_row_to_master_schema.

panoptica.panoptica_result.fn(res: PanopticaResult)
panoptica.panoptica_result.fp(res: PanopticaResult)
panoptica.panoptica_result.prec(res: PanopticaResult)
panoptica.panoptica_result.rec(res: PanopticaResult)
panoptica.panoptica_result.rq(res: PanopticaResult)

Calculate the Recognition Quality (RQ) based on TP, FP, and FN.

Returns:

Recognition Quality (RQ).

Return type:

float

panoptica.panoptica_statistics module

Aggregate statistics and plots computed over many PanopticaResult evaluations.

class panoptica.panoptica_statistics.FloatDistribution(value_list: list[float])

Bases: object

property avg: float
get_string_repr(ndigits: int = 3)
property max: float
property min: float
property std: float
property values: list[float]
z_score(value: float) → float

Calculates the z-score of a value based on the summary statistics.

class panoptica.panoptica_statistics.Panoptica_Statistic(subj_names: list[str], value_dict: dict[str, dict[str, list[float | None]]])

Bases: object

_assertgroup(group)
_assertmetric(metric)
_assertsubject(subjectname)
_remove_subject(subjectname)
_resolve_single_group(group: str | None) → str

Validate group, or pick the only group when group is omitted.

property base_metric_names: list[str]

Returns metric names that are not thresholded

classmethod from_file(file: str | Path, verbose: bool = True, file_type: Literal['tsv', 'jsonl'] = 'jsonl')

Loads a Panoptica_Statistic from a results file produced by Panoptica_Aggregator.

Dispatches to the appropriate backend based on the file extension.

Parameters:

  • file (str | Path) – Path to a TSV or JSONL results file.

  • verbose (bool, optional) – If True, prints a short summary of the metrics/groups discovered. Defaults to True.

  • file_type (FileType, optional) – Format used when file has no extension. An explicit .tsv / .jsonl suffix on file always takes precedence. Defaults to "jsonl".

Returns:

Statistic populated from the file.

Return type:

Panoptica_Statistic

Raises:

ValueError – If the resolved path has an unsupported extension, or if the file contains no records (e.g. header-only TSV or empty JSONL produced by an aborted run).

get(group: str, metric: str, remove_nones: Literal[True]) → list[float]
get(group: str, metric: str, remove_nones: Literal[False] = False) → list[float | None]
get(group: str, metric: str, remove_nones: bool) → list[float | None]

Returns the list of values for given group and metric.

Missing values are returned as None unless remove_nones=True.

get_across_groups(metric) → list[float | None]

Given metric, gives list of all values (even across groups!) Treat with care!

Parameters:

metric (str) – Name of the metric to pool.

Returns:

All values of the metric concatenated across every group.

Return type:

list[float | None]

get_best_worst_k_entries(groups: list[str] | str | None = None, metrics: list[str] | str | None = None, k: int = 3)
get_dict(group, metric, remove_nones, sort_ascending: bool = True)
get_one_group(groupname: str)

Gets the dictionary mapping metric to values for ONE group

Parameters:

groupname (str) – Name of the group to extract.

Returns:

Metric -> list of values for this group.

Return type:

dict[str, list[float | None]]

get_one_metric(metricname: str)

Gets the dictionary mapping the group to the metrics specified

Parameters:

metricname (str) – Name of the metric to extract.

Returns:

Group -> list of values for this metric.

Return type:

dict[str, list[float | None]]

get_one_subject(subjectname: str)

Gets the values for ONE subject for each group and metric

Parameters:

subjectname (str) – Name of the subject to extract.

Returns:

Group -> metric -> value for this subject.

Return type:

dict[str, dict[str, float | None]]

get_subject_wise_difference_to(other: Panoptica_Statistic, group: str, metric: str) → dict[str, float | None]

Calculates the subject-wise difference in metric for given group to another Panoptica_Statistic object

Parameters:

  • other (Panoptica_Statistic) – The other statistic to compare against (must share subject names).

  • group (str) – Group to compare.

  • metric (str) – Metric to compare.

Returns:

Subject -> (self value minus other value); None where either is missing.

Return type:

dict[str, float | None]

get_subject_wise_paired_values_to(other: Panoptica_Statistic, group: str, metric: str) → tuple[list[str], list[float | None], list[float | None]]

Calculates the subject-wise paired values in metric for given group to another Panoptica_Statistic object

Parameters:

  • other (Panoptica_Statistic) – The other statistic to compare against (must share subject names).

  • group (str) – Group to compare.

  • metric (str) – Metric to compare.

get_summary(group, metric, master_only: bool = True) → FloatDistribution

Gets a FloatDistribution for a given group and metric. If master_only is True, ignores individual instance rows to prevent double counting.

get_summary_across_groups() → dict[str, FloatDistribution]

Calculates the average and std over all groups (so group-wise avg first, then average over those)

Returns:

Metric -> distribution of the per-group averages.

Return type:

dict[str, FloatDistribution]

get_summary_dict(include_across_group: bool = True) → dict[str, dict[str, FloatDistribution]]
get_summary_figure(metric: str, groups: list[str] | str | None = None, manual_metric_range: None | tuple[float, float] = None, name_method: str = 'Structure', horizontal: bool = True, sort: bool = True, title: str = '', master_only: bool = True)

Returns a figure object that shows the given metric for each group and its std

get_thresholds_for_metric(metric: str) → list[float]

Returns available thresholds for a specific metric (e.g. ‘pq’).

property groupnames
property instance_subjects: list[str]

Returns only the individual instance rows.

property master_subjects: list[str]

Returns only the primary subject names (ignoring instance rows).

master_values(values: list[float | None]) → list[float]

Pair each value with its subject name and filter out the instance rows.

property metricnames
print_summary(ndigits: int = 3, only_across_groups: bool = True, include_thresholds: bool = False)
recall_by_volume(thresholds: list[float], group: str | None = None, volume_metric: str = 'instance_volume_ref') → dict[str, float]

Instance detection recall stratified by reference-instance volume.

Splits reference instances into user-supplied volume bins and reports the detection recall (fraction matched) within each — e.g. to check whether a model detects small instances as well as large ones. Recall here is the standard instance/lesion-wise detection recall (matched references over all references) restricted to each volume bin.

Requires a results file written with output_individual_instance_metrics=True so the per-instance rows carrying volume_metric and is_matched exist.

Parameters:

  • thresholds – Volume bin edges given by the user (e.g. [160, 271, 451]). n thresholds define n + 1 bins (rec_q0 .. rec_qn). They are applied as-is, never estimated from the evaluated data.

  • group – Class group to evaluate. May be omitted only when the statistic has a single group.

  • volume_metric – Per-instance volume metric to bin on. Defaults to "instance_volume_ref" (physical volume; equals the voxel count under unit voxel spacing); pass "instance_voxel_count_ref" to bin on raw voxel counts.

Returns:

{"rec_q0": ..., "rec_q1": ..., ...}, one entry per bin (nan for an empty bin).

Return type:

dict[str, float]

Raises:

  • ValueError – If group is omitted but the statistic has multiple groups, or if no per-instance rows are present.

  • KeyError – If volume_metric or is_matched is not in the file.

recall_by_volume_percentiles(percentiles: list[float], group: str | None = None, volume_metric: str = 'instance_volume_ref') → dict[str, float]

Instance detection recall stratified by reference-instance percentile volumes.

Parameters:

  • percentiles (list[float]) – _description_

  • group (str | None, optional) – _description_. Defaults to None.

  • volume_metric (str, optional) – _description_. Defaults to “instance_volume_ref”.

Returns:

_description_

Return type:

dict[str, float]

property subjectnames
to_dataframe() → DataFrame

Converts the statistic to a pandas dataframe

Returns:

One row per (subject, group), with a column per metric.

Return type:

pd.DataFrame

to_file(file: str | Path, file_type: Literal['tsv', 'jsonl'] = 'jsonl') → None

Writes the full statistic to disk, format chosen from the file extension (.tsv or .jsonl). Overwrites any existing file at the resolved path; raises before touching disk if the extension is unsupported.

Parameters:

  • file (str | Path) – Path with a supported extension, or without one — in which case file_type decides the format.

  • file_type (FileType, optional) – Format used when file has no extension. An explicit .tsv / .jsonl suffix on file always takes precedence. Defaults to "jsonl".

Raises:

ValueError – If the resolved path has an unsupported extension.

class panoptica.panoptica_statistics.ValueSummary(value_list: list[float])

Bases: FloatDistribution

Deprecated alias for FloatDistribution.

panoptica.panoptica_statistics._flatten_extend(matrix)
panoptica.panoptica_statistics.make_autc_plots(statistics_dict: dict[str | int | float, Panoptica_Statistic], metric: str, groups: list[str] | str | None = None, alternate_groupnames: list[str] | str | None = None, fig: Figure | None = None, plot_std: bool = True, figure_title: str = '', width: int = 850, height: int = 1200, xaxis_title: str | None = None, yaxis_title: str | None = None, manual_metric_range: None | tuple[float, float] = None) → Figure
panoptica.panoptica_statistics.make_curve_over_setups(statistics_dict: dict[str | int | float, Panoptica_Statistic], metric: str, groups: list[str] | str | None = None, alternate_groupnames: list[str] | str | None = None, fig: Figure | None = None, plot_as_barchart=True, plot_std: bool = True, figure_title: str = '', width: int = 850, height: int = 1200, xaxis_title: str | None = None, yaxis_title: str | None = None, manual_metric_range: None | tuple[float, float] = None)
panoptica.panoptica_statistics.plot_box(data: dict[str, ndarray], sort=True, orientation_horizontal: bool = True, name_method: str = 'Structure', name_metric: str = 'Dice-Score', figure_title: str = '', width=850, height=1200, manual_metric_range: None | tuple[float, float] = None)