SR-Forge

Classification¶

Binary classification metrics, applied per element.

Anywhere a model answers yes/no about each pixel — semantic segmentation, change detection, anomaly and defect detection, cloud or shadow masking, water and road extraction, any foreground/background split. Nothing here is specific to a task; it is specific to the shape of the question.

They live under "classification" rather than "segmentation" or "detection" because that is what they are: F1, precision, recall and MCC do not know they are looking at an image, and would behave identically on a flat array of yes/no decisions. torchmetrics draws the same line — these are in its classification package, while its segmentation package holds the metrics that genuinely need region or boundary structure (generalized Dice, Hausdorff distance), and its detection package is box-based and unrelated.

Everything is built from the four confusion counts, and the counts are the honest primitive: they are additive, so they survive pooling over an epoch and across ranks without any weighting, whatever size the images are. The ratios (F1, precision, recall, IoU, MCC) are functions of those counts, which is why none of them can be a per-pixel quantity — see :meth:pointwise on the count classes for the one thing here that genuinely is.

Micro or macro. Every ratio takes aggregate (a :class:Metric option, like on any other metric):

"micro" (default) pools every element seen in the epoch and computes the ratio once. This is what the literature usually reports.
"macro" scores each image and averages.

They are not close whenever the positive class is rare or absent from some images. An image containing no positives at all scores 0 under macro however well the model behaves, so macro is dominated by those images; measured gaps of tens of percent are ordinary on sparse masks. That case is the norm in change detection and in defect or anomaly detection, and common in segmentation of small structures. Pick deliberately; the name appears in your logs either way.

Input convention. x is a score or probability map, thresholded at threshold; y is the target, treated as positive where >= 0.5. Deliberately not logits: a model emitting multi-channel logits applies its own softmax or argmax upstream, so this module holds no assumption about how a particular network spells its output.

`Counts` `dataclass` ¶

The four confusion cells, as attributes rather than dictionary keys.

Passed to :meth:ConfusionRatio.ratio. Per image under aggregate="macro"; already totalled over the epoch under "micro" — the same object either way, so a ratio is written once.

Attributes are used instead of string keys deliberately. c.tpp raises immediately and names the attribute; c["tpp"] would raise a KeyError somewhere downstream, or with a mistyped declaration would quietly compute the wrong thing. All four are always populated, so there is no second place to declare which ones a metric uses and no way for that declaration to drift out of step with the formula.

`predicted_positive: torch.Tensor` `property` ¶

TP + FP — everything the model flagged.

`actual_positive: torch.Tensor` `property` ¶

TP + FN — everything that really changed.

`total: torch.Tensor` `property` ¶

Every pixel looked at.

`ConfusionMetric` ¶

Bases: Metric

Thresholding shared by every metric here.

Rarely subclassed directly — reach for :class:ConfusionCount or :class:ConfusionRatio, which build on it. Exposed because :meth:predicted and :meth:actual are what a subclass of either uses to turn score maps into the boolean masks it reasons about.

`ConfusionCount` ¶

Bases: ConfusionMetric

A confusion cell, reported as a total over the epoch.

Implemented at Level 2 — :meth:pointwise returns a 0/1 indicator map — for two reasons. Masking then comes from the framework rather than from four hand-written copies of the same masking code. And a count is the one quantity in this module that genuinely is per-pixel: every pixel falls into exactly one of the four cells, independently of every other pixel.

Because a finalized Level-2 metric receives {"sum", "count"}, the sum of the indicator map is exactly the number of pixels in this cell, and summing that across images and ranks needs no weighting.

Writing your own means one method returning a boolean map::

@register_class
class PredictedPositives(ConfusionCount):
    # Every pixel the model flagged positive, right or wrong.
    @property
    def best_min(self) -> bool:
        return False

    def indicator(self, x, y):
        return self.predicted(x)

`TruePositives` ¶

Bases: ConfusionCount

Positive pixels the model found.

`FalsePositives` ¶

Bases: ConfusionCount

Pixels flagged positive that were not — false alarms.

`FalseNegatives` ¶

Bases: ConfusionCount

Positive pixels the model missed.

`TrueNegatives` ¶

Bases: ConfusionCount

Negative pixels correctly left alone.

Usually the overwhelming majority when the positive class is sparse, which is why plain accuracy is uninformative on such data and F1 or MCC are reported instead.

`ConfusionRatio` ¶

Bases: ConfusionMetric

A ratio of confusion counts, micro- or macro-averaged.

Level 1 rather than Level 2: a ratio of sums has no per-pixel value. The mean of a per-pixel 'IoU' is the true-positive rate, not IoU — the same reason :class:~srforge.metrics.regression.PSNR has no pointwise.

Writing your own is one method::

@register_class
class Specificity(ConfusionRatio):
    # TN / (TN + FP): of the negative pixels, how many were
    # correctly left alone.
    def ratio(self, c: Counts):
        return c.tn / (c.tn + c.fp + 1e-12)

:meth:ratio receives a :class:Counts — per image under aggregate="macro", already totalled over the epoch under "micro" — so the same method serves both and there is no second code path to keep in step. Micro/macro, thresholding, masking and the epoch arithmetic are all inherited.

All four cells are always available as attributes, so there is nothing to declare about which ones you use: a typo raises at the point of use instead of silently selecting the wrong counts, and a formula cannot drift out of step with a separate declaration of its inputs.

Training on it: soft=True. The yes/no decision stops any gradient, so a hard ratio cannot be a loss. With soft=True the counts use the probability itself (TP = Σ p·y, …; see :meth:_soft_counts) and the same :meth:ratio becomes differentiable — every ratio, including your own, has a trainable form. It keeps its direction: soft F1 is higher-is-better, and the training total negates it like any such metric. :class:DiceLoss, :class:JaccardLoss and :class:TverskyLoss are the common ones by name.

Parameters:

Name	Type	Description	Default
`threshold`	`float`	Score above which a pixel counts as predicted-positive. Default 0.5; refused with `soft=True`, which has no decision to make.	`None`
`smooth`	`float`	Added to numerator and denominator. `smooth=1` makes an image with nothing to find and nothing predicted score 1.0 rather than 0.0 — defensible, and the usual choice in segmentation code. Safe at any value here because the counts are summed before the ratio is taken; it is only unsafe in designs where the formula has to survive an averaging it did not choose. Default 0, or 1 with `soft=True`: without it a batch with no positives scores a constant `0 / FP`, with no gradient to reduce the false alarms.	`None`
`soft`	`bool`	Count with probabilities instead of decisions, for training. The default name becomes `Soft<Class>` (`SoftF1Score`) so the logs never show a soft value under the hard metric's name; an explicit `name` is kept as given.	`False`
`**kwargs`	`Any`	Passed to :class:`Metric` — in particular `aggregate`: `"micro"` (default) pools every pixel and computes the ratio once; `"macro"` scores each image and averages.	`{}`

`ratio(c: Counts) -> torch.Tensor` ¶

Combine the four cells into this metric's value.

`Precision` ¶

Bases: ConfusionRatio

TP / (TP + FP) — of the pixels flagged positive, how many were right.

`Recall` ¶

Bases: ConfusionRatio

TP / (TP + FN) — of the positive pixels, how many were found.

`F1Score` ¶

Bases: ConfusionRatio

2TP / (2TP + FP + FN) — the harmonic mean of precision and recall.

Identical to the Dice coefficient for binary masks: 2|A∩B|/(|A|+|B|) is the same expression. They are one metric under two names, so reporting both is reporting the same number twice.

`IoU` ¶

Bases: ConfusionRatio

TP / (TP + FP + FN) — intersection over union, a.k.a. Jaccard.

A monotone function of :class:F1Score (IoU = F1 / (2 - F1)), so it ranks models identically. Worth reporting because it is conventional, not because it is independent evidence.

`MCC` ¶

Bases: ConfusionRatio

Matthews correlation coefficient, in [-1, 1].

Uses all four cells, which is why it is the metric of choice for heavily imbalanced masks — sparse defects, rare changes, thin structures. Unlike F1 it cannot be inflated by ignoring the negative class, and 0 means "no better than chance" rather than "no overlap".

`TverskyIndex` ¶

Bases: ConfusionRatio

TP / (TP + α·FP + β·FN) — F1 and IoU with the two errors priced apart.

alpha prices false alarms, beta missed positives. α = β = 0.5 is :class:F1Score and α = β = 1 is :class:IoU; raising beta above alpha makes a missed change cost more than a false alarm, the usual choice when positives are rare. To train on it see :class:TverskyLoss.

Parameters:

Name	Type	Description	Default
`alpha`	`float`	Weight of false positives.	`0.5`
`beta`	`float`	Weight of false negatives.	`0.5`
`**kwargs`	`Any`	`threshold`, `smooth` and the :class:`Metric` options.	`{}`

`DiceLoss` ¶

Bases: F1Score

Soft Dice for training — exactly F1Score(soft=True).

Dice is F1 on masks; this is its trainable form, counted with probabilities (see :class:ConfusionRatio). Higher is better, like F1: the training total maximises it. Report the hard :class:F1Score.

Parameters:

Name	Type	Description	Default
`**kwargs`	`Any`	As :class:`F1Score` — `smooth` (default 1), `aggregate` (`"micro"`: batch Dice, or `"macro"`) and the :class:`Metric` options.	`{}`

`JaccardLoss` ¶

Bases: IoU

Soft IoU (Jaccard) for training — exactly IoU(soft=True).

Parameters:

Name	Type	Description	Default
`**kwargs`	`Any`	As :class:`IoU` — `smooth` (default 1), `aggregate` and the :class:`Metric` options.	`{}`

`TverskyLoss` ¶

Bases: TverskyIndex

Soft Tversky index for training — exactly TverskyIndex(soft=True).

Parameters:

Name	Type	Description	Default
`alpha`	`float`	Weight of false alarms.	`0.5`
`beta`	`float`	Weight of missed positives. `beta > alpha` (e.g. 0.3 / 0.7) pushes recall up when positives are rare.	`0.5`
`**kwargs`	`Any`	As :class:`TverskyIndex` — `smooth` (default 1), `aggregate` and the :class:`Metric` options.	`{}`

`BinaryCrossEntropy` ¶

Bases: Metric

Per-pixel binary cross-entropy: −[y·log p + (1−y)·log(1−p)].

The per-pixel loss for a yes/no map — change, defect, cloud, water. x is the probability of "yes" (a sigmoid output), y the target in [0, 1] with the same shape; soft targets are fine.

Commonly summed with :class:DiceLoss: cross-entropy gives every pixel a steady gradient, Dice keeps a rare positive class from being drowned by the background.

Parameters:

Name	Type	Description	Default
`class_weights`		Optional `(negative, positive)` weights multiplying the two terms — e.g. `[1.0, 10.0]` to make a missed positive ten times as costly. A constant of your data: compute it once from the class balance and put it in the config. `None` weighs both classes equally.	`None`
`eps`	`float`	Probabilities are clamped to `[eps, 1 − eps]` before the log.	`1e-07`
`**kwargs`	`Any`	:class:`Metric` options.	`{}`

`CrossEntropy` ¶

Bases: Metric

Per-pixel cross-entropy over classes: −Σ_c w_c·y_c·log p_c.

For a map with one channel per class: x holds the class probabilities (a softmax output), shape [B, C, ...] with C ≥ 2. y is either the same shape (one-hot or soft labels) or a map of class indices — [B, ...] or [B, 1, ...], integer dtype. The sum runs over the class axis, so each pixel contributes one value.

A one-channel map is refused: with one channel −y·log p has no term for the negative class, and predicting "yes" everywhere scores a perfect 0. Use :class:BinaryCrossEntropy for a yes/no map.

Parameters:

Name	Type	Description	Default
`class_weights`		Optional per-class weights, one per channel in channel order — e.g. inverse class frequency. A constant of your data: compute it once and put it in the config. The loss is the plain mean over pixels of the weighted sum, not torch's `F.cross_entropy(weight=...)` mean, which divides by the total weight of the targets instead — the two differ by that factor.	`None`
`eps`	`float`	Probabilities are clamped from below at `eps` before the log.	`1e-12`
`**kwargs`	`Any`	:class:`Metric` options.	`{}`

Classification¶

Counts dataclass ¶

predicted_positive: torch.Tensor property ¶

actual_positive: torch.Tensor property ¶

total: torch.Tensor property ¶

ConfusionMetric ¶

ConfusionCount ¶

TruePositives ¶

FalsePositives ¶

FalseNegatives ¶

TrueNegatives ¶

ConfusionRatio ¶

ratio(c: Counts) -> torch.Tensor ¶

Precision ¶

Recall ¶

F1Score ¶

IoU ¶

MCC ¶

TverskyIndex ¶

DiceLoss ¶

JaccardLoss ¶

TverskyLoss ¶

BinaryCrossEntropy ¶

CrossEntropy ¶

`Counts` `dataclass` ¶

`predicted_positive: torch.Tensor` `property` ¶

`actual_positive: torch.Tensor` `property` ¶

`total: torch.Tensor` `property` ¶

`ConfusionMetric` ¶

`ConfusionCount` ¶

`TruePositives` ¶

`FalsePositives` ¶

`FalseNegatives` ¶

`TrueNegatives` ¶

`ConfusionRatio` ¶

`ratio(c: Counts) -> torch.Tensor` ¶

`Precision` ¶

`Recall` ¶

`F1Score` ¶

`IoU` ¶

`MCC` ¶

`TverskyIndex` ¶

`DiceLoss` ¶

`JaccardLoss` ¶

`TverskyLoss` ¶

`BinaryCrossEntropy` ¶

`CrossEntropy` ¶