SR-Forge

Metrics

This page is a manual. It starts with using the metrics that ship with SR-Forge, then explains how a metric works inside, and ends with writing your own. Each section builds on the ones before it, so read it top to bottom the first time; afterwards the recipes at the end are the fastest way back in.

Just want to train? You need very little of this page

List your metrics and connect each to your data with io:. That's it: no metric has a setting you must remember, and every option has a sensible default. What you have to set, and what you don't is the whole story in one table; the rest of the page is for when you want more.

---

1. What a metric is

In SR-Forge a metric is anything that compares tensors and produces a score: L1, PSNR, SSIM, an F1 score, a perceptual distance. Every one of them is a Metric. A loss is simply a metric you train on — there is no separate loss class.

Why one class for both? Because a training objective and an evaluation metric are the same computation used for different purposes. L1 is something you minimise during training and something you report on the validation set. Keeping them as one class means the number you train on and the number you report are computed by the same code — they cannot drift apart. Whether a metric drives the gradient (and so acts as a loss) or is only watched is a setting (its weight), not a different type.

A metric is a read-only consumer of the data flowing through your pipeline:

Dataset ──► Transforms ──► Model ──► Entry {lr, sr, hr, mask, ...} ──► Metric ──► MetricScores
                                                                        (reads,
                                                                     never writes)

It reads fields from the Entry, computes, and returns a MetricScores object holding the result. It never modifies the Entry — an evaluation that could change the data it evaluates would be a source of very confusing bugs.

---

2. Using a built-in metric

In a config

loss:
  _target: srforge.metrics.regression.L1
  io:
    inputs:
      x: sr        # the metric's `x` reads entry["sr"]
      y: hr        # the metric's `y` reads entry["hr"]

_target names the class; params: (absent here, because L1 needs none) would hold its constructor arguments; io: connects the metric to your data. See Configuration for the general syntax.

What you have to set, and what you don't

A typical training — train on L1, watch SSIM, report a torchmetrics PSNR:

loss:
  _target: srforge.metrics.LossCombiner
  params:
    losses:
      - _target: srforge.metrics.regression.L1
        io: {inputs: {x: sr, y: hr}}
      - _target: srforge.metrics.image.SSIM
        params: {weight: 0.0}                # optional: watch it, don't train on it
        io: {inputs: {x: sr, y: hr}}
      - _target: torchmetrics.image.PeakSignalNoiseRatio
        params: {data_range: 1.0}            # torchmetrics' own required argument
        io: {inputs: {x: sr, y: hr}}

You always write: io:, to connect the metric's inputs to your Entry fields — the same for every metric, like it is for models. Leave it out only if your fields are literally called x and y.

You write when the metric can't work without it:

• a wrapper's inner metric — base_metric for CorrectedLoss and UncertaintyLoss;
• losses for LossCombiner;
• whatever a library metric itself requires, like torchmetrics' data_range. That's the library's rule, not SR-Forge's.

No other built-in metric has a required argument.

Everything else is optional, the same for every metric, and has a default that suits a standard training:

Option	Default	Change it when
weight	1.0	0.0 watches a metric without training on it or letting it pick the best checkpoint
name	the class name	you want a different label in logs and charts
aggregate	macro	you want the pooled epoch value — e.g. F1 with sparse positives
reduction	mean	almost never; for a loss whose math needs a per-image sum

Get one wrong — a typo, or an option a metric can't honour — and the config is refused when it loads, before training starts, with a message listing what the metric does accept.

Why the io block exists

A metric's inputs are simply the parameters of its scoring method — any number, under any names. The built-in metrics follow a convention: x is the prediction, y is the target, and y_mask (optional) marks the valid pixels of the target. A few depart from it where the convention does not fit: TBE needs only x (it judges an image without a reference), and UncertaintyLoss adds a log_var input. Your own metrics can declare whatever inputs they need — see §9.

Either way, the metric cannot know what your pipeline calls those things — sr and hr, or pred and gt, or change_map and labels.

The io block is the translation: metric parameter → Entry field. It is the same IO binding every SR-Forge component uses, with one difference: a metric has inputs only. It produces a score, not new Entry fields, so an outputs: key is an error.

If your Entry already uses the names x and y, you can omit io — each parameter then reads the field of the same name.

In Python

The same metric, bound the same way — set_io returns the metric, so it chains:

import torch
from srforge.data import Entry
from srforge.metrics.regression import L1

loss = L1().set_io({"inputs": {"x": "sr", "y": "hr"}})

sr = torch.rand(4, 3, 64, 64)       # a batch of 4 predictions
hr = torch.rand(4, 3, 64, 64)       # and their targets

There are three ways to call it. All three give the same result:

scores = loss(Entry(sr=sr, hr=hr))  # 1. an Entry — what the trainer does
scores = loss(sr=sr, hr=hr)         # 2. keywords
scores = loss(sr, hr, None)         # 3. positional

scores["L1"].raw                    # tensor of shape [4]: one L1 per image

1. An Entry. The metric looks up the fields its io mapping names: x reads entry["sr"], y reads entry["hr"]. This is how the metric is called during training and validation.
2. Keywords. Handy in tests and notebooks. The keywords are the field names — sr and hr, the right-hand side of the mapping — not the metric's parameter names. So a metric behaves the same whether it is given an Entry or keywords, and loss(x=sr, y=hr) is an error here: no field called x was bound.
3. Positional. Tensors in the metric's own parameter order. L1's parameters are x, y, y_mask, and positional calls need every parameter, so the optional mask is passed as None.

The same metric without set_io

Here is a second L1, created without calling set_io. With no mapping, each parameter reads the field that has its own name: x reads x, y reads y. Called with the same sr and hr tensors:

plain = L1()                           # no set_io: x reads "x", y reads "y"

scores = plain(Entry(x=sr, y=hr))      # 1. the Entry's fields must be called x and y
scores = plain(x=sr, y=hr)             # 2. keywords are now x and y
scores = plain(sr, hr, None)           # 3. positional: unchanged

plain(sr=sr, hr=hr)                    # KeyError: Parameter 'x' not found!
plain(Entry(sr=sr, hr=hr))             # KeyError: Parameter 'x' not found!

The three working calls give the same scores as before. Only the names changed, because the mapping changed; positional calls never use names, so they behave identically with or without set_io.

The rule, in one sentence: a keyword — like an Entry field — must be the name the parameter reads from: the parameter's own name, unless the io mapping changed it. Mapping only some parameters changes only those: after set_io({"inputs": {"x": "sr"}}), the call is loss(sr=sr, y=hr).

---

3. Several metrics at once: LossCombiner

A real training objective is often a sum — L1 for fidelity plus a little SSIM for structure — and you usually want to watch a few more numbers, like PSNR, without training on them. LossCombiner runs a list of metrics on the same Entry and collects all their scores:

loss:
  _target: srforge.metrics.LossCombiner
  params:
    losses:
      - _target: srforge.metrics.regression.L1
        params:
          weight: 1.0
        io:
          inputs: {x: sr, y: hr}
      - _target: srforge.metrics.image.SSIM
        params:
          weight: 0.01
        io:
          inputs: {x: sr, y: hr}
      - _target: srforge.metrics.regression.PSNR
        params:
          weight: 0.0          # watched, not trained on
        io:
          inputs: {x: sr, y: hr}

Two constructor parameters that every metric accepts now start to matter.

Weights: what gets optimised

The number the trainer backpropagates — the training loss — is the weighted total of every metric in the criterion:

total = Σ  weight_i × sign_i × score_i        sign_i = +1 if lower is better
                                                      -1 if higher is better

• weight scales a metric's contribution. The default is 1.0.
• weight: 0 keeps the metric out of the gradient entirely while still computing and logging it. This is how you watch a metric without training on it.
• The sign comes from the metric itself: each metric declares whether lower or higher is better (its best_min property). PSNR and SSIM are "higher is better", so they enter the total negated — minimising the total then raises them. You never flip signs by hand.

For the example above: total = 1.0·L1 − 0.01·SSIM + 0·PSNR.

Names: how scores are labelled

Each metric's score is stored — and logged — under its name, which defaults to the class name (L1, SSIM, PSNR). Names must be unique within a combiner, so you only need to set one when the same metric appears twice:

- _target: srforge.metrics.regression.L1
  params: {name: L1_image}
  io:
    inputs: {x: sr, y: hr}
- _target: srforge.metrics.regression.L1
  params: {name: L1_edges, weight: 0.1}
  io:
    inputs: {x: sr_edges, y: hr_edges}

A duplicate name is refused when the combiner is built rather than letting one score silently overwrite the other.

---

4. Training and validation criteria

The trainer takes two criteria:

trainer:
  _target: srforge.training.PyTorchTrainer
  params:
    training_criterion: ${ref:loss}                 # backpropagated every step
    validation_criterion: ${ref:validation_metrics} # evaluated after each epoch
    # model, runners, lr_scheduler, ...

They are separate because they answer different questions. The training criterion is what the model learns from; the validation criterion is how you judge it. You might train on L1 alone but validate on PSNR and SSIM.

The validation weights decide which checkpoint is 'best'

After every epoch the trainer computes the validation criterion's weighted total — the same formula as in §3. The checkpoint saver keeps the epoch where that total is lowest, and plateau LR schedulers and early stopping read the same number.

So validation weights are not only for display. To select the model with the best PSNR, give PSNR a non-zero weight and everything else 0:

validation_metrics:
  _target: srforge.metrics.LossCombiner
  params:
    losses:
      - _target: srforge.metrics.regression.PSNR
        params: {weight: 1.0}        # selects the checkpoint
        io:
          inputs: {x: sr, y: hr}
      - _target: srforge.metrics.image.SSIM
        params: {weight: 0.0}        # reported only
        io:
          inputs: {x: sr, y: hr}

Mixing several non-zero weights sums quantities in different units (dB, unitless SSIM, pixel error). That is allowed, but it is a choice you should make deliberately.

Both criteria can be the same object — ${ref:loss} in both places — which is the simplest setup when what you train on is also what you judge by.

What the run summary records

The checkpoint saver writes two kinds of "best" to the tracker's summary (the run table in W&B). They answer different questions:

Keys	Question	Values
best_epoch, best_losses.*	What does the model I keep score?	every metric at the one epoch with the lowest weighted validation total — the _best checkpoint
best_per_metric.<metric>.value / .epoch	How good did each metric ever get, and when?	each metric's own best raw value, in its own direction (lowest L1, highest PSNR), and its epoch

best_epoch                       9      <- the kept checkpoint
best_losses.raw.PSNR         48.83      <- PSNR of that checkpoint
best_per_metric.PSNR.value   49.12      <- PSNR's own peak ...
best_per_metric.PSNR.epoch      14      <- ... at a different epoch

For dict-of-bands inputs each band gets its own peak too: best_per_metric.PSNR.b8a.value and .epoch.

Peaks usually come from different epochs

No single model reaches every best_per_metric value at once. Report a model with best_losses.* — that is the checkpoint you actually have. best_per_metric is for comparing runs on one metric and for spotting trade-offs: PSNR peaking long before the kept checkpoint says the validation weights favour something else.

Both are saved in the checkpoint, so a crashed and resumed run keeps them.

---

5. Masking invalid pixels: y_mask

Real targets often contain pixels that should not count: image borders after registration, clouds and their shadows, no-data regions in satellite tiles, placeholder samples with no ground truth at all. Scoring the prediction there would reward or punish the model for pixels nobody can judge.

Metrics that support masking take an optional y_mask. Bind it like any other input:

- _target: srforge.metrics.regression.L1
  io:
    inputs: {x: sr, y: hr, y_mask: valid_mask}

What the mask means:

• 1 = valid, 0 = ignore. Soft values in between weight a pixel partially.
• Shape. [B, 1, H, W] (or [B, H, W]) applies one mask to every channel; [B, C, H, W] masks each channel separately. A mask that fits neither is refused with an error rather than stretched to fit.
• The score is averaged over valid elements only — not over the whole image with zeros mixed in. A tile that is 90% cloud is scored on its 10% of clear pixels, not diluted by the clouds. With a one-channel mask on three-channel data, that is three values per valid pixel.
• A value computed from all channels needs all of them valid. Some metrics produce one value per pixel from every channel together — SAM's spectral angle, LPIPS' perceptual distance. With a per-channel mask, such a pixel counts only where every channel is valid, since one bad channel spoils the value computed from it.
• Windowed metrics keep a pixel only if its whole window is valid. SSIM and MGE judge each pixel by a small window around it, so a window that reaches into a masked area would compare values nobody vouches for. Near a mask edge the valid area therefore shrinks by the window's radius: 5 pixels for SSIM's default 11×11 window, 1 pixel for MGE. A valid region narrower than the window leaves nothing to score.
• A fully masked sample scores 0, not NaN, so a placeholder sample in a batch contributes nothing instead of poisoning the gradient.

Which metrics accept a mask is in the catalogue (look for y_mask in the parameters). Metrics adapted from torchmetrics or pyiqa deliberately refuse a mask binding: those libraries have no notion of one, and silently computing an unmasked score on masked data would report the wrong number as if it were right.

---

6. Multi-band inputs

Multispectral and hyperspectral data often arrive as a dict of bands — {"B02": tensor, "B03": tensor, ...} — rather than one stacked tensor, for example because bands have different resolutions. Metrics accept both.

When a metric receives dicts, it scores each band separately and keeps the results per band:

scores = L1()(x={"red": sr_r, "nir": sr_n}, y={"red": hr_r, "nir": hr_n})
scores.as_raw_flat_dict()
# {"L1.red": ..., "L1.nir": ..., "L1": ...}      L1 = average over the bands

Per-band scores are logged individually (L1.red, L1.nir), which is usually what you want to see: a model can be excellent on visible bands and poor on infrared, and an average hides that. Only bands present in both x and y are compared.

This holds for every metric, whatever it computes: each band gets the metric's full rule on its own, exactly as if it had been passed alone. The combined value (L1 above) is the average of the band values, so each band counts equally — bands often have different resolutions, and a 10 m band has 36 times the pixels of a 60 m one. The exception is a metric whose epoch value is not an average at all, such as a count of pixels: its bands combine the way its images do, so a count is summed over the bands.

Some metrics cannot be split per band because they are defined across bands — SAM measures the angle between spectral vectors, so it needs every band at once. Such a metric receives the whole dict in one call instead. As a user you do not choose this; the metric does (how, is in §9).

---

7. How a metric turns pixels into a number

So far a metric has been a black box that produces "a score". To choose between options like reduction and aggregate — and to write your own metric — you need to see what happens inside. A metric answers up to three questions, in order:

  prediction, target            per PIXEL
        │
        │  ① pointwise(x, y)     "how wrong is each pixel?"           ← Level 2
        ▼
  score map  [B, C, H, W]
        │
        │  ② reduce(map, mask)   "how wrong is each image?"
        ▼
  per-image scores  [B]                                              ← Level 1
        │
        │  ③ aggregate / finalize   "how wrong is the whole batch / epoch?"
        ▼
  one number

Why "one score per image" is the centre

Stage ② is the contract every metric must meet: one value per sample, a tensor of shape [B]. It is the natural unit in SR-Forge because:

• scores from different metrics can be added sample by sample into the weighted total;
• the framework can stack them across batches into an epoch, and across GPUs in distributed training, without knowing anything about the metric;
• you can inspect individual images — find the worst samples, compare models image by image.

① The per-pixel stage — and the two "levels"

Some metrics have a natural value at every pixel: the absolute error of L1, the squared error of MSE, the spectral angle of SAM. Others do not: SSIM compares local windows, LPIPS runs a feature network, PSNR takes a logarithm of a mean. This difference decides how a metric is written:

• Level 2 metrics implement stage ① (pointwise) and inherit stage ②. The framework does the reduction — and therefore the masking — for them.
• Level 1 metrics write stage ② (calculate_score) themselves: because they have no per-pixel value to offer, or because their image score is not simply the average of one.

§9 turns this into three questions you can answer for your own metric.

Level 2 is preferred whenever it is possible, and the reasons come up repeatedly below: masking is guaranteed to be applied correctly, micro-averaging comes for free, and wrappers such as UncertaintyLoss can reuse the per-pixel map.

② The per-image stage: reduction

For a Level-2 metric, the reduction parameter controls how the score map becomes [B]:

reduction	Per-image value	When
mean (default)	average over valid elements	Almost always — the score does not depend on image size
sum	sum over valid elements	When the math calls for a sum, e.g. a proper Gaussian negative log-likelihood
none	the map itself, unreduced	Rarely; for code that post-processes the map

A Level-1 metric does this stage itself, so it has no map for reduction to collapse, and it refuses sum and none instead of silently ignoring them. The exceptions are PSNR and Regularizer, which call reduce() in their own calculate_score and accept all three. The built-in table shows which metric takes it; for your own metric, see reduction in your own calculate_score.

③ The batch and epoch stage

Per-image scores still have to become one number: once per training step (the value backpropagated) and once per epoch (the value reported and used to pick checkpoints). By default that is simply their average. Whether the average is the right answer is the subject of the next section.

---

8. Micro or macro: aggregate

The problem

Suppose your validation set has two images:

	size	valid pixels	L1
A	512 × 512	262 144	0.02
B	64 × 64	4 096	0.10

What is "the L1 of the validation set"? There are two defensible answers:

• Macro — average the per-image scores: (0.02 + 0.10) / 2 = 0.060. Every image counts equally.
• Micro — pool every pixel, then compute once: (262144·0.02 + 4096·0.10) / 266240 ≈ 0.021. Every pixel counts equally.

Neither is a bug; they answer different questions. Macro asks "how well do I do on a typical image?"; micro asks "how well do I do on the data as a whole?". They agree only when every image has the same number of valid pixels. They drift apart when image sizes differ, when masks cover some images more than others, and — most dramatically — when the metric is a ratio or a logarithm:

• PSNR: the log of the pooled error is not the mean of per-image logs. A few near-perfect images produce huge per-image PSNRs and dominate a macro average; gaps over 10 dB are measured on real SR data.
• F1 on sparse masks: an image with nothing to detect scores 0 under macro however well the model behaves. See Classification Metrics.

Choosing

Every metric takes aggregate, set under params: like any other argument:

- _target: srforge.metrics.regression.PSNR
  params:
    aggregate: micro          # one PSNR from the pooled squared error
  io:
    inputs: {x: sr, y: hr, y_mask: valid_mask}

Rules of thumb:

• Follow your field's convention when you report. SR benchmarks usually report macro PSNR/SSIM (per image, averaged); change detection and segmentation usually report micro F1/IoU.
• With masks or mixed image sizes, micro is often what you actually mean — a mostly-clouded tile should not weigh as much as a clear one.

It applies to training too

aggregate is used at both scales from §7 ③:

• during training, the backpropagated value of a micro metric is pooled over the batch — for L1, the mean over every valid pixel in the batch;
• for the epoch, the same formula runs over every batch.

Batch-level micro is an ordinary, differentiable objective (it is what torch.nn.functional.l1_loss computes on a whole batch tensor). It weights pixels equally where macro weights images equally.

Micro with a dict of bands

With dict-of-bands inputs, micro pools within each band: F1Score.red is the micro F1 over every red pixel, F1Score.nir over every NIR pixel. The combined F1Score is the average of the band values, as for macro. To pool every pixel of every band into one number instead, stack the bands into one tensor before the metric.

What supports what

A metric can be pooled only if it can be computed from totals that add up: L1 from the sum of errors and the pixel count, PSNR from the sum of squared errors and the count, F1 from the counts of true and false positives. A metric computed by a network over a whole image (NIQE) has nothing to add up, so it can only be macro.

Metric	aggregate (first = default)	Why
Level-2 metrics: L1, MSE, Charbonnier, LPIPS, CrossEntropy, SAM, UncertaintyLoss, your own	macro, micro	Automatic: the framework keeps the sum and the count
PSNR	macro, micro	Defines how it pools
Precision, Recall, F1Score, IoU, MCC	micro, macro	Micro is what that literature reports
torchmetrics metrics (via the adapter)	macro, micro	Pools the metric's own state
SSIM, MGE, TotalVariation, pyiqa metrics, CorrectedLoss	macro	Nothing that adds up
Metrics with their own epoch rule (confusion counts, custom totals)	custom	Not a choice — see §9

Asking for something a metric does not support is an error when the config is built, and the message says what is supported:

SSIM does not support aggregate='micro'. Supported: 'macro'.
It reports a finished value per image and defines no pooled form. ...

The same applies to typos (Micro), to aggregate: micro combined with reduction: sum (a pooled sum is not a mean), and to a wrapper holding a micro metric (CorrectedLoss around PSNR(aggregate="micro") would silently use per-image values).

---

9. Writing your own metric

Writing a metric means choosing which of the three stages from §7 to implement: pointwise, calculate_score, finalize. That choice has its own page — Writing a Metric — which covers:

• which methods to write: three plain questions;
• one metric written every possible way, with what each combination gets and gives up;
• inputs, masks and epoch rules.

---

10. Wrapping and scheduling metrics

These build on an existing metric instead of replacing it.

CorrectedLoss — tolerate small misalignments

In multi-frame and satellite super-resolution the target is often shifted from the prediction by a pixel or two, and differs slightly in brightness. A plain L1 then punishes the model for a registration error that is not its fault. CorrectedLoss evaluates its base metric over every shift within border pixels, optionally removes the mean brightness difference first, and keeps the best score for each image:

- _target: srforge.metrics.wrappers.CorrectedLoss
  params:
    base_metric:
      _target: srforge.metrics.regression.PSNR
    border: 3                  # search shifts of up to ±3 pixels
    do_correction: true        # remove the brightness bias first
  io:
    inputs: {x: sr, y: hr, y_mask: valid_mask}

The outer io binds your fields to the wrapper. Inside, the base metric reads the canonical names x, y, y_mask; if your base metric names its parameters differently, give it its own io that maps them to those names.

The score is logged as CorrectedLoss. Set name (say cPSNR) to make the log readable — and you must, as soon as a combiner holds two corrected metrics, because names must be unique.

UncertaintyLoss and Regularizer

Some models predict, alongside the image, how uncertain they are at each pixel (a log-variance map s). Training them uses the heteroscedastic negative log-likelihood:

loss = base(x, y) · exp(−s)   +   s
       └── data term ──────┘     └ regularizer ┘

The data term lets the model down-weight pixels it declares uncertain; the regularizer stops it from declaring everything uncertain. SR-Forge makes them two separate metrics, both trained on:

loss:
  _target: srforge.metrics.LossCombiner
  params:
    losses:
      - _target: srforge.metrics.wrappers.UncertaintyLoss
        params:
          base_metric:
            _target: srforge.metrics.regression.MSE
        io:
          inputs: {x: sr, y: hr, log_var: log_var, y_mask: valid_mask}
      - _target: srforge.metrics.regularization.Regularizer
        params: {penalty: identity}
        io:
          inputs: {x: log_var, y_mask: valid_mask}

Why split them? So that several data terms can share one uncertainty map with exactly one regularizer. With two UncertaintyLoss entries (say MSE and LPIPS) and one Regularizer, the total is (e_MSE + e_LPIPS)·exp(−s) + s — the right objective. If each data term carried its own +s, you would get 2s and the model would learn uncertainties that are systematically too small.

Any pointwise metric can be the base, whatever its inputs, because the data term multiplies the base's per-pixel map. UncertaintyLoss takes the base's inputs plus log_var (and y_mask): x, y, log_var around MSE, x, log_var around a metric with no reference, the base's own names around a custom one. Bind them on the UncertaintyLoss itself. A base without a pointwise — SSIM, MGE — is refused when the wrapper is built.

When is this a likelihood?

base · exp(−s) + s is a true negative log-likelihood when the base is a residual between prediction and target: squared error gives the Gaussian, absolute error the Laplacian (family: gauss or laplace sets the matching constant). Around any other per-pixel term — a no-reference score, a perceptual distance — it is learned loss attenuation: the model learns where to discount the term and pays s for it. That is a common and useful objective, just not a likelihood.

Never use UncertaintyLoss without the Regularizer

Without the +s term, predicting infinite uncertainty drives the loss to minus infinity. Training shows it within a few epochs as a loss that falls to large negative values.

Regularizer on its own

Regularizer penalises the magnitude of a single field — any auxiliary output that should stay small or sparse. It honours y_mask like the other built-in metrics.

penalty	Per-element function	Typical use
identity	x	The uncertainty regularizer above
abs	|x|	Sparsity (attention maps, latent codes)
square	x²	Keeping predicted offsets or residuals small
exp	exp(x)	Penalising positive log-quantities
entropy	−x·log(x)	Encouraging confident probability maps
huber	smooth L1, with huber_delta	Robust magnitude penalty

- _target: srforge.metrics.regularization.Regularizer
  params: {penalty: abs, weight: 0.001}
  io:
    inputs: {x: attention_map}

In Python, penalty can also be any function returning a tensor of the same shape.

LossScheduler — change the loss during training

Curricula are common: start with a simple fidelity loss, add a perceptual term once the model is roughly right. LossScheduler holds one criterion per starting epoch; the trainer switches to the right one at the start of each epoch:

loss:
  _target: srforge.metrics.LossScheduler
  params:
    schedule:
      0:                                    # epochs 0-49
        _target: srforge.metrics.regression.L1
        io:
          inputs: {x: sr, y: hr}
      50:                                   # epoch 50 onwards
        _target: srforge.metrics.LossCombiner
        params:
          losses:
            - _target: srforge.metrics.regression.L1
              io:
                inputs: {x: sr, y: hr}
            - _target: srforge.metrics.image.LPIPS
              params: {weight: 0.1}
              io:
                inputs: {x: sr, y: hr}

The schedule must start at epoch 0. Pass it as the training_criterion like any other criterion.

---

11. What a metric returns: MetricScores

You need this section when writing hooks, custom training loops, or analysis code; configs never touch it.

A metric call returns MetricScores, which holds one MetricEntry per metric name. An entry keeps the metric's raw result together with its weight, its direction (best_min), and its aggregate, so everything needed to interpret the numbers travels with them.

scores = criterion(entry)
total = scores.total_weighted()      # [B] — the weighted total from §3
total.mean().backward()              # what the training runner does

scores["L1"].raw                     # [B] per-image scores (or a dict of bands)
scores["L1"].final_value()           # the single number for what the entry holds

What each kind of entry contributes to total_weighted():

aggregate	Contribution
macro	its per-image scores
micro	its pooled value over what the entry holds — the batch during training, the whole epoch after accumulation
custom	nothing — a count or a worst case is not a loss term

MetricScores methods

Method	Returns	Purpose
total_weighted()	Tensor[B]	Weighted, sign-corrected sum — the optimisation objective
total_raw()	Tensor[B]	Plain sum of raw scores
mean_raw() / mean_weighted()	{name: Tensor}	One number per metric
as_raw_dict() / as_weighted_dict()	{name: Tensor \| dict}	Structured views
as_raw_flat_dict() / as_weighted_flat_dict()	{name: Tensor}	Flattened; bands become name.band
as_summary_dict()	dict	What the loggers record
merge(other)	MetricScores	Combine scores of different metrics (what LossCombiner does)
add_scores(other)	None	Append another batch in place (what the runners do over an epoch)
detached(cpu=True)	MetricScores	A copy cut from the autograd graph, for accumulation and logging
scores[name], name in scores, iteration		Access entries
scores[name] = MetricEntry(...)		Inject an extra term from a hook — see Hooks

MetricEntry methods

Method	Returns	Purpose
raw / as_raw()	Tensor \| dict	The value the metric returned
as_opt()	Tensor \| dict	Sign-corrected so lower is always better
as_weighted()	Tensor \| dict	Sign-corrected and weighted
raw_batch() / opt_batch() / weighted_batch()	Tensor[B]	The same views, bands averaged; refused for micro and custom entries, which hold ingredients rather than per-image scores
final_value()	Tensor	The single number, using the entry's own rule

---

12. Built-in metrics

All live in srforge.metrics — in its regression, image, wrappers, regularization and classification modules — and accept weight, name and aggregate. reduction is accepted by L1, MSE, Charbonnier, PSNR, SAM, LPIPS, CrossEntropy, BinaryCrossEntropy, the confusion counts, UncertaintyLoss and Regularizer: the metrics that turn a per-pixel map into each image's score. The others refuse it, having no such map.

Metric	Inputs	Better	aggregate	What it measures
L1	x, y, y_mask	lower	macro, micro	Mean absolute error
MSE	x, y, y_mask	lower	macro, micro	Mean squared error
Charbonnier	x, y, y_mask	lower	macro, micro	Smooth L1, robust near zero
PSNR	x, y, y_mask	higher	macro, micro	Peak signal-to-noise ratio (data_range sets the peak)
SSIM	x, y, y_mask	higher	macro	Structural similarity
MGE	x, y, y_mask	lower	macro	Mean gradient (edge) error
TotalVariation	x, y, y_mask (y optional)	lower	macro	Spatial smoothness
SAM	x, y, y_mask	lower	macro, micro	Spectral angle; stacked [B,C,H,W] or dict of bands
LPIPS	x, y, y_mask	lower	macro, micro	Learned perceptual distance
BinaryCrossEntropy	x, y, y_mask	lower	macro, micro	Per-pixel cross-entropy of a yes/no map; optional class_weights
CrossEntropy	x, y, y_mask	lower	macro, micro	Per-pixel cross-entropy over classes (one channel each); optional class_weights
TBE	x (single channel)	lower	macro	The Blur Effect — no reference; computed on the CPU without a gradient, so for evaluation only
CorrectedLoss	x, y, y_mask	as base	macro	Shift-tolerant wrapper
UncertaintyLoss	the base's inputs, log_var, y_mask	lower	macro, micro	Uncertainty-weighted data term
Regularizer	x, y_mask	lower	macro	Magnitude penalty

Classification metrics — true/false positives and negatives, Precision, Recall, F1Score, IoU, MCC — are for models that answer yes/no per pixel (segmentation, change detection, cloud masking). They live in srforge.metrics.classification: see Classification Metrics. They can't be trained on as they are; with soft: true they can, and DiceLoss, JaccardLoss and TverskyLoss are the common ones by name — see Training with them.

Metrics from other libraries — torchmetrics and pyiqa (NIQE and other no-reference metrics) — can be used directly in configs: see External Metrics.

---

Recipes

I want to…	Do this
Train on L1, watch PSNR and SSIM	LossCombiner with PSNR and SSIM at weight: 0 — §3
Pick the checkpoint with the best PSNR	Validation criterion with PSNR weight: 1, the rest 0 — §4
Ignore clouds or no-data pixels	Bind y_mask — §5
See per-band scores for multispectral data	Pass dicts of bands — §6
Report PSNR over the pooled error	params: {aggregate: micro} on PSNR — §8
Weight every valid pixel equally when training	aggregate: micro on a Level-2 metric — §8
Tolerate 1–2 px misregistration	CorrectedLoss — §10
Train a model that predicts its own uncertainty	UncertaintyLoss + Regularizer — §10
Switch losses mid-training	LossScheduler — §10
Write a new per-pixel metric	Implement pointwise — §9
Report a total over the epoch	Override finalize with declared reductions — §9

---

Next: Writing Scripts — training and test scripts that wire everything together.