The Anatomy of a Training¶
Every deep-learning training — in any framework, for any task — needs the same ingredients. This page walks through each one: what it is, why training can't work without it, and which SR-Forge abstraction handles it. If you understand this page, you understand why the framework is shaped the way it is.
The ingredient list¶
| A training needs… | Because… | In SR-Forge that's… |
|---|---|---|
| Examples to learn from | no data, no learning | Dataset |
| A container for one example | a sample's pieces must travel together | Entry |
| Batches | GPUs are efficient on many samples at once | Collation + DataLoaderFactory |
| Data preparation | raw files are rarely network-ready | Transforms |
| A model | the thing that actually learns | Model |
| Wiring | the model must find its inputs in your data | IO Binding |
| A loss | learning needs a measurable definition of "wrong" | Loss → MetricScores |
| An optimizer | gradients must become weight updates | torch.optim.* via YAML |
| The loop | forward → loss → backward → update, thousands of times | Trainer + EpochRunner |
| A stop signal | training too long makes the model worse | stop_condition |
| Eyes on the process | otherwise you fly blind for hours | Hooks + Tracker |
| Saved progress | crashes happen; the best epoch isn't the last | PyTorchModelSaver + resume |
| A reproducible definition | you'll run hundreds of variants | YAML config |
The loop at the heart of it all:
flowchart LR
D[batch of data] --> F["model.forward()"]
F --> L["loss: how wrong?"]
L --> B["backward: whose fault?"]
B --> O["optimizer: adjust weights"]
O -->|next batch| D
style L stroke:#c9a84c,stroke-width:2pxNow each ingredient, one at a time.
1. Examples to learn from¶
Neural networks learn by seeing examples: an input, and (usually) the answer we wanted. Thousands of them. Something has to find those examples on disk, load one when asked, and say how many there are.
In SR-Forge: a Dataset. You implement two methods — __getitem__(i) (load example number i) and __len__() (how many exist). Built-ins like LazyDataset cover the common "one folder per scene" layout so you often don't write any code at all. Datasets can also cache preprocessed examples so slow preparation runs only once.
2. A container for one example¶
One training example is rarely one tensor. A super-resolution sample is a low-res image + a high-res target + maybe a cloud mask + a scene name. These pieces must travel through the pipeline together — hand a function three loose tensors and a string and you'll spend your life keeping their order straight.
In SR-Forge: the Entry — a dictionary with named fields (entry.lr, entry.hr, entry.mask, entry.name). Every component reads from and writes to Entries, which is exactly what makes components swappable. One call (entry.to("cuda")) moves everything to the GPU.
3. Batches¶
GPUs are throughput machines: processing 8 images at once costs barely more than processing 1. So training processes batches — stacks of examples with an extra leading dimension ([8, C, H, W] instead of [C, H, W]).
In SR-Forge: collation turns a list of Entries into one batched Entry — tensors are stacked, names become lists, and every field type has a defined rule. It happens automatically inside the DataLoader (DataLoaderFactory picks the right one, including for graph data). Your job is only to return natural shapes from the dataset — never add the batch dimension yourself.
4. Data preparation¶
Raw files are rarely what the network should see. Pixel values need normalizing (networks train badly on values in [0, 255]), types need converting, and — during training — random modifications (crops, flips, noise) make the model robust instead of memorizing.
In SR-Forge: Transforms. Deterministic preparation goes on the dataset (and gets cached); random augmentation goes in the model pipeline so it's fresh every epoch. Two flavors: DataTransform changes field values, EntryTransform changes the Entry's structure.
5. A model¶
The model is the function being learned: it takes the input and produces a prediction, and its millions of internal weights are what training adjusts.
In SR-Forge: a Model — a regular PyTorch nn.Module where you write forward() as usual. The framework adds one thing on top: the ability to read inputs from Entry fields and write outputs back. Multi-stage pipelines chain via SequentialModel; adversarial setups via GANModel.
6. Wiring between data and model¶
Your model's forward(image) expects a parameter called image; your dataset produced a field called lr. Someone has to connect them — and hard-coding the connection inside the model would mean rewriting the model for every dataset.
In SR-Forge: IO Binding — a small mapping ({"inputs": {"image": "lr"}, "outputs": "sr"}) declared in YAML or Python, kept outside the model code. Same model, any data layout. This is the framework's central trick; the step-by-step explainer walks through it.
7. A loss¶
"Learning" needs a precise, differentiable definition of wrong. The loss compares prediction to target and produces a single number — and because it's differentiable, PyTorch can compute how much each weight contributed to the error (the gradient). That gradient is the learning signal. No loss, no direction to improve in.
In SR-Forge: a Loss reads its inputs from the Entry (via IO binding, like a model) and returns MetricScores — a container of named, weighted scores. Combine several with LossCombiner (weight 0.0 = track as a metric without training on it); change weights mid-training with LossScheduler. The backprop chain is one line: scores.total_weighted().mean().backward().
8. An optimizer, and a learning-rate schedule¶
The gradient says which direction each weight should move; the optimizer decides how far, using strategies (momentum, adaptive step sizes) that make training converge instead of oscillating. The learning rate — the master step size — usually needs to shrink over time: big steps early to make progress, small steps late to fine-tune.
In SR-Forge: plain PyTorch — no wrapper needed. You declare torch.optim.AdamW and a scheduler in YAML; the only SR-Forge-specific bit is ${ref:model}.trainable_params(), which hands the instantiated model's weights to the optimizer at config-resolution time.
9. The loop itself¶
Training is a nested loop: for each epoch (one full pass over the data), for each batch: forward pass → loss → backward pass → optimizer step. After each training epoch, a validation pass — same forward and loss, but on unseen data and without learning — tells you whether the model is genuinely improving or just memorizing.
In SR-Forge: the Trainer owns the outer loop (alternating train/validation epochs, stepping the scheduler, checking the stop condition), and delegates each epoch to an EpochRunner — TrainingEpochRunner with gradients, ValidationEpochRunner without. Mixed precision, gradient accumulation, and multi-GPU are runner config switches, not code you write.
10. Knowing when to stop¶
Train too long and the model starts memorizing the training data — validation performance degrades while training loss keeps falling (overfitting). Watching validation and stopping when it stops improving is both a quality and a compute win.
In SR-Forge: a stop_condition on the trainer — e.g. ValidationLossDidNotImprove(patience=50) ends training after 50 epochs without progress. See Stop Conditions.
11. Eyes on the process¶
A training runs for hours. Without instrumentation you learn about a diverged loss or a dead GPU at hour six. You want: a progress bar (is it moving?), loss curves (is it learning?), and sample predictions (does the output look right? — curves can lie).
In SR-Forge: Hooks — small plug-ins attached to the runner or trainer: ProgressBar, LossLogger, BatchImageLogger. They log through the ExperimentTracker, which speaks to Weights & Biases — or to nothing (NullTracker) when you're debugging. Hooks can also act, not just watch: GradientClip caps exploding gradients; custom hooks can add penalty terms or skip steps.
12. Saved progress¶
Two separate reasons to save: insurance (a crash at epoch 190 of 200 must not cost you the run) and selection (the best model is usually not the last epoch — you want the checkpoint from when validation peaked).
In SR-Forge: the PyTorchModelSaver hook keeps both _best and _last checkpoints, uploaded via the tracker. resume_from_checkpoint() restores everything — weights, optimizer state, scheduler, RNG — with one call, so a resumed run continues as if never interrupted.
13. A reproducible definition of the experiment¶
Research means running variants: a different loss weight, a bigger model, another dataset split. If each variant means editing Python, you accumulate untracked, unreproducible mutations. The experiment definition should be a document — diffable, shareable, re-runnable.
In SR-Forge: the YAML config. Every ingredient above — dataset, transforms, model, loss, optimizer, runners, trainer, hooks — is declared as _target + params (+ io for the wiring), and ConfigResolver builds and connects the live objects. Swap the model, reweight the loss, or add a hook by editing YAML; override anything from the command line for quick sweeps.
Seeing it all together¶
The scaffold generated by srforge init is exactly this list made concrete — open its train-cfg.yaml and you'll recognize every section. The minimal example is the same anatomy at its smallest: synthetic data, a tiny model, one loss, a bare loop.
Where to next: Core Concepts for the compact glossary, or dive into the Guide piece by piece — you now know why each piece exists.