SR-Forge

Transforms¶

In the previous section, you learned that Entry carries data through the pipeline. Transforms are the processing steps that modify that data — normalizing pixel values, augmenting images, selecting spectral bands, computing derived features, and more.

Two Types of Transform¶

SR-Forge has two types of transform, separated by what they can see:

A DataTransform is a function on values. It receives the content of Entry fields as arguments (tensors, strings, dicts, etc.), processes them, and returns results. The framework handles extracting values from the Entry, calling your method, and storing results back. Example: normalizing pixel values, resizing an image, converting tensor types.

An EntryTransform receives the entire Entry object. It can read any field, add new fields, remove fields, rename fields, make decisions based on which fields exist (entry.keys(), entry.is_batched), and even convert the Entry into a different type. Example: splitting a multispectral dict into separate band fields, renaming fields, removing temporary metadata.

The two types compose naturally: because DataTransforms operate on raw values, you can call them inside an EntryTransform — combining Entry-level logic with reusable value-processing components. The reverse doesn't work — a DataTransform only receives values, so it has no Entry to pass to an EntryTransform.

Rule of thumb for implementing custom transforms: Start with DataTransform. It's simpler, more reusable, and IO binding controls where results go. Switch to EntryTransform only when you need structural access: removing fields, inspecting field names, or changing the Entry type.

Why two types?

DataTransform is simpler and more reusable — it works on raw values, supports automatic dict/list recursion, and can be called standalone without an Entry. But it can only return values to pre-declared output fields. EntryTransform gives full structural access — adding/removing fields dynamically, inspecting field names, changing the Entry type — at the cost of being coupled to Entry. The split keeps the common case (value processing) simple while still supporting structural operations when needed.

Where Do Transforms Run?¶

Transforms can be used anywhere you have an Entry:

In a Dataset — for preprocessing that only needs to happen once per sample. Datasets support caching, so subsequent epochs skip recomputation entirely:

ds = MyDataset(paths, transforms=[Normalize().set_io({"inputs": {"image": "image"}})])
ds.cache("/tmp/cache")  # expensive transforms run once, cached on disk

In a SequentialModel — as part of the model pipeline. Use this for augmentations (random modifications like cropping or adding noise that make training more robust) or processing that depends on model outputs:

model = SequentialModel(
    modules={
        # RandomCrop is an EntryTransform — field selection lives in the
        # constructor (field_key=), not in flow. EntryTransforms use the
        # empty-segment form " -> name -> " because they read & mutate
        # the Entry opaquely.
        "crop": RandomCrop(patch_size=64, field_key="image"),
        "net": MyModel(),
    },
    flow=[" -> crop -> ", "image -> net -> output"],
)

Directly on an Entry — for prototyping, debugging, or one-off processing:

transform = Normalize().set_io({"inputs": {"image": "image"}})
entry = transform(entry)

In hooks — for postprocessing during evaluation, such as denormalizing predictions before saving to disk or converting tensors to images for logging (e.g. the transforms: parameter of the BatchImageLogger hook).

In a typical experiment you'll use several of these together — heavy preprocessing cached in the dataset, augmentations in the model pipeline, and denormalization in an image-logging hook.

Batched vs Unbatched: Pick One¶

Where a transform runs determines what shape the data arrives in:

In a Dataset — data is unbatched. Tensors are [C, H, W] (natural shapes), names are bare strings, and entry.is_batched is False.
In a SequentialModel (after the DataLoader) — data is batched. Tensors are [B, C, H, W], names are lists of strings, and entry.is_batched is True.

You might expect to need separate code for each case. You don't. Both DataTransform and EntryTransform let you pick one method and the framework handles the other case automatically:

transform() — write your logic for batched data. This is the performance choice: PyTorch operations vectorize across the batch, so one kernel launch processes all samples. When the framework encounters an unbatched entry, it auto-wraps it via collate([entry]), calls your method, and unwraps the result — you don't need to think about it.
transform_unbatched() — write your logic for a single sample. This is the simplicity choice: no batch dimension to manage, natural shapes, straightforward code. When the framework encounters a batched entry, it loops over each sample, calls your method, and re-batches the results.

You only implement one. The framework adapts to whichever you chose. There's no need to implement both — pick based on what matters more for your transform:

	`transform()`	`transform_unbatched()`
Data shape	`[B, C, H, W]`	`[C, H, W]`
Strength	Performance — batch-parallel GPU ops	Simplicity — natural single-sample code
Best for	Math, convolutions, interpolation — anything that vectorizes	Per-sample conditionals, variable-size processing, prototyping
Fallback cost	Negligible (wrapping one entry)	Sequential loop over batch

See Collation for how batching works and Per-Sample Processing for a deeper dive.

If both methods are implemented

transform() always wins — even for unbatched entries. The framework wraps the single sample, calls transform(), and unwraps the result. The wrapping cost is negligible compared to the performance gain of a vectorized implementation, so there's no reason to fall back to transform_unbatched(). You can still call transform_unbatched() directly if needed — it just won't be picked by the automatic dispatch.

Decision Tree¶

I want to create a new transformation...
                  |
    Does it need access to Entry STRUCTURE?
    (add/remove fields, inspect field names,
     change Entry type)
               |
        +------+------+
        |             |
       YES           NO
        |             |
        v             v
  EntryTransform   DataTransform

Quick Comparison¶

Characteristic	DataTransform	EntryTransform
Receives	Field values as arguments	Entire Entry object
IO declaration	Method signature (parameter names + annotations)	Constructor `_key` params
Write to new fields	Yes (via IO binding)	Yes
Remove / rename fields	No	Yes
Inspect field names / Entry state	No	Yes (`is_batched`, `keys()`, etc.)
Change Entry type	No	Yes
Recursive application	Yes — recurses into dicts/lists automatically¹	No
Standalone use	Yes (raw values, no Entry needed)	No — requires Entry
Multi-field input	Yes	Yes

Read these next, in order:

DataTransform — The simpler type: pure functions on values with automatic recursion
EntryTransform — Full Entry control: structural operations and constructor field keys

When a DataTransform's field contains a dict of tensors (e.g., {"b2": tensor, "b8": tensor}), the framework can automatically apply the transform to each value inside the dict. Whether it recurses or passes the whole dict depends on type annotations — see DataTransform: How Type Annotations Control Container Handling. ↩