SR-Forge

Lazy Datasets

LazyDataset

Bases: Dataset

A dataset that lazily loads data from a directory structure.

Each example is a subdirectory containing files that match the provided mappings. Files are loaded on demand when accessed via __getitem__.

Parameters:

Name	Type	Description	Default
root	Union[Path, str]	Root directory containing the dataset.	required
mappings	Dict[str, str]	Dictionary mapping Entry field names to file name patterns within each example directory (e.g., {"hr": "HR", "lrs": "LR"}).	required
depth	int	Subdirectory depth for discovering examples. 0 means immediate children of root are examples.	0
name	str	Optional dataset name.	None
allowed_dirs	List[str]	Optional list of directory names to include. If empty, all directories are included.	None
transforms	list	Optional list of callables applied to each loaded Entry.	None

name: str property

Return the dataset name.

examples: List[str] property

Return the list of example names.

__get_example_names(depth: int) -> List[str]

Derive human-readable names from example paths.

__len__() -> int

Return the number of examples in the dataset.

__getitem__(idx) -> Entry

Load and return the Entry at the given index.

__exclude_nones(loaded: Any) -> Any staticmethod

Remove None values from dicts or lists, returning None if empty.

LazyMultiSpectralDataset

Bases: LazyDataset

A LazyDataset variant for multispectral data. Each scene folder contains one subfolder per spectral band. For each band folder, an Entry is created (using the same mapping logic as LazyDataset, but relative to the band folder). Finally, a single Entry is created where each field is a dict keyed by band name.

__getitem__(idx) -> Entry

Load and return a multispectral Entry at the given index.

Iterates over band subdirectories within the scene folder, loads each band's data using the parent mappings, merges band-level and scene-level metadata, and returns a single Entry where each field is a dict keyed by band name.

Parameters:

Name	Type	Description	Default
idx	int	Index of the scene to load.	required

Returns:

Name	Type	Description
Entry	Entry	An Entry with dict-valued fields keyed by band name.

LazyConfigDataset

Bases: Dataset

A dataset driven by a JSON or dict config mapping field names to file paths.

Each key in the config maps to a list of file paths. All lists must have the same length. When accessed, the dataset loads files at the given index and groups them into an Entry. All files at the same index must share the same stem (base filename without extension).

Parameters:

Name	Type	Description	Default
config	Union[str, Path, Dict[str, List[str]]]	A dict mapping field names to lists of file paths, or a path to a JSON file containing such a dict.	required
name	str	Optional dataset name.	None
transforms	list	Optional list of callables applied to each loaded Entry.	None

name: str property

Return the dataset name.

examples: List[str] property

Return the list of example names (not implemented for config datasets).

__load_config(config: Any) -> Union[Dict[str, List[str]]] staticmethod

Load and validate the config from a dict, file path, or JSON string.

Parameters:

Name	Type	Description	Default
config	Any	A dict, Path, or string path to a JSON config file.	required

Returns:

Type	Description
Union[Dict[str, List[str]]]	A dictionary mapping field names to lists of file paths.

Raises:

Type	Description
ValueError	If config is not a supported type.

__len__() -> int

Return the number of examples in the dataset.

__getitem__(idx) -> Entry

Load and return the Entry at the given index.

Parameters:

Name	Type	Description	Default
idx	int	Index of the example to load.	required

Returns:

Name	Type	Description
Entry	Entry	An Entry with fields populated from the loaded files.

Raises:

Type	Description
AssertionError	If files at the same index have different stems.