Balancing Class Frequencies in Chip Datasets

To balance a skewed chip dataset, weight each chip by the rarity of the classes it contains and weight the loss by inverse class frequency:

import numpy as np

pixel_counts = np.bincount(masks.reshape(-1), minlength=n_classes)[1:]
class_weight = pixel_counts.sum() / (len(pixel_counts) * pixel_counts)   # inverse frequency

# per-chip sampling weight: driven by the rarest class present in the chip
chip_weight = np.array([class_weight[np.unique(m[m > 0]) - 1].max() for m in masks])
chip_weight /= chip_weight.sum()

Both halves matter. The sampling weight decides which chips the model sees; the class weight decides how much each pixel inside them counts. This page sits under building training datasets from satellite imagery in Raster Machine Learning & Model Inference.

Imbalance in Satellite Data Is Usually Extreme

Land cover follows a power law almost everywhere. A typical agricultural scene is half cropland and a few per cent water, and the class that motivated the project — informal settlement, burn scar, greenhouse — is often well under one per cent of pixels.

The consequence is that overall accuracy stops being a useful training signal. A model that predicts “not greenhouse” everywhere scores 99.96%, and gradient descent finds that solution immediately.

Environment & Setup

pip install "numpy>=1.23" "scikit-learn>=1.3" "torch>=2.1"

Complete Working Example

import numpy as np


def class_pixel_counts(masks: np.ndarray, n_classes: int) -> np.ndarray:
    """Labelled pixels per class across the whole chip set (class 0 excluded)."""
    flat = masks.reshape(-1)
    return np.bincount(flat[flat > 0], minlength=n_classes + 1)[1:]


def inverse_frequency_weights(counts: np.ndarray, *, clip: float = 50.0) -> np.ndarray:
    """Inverse-frequency class weights, clipped so one rare class cannot dominate."""
    counts = np.maximum(counts, 1)
    w = counts.sum() / (len(counts) * counts)
    return np.clip(w, 1.0 / clip, clip).astype("float32")


def chip_sampling_weights(masks: np.ndarray, class_w: np.ndarray) -> np.ndarray:
    """One sampling weight per chip, driven by the rarest class it contains."""
    weights = np.ones(len(masks), dtype="float32")
    for i, m in enumerate(masks):
        present = np.unique(m[m > 0])
        if present.size:
            weights[i] = class_w[present - 1].max()
    return weights / weights.sum()


if __name__ == "__main__":
    masks = np.load("chips_fold0.npz")["masks"]
    counts = class_pixel_counts(masks, n_classes=6)
    cw = inverse_frequency_weights(counts)
    sw = chip_sampling_weights(masks, cw)

    print("pixel share :", (counts / counts.sum()).round(4))
    print("class weight:", cw.round(2))
    print("chips whose weight exceeds 10x the median:",
          int((sw > 10 * np.median(sw)).sum()))

The clip is the part people leave out and later regret. Unclipped inverse frequency gives the greenhouse class a weight of roughly 400, which turns every greenhouse pixel into a 400-pixel gradient and makes training unstable. Clipping at 50 keeps the emphasis while leaving the optimisation well behaved; the square root of the inverse frequency is another common, gentler choice.

Variant Patterns

1. A weighted sampler for segmentation training

import torch
from torch.utils.data import DataLoader, WeightedRandomSampler

sampler = WeightedRandomSampler(
    weights=torch.as_tensor(sw, dtype=torch.double),
    num_samples=len(sw),        # one epoch is still len(dataset) draws
    replacement=True,
)
loader = DataLoader(train_dataset, batch_size=16, sampler=sampler, num_workers=4)

# The validation loader must NOT be weighted
val_loader = DataLoader(val_dataset, batch_size=16, shuffle=False, num_workers=4)

2. Class weights in the loss

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss(
    weight=torch.as_tensor(cw),
    ignore_index=255,            # the ignore value burned with the mask
)

ignore_index ties this back to the ignore regions produced in rasterizing labels into segmentation masks: uncertain boundary pixels contribute nothing to the gradient, which is a different mechanism from weighting and composes cleanly with it.

3. Sample weights for a pixel-based model

For a tabular model the weight applies per row, and no resampling is needed at all.

import numpy as np
from sklearn.ensemble import HistGradientBoostingClassifier

sample_weight = cw[y - 1]
model = HistGradientBoostingClassifier(max_iter=400, learning_rate=0.06)
model.fit(X, y, sample_weight=sample_weight)

When to Leave the Imbalance Alone

Balancing is a trade, not an improvement, and it is worth being explicit about what is being traded. An unbalanced model is well calibrated: when it says a pixel is 70% likely to be cropland, roughly 70% of such pixels are cropland, because the training distribution matched the landscape. A balanced model is deliberately miscalibrated — it has been told the rare class is more common than it is, so its probabilities overstate rarity everywhere.

That matters whenever the output is thresholded or converted into an area estimate. If you balance during training and then threshold at 0.5, the map over-predicts the rare class by a factor related to the weighting, and the area figures inherit it. Two fixes are available: recalibrate the probabilities on an unweighted validation set, or choose the operating threshold from that validation set rather than assuming 0.5 means anything.

So the decision rule is simple. If the deliverable is a probability, or an area estimate, or anything that a downstream consumer will threshold themselves, prefer an unbalanced model with per-class metrics reported honestly. If the deliverable is a detection map where missing the rare class is the failure mode that matters, balance, and document that the probabilities are no longer calibrated.

Common Errors

Training loss drops but the rare class is never predicted

The weights were computed from chip counts rather than pixel counts. A chip containing three greenhouse pixels counts as a full greenhouse chip and the real skew is hidden. Always count at the pixel level.

Loss becomes NaN after adding weights

Unclipped inverse frequency produced an enormous weight, and one batch of rare pixels blew up the gradient. Clip the weights, lower the learning rate, or use the square-root variant.

Validation accuracy collapses after balancing

The validation loader is also weighted, so it no longer measures performance on the real landscape. Weight training only; validate on the natural distribution, then report per-class metrics as described in validating raster predictions and accuracy assessment.

Frequently Asked Questions

Q: Should I always balance an imbalanced raster dataset? No. If the imbalance reflects the real landscape and you care about overall accuracy, leaving it alone gives a better-calibrated model. Balance when the rare class is the point of the exercise, and expect the calibration to shift when you do.

Q: Is oversampling or class weighting better? Weighting for pixel-based models, oversampling for segmentation. A pixel model sees each pixel independently so a weight is exact; a segmentation model sees whole chips, and if rare-class chips are never drawn no weight can help.

Q: Why did balancing make my accuracy worse? Because overall accuracy is dominated by the common classes you just down-weighted. Check the per-class recall instead: it almost certainly improved for the rare class, which is the trade you asked for.

Related

• Building Training Datasets from Satellite Imagery — chip filtering decisions that create or reduce the skew.
• Rasterizing Labels into Segmentation Masks — where the ignore value that composes with weighting is set.
• Splitting Chips into Spatially Disjoint Folds — clustered rare classes make fold assignment harder.
• Validating Raster Predictions and Accuracy Assessment — per-class metrics that show whether balancing worked.