Pinning GDAL and PROJ Versions Reproducibly

Two builds of the same commit should produce the same stack. That takes a hashed lock file, a pinned base image, and a recorded version manifest:

pip-compile --generate-hashes --output-file requirements.lock requirements.in
FROM python:3.11.9-slim-bookworm
RUN pip install --require-hashes -r requirements.lock
RUN python -c "import rasterio, pyproj, json; json.dump( \
    {'rasterio': rasterio.__version__, 'gdal': rasterio.__gdal_version__, \
     'proj': pyproj.proj_version_str}, open('/app/versions.json','w'))"

Reproducibility is the reason the images in Containerizing Geospatial Python Environments exist at all.

Why This Arises in Remote Sensing Workflows

The geospatial stack has more moving parts below Python than most. A reprojection’s numeric result depends on the PROJ version and on which datum grids are installed. A COG read’s request pattern depends on the GDAL version’s defaults. A compression setting may be unavailable in one build and present in another. None of these appear in a pip freeze if you are looking only at Python packages.

The consequence is a specific, frustrating class of bug: the same code, the same inputs, and different numbers on two machines — or on the same machine before and after a rebuild. It presents as a data problem, gets investigated as a data problem, and turns out to be a library problem that nobody recorded.

Pinning does not prevent change; it makes change deliberate. The goal is that any difference in behaviour between two runs can be traced to a commit that someone reviewed, rather than to the day the image happened to be built.

Four layers, four different pinning mechanisms The application is pinned by the commit, the Python packages by a hashed lock file, the native GDAL and PROJ by the wheel or the base image, and the datum grids by what is shipped in the image. A pin at one level says nothing about the levels below it. Each layer needs its own pin your pipeline code pinned by: the commit SHA in the image tag rasterio · rioxarray · pyproj · numpy pinned by: hashed lock file GDAL · PROJ · libtiff · libcurl pinned by: the wheel, or the base image tag proj.db + datum grids pinned by: what you ship — nothing, by default the layer most often left floating A pinned rasterio with a network-downloaded grid is still not reproducible.

Environment & Setup

Tool Version Why
pip-tools or uv current Produces a fully-resolved, hashed lock file
pip ≥23.1 --require-hashes enforcement
rasterio ≥1.3.0 Exposes __gdal_version__
pyproj ≥3.4 Exposes proj_version_str and the data directory 
pip install pip-tools

Complete Working Example

The three pieces: a lock file, an expectation file, and a check that fails when they diverge.

Pinning strength by mechanism A floating tag pins nothing, a version pin identifies a release, a hash identifies an exact artefact and a digest identifies an exact image. Each step up removes a class of silent change, and only the last two survive a re-upload or a rebuild. How strong is each pin, really mechanism pins survives python:3.11-slim nothing below the minor version nothing python:3.11.9-slim-bookworm the interpreter and distribution a base rebuild? no rasterio==1.3.9 the release a re-upload? no --require-hashes the exact wheel bytes yes FROM image@sha256:… the exact image yes Use the bottom two for anything whose numeric output you will have to defend. 
"""stack_check.py — record and verify the resolved geospatial stack."""
import json
import os
import sys

import pyproj
import rasterio

EXPECTED_PATH = os.environ.get("STACK_EXPECTED", "stack_expected.json")
RECORDED_PATH = os.environ.get("STACK_RECORDED", "/app/versions.json")


def resolved() -> dict:
    """Everything that can change behaviour, as far down the stack as we can see."""
    return {
        "python": sys.version.split()[0],
        "rasterio": rasterio.__version__,
        "gdal": rasterio.__gdal_version__,
        "proj": pyproj.proj_version_str,
        "proj_data_dir": pyproj.datadir.get_data_dir(),
        "numpy": __import__("numpy").__version__,
        "gdal_drivers": len(rasterio.drivers.raster_driver_extensions()),
    }


def record(path: str = RECORDED_PATH) -> dict:
    info = resolved()
    os.makedirs(os.path.dirname(path), exist_ok=True)
    with open(path, "w") as fh:
        json.dump(info, fh, indent=2, sort_keys=True)
    return info


def verify(expected_path: str = EXPECTED_PATH) -> None:
    """Fail loudly when the resolved stack differs from the reviewed one."""
    with open(expected_path) as fh:
        expected = json.load(fh)
    actual = resolved()

    drift = {k: (expected[k], actual[k])
             for k in expected
             if k in actual and expected[k] != actual[k]}
    if drift:
        lines = [f"  {k}: expected {exp!r}, got {act!r}" for k, (exp, act) in drift.items()]
        raise SystemExit("geospatial stack drifted:\n" + "\n".join(lines))
    print("stack matches expectations:", json.dumps(actual, indent=2, sort_keys=True))


if __name__ == "__main__":
    if len(sys.argv) > 1 and sys.argv[1] == "record":
        print(json.dumps(record(), indent=2, sort_keys=True))
    else:
        verify()

Wire it into the build so a drifted stack cannot ship:

COPY stack_expected.json /app/stack_expected.json
COPY stack_check.py /app/stack_check.py
RUN python /app/stack_check.py record && python /app/stack_check.py

The proj_data_dir field is worth including even though it is a path rather than a version: it reveals whether PROJ is reading the data bundled with the wheel or a system copy, which is the usual explanation when two images with identical version numbers still disagree.

Variant Patterns

1. Pinning when using a system GDAL

Building rasterio against a system GDAL moves the pin from the wheel to the base image tag, so the tag has to be exact.

# Not :latest, not :3.8 — the full version, so a rebuild in six months is identical
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.8.4

RUN pip install --require-hashes --no-binary rasterio -r requirements.lock

Digest pinning goes further and removes even the possibility of a retagged image:

FROM ghcr.io/osgeo/gdal@sha256:9c1e0f...   # immutable

Digests are unreadable and unambiguous, which is exactly the trade a reproducible build wants.

2. Shipping datum grids instead of downloading them

Where the transformation grid comes from With network access enabled, PROJ may fetch a grid at runtime, so the same job can pick a different transformation path depending on connectivity and cache state. Shipping the grids in the image and disabling network access makes the choice deterministic and removes a runtime dependency. PROJ_NETWORK: two behaviours PROJ_NETWORK=ON transform call grid present? no → download at runtime result depends on connectivity PROJ_NETWORK=OFF + shipped grids transform call grid in image always the same path chosen reproducible, and no runtime egress Ship only the grids your area needs — the full set is large, and most pipelines use two or three. 
ENV PROJ_NETWORK=OFF
COPY proj_grids/ /usr/local/share/proj/

Which grids matter depends on the CRSs in use; the transformation-path discussion in Transforming Point Coordinates with pyproj explains how to find out which ones your operations select.

3. Recording the stack in every output

The image records its stack; outputs should too, so a file can be traced back to what produced it.

import json
import rasterio

stack = json.load(open("/app/versions.json"))
with rasterio.open(dst_path, "w", **profile) as dst:
    dst.write(arr, 1)
    dst.update_tags(**{f"stack_{k}": str(v) for k, v in stack.items()})

When two products disagree, comparing their tags takes seconds; reconstructing which image built each takes hours.

Detecting Drift Before It Ships

The check above is a build gate, but drift can also be monitored across the fleet.

Compare recorded stacks across images. If several pipelines share a base, their versions.json files should be identical, and a diff is the fastest way to find the one that was rebuilt against something newer.

Watch for silent platform changes. A wheel built for a new manylinux tag can bring a different bundled GDAL under the same rasterio version, which the version pin does not catch but the recorded gdal field does.

Schedule the update rather than absorbing it. A monthly bump of the lock file, run through the same smoke test and a numeric comparison on a fixed test scene, turns library updates into a reviewed change with evidence — the same discipline applied to data conventions in Auditing CRS and nodata Drift Across a Collection.

Common Errors

ERROR: In --require-hashes mode, all requirements must have their versions pinned

A transitive dependency is missing from the lock file. Regenerate it with --generate-hashes rather than hand-editing; hand-maintained lock files drift within a week.

The lock file resolves differently on another machine

The lock was generated on a different Python version or platform. Generate it inside the same base image the build uses, so the resolution sees the same environment.

GDAL version changed without any pin changing

The wheel was rebuilt for a new platform tag, or the base image’s floating tag moved. Pin the base image by digest and record the GDAL version so the change is visible.

Frequently Asked Questions

Q: Does pinning rasterio pin GDAL? When installing from binary wheels, effectively yes: each rasterio wheel bundles a specific GDAL and PROJ build, so a pinned rasterio version and platform give the same libraries. When building against a system GDAL, it pins nothing below the Python layer.

Q: Why do hashes matter if versions are pinned? A version pin identifies a release; a hash identifies the exact artefact. Hashes protect against a re-uploaded file, a mirror serving something different, and a wheel rebuilt for a new platform tag — all of which change behaviour without changing the version.

Q: How often should pins be updated? On a schedule you control rather than by accident — monthly is common. The point of pinning is not to freeze forever but to make every change deliberate, reviewed and attributable to a commit.