Reading a COG over S3 Without Downloading It

To read pixels from a Cloud-Optimized GeoTIFF on S3 without pulling the whole object, open the s3:// URI with rasterio and pass a Window to src.read() — GDAL’s /vsis3/ backend then issues HTTP Range requests for only the internal tiles that overlap your window:

import rasterio
from rasterio.windows import from_bounds

uri = "s3://sentinel-cogs/sentinel-s2-l2a-cogs/36/N/YF/2023/6/S2A_36NYF_20230615_0_L2A/B04.tif"

with rasterio.Env(AWS_NO_SIGN_REQUEST="YES"):
    with rasterio.open(uri) as src:
        window = from_bounds(499980, 79000, 509980, 89000, transform=src.transform)
        red = src.read(1, window=window)  # only the overlapping tiles are fetched
        print(red.shape)

Reading pixels this way — rather than whole files — is the payoff of the tiled layout described in Understanding Cloud-Optimized GeoTIFF Structure.

Why This Arises in Remote Sensing Workflows

A single Sentinel-2 band on the AWS sentinel-cogs bucket is roughly 100–150 MB, and a full scene across every band is well over a gigabyte. When your area of interest is a 10 km field block inside a 110 km tile, downloading the whole file to extract a few thousand pixels wastes egress budget, disk, and wall-clock time. Multiply that by thousands of scenes in a time series and the naive “download then crop” approach becomes the dominant cost in the pipeline.

This page is specifically about reading pixel data over S3. Its sibling, How to Read COG Headers Without Downloading Full Files, stops at the header — CRS, bounds, overviews — and never touches a tile. Here we go one step further: after validating the header, we fetch the actual reflectance values for a sub-region, still transferring only kilobytes to a few megabytes instead of the entire object.

The mechanism that makes this possible is the same tiled, indexed layout that defines a COG. Because pixels are stored in independently addressable tiles (typically 512×512) and the header carries a byte-offset table pointing at each one, GDAL can translate “read this window” into a small set of Range: bytes=start-end requests. This sits inside Understanding Cloud-Optimized GeoTIFF Structure and, for the broader picture of catalog-driven ingestion, Core Raster Fundamentals & STAC Mapping.

How a Windowed S3 Read Maps to Range Requests

The diagram traces what happens between your src.read(window=...) call and the tile bytes that actually cross the network.

The key insight is that the size of the transfer scales with the window, not with the file. A 1024×1024 window over a 10980×10980 band touches a handful of tiles regardless of how large the full band is. To go even smaller for previews, read from an overview level so GDAL fetches decimated tiles — the same principle covered in Optimizing Rasterio Window Reads for Memory Efficiency.

Environment & Setup

pip install "rasterio>=1.3.0" "boto3>=1.26"

For a public bucket like sentinel-cogs, no credentials are needed — set AWS_NO_SIGN_REQUEST=YES. For a private bucket, export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION, or attach an IAM role; GDAL’s /vsis3/ resolves them through the standard AWS chain.

Complete Working Example

The function below reads a reflectance sub-array for a geographic bounding box directly from S3. It validates the header first, clips the window to the dataset extent so an out-of-bounds request cannot fault, and returns both the array and the window’s affine transform so the result stays georeferenced.

import rasterio
from rasterio.windows import from_bounds, Window
from rasterio.errors import RasterioIOError
from affine import Affine
import numpy as np


def read_s3_cog_window(
    uri: str,
    bbox: tuple[float, float, float, float],
    band: int = 1,
    *,
    unsigned: bool = False,
    region: str = "us-west-2",
) -> tuple[np.ndarray, Affine]:
    """
    Read a windowed sub-array from a COG on S3 without downloading the full object.

    Parameters
    ----------
    uri : str
        s3:// or /vsis3/ path to the COG.
    bbox : (left, bottom, right, top)
        Bounding box in the COG's own CRS (not lon/lat unless the COG is EPSG:4326).
    band : int
        1-based band index to read.
    unsigned : bool
        True for public Open Data buckets that reject signed requests.
    region : str
        AWS region hosting the bucket.

    Returns
    -------
    (array, transform)
        The pixel array for the window and its affine transform.
    """
    env = {
        # Skip the bucket LIST that GDAL would otherwise run before opening
        "GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR",
        # Merge adjacent tile byte ranges into a single request
        "GDAL_HTTP_MERGE_CONSECUTIVE_RANGES": "YES",
        "AWS_REGION": region,
    }
    if unsigned:
        env["AWS_NO_SIGN_REQUEST"] = "YES"

    try:
        with rasterio.Env(**env):
            with rasterio.open(uri) as src:
                # Build a floating window from the bbox, then clip to the raster
                win = from_bounds(*bbox, transform=src.transform)
                full = Window(0, 0, src.width, src.height)
                win = win.intersection(full).round_offsets().round_lengths()
                if win.width <= 0 or win.height <= 0:
                    raise ValueError("bbox does not overlap the raster extent")

                # Only the tiles overlapping `win` are pulled from S3 here
                array = src.read(band, window=win)
                # Transform for the sub-array, so downstream code stays georeferenced
                win_transform = src.window_transform(win)
                return array, win_transform
    except RasterioIOError as exc:
        raise RuntimeError(f"Could not read window from {uri!r}: {exc}") from exc


if __name__ == "__main__":
    s2_red = (
        "s3://sentinel-cogs/sentinel-s2-l2a-cogs/36/N/YF/2023/6"
        "/S2A_36NYF_20230615_0_L2A/B04.tif"
    )
    # UTM 36N coordinates (metres) — a 10 km block inside the tile
    arr, tf = read_s3_cog_window(
        s2_red, bbox=(499980, 79000, 509980, 89000), unsigned=True
    )
    print(f"window shape : {arr.shape}")
    print(f"dtype        : {arr.dtype}")
    print(f"transform    : {tf}")
    print(f"valid pixels : {int(np.count_nonzero(arr))}")

Passing coordinates in the COG’s own CRS is deliberate: from_bounds interprets the box against src.transform, which is in projected units. If your area of interest is in longitude/latitude, reproject it to the raster CRS first — see Reprojecting a Raster from UTM to WGS84 and Fixing EPSG Mismatches in rasterio.open.

Variant Patterns

1. Overview-aware preview read

For a quick thumbnail or a coarse coverage check, read from a decimated overview by passing out_shape. GDAL selects the nearest overview level and fetches its (much smaller) tiles instead of full-resolution ones.

import rasterio
from rasterio.enums import Resampling

uri = "s3://sentinel-cogs/sentinel-s2-l2a-cogs/36/N/YF/2023/6/S2A_36NYF_20230615_0_L2A/B04.tif"

with rasterio.Env(AWS_NO_SIGN_REQUEST="YES"):
    with rasterio.open(uri) as src:
        # Decimate the full band to 512x512 — reads an overview, not the base level
        preview = src.read(
            1,
            out_shape=(512, 512),
            resampling=Resampling.average,
        )
        print(preview.shape)  # (512, 512), a few hundred KB transferred

2. Private bucket with the /vsis3/ prefix

For an authenticated bucket, use the explicit /vsis3/ prefix (or an s3:// URI) and let GDAL resolve credentials from the environment or an IAM role. Never embed keys in the path.

import os
import rasterio
from rasterio.windows import Window

vsis3_path = "/vsis3/my-private-bucket/tiles/scene_042/B08.tif"

with rasterio.Env(
    GDAL_DISABLE_READDIR_ON_OPEN="EMPTY_DIR",
    AWS_REGION=os.environ["AWS_REGION"],
    # No AWS_NO_SIGN_REQUEST — GDAL signs with the resolved credentials
):
    with rasterio.open(vsis3_path) as src:
        win = Window(col_off=2048, row_off=2048, width=1024, height=1024)
        nir = src.read(1, window=win)
        print(nir.mean())

3. Aligning windows to tile boundaries

Reads that start and end on internal tile edges avoid fetching partial tiles you then discard. Snap the window offsets and lengths to the block size reported by the dataset.

import rasterio
from rasterio.windows import Window

uri = "s3://sentinel-cogs/sentinel-s2-l2a-cogs/36/N/YF/2023/6/S2A_36NYF_20230615_0_L2A/B04.tif"

with rasterio.Env(AWS_NO_SIGN_REQUEST="YES"):
    with rasterio.open(uri) as src:
        bh, bw = src.block_shapes[0]  # e.g. (512, 512) for a tiled COG

        def snap(value: int, size: int) -> int:
            return (value // size) * size

        col = snap(3000, bw)
        row = snap(3000, bh)
        win = Window(col, row, bw * 2, bh * 2)  # two tiles square, edge-aligned
        data = src.read(1, window=win)
        print(win, data.shape)

Common Errors

CPLE_AWSAccessDenied: HTTP response code: 403

The request was signed but the credentials lack access, or the bucket is public and rejected a signed request. For public Open Data buckets set AWS_NO_SIGN_REQUEST=YES; for private buckets confirm the IAM policy grants s3:GetObject on the key and that AWS_REGION matches the bucket.

RasterioIOError: ... not recognized as a supported file format

The s3:// path resolved to an error document (an XML AccessDenied or NoSuchKey body) rather than TIFF bytes. Verify the exact key with aws s3 ls s3://bucket/key and check the region — a cross-region request to the wrong endpoint often returns HTML.

Whole-file transfer despite passing a window

The file is not actually a tiled COG — a striped GeoTIFF forces GDAL to read entire scanlines, so a small window still pulls large byte ranges. Confirm src.block_shapes[0] has a block height greater than 1, and see How to Read COG Headers Without Downloading Full Files for a validity check.

Related

• Understanding Cloud-Optimized GeoTIFF Structure — the parent section on tiled layout and overview pyramids that makes partial reads possible.
• How to Read COG Headers Without Downloading Full Files — the header-only sibling; validate CRS and bounds before you read any pixels.
• Optimizing Rasterio Window Reads for Memory Efficiency — tile-aligned windows and overview selection for large-array reads.
• Querying STAC Catalogs Programmatically — resolve the S3 asset URIs you feed into these windowed reads.