Converting a GeoTIFF to a COG with rio-cogeo

To convert an existing GeoTIFF into a valid Cloud-Optimized GeoTIFF, translate it with rio-cogeo, which tiles the data, builds overviews and writes the header-first layout in one pass:

from rio_cogeo.cogeo import cog_translate
from rio_cogeo.profiles import cog_profiles

cog_translate(
    "scene.tif",
    "scene_cog.tif",
    cog_profiles.get("deflate"),
    overview_resampling="average",
    web_optimized=False,
    in_memory=False,
)

That single call is the shortest correct version of the workflow described in Writing and Validating Cloud-Optimized GeoTIFFs.

Why This Arises in Remote Sensing Workflows

Most rasters in circulation were not written for cloud access. Archives predate the COG convention, desktop tools export strip-organised files by default, and pipelines that use rasterio.open(..., "w") without a tiling profile produce exactly the kind of file that makes remote reads expensive. Conversion is therefore a routine ingest step rather than a one-off migration: data arrives, gets converted, and only then enters the archive.

The reason it matters is measurable rather than aesthetic. A windowed read against a striped file transfers whole rows; a preview against an overview-free file decimates full-resolution data. Both are invisible locally, where the file is on a fast disk, and both dominate the cost as soon as the file moves to object storage — the mechanics are laid out in Reading a COG over S3 Without Downloading.

What the conversion actually rearranges The source file stores strips of pixels with its directory structures interleaved and no overviews. After conversion the file begins with its header and image file directories, then the overview levels from coarsest to finest, then the full-resolution tiles, so a reader can describe the file in one request and preview it in two. Same pixels, different order on disk before strips of full-resolution pixels trailing directories reader needs several requests to describe the file, and has no coarse level to preview from cog_translate after IFDs overviews tiled full-resolution pixels one request describes the file; a preview reads only the shaded overview block Nothing about the pixel values changes — only where the bytes live and how they are indexed.

Environment & Setup

Package Version Why
rio-cogeo ≥5.0 cog_translate, cog_validate and the named profiles
rasterio ≥1.3.0 Underlying I/O and the GDAL COG driver
GDAL ≥3.4.0 Provides the codecs and the layout writer 
pip install "rio-cogeo>=5.0" "rasterio>=1.3.0"

Complete Working Example

This function converts one file, validates the result, and only then replaces the original — the ordering that makes a batch run safe to re-run after a failure.

import os
import shutil
import tempfile

import rasterio
from rio_cogeo.cogeo import cog_translate, cog_validate
from rio_cogeo.profiles import cog_profiles


def to_cog(
    src_path: str,
    dst_path: str,
    *,
    profile_name: str = "deflate",
    blocksize: int = 512,
    overview_resampling: str = "average",
) -> dict:
    """Convert a GeoTIFF to a validated COG. Returns the output's structural facts."""
    profile = cog_profiles.get(profile_name)
    profile.update(blockxsize=blocksize, blockysize=blocksize, BIGTIFF="IF_SAFER")

    # Write to a temp file first: a crashed conversion must not leave a half-written COG
    tmp_fd, tmp_path = tempfile.mkstemp(suffix=".tif", dir=os.path.dirname(dst_path) or ".")
    os.close(tmp_fd)
    try:
        cog_translate(
            src_path,
            tmp_path,
            profile,
            overview_resampling=overview_resampling,
            in_memory=False,          # stream through disk; large scenes will not fit in RAM
            quiet=True,
            config={"GDAL_NUM_THREADS": "ALL_CPUS"},
        )
        is_valid, errors, _ = cog_validate(tmp_path)
        if not is_valid:
            raise ValueError(f"conversion produced an invalid COG: {errors}")

        with rasterio.open(tmp_path) as src:
            facts = {
                "block": src.block_shapes[0],
                "overviews": src.overviews(1),
                "compress": src.profile.get("compress"),
                "nodata": src.nodata,
            }
        shutil.move(tmp_path, dst_path)   # atomic on the same filesystem
        return facts
    finally:
        if os.path.exists(tmp_path):
            os.remove(tmp_path)


if __name__ == "__main__":
    print(to_cog("S2A_36NYF_20230615_B04.tif", "S2A_36NYF_20230615_B04_cog.tif"))

Two arguments carry more weight than the rest. in_memory=False forces streaming through a temporary file rather than buffering the whole raster, which is the difference between converting a Sentinel-2 band on a small container and being killed by the OOM reaper. overview_resampling must match the data class, for the reasons developed in Adding Internal Overviews with the Right Resampling.

Variant Patterns

1. Choosing a profile by dtype and audience

cog_profiles ships named presets, and the choice comes down to who reads the output and what the values are.

Which named profile to pass The deflate profile suits integer data destined for public archives. The zstd profile suits internal intermediates where write speed matters. The lzw profile exists for maximum compatibility with very old readers, and the raw profile for uncompressed scratch outputs where read speed dominates. cog_profiles.get(name) profile good for why deflate public archives, any dtype every GDAL build in circulation can read it zstd internal intermediates similar size at roughly a third of the write time lzw legacy consumers widest compatibility, weakest ratio raw scratch, read-dominated no decode cost, largest files, rarely worth it remotely Every preset can be overridden — the presets set sensible predictors per dtype, which is their real value.

2. Web-optimized output for tile serving

If the file will back a map, web_optimized=True reprojects to Web Mercator and aligns the overview levels to the standard tile pyramid, so a tile request maps to exactly one overview level.

cog_translate(
    "scene.tif",
    "scene_web.tif",
    cog_profiles.get("deflate"),
    web_optimized=True,      # reproject to EPSG:3857 and snap to the tile grid
    zoom_level_strategy="upper",
    overview_resampling="average",
    in_memory=False,
)

This is a genuine reprojection, so it resamples: keep the analysis copy in its native CRS and treat the web copy as a derived product rather than a replacement, following the reasoning in Reprojecting a Raster from UTM to WGS84.

3. Batch conversion over an archive

Conversion is CPU-bound, so processes beat threads here — the opposite of the metadata sweeps in Automating Metadata Extraction for Batch Raster Jobs.

from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path


def convert_tree(src_dir: str, dst_dir: str, workers: int = 8) -> list[tuple[str, str]]:
    """Convert every .tif under src_dir, returning (path, error) for the failures."""
    src_files = sorted(Path(src_dir).rglob("*.tif"))
    failures: list[tuple[str, str]] = []
    with ProcessPoolExecutor(max_workers=workers) as pool:
        futures = {}
        for src in src_files:
            dst = Path(dst_dir) / src.relative_to(src_dir)
            dst.parent.mkdir(parents=True, exist_ok=True)
            futures[pool.submit(to_cog, str(src), str(dst))] = src

        for fut in as_completed(futures):
            src = futures[fut]
            try:
                fut.result()
            except Exception as exc:            # keep going; record what failed
                failures.append((str(src), repr(exc)))
    return failures
Where batch conversion stops scaling Conversion throughput rises nearly linearly with worker processes up to about the physical core count, then flattens as compression saturates memory bandwidth and the temporary files saturate the disk. Adding workers past that point increases memory use and failure probability without increasing throughput. Files converted per minute, 8-core machine high 0 8 workers = core count past here, throughput is flat and memory pressure rises 1 4 12 24 worker processes Each worker holds a temporary file and a compression buffer; disk space, not CPU, is the usual first limit. Set the temp directory explicitly if the default lives on a small container filesystem.

Proving the Conversion Was Worth It

A conversion that nobody measures tends to drift: someone disables overviews to save space, a profile changes, and six months later the archive reads no better than it did before. Two measurements, taken once per archive, keep that honest.

The first is request count and bytes on a representative read. Open the object over HTTPS with GDAL’s curl reporting enabled and count what a single windowed read costs before and after conversion. A converted file should describe itself in one or two requests and satisfy a small window with a handful of small ranged GETs; the original will typically issue similar request counts but transfer one to two orders of magnitude more bytes, because each request pulls an entire strip.

import rasterio
from rasterio.windows import Window

CFG = {
    "GDAL_DISABLE_READDIR_ON_OPEN": "EMPTY_DIR",
    "CPL_CURL_VERBOSE": "YES",          # request log goes to stderr
    "CPL_DEBUG": "ON",
}

with rasterio.Env(**CFG):
    with rasterio.open("https://example-bucket.s3.amazonaws.com/scene_cog.tif") as src:
        print(src.profile["tiled"], src.block_shapes[0], src.overviews(1))
        arr = src.read(1, window=Window(2048, 2048, 512, 512))
        print("window read:", arr.shape, arr.dtype)

The second is preview cost. Request a 512-pixel version of the whole scene with out_shape and compare the bytes transferred against the same request on the original. On a converted file GDAL serves it from an overview level and moves under a megabyte; on the original it decimates full-resolution data and moves the whole band. This is the measurement that justifies overviews to anyone questioning the extra storage, and the mechanics behind it are covered in Reading a COG over S3 Without Downloading.

Record both numbers in the conversion job’s output alongside the file inventory. When a later change regresses them, the diff is visible immediately rather than being discovered through a support ticket about a slow map.

What Conversion Cannot Fix

Conversion rewrites layout, not semantics, and it is worth being explicit about the problems it leaves exactly where it found them.

A missing or wrong nodata declaration survives conversion untouched. The converted file will have the same undeclared fill, and every downstream statistic will keep averaging fill values into real data. If the source is known to use a sentinel that it never declared, set it during conversion by passing an explicit nodata to cog_translate, and record that you did — the diagnosis is in Extracting nodata and dtype from a GeoTIFF.

A mislabelled CRS also survives. Conversion neither validates nor corrects the projection tag, so a file that claims EPSG:4326 while holding UTM metres becomes a beautifully structured file that is still thousands of kilometres out of position. That check belongs in the ingest step before conversion, using the bounds-versus-CRS test from Fixing EPSG Mismatches in rasterio.open.

An unsuitable dtype survives as well. Converting a float64 array that only ever holds small integers produces a COG that is four times larger than it needs to be, and no compression setting recovers that. Casting belongs upstream of conversion, in the code that produced the array.

Finally, conversion does nothing about where the file lives. A perfectly formed COG in the wrong region is slower and more expensive to read than a mediocre one in the right region, which is why region placement is decided before any of this matters.

Common Errors

MemoryError or an OOM kill during conversion

in_memory defaulted to true for a raster large enough to matter. Pass in_memory=False so the translation streams through a temporary file, and point TMPDIR at a filesystem with room for a full copy.

The source had no overviews and overview_resampling was left at a level that produced none. Pass an explicit overview_level or let rio-cogeo compute the levels, then re-validate.

The output opens locally but 403s over HTTPS

Nothing to do with the conversion: the object was uploaded without public read, or the signed URL expired. Check the object policy before re-converting anything.

Frequently Asked Questions

Q: Can rio-cogeo convert a file in place? Not safely. cog_translate reads the source while writing the destination, so writing to the same path risks a truncated file. Write to a temporary path and replace the original after validation succeeds.

Q: Why is my converted file larger than the original? Overviews add roughly a third to the size, and an unsuitable predictor can add more. A COG is expected to be larger than a bare compressed GeoTIFF; the extra bytes are what make partial reads cheap.

Q: Does conversion change my pixel values? No, unless you change dtype, nodata or resampling. The full-resolution data is copied as-is; only new overview levels are computed, and they live alongside the original pixels.