Logging Structured Events from Raster Tasks

Configure JSON output once, bind context per task, and log one event at the end of each stage:

import structlog

structlog.configure(processors=[
    structlog.processors.TimeStamper(fmt="iso"),
    structlog.processors.add_log_level,
    structlog.processors.JSONRenderer(),
])
log = structlog.get_logger().bind(run="2026-09-18T02", stage="ndvi", tile="33UVP_0412")
log.info("tile_done", seconds=38.4, scenes=9, nodata_frac=0.021, vmax=0.93)

Free-text log lines are written for people reading one at a time; structured events are written for queries over thousands. This page belongs to monitoring and observability for raster pipelines in Cloud Execution & Orchestration.

Anatomy of a Useful Event

Three groups of fields Identity fields — run, stage, tile, scene — say which unit of work the event describes and allow joining. Outcome fields — status, seconds, error type — say what happened. Data-quality fields — scene count, nodata fraction, value range, bytes read — say whether the output is sound. Every event carries all three groups. One event, three kinds of field identity run, stage tile, scene / date attempt outcome status seconds error type, message data quality scenes, bytes read nodata fraction min / max value The right-hand group is what generic logging setups leave out, and what catches quiet failures.

Identity fields make events joinable: with a run and a tile on every line, “which tiles failed at the write stage last night” is one query. Outcome fields make them countable. Data-quality fields make them meaningful for rasters specifically — an event that says status=ok, nodata_frac=1.0 is a failure that no exit code would reveal.

Environment & Setup

Package Version pin Used for
structlog >=24.1 Structured, context-bound logging
dask[distributed] >=2023.1 Worker plugins to configure logging
numpy >=1.23 Data-quality metrics
duckdb >=0.10 Querying JSON lines logs locally 
pip install "structlog>=24.1" "dask[distributed]>=2023.1" "numpy>=1.23" "duckdb>=0.10"

Complete Working Example

import logging
import sys
import time
from contextlib import contextmanager

import numpy as np
import structlog


def configure_logging(level: int = logging.INFO) -> None:
    logging.basicConfig(format="%(message)s", stream=sys.stdout, level=level)
    structlog.configure(
        processors=[
            structlog.contextvars.merge_contextvars,
            structlog.processors.TimeStamper(fmt="iso", utc=True),
            structlog.processors.add_log_level,
            structlog.processors.format_exc_info,
            structlog.processors.JSONRenderer(),
        ],
        logger_factory=structlog.stdlib.LoggerFactory(),
        cache_logger_on_first_use=True,
    )


def quality(arr: np.ndarray, nodata=None) -> dict:
    a = np.asarray(arr)
    invalid = np.isnan(a) if a.dtype.kind == "f" else np.zeros(a.shape, bool)
    if nodata is not None:
        invalid |= a == nodata
    valid = a[~invalid]
    return {
        "nodata_frac": round(float(invalid.mean()), 4),
        "vmin": float(valid.min()) if valid.size else None,
        "vmax": float(valid.max()) if valid.size else None,
    }


@contextmanager
def stage(name: str, **ctx):
    log = structlog.get_logger().bind(stage=name, **ctx)
    t0 = time.perf_counter()
    extra: dict = {}
    try:
        yield extra
    except Exception as exc:
        log.error("stage_failed", seconds=round(time.perf_counter() - t0, 2),
                  error=type(exc).__name__, message=str(exc)[:500], **extra)
        raise
    else:
        log.info("stage_done", seconds=round(time.perf_counter() - t0, 2), **extra)


def process_tile(run_id: str, tile_id: str):
    structlog.contextvars.bind_contextvars(run=run_id, tile=tile_id)
    with stage("load") as ev:
        cube, items = load_tile(tile_id)
        ev.update(scenes=len(items))
    with stage("ndvi") as ev:
        ndvi = compute_ndvi(cube)
        ev.update(quality(ndvi))
    with stage("write") as ev:
        path = write_cog(ndvi, tile_id)
        ev.update(path=path)

The context manager guarantees exactly one event per stage — success or failure — with the duration measured the same way every time, and lets the stage body add whatever data fields it knows. Binding run and tile through contextvars means every line emitted inside the task, including from helper functions that know nothing about tiles, carries them automatically.

Getting Worker Logs Out

Configure logging where the code runs Logging configured in the client process does not apply to Dask workers, which are separate processes. A worker plugin runs the configuration on every worker at startup, so tasks emit JSON to stdout. The container platform or cloud agent then ships stdout to a log store such as CloudWatch. Client config does not reach workers client register plugin worker 1 → stdout worker 2 → stdout worker n → stdout log agent log store

This is the step most setups miss. The logging configuration applied in the notebook or driver script only affects that process; Dask workers are separate processes, often on other machines, and log with their own defaults. A worker plugin applies the same configuration on every worker, including ones added later by autoscaling:

from dask.distributed import WorkerPlugin


class JSONLogging(WorkerPlugin):
    def setup(self, worker):
        configure_logging()

client.register_plugin(JSONLogging())

On AWS Batch and in containers generally, writing to stdout is enough: the platform’s log driver ships it to CloudWatch or equivalent. Writing log files inside containers is an anti-pattern — they disappear with the container.

Querying the Events

Once events are JSON lines, questions become queries. Locally, DuckDB reads a directory of log files directly:

SELECT stage, status, count(*) AS n, median(seconds) AS p50
FROM read_json_auto('logs/*.jsonl')
WHERE run = '2026-09-18T02'
GROUP BY ALL ORDER BY stage;

The same query structure works in CloudWatch Logs Insights or any log store that parses JSON. Keep field names stable and typed consistently — a field that is sometimes a number and sometimes a string breaks aggregation — and treat the event schema as an interface that changes deliberately.

What Not to Log

Structured logging makes it easy to add fields, which makes it easy to add the wrong ones. Do not log credentials, signed URLs or full request headers; a presigned S3 URL in a log line is a working credential for its lifetime. Do not log arrays or large objects — log their summary statistics instead. And do not log per-chunk or per-pixel events at INFO: the volume grows with data size rather than with the number of tiles, and the log bill follows. A useful rule is that the number of INFO events per run should be a small multiple of the number of tiles.

Logging Retries and Attempts

Orchestrators retry failed tasks, and without care each attempt looks like a separate, contradictory event for the same tile and stage. Bind an attempt field from the orchestrator’s retry counter — Prefect exposes it on the task run context, AWS Batch through the AWS_BATCH_JOB_ATTEMPT environment variable — so that a failure followed by a success reads as exactly that. Tiles that routinely need a second attempt are worth listing in each run’s summary: flaky reads from one bucket region or one oversized scene tend to show up there weeks before they fail outright.

Verification

Every tile, every stage, exactly once After a run, count events per tile and stage. Every tile should have exactly one event for each stage it reached. Missing events mean a stage died without logging; duplicates mean retries are not distinguished by an attempt field. Event completeness 1 per stage healthy 0 for a stage died without logging 2+ for a stage retries need an attempt field A process killed for memory is the usual cause of a missing event. 
import duckdb

gaps = duckdb.sql("""
    SELECT tile, count(DISTINCT stage) AS stages
    FROM read_json_auto('logs/*.jsonl') WHERE run = '2026-09-18T02'
    GROUP BY tile HAVING stages < 3
""").df()
print(f"{len(gaps)} tiles missing stage events")

Common Errors

Worker logs are plain text, not JSON

Logging was configured only on the client. Register a worker plugin that configures it on every worker.

Every line is logged twice

Both structlog and a root handler emit. Route structlog through the standard library logger and configure a single handler.

Context from one tile appears on another’s events

Context was bound globally in a thread pool. Use contextvars and clear them at the start of each task.

Log volume is enormous

Events are emitted per chunk or at DEBUG. Return to one event per tile and stage at INFO.

Frequently Asked Questions

Q: Why structured logs instead of plain text? Because structured events can be filtered, counted and joined by field, which turns questions like which tiles failed in the last run into queries rather than searches through text.

Q: Should I use structlog or the standard logging module? Either can emit JSON. structlog makes binding context and rendering JSON simpler; the standard module with a JSON formatter works if you prefer no extra dependency.

Q: What data-quality fields are most useful? Nodata fraction, minimum and maximum value, scene count and bytes read. Together they catch inverted masks, missing scaling, empty searches and full-file downloads.

Q: Where should containerised tasks write logs? To stdout. The container platform ships stdout to the log store; files inside containers disappear with them.