Skip to content

Wire SlidingWindowTrigger to a streaming hourly-epoch mosaic driver #78

Description

@jakobtfaber

What to build

dsa110_continuum/mosaic/trigger.py::SlidingWindowTrigger (with module-level WINDOW=12, STRIDE=6, DEC_TOLERANCE, SLEW_STABILITY_COUNT) is the documented design target for the sliding operational mode of hourly-epoch mosaics (per CONTEXT.md "Hourly-epoch mosaic"). Today no production path drives it.

What already exists (do NOT re-implement)

  • dsa110_continuum/mosaic/orchestrator.py:455-551MosaicOrchestrator.create_sliding_window_mosaic(tile_image_ids, tier). Given a list of tile image DB IDs, it queries paths, calls create_mosaic_from_images(...), then calls SlidingWindowTrigger.mark_mosaic_complete(...) to update trigger state. This is a mosaic receiver — no sensor or loop calls it. Caveat: line 484 still imports dsa110_contimg.infrastructure.database.unified (legacy); the design must decide whether to migrate this or keep it.
  • scripts/mosaic_day.py::generate_windows(tiles, window_tiles=12, stride_tiles=6) — pure function that partitions a list of tiles into sliding windows. Tested in tests/test_mosaic_windowing.py (which tests generate_windows, NOT SlidingWindowTrigger — the previous draft of this ticket misattributed the test).

So the missing piece is the driver loop: a sensor that records each new tile via SlidingWindowTrigger.record_tile(...), decides when a window is ready, and calls MosaicOrchestrator.create_sliding_window_mosaic(...) with the right tile IDs.

Design (HITL)

Choose how the driver loop integrates:

  • Standalone streaming daemon: long-running service that records new tiles, fires a mosaic build when the trigger returns ready, archives outputs.
  • Opt-in flag in batch_pipeline.py: --mode {batch,sliding} selects between hour-bin batching and sliding-window grouping; reuses the existing orchestration scaffolding.
  • Dagster sensor + job: a sensor-driven Dagster job that wraps SlidingWindowTrigger and MosaicOrchestrator.create_sliding_window_mosaic.

Capture the choice in an ADR. The ADR must also resolve:

  • Filename / archive layout. The previous draft of this ticket required {date}T{HH}00_mosaic.fits, but that convention encodes only the hour. Sliding mode produces ~30-min-stride windows (STRIDE=6 × ~5 min/tile), so two windows per hour collide on that name. Pick a stride-aware naming scheme (e.g. {date}T{HHMM}_mosaic.fits keyed off the window's mid-MJD, or a window index suffix).
  • Coexistence with batch outputs. If both batch and sliding mosaics run for the same date, do they share an archive root or split? Does the QA gate apply per-window or per-hour?
  • Whether to migrate orchestrator.py:484's legacy dsa110_contimg.infrastructure.database.unified import, since the streaming driver will exercise this path repeatedly.

Implementation (AFK)

Wire the chosen driver. Reuse the canonical hourly-epoch coadd entry (i.e. the resolution of #76) so we don't end up with a third coadd implementation.

Tests

  • Add a sibling to tests/test_mosaic_windowing.py that tests SlidingWindowTrigger itself (the existing file tests mosaic_day.generate_windows).
  • Add an end-to-end driver test: simulate the arrival of N synthetic tiles → trigger fires after WINDOW tiles → mosaic FITS gets written at the expected sliding-window cadence.

Acceptance criteria

  • ADR in docs/adr/ covering integration choice, filename/archive layout, batch/sliding coexistence, and the orchestrator-legacy-DB-import decision.
  • Driver loop exists in code with at least one production-shaped invocation path (script, Dagster sensor, or service entry).
  • End-to-end test exercises the driver loop on synthetic tiles.
  • New unit tests cover SlidingWindowTrigger directly (not via mosaic_day.generate_windows).
  • Driver uses the canonical coadd entry from ADR: choose canonical hourly-epoch coadd implementation (mosaic_day.coadd_tiles vs builder.build_mosaic) #76; no new coadd implementation introduced.
  • CONTEXT.md "Hourly-epoch mosaic" entry's sliding sub-bullet is updated from "design target" to point at the new driver, including the chosen filename convention.

Blocked by

Pointers for a fresh agent

  • dsa110_continuum/mosaic/trigger.py:110SlidingWindowTrigger class.
  • dsa110_continuum/mosaic/trigger.py:66, 90TileRecordResult, TriggerState.
  • dsa110_continuum/mosaic/orchestrator.py:455-551 — existing partial create_sliding_window_mosaic to reuse, including the legacy DB import at line 484.
  • scripts/mosaic_day.py::generate_windows and validate_window_params — pure window-partitioning helpers.
  • tests/test_mosaic_windowing.py — exists but tests mosaic_day.generate_windows, not SlidingWindowTrigger. Add a sibling for the trigger; do not assume the existing file covers it.
  • CONTEXT.md "Hourly-epoch mosaic" — defines batch vs sliding modes; new driver implements the sliding mode.
  • scripts/batch_pipeline.py::bin_tiles_by_hour, build_epoch_tile_sets — the batch mode reference; the ADR must decide whether the sliding driver mirrors its archive layout or chooses a stride-aware one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitectural decision (HITL); typically produces an ADRneeds-triageA maintainer needs to evaluate the issue.refactorRestructuring code without changing external behavior

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions