Implementation Plan

Implementation Plan

Migrated from PLAN.md in the main repository.

Overview

Complete refactor of MELODIES-MONET addressing:

• driver.py: 3,116 lines → ~50 modules (<500 lines each)
• plotting(): 1,321-line method → individual plotter classes
• No type hints → Full type hints + mypy strict
• 211 print() → Structured logging
• 4 test files → Comprehensive test suite (>80% coverage)

Key Design Principles

1. Uniform Pairing Logic: A single, consistent pairing engine that works across all observation types (surface, aircraft, satellite, sonde). The pairing strategy is determined by data geometry (point, profile, swath, grid) rather than data source.

2. Synthetic Data for Testing: Generate test data programmatically - no dependency on external datasets. Tests are self-contained and reproducible.

3. xarray-Only Data Model: All data flows through the system as xr.Dataset. No pandas in core pairing/analysis logic.

   Component	Data Structure
   Model data	xr.Dataset with dims (time, level, lat, lon)
   Point obs	xr.Dataset with dims (time, site) + lat/lon coords
   Track obs	xr.Dataset with dims (time,) + lat/lon/alt coords
   Profile obs	xr.Dataset with dims (time, level) + lat/lon coords
   Swath obs	xr.Dataset with dims (time, scanline, pixel)
   Grid obs	xr.Dataset with dims (time, lat, lon)
   Paired output	xr.Dataset

   pandas limited to:

   • I/O adapters: Read CSV/tabular → convert to xarray
   • Statistics output: Final tables for export

Implementation Phases

Phase	Description	Status
1	Foundation	COMPLETE
2	Synthetic Data & Testing Infrastructure	COMPLETE
3	Configuration	COMPLETE
4	Core Data Classes	COMPLETE
5	Unified Pairing Engine	COMPLETE
6	Model Implementations	COMPLETE
7	Observation Implementations	COMPLETE
8	Pipeline	COMPLETE
9	Plotting	COMPLETE
10	Statistics	COMPLETE
11	CLI	COMPLETE
12	Documentation & Polish	PENDING

Technical Decisions

Aspect	Decision
Python version	3.11+
Type checking	mypy with selected stricter checks
Validation	Pydantic v2
Logging	Python logging module
CLI	Typer
Testing	pytest + pytest-cov + synthetic data
Formatting	Black + isort
Data model	xarray-only (pandas for I/O adapters and stats tables only)
Dependencies	Continue using monet/monetio